An Introduction To Stochastic Epidemic Models
An Introduction to Stochastic EpidemicModelsLinda J. S. AllenDepartment of Mathematics and StatisticsTexas Tech UniversityLubbock, Texas 79409-1042, U.S.A.linda.j.allen@ttu.edu1 IntroductionThe goals of this chapter are to provide an introduction to three differentmethods for formulating stochastic epidemic models that relate directly totheir deterministic counterparts, to illustrate some of the techniques for analyzing them, and to show the similarities between the three methods. Threetypes of stochastic modeling processes are described: 1) a discrete time Markovchain (DTMC) model, 2) a continuous time Markov chain (CTMC) model,and 3) a stochastic differential equation (SDE) model. These stochastic processes differ in the underlying assumptions regarding the time and the statevariables. In a DTMC model, the time and the state variables are discrete. Ina CTMC model, time is continuous, but the state variable is discrete. Finally,the SDE model is based on a diffusion process, where both the time and thestate variables are continuous.Stochastic models based on the well-known SIS and SIR epidemic modelsare formulated. For reference purposes, the dynamics of the SIS and SIRdeterministic epidemic models are reviewed in the next section. Then theassumptions that lead to the three different stochastic models are describedin Sects. 3, 4, and 5. The deterministic and stochastic model dynamics areillustrated through several numerical examples. Some of the MatLab programsused to compute numerical solutions are provided in the last section of thischapter.One of the most important differences between the deterministic andstochastic epidemic models is their asymptotic dynamics. Eventually stochastic solutions (sample paths) converge to the disease-free state even thoughthe corresponding deterministic solution converges to an endemic equilibrium.Other properties that are unique to the stochastic epidemic models includethe probability of an outbreak, the quasistationary probability distribution,the final size distribution of an epidemic and the expected duration of anepidemic. These properties are discussed in Sect. 6. In Sect. 7, the SIS epi-
2Linda J. S. Allendemic model with constant population size is extended to one with a variablepopulation size and the corresponding SDE model is derived.The chapter ends with a discussion of two well-known DTMC epidemicprocesses that are not directly related to any deterministic epidemic model.These two processes are chain binomial epidemic processes and branchingepidemic processes.2 Review of Deterministic SIS and SIR Epidemic ModelsIn SIS and SIR epidemic models, individuals in the population are classifiedaccording to disease status, either susceptible, infectious, or immune. Theimmune classification is also referred to as removed because individuals areno longer spreading the disease when they are removed or isolated from theinfection process. These three classifications are denoted by the variables S,I, and R, respectively.In an SIS epidemic model, a susceptible individual, after a successful contact with an infectious individual, becomes infected and infectious, but doesnot develop immunity to the disease. Hence, after recovery, infected individuals return to the susceptible class. The SIS epidemic model has been appliedto sexually transmitted diseases. We make some additional simplifying assumptions. There is no vertical transmission of the disease (all individuals areborn susceptible) and there are no disease-related deaths. A compartmentaldiagram in Fig. 1 illustrates the dynamics of the SIS epidemic model. Solidarrows denote infection or recovery. Dotted arrows denote births or deaths.SIFig. 1. SIS compartmental diagram.Differential equations describing the dynamics of an SIS epidemic modelbased on the preceding assumptions have the following form:dSβ SI (b γ)IdtNβdI SI (b γ)I,dtN(1)
An Introduction to Stochastic Epidemic Models3where β 0 is the contact rate, γ 0 is the recovery rate, b 0 is thebirth rate, and N S(t) I(t) is the total population size. The initial conditions satisfy S(0) 0, I(0) 0, and S(0) I(0) N . We assume that thebirth rate equals the death rate, so that the total population size is constant,dN/dt 0. The dynamics of model (1) are well-known [25]. They are determined by the basic reproduction number. The basic reproduction number isthe number of secondary infections caused by one infected individual in anentirely susceptible population [10, 26]. For model (1), the basic reproductionnumber is defined as follows:R0 β.b γ(2)The fraction 1/(b γ) is the length of the infectious period, adjusted fordeaths. The asymptotic dynamics of model (1) are summarized in the followingtheorem.Theorem 1. Let S(t) and I(t) be a solution to model (1).i) If R0 1, then lim (S(t), I(t)) (N, 0) (disease-free equilibrium).t 1N(endemic equilib,N 1 ii) If R0 1, then lim (S(t), I(t)) t R0R0rium).In an SIR epidemic model, individuals become infected, but then developimmunity and enter the immune class R. The SIR epidemic model has beenapplied to childhood diseases such as chickenpox, measles, and mumps. Acompartmental diagram in Fig. 2 illustrates the relationship between the threeclasses. Differential equations describing the dynamics of an SIR epidemicSIFig. 2. SIR compartmental diagram.model have the following form:R
4Linda J. S. AllendSβ SI b(I R)dtNβdI SI (b γ)I(3)dtNdR γI bR,dtwhere β 0, γ 0, b 0, and the total population size satisfies N S(t) I(t) R(t). The initial conditions satisfy S(0) 0, I(0) 0, R(0) 0,and S(0) I(0) R(0) N . We assume that the birth rate equals the deathrate so that the total population size is constant, dN/dt 0.The basic reproduction number (2) and the birth rate b determine the dynamics of model (3). The dynamics are summarized in the following theorem.Theorem 2. Let S(t), I(t), and R(t) be a solution to model (3).i) If R0 1, then lim I(t) 0 (disease-free equilibrium).t ii) If R0 1, thenlim (S(t), I(t), R(t)) t N bN,R0 b γ 11 R0 γN,b γ 11 R0 (endemic equilibrium).S(0)iii) Assume b 0. If R0 1, then there is an initial increase in theNS(0)number of infected cases I(t) (epidemic), but if R0 1, then I(t)Ndecreases monotonically to zero (disease-free equilibrium).The quantity R0 S(0)/N is referred to as the initial replacement number,the average number of secondary infections produced by an infected individualduring the period of infectiousness at the outset of the epidemic [25, 26].Since the infectious fraction changes during the course of the epidemic, thereplacement number is generally defined as R0 S(t)/N [25, 26]. In case iii) ofTheorem 2, the disease eventually disappears from the population but if theinitial replacement number is greater than one, the population experiences anoutbreak.3 Formulation of DTMC Epidemic ModelsLet S(t), I(t), and R(t) denote discrete random variables for the number ofsusceptible, infected, and immune individuals at time t, respectively. (Calligraphic letters denote random variables.) In a DTMC epidemic model,t {0, t, 2 t, . . .} and the discrete random variables satisfyS(t), I(t), R(t) {0, 1, 2, . . . , N }.The term “chain” (letter C) in DTMC means that the random variables arediscrete. The term “Markov” (letter M) in DTMC is defined in the nextsection.
An Introduction to Stochastic Epidemic Models53.1 SIS Epidemic ModelIn an SIS epidemic model, there is only one independent random variable,I(t), because S(t) N I(t), where N is the constant total population size.The stochastic process {I(t)} t 0 has an associated probability function,pi (t) Prob{I(t) i},for i 0, 1, 2, . . . , N and t 0, t, 2 t, . . . ,whereNXpi (t) 1.i 0Let p(t) (p0 (t), p1 (t), . . . , pN (t))T denote the probability vector associatedwith I(t). The stochastic process has the Markov property ifProb{I(t t) I(0), I( t), . . . , I(t)} Prob{I(t t) I(t)}.The Markov property means that the process at time t t only depends onthe process at the previous time step t.To complete the formulation for a DTMC SIS epidemic model, the relationship between the states I(t) and I(t t) needs to be defined. Thisrelationship is determined by the underlying assumptions in the SIS epidemicmodel and is defined by the transition matrix. The probability of a transitionfrom state I(t) i to state I(t t) j, i j, in time t, is denoted aspji (t t, t) Prob{I(t t) j I(t) i}.When the transition probability pji (t t, t) does not depend on t, pji ( t),the process is said to be time homogeneous. For the stochastic SIS epidemicmodel, the process is time homogeneous because the deterministic model isautonomous.To reduce the number of transitions in time t, we make one more assumption. The time step t is chosen sufficiently small such that the numberof infected individuals changes by at most one during the time interval t,that is,i i 1, i i 1 or i i.Either there is a new infection, a birth, a death, or a recovery during the timeinterval t. Of course, this latter assumption can be modified, if the time stepcannot be chosen arbitrarily small. In this latter case, transition probabilitiesneed to be defined for all possible transitions that may occur, i i 2,i i 3, etc. In the simplest case, with only three transitions possible, thetransition probabilities are defined using the rates (multiplied by t) in thedeterministic SIS epidemic model. This latter assumption makes the DTMCmodel a useful approximation to the CTMC model, described in Sect. 4. Thetransition probabilities for the DTMC epidemic model satisfy
6Linda J. S. Allen βi(N i) t, (b N γ)i t,pji ( t) βi(N i) (b γ)i t,1 N 0,j i 1j i 1j ij 6 i 1, i, i 1.The probability of a new infection, i i 1, is βi(N i) t/N. The probabilityof a death or recovery, i i 1, is (b γ)i t. Finally, the probability of nochange in state, i i, is 1 [βi(N i)/N (b γ)i] t. Since a birth ofa susceptible must be accompanied by a death, to keep the population sizeconstant, this probability is not needed in either the deterministic or stochasticformulations.To simplify the notation and to relate the SIS epidemic process to a birthand death process, the transition probability for a new infection is denoted asb(i) t and for a death or a recovery is denoted as d(i) t. Then b(i) t,j i 1 d(i) t,j i 1pji ( t) 1 [b(i) d(i)] t,j i 0,j 6 i 1, i, i 1.The sum of the three transitions equals one because these transitions representall possible changes in the state i during the time interval t. To ensure thatthese transition probabilities lie in the interval [0, 1], the time step t mustbe chosen sufficiently small such thatmax{[b(i) d(i)] t} 1.i {1,.,N }Applying the Markov property and the preceding transition probabilities,the probabilities pi (t t) can be expressed in terms of the probabilities attime t. At time t t,pi (t t) pi 1 (t)b(i 1) t pi 1(t)d(i 1) t pi(t)(1 [b(i) d(i)] t) (4)for i 1, 2, . . . , N , where b(i) βi(N i)/N and d(i) (b γ)i.A transition matrix is formed when the states are ordered from 0 to N . Forexample, the (1, 1) element in the transition matrix is the transition probability from state zero to state zero, p00 ( t) 1, and the (N 1, N 1)element is the transition probability from state N to state N , pN N ( t) 1 [b γ]N t 1 d(N ) t. Denote the transition matrix as P ( t). MatrixP ( t) is a (N 1) (N 1) tridiagonal matrix given by
An Introduction to Stochastic Epidemic Models01d(1) t0B0 1 (b d)(1) td(2) tBB0b(1) t1 (b d)(2) tBB00b(2) tBB.B 00CC,.C.CC···d(N 1) t0C· · · 1 (b d)(N 1) t d(N ) t A···b(N 1) t1 d(N ) t1where the notation (b d)(i) [b(i) d(i)] for i 1, 2, . . . , N . Matrix P ( t)is a stochastic matrix, i.e., the column sums equal one.The DTMC SIS epidemic process {I(t)} t 0 is now completely formulated.Given an initial probability vector p(0), it follows that p( t) P ( t)p(0).The identity (4) expressed in matrix and vector notation isp(t t) P ( t)p(t) P n 1 ( t)p(0),(5)where t n t.Difference equations for the mean and the higher order moments of theepidemic process can be obtained directly from the difference equations in (4).PNFor example, the expected value for I(t) is E(I(t)) i 0 ipi (t). Multiplyingequation (4) by i and summing on i leads toE(I(t t)) NXipi (t t)i 0 NXi 1 ipi 1 (t)b(i 1) t NXi 0ipi (t) NXi 0N 1Xipi 1 (t)d(i 1) ti 0ipi (t)b(i) t NXipi (t)d(i) t.i 0Simplifying and substituting the expressions βi(N i)/N and (b γ)i for b(i)and d(i), respectively, yieldsE(I(t t)) E(I(t)) NXi 1 N 1Xpi 1 (t)β(i 1)(N [i 1]) tNpi 1 (t)(b γ)(i 1) ti 0 E(I(t)) [β (b γ)] tE(I(t)) β tE(I 2 (t)),NPNwhere E(I 2 (t)) i 0 i2 pi (t) (see e.g., [8]). The difference equation for themean depends on the second moment. Difference equations for the second andthe higher order moments depend on even higher order moments. Therefore,
8Linda J. S. Allenthese equations cannot be solved unless some additional assumptions are maderegarding the higher order moments. However, because E(I 2 (t)) E 2 (I(t)),the mean satisfies the following inequality:βE(I(t t)) E(I(t)) [β (b γ)] E(I(t)) E 2 (I(t)). tN(6)As t 0,βdE(I(t)) [β (b γ)] E(I(t)) E 2 (I(t))dtNβ[N E(I(t))] E(I(t)) (b γ)E(I(t)) N(7)The right side of (7) is the same as the differential equation for I(t) in (1),if, in equation (1), I(t) and S(t) are replaced by E(I(t)) and N E(I(t)),respectively. The differential inequality implies that the mean of the randomvariable I(t) in the stochastic SIS epidemic process is less than the solutionI(t) of the deterministic differential equations in (1).Some properties of the DTMC SIS epidemic model follow easily fromMarkov chain theory [6, 47]. States are classified according to their connectedness in a directed graph or digraph. The digraph of the SIS Markov chainmodel is illustrated in Fig. 3, where i 0, 1, . . . , N are the infected states. The012NFig. 3. Digraph of the stochastic SIS epidemic model.states {0, 1, . . . , N } can be divided into two sets consisting of the recurrentstate, {0}, and the transient states, {1, . . . , N }. The zero state is an absorbing state. It is clear from the digraph that beginning from state 0 no otherstate can be reached; the set {0} is closed. In addition, any state in the set{1, 2, . . . , N } can be reached from any other state in the set, but the set is notclosed because p01 ( t) 0. For transient states it can be shown that elements(n)of the transition matrix have the following property [6, 47]: Let P n (pij ),(n)where pij is the (i, j) element of the nth power of the transition matrix, P n ,then(n)lim pij 0n for any state j and any transient state i. The limit of P n as n is astochastic matrix; all rows are zero except the first one which has all ones.From the relationship (5) and Markov chain theory, it follows thatlim p(t) (1, 0, . . . , 0)T ,t
An Introduction to Stochastic Epidemic Models9where t n t. The preceding result implies, in the DTMC SIS epidemicmodel, the population approaches the disease-free equilibrium (probabilityof absorption is one), regardless of the magnitude of the basic reproductionnumber. Compare this stochastic result with the asymptotic dynamics of thedeterministic SIS epidemic model (Theorem 1). Because this stochastic resultis asymptotic, the rate of convergence to the disease-free equilibrium can bevery slow. The mean time until the disease-free equilibrium is reached (absorption) depends the initial conditions and the parameter values, but canbe extremely long (as shown in the numerical example in the next section).The expected duration of an epidemic (mean time until absorption) and theprobability distribution conditioned on nonabsorption are discussed in Sect. 6.3.2 Numerical ExampleA sample path or stochastic realization of the stochastic process {I(t)} t 0 fort {0, t, 2 t, . . .} is an assignment of a possible value to I(t) based on theprobability vector p(t). A sample path is a function of time, so that it canbe plotted against the solution of the deterministic model. For illustrativepurposes, we choose a population size of N 100, t 0.01, β 1, b 0.25,γ 0.25 and an initial infected population size of I(0) 2. In terms of thestochastic model,Prob{I(0) 2} 1.In this example, the basic reproduction number is R0 2. The deterministicsolution approaches an endemic equilibrium given by I 50.Three sample paths of the stochastic model are compared to the deterministic solution in Fig. 4. One of the sample paths is absorbed before 200 timesteps (the population following this path becomes disease-free) but two sample paths are not absorbed during 2000 time steps. These latter sample pathsfollow more closely the dynamics of the deterministic solution. The horizontalaxis is the number of time steps t. For t 0.01 and 2000 time steps, thesolutions in Fig. 4 are graphed over the time interval [0, 20]. Each sample pathis not continuous because at each time step, t t, 2 t, . . . , the sample patheither stays constant (no change in state with probability 1 [b(i) d(i)] t),jumps down one integer value (with probability d(i) t), or jumps up oneinteger value (with probability b(i) t). For convenience, these jumps are connected with vertical line segments. Each sample path is continuous from theright but not from the left.The entire probability distribution, p(t), t 0, t, . . ., associated withthis particular stochastic process can be obtained by applying (5). A MatLab program is provided in the last section that generates the probabilitydistribution as a function of time (Fig. 5). Note that the probability distribution is bimodal, part of the distribution is at zero and the remainder of thedistribution follows a path similar to the deterministic solution. Eventually,the probability distribution at zero approaches one. This bimodal distribution is important; the part of the distribution that does not approach zero (at
10Linda J. S. Allen70Number of Infectives605040302010005001000Time Steps15002000Fig. 4. Three sample paths of the DTMC SIS epidemic model are graphed with thedeterministic solution (dashed curve). The parameter values are t 0.01, N 100,β 1, b 0.25, γ 0.25, and I(0) 2.time step 2000) is known as the quasistationary probability distribution (seeSect. 6.2).3.3 SIR Epidemic ModelLet S(t), I(t), and R(t) denote discrete random variables for the numberof susceptible, infected, and immune individuals at time t, respectively. TheDTMC SIR epidemic model is a bivariate process because there are two independent random variables, S(t) and I(t). The random variable R(t) N S(t) I(t). The bivariate process {(S(t), I(t))} t 0 has a joint probabilityfunction given byp(s,i) (t) Prob{S(t) s, I(t) i}.This bivariate process has the Markov property and is time-homogeneous.Transition probabilities can be defined based on the assumptions in theSIR deterministic formulation. First, assume that t can be chosen sufficientlysmall such that at most one change in state occurs during the time interval t. In particular, there can be either a new infection, a birth, a death, or arecovery. The transition probabilities are denoted as follows:p(s k,i j),(s,i) ( t) Prob{( S, I) (k, j) (S(t), I(t)) (s, i)},
An Introduction to Stochastic Epidemic 52000 100Time StepsInfectivesFig. 5. Probability distribution of the DTMC SIS epidemic model. Parameter valuesare the same as in Fig. 4.where S S(t t) S(t). Hence, βis/N t, γi t, bi t, p(s k,i j),(s,i) ( t) b(N s i) t, 1 βis/N t [γi b(N s)] t, 0,(k, j) ( 1, 1)(k, j)
An Introduction to Stochastic Epidemic Models Linda J. S. Allen Department of Mathematics and Statistics Texas Tech University Lubbock, Texas 79409-1042, U.S.A. linda.j.allen@ttu.edu 1 Introduction The goals of this chapter are to provide an introduction to three different methods for formulating stochastic epidemic models that relate directly to
Jul 09, 2010 · Stochastic Calculus of Heston’s Stochastic–Volatility Model Floyd B. Hanson Abstract—The Heston (1993) stochastic–volatility model is a square–root diffusion model for the stochastic–variance. It gives rise to a singular diffusion for the distribution according to Fell
are times when the fast stochastic lines either cross above 80 or below 20, while the slow stochastic lines do not. By slowing the lines, the slow stochastic generates fewer trading signals. INTERPRETATION You can see in the figures that the stochastic oscillator fluctuates between zero and 100. A stochastic value of 50 indicates that the closing
sion analysis on discrete-time stochastic processes. We now turn our focus to the study of continuous-time stochastic pro-cesses. In most cases, it is di cult to exactly describe the probability dis-tribution for continuous-time stochastic processes. This was also di cult for discrete time stochastic processes, but for them, we described the .
(e.g. bu er stocks, schedule slack). This approach has been criticized for its use of a deterministic approximation of a stochastic problem, which is the major motivation for stochastic programming. This dissertation recasts this debate by identifying both deterministic and stochastic approaches as policies for solving a stochastic base model,
Stochastic Programming Stochastic Dynamic Programming Conclusion : which approach should I use ? Objective and constraints Evaluating a solution Presentation Outline 1 Dealing with Uncertainty Objective and constraints Evaluating a solution 2 Stochastic Programming Stochastic Programming Approach Information Framework Toward multistage program
STOCHASTIC CALCULUS AND STOCHASTIC DIFFERENTIAL EQUATIONS 5 In discrete stochastic processes, there are many random times similar to (2.3). They are non-anticipating, i.e., at any time n, we can determine whether the cri-terion for such a random time is met or not solely by the “history” up to time n.
processes 1.Basics of stochastic processes 2.Markov processes and generators 3.Martingale problems 4.Exisence of solutions and forward equations 5.Stochastic integrals for Poisson random measures 6.Weak and strong solutions of stochastic equations 7.Stochastic equations for Markov processes in Rd 8.Convergenc
Peter Norvig Prentice Hall, 2003 This is the book that ties in most closely with the module Artificial Intelligence (2nd ed.) Elaine Rich & Kevin Knight McGraw Hill, 1991 Quite old now, but still a good second book Artificial Intelligence: A New Synthesis Nils Nilsson Morgan Kaufmann, 1998 A good modern book Artificial Intelligence (3rd ed.) Patrick Winston Addison Wesley, 1992 A classic, but .
