Bayesian Inference For Partially Observed Markov Processes .
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsBayesian inference for partially observed Markovprocesses, with application to systems biologyDarren Wilkinsonhttp://tinyurl.com/darrenjwSchool of Mathematics & Statistics, Newcastle University, UKBayes–250Informatics ForumEdinburgh, UK5th–7th September, 2011Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsSystems biology modelsPopulation dynamicsStochastic chemical kineticsGenetic autoregulationSystems biology modellingUses accurate high-resolution time-course data on a relativelysmall number of bio-molecules to parametrise carefullyconstructed mechanistic dynamic models of a process ofinterest based on current biological understandingTraditionally, models were typically deterministic, based on asystem of ODEs known as the Reaction Rate Equations(RREs)It is now increasingly accepted that biochemical networkdynamics at the single-cell level are intrinsically stochasticThe theory of stochastic chemical kinetics provides a solidfoundation for describing network dynamics using a Markovjump processDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsSystems biology modelsPopulation dynamicsStochastic chemical kineticsGenetic autoregulationStochastic Chemical KineticsStochastic molecular approach:Statistical mechanical arguments lead to a Markov jumpprocess in continuous time whose instantaneous reaction ratesare directly proportional to the number of molecules of eachreacting speciesSuch dynamics can be simulated (exactly) on a computerusing standard discrete-event simulation techniquesStandard implementation of this strategy is known as the“Gillespie algorithm” (just discrete event simulation), butthere are several exact and approximate variants of this basicapproachDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsSystems biology modelsPopulation dynamicsStochastic chemical kineticsGenetic autoregulationLotka-Volterra systemTrivial (familiar) example from population dynamics (in reality, the“reactions” will be elementary biochemical reactions taking placeinside a cell)ReactionsX 2X(prey reproduction)X Y 2Y(prey-predator interaction)Y (predator death)X – Prey, Y – PredatorWe can re-write this using matrix notationDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsSystems biology modelsPopulation dynamicsStochastic chemical kineticsGenetic autoregulationForming the matrix representationThe L-V system in tabular formR1R2R3Rate Lawh(·, c)c1 xc2 xyc3 yLHSX Y1 01 10 1RHSX Y2 00 20 0Net-effectXY10-110-1Call the 3 2 net-effect (or reaction) matrix N. The matrixS N 0 is the stoichiometry matrix of the system. Typically bothare sparse. The SVD of S (or N) is of interest for structuralanalysis of the system dynamics.Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsSystems biology modelsPopulation dynamicsStochastic chemical kineticsGenetic autoregulationStochastic chemical kineticsu species: X1 , . . . , Xu , and v reactions: R1 , . . . , RvRi : pi1 X1 · · · piu Xu qi1 X1 · · · qiu Xu , i 1, . . . , vIn matrix form: PX QX (P and Q are sparse)S (Q P)0 is the stoichiometry matrix of the systemXjt : # molecules of Xj at time t. Xt (X1t , . . . , Xut )0Reaction Ri has hazard (or rate law, or propensity) hi (Xt , ci ),where ci is a rate parameter, c (c1 , . . . , cv )0 ,h(Xt , c) (h1 (Xt , c1 ), . . . , hv (Xt , cv ))0 and the system evolvesas a Markov jump processFor mass-action stochastic kinetics, u YXjthi (Xt , ci ) ci, i 1, . . . , vpijj 1Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsSystems biology modelsPopulation dynamicsStochastic chemical kineticsGenetic autoregulation25The Lotka-Volterra TimeDarren Wilkinson — Bayes–250, Edinburgh, 5/9/201150100150200250300350Y1Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsSystems biology modelsPopulation dynamicsStochastic chemical kineticsGenetic autoregulationExample — genetic auto-regulationPrP2RNAPpqDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011gDNABayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsSystems biology modelsPopulation dynamicsStochastic chemical kineticsGenetic autoregulationBiochemical reactionsSimplified view:Reactionsg P2 g · P2g g rr r P2P P2r P RepressionTranscriptionTranslationDimerisationmRNA degradationProtein degradationDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsSystems biology modelsPopulation dynamicsStochastic chemical kineticsGenetic autoregulation1.0304000200P2600 0 10P500.00.5Rna1.52.0Simulated realisation of the auto-regulatory network010002000300040005000TimeDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsPartially observed Markov process (POMP) modelsBayesian inferenceLikelihood-free algorithms for stochastic model calibrationPartially observed Markov process (POMP) modelsContinuous-time Markov process: X {Xs s 0} (for now,we suppress dependence on parameters, θ)Think about integer time observations (extension to arbitrarytimes is trivial): for t N, Xt {Xs t 1 s t}Sample-path likelihoods such as π(xt xt 1 ) can often (but notalways) be computed (but are often computationally difficult),but discrete time transitions such as π(xt xt 1 ) are typicallyintractablePartial observations: D {dt t 1, 2, . . . , T } wheredt Xt xt π(dt xt ),t 1, . . . , T ,where we assume that π(dt xt ) can be evaluated directly(simple measurement error model)Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsPartially observed Markov process (POMP) modelsBayesian inferenceLikelihood-free algorithms for stochastic model calibrationBayesian inference for POMP modelsMost “obvious” MCMC algorithms will attempt to impute (atleast) the skeleton of the Markov process: X0 , X1 , . . . , XTThis will typically require evaluation of the intractable discretetime transition likelihoods, and this is the problem.Two related strategies:Data augmentation: “fill in” the entire process in some way,typically exploiting the fact that the sample path likelihoodsare tractable — works in principle, but difficult to “automate”,and exceptionally computationally intensive due to the need tostore and evaluate likelihoods of cts sample pathsLikelihood-free (AKA plug-and-play): exploits the fact that itis possible to forward simulate from π(xt xt 1 ) (typically bysimulating from π(xt xt 1 )), even if it can’t be evaluatedLikelihood-free is really just a special kind of augmentationstrategyDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsPartially observed Markov process (POMP) modelsBayesian inferenceLikelihood-free algorithms for stochastic model calibrationBayesian inferenceLet π(x c) denote the (complex) likelihood of the simulationmodelLet π(D x, τ ) denote the (simple) measurement error modelPut θ (c, τ ), and let π(θ) be the prior for the modelparametersThe joint density can be writtenπ(θ, x, D) π(θ)π(x θ)π(D x, θ).Interest is in the posterior distribution π(θ, x D)Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsPartially observed Markov process (POMP) modelsBayesian inferenceLikelihood-free algorithms for stochastic model calibrationMarginal MH MCMC schemeFull model: π(θ, x, D) π(θ)π(x θ)π(D x, θ)Target: π(θ D) (with x marginalised out)Generic MCMC scheme:Propose θ? f (θ? θ)Accept with probability min{1, A}, whereA π(θ? ) f (θ θ? ) π(D θ? ) π(θ)f (θ? θ)π(D θ)π(D θ) is the “marginal likelihood” (or “observed datalikelihood”, or.)Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsPartially observed Markov process (POMP) modelsBayesian inferenceLikelihood-free algorithms for stochastic model calibrationLF-MCMCPosterior distribution π(θ, x D)Propose a joint update for θ and x as follows:Current state of the chain is (θ, x)First sample θ? f (θ? θ)Then sample a new path, x? π(x? θ? )Accept the pair (θ? , x? ) with probability min{1, A}, whereA π(θ? ) f (θ θ? ) π(D x? , θ? ) .π(θ)f (θ? θ)π(D x, θ)Note that choosing a prior independence proposal of the formf (θ? θ) π(θ? ) leads to the simpler acceptance ratioA Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011π(D x? , θ? )π(D x, θ)Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsPartially observed Markov process (POMP) modelsBayesian inferenceLikelihood-free algorithms for stochastic model calibration“Ideal” joint MCMC schemeLF-MCMC works by making the proposed sample pathconsistent with the proposed new parameters, butunfortunately not with the dataIdeally, we would do the joint update as followsFirst sample θ? f (θ? θ)Then sample a new path, x? π(x? θ? , D)Accept the pair (θ? , x? ) with probability min{1, A}, whereπ(θ? ) π(x? θ? ) f (θ θ? ) π(D x? , θ? ) π(x D, θ)π(θ) π(x θ) f (θ? θ) π(D x, θ) π(x? D, θ? )π(θ? ) π(D θ? ) f (θ θ? ) π(θ) π(D θ) f (θ? θ)A This joint scheme reduces down to the marginal scheme (Chib(1995)), but will be intractable for complex models.Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEsParticle MCMC (pMCMC)Of the various alternatives, pMCMC is the only obviouspractical option for constructing global likelihood-free MCMCalgorithms which are exact (Andreiu et al, 2010)Start by considering a basic marginal MH MCMC scheme withtarget π(θ D) and proposal f (θ? θ) — the acceptanceprobability is min{1, A} whereA π(θ? ) f (θ θ? ) π(D θ? ) π(θ)f (θ? θ)π(D θ)We can’t evaluate the final terms, but if we had a way toconstruct a Monte Carlo estimate of the likelihood, π̂(D θ),we could just plug this in and hope for the best:A π(θ? ) f (θ θ? ) π̂(D θ? ) π(θ)f (θ? θ)π̂(D θ)Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEs“Exact approximate” MCMC (the pseudo-marginalapproach)Remarkably, provided only that E[π̂(D θ)] π(D θ), thestationary distribution of the Markov chain will be exactlycorrect (Beaumont, 2003, Andreiu & Roberts, 2009)Putting W π̂(D θ)/π(D θ) and augmenting the state spaceof the chain to include W , we find that the target of thechain must be π(θ)π̂(D θ)π(w θ) π(θ D)w π(w θ)and so then the above “unbiasedness” property implies thatE(W θ) 1, which guarantees that the marginal for θ isexactly π(θ D)Blog post: http://tinyurl.com/6ex4xqwDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEsParticle marginal Metropolis-Hastings (PMMH)Likelihood estimates constructed via importance samplingtypically have this “unbiasedness” property, as do estimatesconstructed using a particle filterIf a particle filter is used to construct the Monte Carloestimate of likelihood to plug in to the acceptance probability,we get (a simple version of) the particle Marginal MetropolisHastings (PMMH) pMCMC algorithmThe full PMMH algorithm also uses the particle filter toconstruct a proposal for x, and has target π(θ, x D) — notjust π(θ D)The (bootstrap) particle filter relies only on the ability toforward simulate from the process, and hence the entireprocedure is “likelihood-free”Blog post: http://bit.ly/kvznmqDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEs500Test problem: Lotka-Volterra lated time series data set consisting of 16 equally spacedobservations subject to Gaussian measurement error with astandard deviation of 10.Darren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEsMarginal posteriors for the Lotka-Volterra 042 0.0046 0.0050 0ACF6000Iteration10 .95Value0.020000.85Lag0.7006 e0Valueth10.8Value0.85 1.05th10.550.600.650.70ValueNote that the true parameters, θ (1, 0.005, 0.6) are wellidentified by the dataDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEsMarginal posteriors observing only eration0.020001.1th20.400.9Value0.820000.7Lag0.800 2 .10.7Valueth10.40.60.81.0ValueNote that the mixing of the MCMC sampler is reasonable, andthat the true parameters, θ (1, 0.005, 0.6) are quite wellidentified by the dataDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEsMarginal posteriors for unknown measurement onDarren Wilkinson — Bayes–250, Edinburgh, CFth20.001.00th20.5020000.90Value10 30 5000.80Lag0.620000 4 alueth10.6Value0.80 1.00th11020304050ValueBayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEsR package: smfsbFree, open source, well-documented software package for R,smfsb, associated with the forthcoming second edition of“Stochastic modelling for systems biology”Code for stochastic simulation and of (biochemical) reactionnetworks (Markov jump processes and chemical Langevin),and pMCMC-based Bayesian inference for POMP modelsFull installation and “getting started” instructions athttp://tinyurl.com/smfsb2eOnce the package is installed and loaded, runningdemo("PMCMC") at the R prompt will run a PMMH algorithmfor the Lotka-Volterra model discussed hereDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEsHitting the data.The above algorithm works well in many cases, and isextremely general (works for any Markov process)In the case of no measurement error, the probability of hittingthe data (and accepting the proposal) is very small (possiblyzero), and so the mixing of the MCMC scheme is very poorABC (approximate Bayesian computation) strategy is toaccept if?kxt 1 dt 1 k εbut this forces a trade-off between accuracy and efficiencywhich can be unpleasant (cf. noisy ABC)Same problem in the case of low measurement errorParticularly problematic in the context of high-dimensionaldataWould like a strategy which copes better in this caseDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEsThe chemical Langevin equation (CLE)The CLE is a diffusion approximation to the true Markovjump processStart with the time change representation Z th(Xτ , c)dτXt X0 S N0and approximate Ni (t) ' t Wi (t), where Wi (t) is anindependent Wiener process for each iSubstituting in and using a little stochastic calculus gives:The CLE as an Itô SDE:dXt Sh(Xt , c) dt pS diag{h(Xt , c)}S 0 dWtDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMMH algorithmPMMHExample: Lotka-Volterra modelImproved filtering for SDEsImproved particle filters for SDEsThe “bootstrap” particle filter uses blind forward simulationfrom the modelIf we are able to evaluate the “likelihood” of sample paths, wecan use other proposalsThe particle filter weights then depend on theRadon-Nikodym derivative of law of the proposed path wrtthe true conditioned processFor SDEs, the weight will degenerate unless the proposedprocess is absolutely continuous wrt the true conditionedprocessIdeally we would like to sample from π(x?t 1 c ? , xt? , dt 1 ), butthis is not tractable for nonlinear SDEs such as the CLEDarren Wilkinson — Bayes–250, Edinburgh, 5/9/2011Bayesian inference for POMP models using pMCMC
Stochastic modelling of dynamical systemsBayesian inferenceParticle MCMCSummary and conclusionsParticle MCMC and the PMM
Stochastic modelling of dynamical systems Bayesian inference Particle MCMC Summary and conclusions Systems biology models Population dynamics Stochastic chemical kinetics Genetic autoregulation Lotka-Volterra system Trivial (familiar) example from population dynamics (in reality, the \reactio
Computational Bayesian Statistics An Introduction M. Antónia Amaral Turkman Carlos Daniel Paulino Peter Müller. Contents Preface to the English Version viii Preface ix 1 Bayesian Inference 1 1.1 The Classical Paradigm 2 1.2 The Bayesian Paradigm 5 1.3 Bayesian Inference 8 1.3.1 Parametric Inference 8
Bruksanvisning för bilstereo . Bruksanvisning for bilstereo . Instrukcja obsługi samochodowego odtwarzacza stereo . Operating Instructions for Car Stereo . 610-104 . SV . Bruksanvisning i original
Bayesian" model, that a combination of analytic calculation and straightforward, practically e–-cient, approximation can ofier state-of-the-art results. 2 From Least-Squares to Bayesian Inference We introduce the methodology of Bayesian inference by considering an example prediction (re-gression) problem.
Bayesian Modeling Using WinBUGS, by Ioannis Ntzoufras, New York: Wiley, 2009. 2 PuBH 7440: Introduction to Bayesian Inference. Textbooks for this course Other books of interest (cont’d): Bayesian Comp
Comparison of frequentist and Bayesian inference. Class 20, 18.05 Jeremy Orloff and Jonathan Bloom. 1 Learning Goals. 1. Be able to explain the difference between the p-value and a posterior probability to a doctor. 2 Introduction. We have now learned about two schools of statistical inference: Bayesian and frequentist.
Why should I know about Bayesian inference? Because Bayesian principles are fundamental for statistical inference in general system identification translational neuromodeling ("computational assays") - computational psychiatry - computational neurology
of inference for the stochastic rate constants, c, given some time course data on the system state, X t.Itis therefore most natural to first consider inference for the earlier-mentioned MJP SKM. As demonstrated by Boys et al. [6], exact Bayesian inference in this settin
10 tips och tricks för att lyckas med ert sap-projekt 20 SAPSANYTT 2/2015 De flesta projektledare känner säkert till Cobb’s paradox. Martin Cobb verkade som CIO för sekretariatet för Treasury Board of Canada 1995 då han ställde frågan
