Introduction To Bayesian Inference
Introduction to Bayesian InferenceDipankar BandyopadhyayDiv. of Biostatistics, School of Public Health, University ofMinnesota, Minneapolis, Minnesota, U.S.A.1
Textbooks for this courseRequired:Bayesian Methods for Data Analysis, 3rd ed., by B.P. Carlinand T.A. Louis, Boca Raton, FL: Chapman and Hall/CRCPress, 2009. We will call it CL.Other books of interest:Bayesian Data Analysis, 3rd ed., by A. Gelman, J.B. Carlin,H.S. Stern, D.B. Dunson, A. Vehtari and D.B. Rubin, BocaRaton, FL: Chapman and Hall/CRC Press, 2014. This is areally nice reference.Bayesian Modeling Using WinBUGS, by Ioannis Ntzoufras,New York: Wiley, 2009.2PuBH 7440: Introduction to Bayesian Inference
Textbooks for this courseOther books of interest (cont’d):Bayesian Computation with R, by J.H. Albert, New York:Springer, 2007.Bayesian Statistical Modelling, Applied Bayesian Modelling,and Bayesian Models for Categorical Data, by PeterCongdon, New York: Wiley, 2003/2006/2009.Bayesian Approaches to Clinical Trials and Health-CareEvaluation, by D.J. Spiegelhalter, K.R. Abrams, and J.P.Myles: Chichester: Wiley, 2004.(“BCLM”): Bayesian Adaptive Methods for Clinical Trials(ISBN 978-1-4398-2548-8) by S.M. Berry, B.P. Carlin, J.J.Lee, and P. Müller, Boca Raton, FL: Chapman andHall/CRC Press, 2010.3PuBH 7440: Introduction to Bayesian Inference
Chapter 1: Introduction and OverviewBiostatisticians in the drug and medical device industriesare increasingly faced with data that are:highly multivariate, with many important predictors andresponse variablestemporally correlated (longitudinal, survival studies)costly and difficult to obtain, but often with historical data onprevious but similar drugs or devicesRecently, the FDA Center for Devices has encouragedhierarchical Bayesian statistical approaches –Methods are not terribly novel: Bayes (1763)!But their practical application has only become feasible inthe last decade or so due to advances in computing viaMarkov chain Monte Carlo (MCMC) methods and relatedWinBUGS software4PuBH 7440: Introduction to Bayesian Inference
Some preliminary Q&AWhat is the philosophical difference between classical(“frequentist”) and Bayesian statistics?To a frequentist, unknown model parameters are fixed andunknown, and only estimable by replications of data fromsome experiment.A Bayesian thinks of parameters as random, and thushaving distributions (just like the data). We can thus thinkabout unknowns for which no reliable frequentistexperiment exists, e.g.θ5 proportion of US men withuntreated atrial fibrillationPuBH 7440: Introduction to Bayesian Inference
Some preliminary Q&AHow does it work?A Bayesian writes down a prior guess for θ , p(θ ), thencombines this with the information that the data X provideto obtain the posterior distribution of θ , p(θ X). All statisticalinferences (point and interval estimates, hypothesis tests)then follow as appropriate summaries of the posterior.Note thatposterior information prior information 0 ,with the second “ ” replaced by “ ” only if the prior isnoninformative (which is often uniform, or “flat”).6PuBH 7440: Introduction to Bayesian Inference
Some preliminary Q&AIs the classical approach “wrong”?While a “hardcore” Bayesian might say so, it is probablymore accurate to think of classical methods as merely“limited in scope”!The Bayesian approach expands the class of models wecan fit to our data, enabling us to handle:any outcome (binary, count, continuous, censored)repeated measures / hierarchical structurecomplex correlations (longitudinal, spatial, or cluster sample)/ multivariate dataunbalanced or missing data– and many other settings that are awkward or infeasible froma classical point of view.The approach also eases the interpretation of and learningfrom those models once fit.7PuBH 7440: Introduction to Bayesian Inference
Simple example of Bayesian thinkingFrom Business Week, online edition, July 31, 2001:“Economis p1 (x θ ) nx θ x (1 θ )n x 93 129 θ (1 θ ) .Negative Binomial: Flip until we get r 3 tails x X NB(3, θ ) L2 (θ ) p2 (x θ ) r x 1θ (1 θ )rx 11 93 9 θ (1 θ ) .Adopt the rejection region, “Reject H0 if X c.”22PuBH 7440: Introduction to Bayesian Inference
Binomial vs. Negative Binomialp-values:α1 Pθ 1 (X 9) Σ12j 92α2 Pθ 1 (X 9) 212 j12 j .075j θ (1 θ ) 2 j j3Σ j 9 j θ (1 θ ) .0325So at α .05, two different decisions! Violates theLikelihood Principle, since L1 (θ ) L2 (θ )!!What happened? Besides the observed x 9, we also tookinto account the “more extreme” X 10.Jeffreys (1961): “.a hypothesis which may be true may berejected because it has not predicted observable resultswhich have not occurred.”In our example, the probability of the unpredicted andnon-occurring set X 10 has been used as evidenceagainst H0 !23PuBH 7440: Introduction to Bayesian Inference
Bayesians have a problem with p-valuesp Pr(results as surprising as you got or more so).The “or more so" part gets us in trouble with:The Likelihood Principle: When making decisions, only theobserved data can play a role.This can lead to bad decisions (esp. false positives)Are p-values at least more objective, because they are notinfluenced by any prior distribution?No, because they are influenced crucially by the design ofthe experiment, which determines the reference space ofevents for the calculation.Purely practical problems also plague p-values:Ex: Unforeseen events: First 5 patients develop a rash, andthe trial is stopped by clinicians. this aspect of design wasn’t anticipated, so strictlyspeaking, the p-value is not computable!24PuBH 7440: Introduction to Bayesian Inference
Conditional (Bayesian) PerspectiveAlways condition on data which has actually occurred; thelong-run performance of a procedure is of (at most)secondary interest. Fix a prior distribution p(θ ), and useBayes’ Theorem (1763):p(θ x) f (x θ )p(θ )(“posterior likelihood prior”)Indeed, it often turns out that using the Bayesian formalismwith relatively vague priors produces procedures whichperform well using traditional frequentist criteria (e.g., lowmean squared error over repeated sampling)!– several examples in Chapter 4 of the C&L text!25PuBH 7440: Introduction to Bayesian Inference
Frequentist criticisms[from B. Efron (1986, Amer. Statist.), “Why isn’t everyone aBayesian?”]Shouldn’t condition on x (but this renews conflict with theLikelihood Principle)Not easy/automatic (but computing keeps improving:MCMC methods, WinBUGS software.)How to pick the prior p(θ ): Two experimenters could getdifferent answers with the same data! How to controlinfluence of the prior? How to get objective results (say, fora court case, scientific report,.)? Clearly this final criticism is the most serious!26PuBH 7440: Introduction to Bayesian Inference
Bayesian Advantages in InferenceAbility to formally incorporate prior informationThe reason for stopping experimentation does not affectthe inferenceAnswers are more easily interpretable by nonspecialists(e.g. confidence intervals)All analyses follow directly from the posterior; no separatetheories of estimation, testing, multiple comparisons, etc.are neededAny question can be directly answered (bioequivalence,multiple comparisons/hypotheses, .)Inferences are conditional on the actual dataBayes procedures possess many optimality properties(e.g. consistent, impose parsimony in model choice, definethe class of optimal frequentist procedures, .)27PuBH 7440: Introduction to Bayesian Inference
Role of Bayes in drug/device settingsSafety/efficacy studies: Historical data and/or informationfrom published literature can be used to reduce samplesize, reducing time and expense. Unlimited looks ataccumulating data are also permitted (due to differentframework for testing).Equivalence studies: Bayes allows one to make directstatements about the probability that one drug isequivalent to another, rather than merely “failing to reject”the hypothesis of no difference.Meta-analysis: Bayes facilitates combining disparate butsimilar studies of a common drug or device.Hierarchical models: Realistic models can be fit tocomplicated, multilevel data (e.g., multiple observationsper patient, or multiple patients per clinical site),accounting for all sources of uncertainty.28PuBH 7440: Introduction to Bayesian Inference
Bayesian design of experimentsIn traditional sample size formulae, one often plugs in a“best guess" or “smallest clinically significant difference" forθ “Everyone is a Bayesian at the design stage."In practice, frequentist and Bayesian outlooks arise:Applicants may have a more Bayesian outlook:to take advantage of historical data or expert opinion (andpossibly stop the trial sooner), orto “peek" at the accumulating data without affecting theirability to analyze it laterRegulatory agencies may appreciate this, but also retainmany elements of frequentist thinking:to ensure that in the long run they will only rarely approve auseless or harmful productApplicants must thus design their trials accordingly!29PuBH 7440: Introduction to Bayesian Inference
Bayesian Modeling Using WinBUGS, by Ioannis Ntzoufras, New York: Wiley, 2009. 2 PuBH 7440: Introduction to Bayesian Inference. Textbooks for this course Other books of interest (cont’d): Bayesian Comp
Computational Bayesian Statistics An Introduction M. Antónia Amaral Turkman Carlos Daniel Paulino Peter Müller. Contents Preface to the English Version viii Preface ix 1 Bayesian Inference 1 1.1 The Classical Paradigm 2 1.2 The Bayesian Paradigm 5 1.3 Bayesian Inference 8 1.3.1 Parametric Inference 8
Bayesian" model, that a combination of analytic calculation and straightforward, practically e–-cient, approximation can ofier state-of-the-art results. 2 From Least-Squares to Bayesian Inference We introduce the methodology of Bayesian inference by considering an example prediction (re-gression) problem.
Comparison of frequentist and Bayesian inference. Class 20, 18.05 Jeremy Orloff and Jonathan Bloom. 1 Learning Goals. 1. Be able to explain the difference between the p-value and a posterior probability to a doctor. 2 Introduction. We have now learned about two schools of statistical inference: Bayesian and frequentist.
Why should I know about Bayesian inference? Because Bayesian principles are fundamental for statistical inference in general system identification translational neuromodeling ("computational assays") - computational psychiatry - computational neurology
of inference for the stochastic rate constants, c, given some time course data on the system state, X t.Itis therefore most natural to first consider inference for the earlier-mentioned MJP SKM. As demonstrated by Boys et al. [6], exact Bayesian inference in this settin
variety of modeling problems. With this work, we provide a general introduction to amortized Bayesian parameter estima-tion and model comparison and demonstrate the applicability of the proposed methods on a well-known class of intractable response-time models. Keywords: Bayesian inference; Neural netwo
value of the parameter remains uncertain given a nite number of observations, and Bayesian statistics uses the posterior distribution to express this uncertainty. A nonparametric Bayesian model is a Bayesian model whose parameter space has in nite dimension. To de ne a nonparametric Bayesian model, we have
procedure ASTM C 1202 can be improved by relying on multiple measurements that indicate both chloride ions penetrability and concrete resistivity. These are chief factors in the service life .
