Analysis With R. Introduction To Bayesian Data

?Introduction to Bayesian DataAnalysis with R.Rasmus Bååth, Lund University@rabaath rasmus.baath@gmail.comhttp://www.sumsar.net

Source: I borrowed these three examples from apresentation. But which presentation I can’t rememberor find. I you know what presentation it could havebeen, please let me know and I will credit it here.

What do these have in common? Complex problems Large inherent uncertainty that needs to bequantified. Requires efficient integration of manysources of information. They all use Bayesian data analysis.

Bayesian data analysis is a great tool! and R is a great tool for doing Bayesian dataanalysis.But if you google “Bayesian” you getphilosophy:Subjective vs ObjectiveFrequentism vs Bayesianismp-values vs subjective probabilities

?Bayesian data analysisWhat? Why? How?

Overview of this tutorial What is Bayesian data analysis?Prediction contest Why use Bayesian data analysis?Exercises How to interpret and perform a Bayesiandata analysis in R?More Exercises

Why am I here? I use Bayesian methods in myresearch at Lund University whereI also run a network for peopleinterested in Bayes. I’m working on an R-package tomake simple Bayesian analysessimple to run. I blog about Bayesian dataanalysis.www.sumsar.net

What is Bayesian dataanalysis? It is when you use probability to representuncertainty in all parts of a statisticalmodel. A flexible extension of maximum likelihood. Potentially the most information-efficientmethod to fit a statistical model.(But potentially also the mostcomputationally intensive method )

Bayesian models as generative modelsParametersµρθσGenerative model5, 2, 7, 8, 3, 9, 1, 2, .DataIf we know theparameters

Bayesian models as generative modelsParametersµρθσGenerative model5, 2, 7, 8, 3, 9, 1, 2, .DataWhen we knowthe data.

How many fish are in the lake? An actual problem in Abundanceestimation. Use in, for example, wildlifemanagement. Also other uses, for example, to estimatehow many DKK 1,000 bills are in circulation.

How many fish are in the lake? The problem: We can’t catch them all. But we can catch some of them.

Mark and Re-capture1. Catch a couple of fish.2. Mark them and throw them back.

Mark and Re-capture1. Catch a couple of fish.2. Mark them and throw them back.

Mark and Re-capture1.2.3.4.Catch a couple of fish.Mark them and throw them back.At a later point, catch a couple of fish again.Count how many are marked.20 were marked and five out of the 20 thatwere caught the second time were marked.

So, how many fish are in the lake? What are the probable number of fish in thelake? We have almost already described thesolution! (If we know about Bayesian DataAnalysis, that is.)

ParametersµρθσGenerative model5, 2, 7, 8, 3, 9, 1, 2, .Data

ParametersNo. of Fish1. Mark 20 “fish”2. Sample 20 “fish”3. Count the no. marked fish5 marked fishData

ParametersUncertaintyNo. of Fish1. Mark 20 “fish”2. Sample 20 “fish”3. Count the no. marked fish5 marked fishData

Uniform(0, 250)UncertaintyNo. of Fish1. Mark 20 “fish”2. Sample 20 “fish”3. Count the no. marked fish5 marked fishData

One simple way of fitting the model1. Draw a large random sample from the “prior” probabilitydistribution on the parameters. Here, for example:no fish: [63, 30, 167, 30, 164, 222, 225, 42, 122, ]2. Plug in each draw into the generative model whichgenerates a vector of “fake” data. For example:fish 63fish 30fishpickfishpick413fish 167 fish 30fishpick5fishpick15

One simple way of fitting the model3. Keep only those parameter values that generated the datathat was actually observed.fish 63fish 30fishpickfishpick4 5?fish 167 fish 3013 5 ?fishpick5 5?fishpick 15 5 ?

One simple way of fitting the model3. Keep only those parameter values that generated the datathat was actually observed.4. The distribution of the retained parameters now representthe probability that the data was produced by a certainparameter value. For example:[167, 135, 148, 90, 162, 88, 98, 110, 176, ]fish 63fish 30fishpickfishpick4 5?fish 167 fish 3013 5 ?fishpick5 5?fishpick 15 5 ?

Time for a demonstrationThe script that was “live coded” can be found here:http://rpubs.com/rasmusab/live coding user 2015 bayes tutorial

PriorMaximum likelihoodestimatePosterior50 %CredibleIntervallPosterior median

P(100 5 o ) P(100 ) P(5 o 100 )n fishpick5o

P(100 5 o ) P(100 ) P(5 o 100 )Σ P(n ) · P(5 o n )n fishpick5o

n fishpick5oP(100 5 o ) P(100 ) P(5 o 100 )Σ P(n ) · P(5 o n )Parameters ΘGenerativemodelData DP(Θ D) P(Θ) · P(D Θ)Σ P(Θ) · P(D Θ)Bayes theorem

What have we done? We have specified prior informationP(Θ) A generative modelP(D Θ) And have calculated the probability ofof different parameter valuesP(Θ D)

What have we done? In this example we used a capturerecapture model with one parameter. But the general method works on anygenerative model and with anynumber of parameters. The specific computational methodwe used only works in rare cases.

What is not Bayesian dataanalysis? A category of models Subjective Not necessarily the most computationallyefficient method of fitting a model. Anything new.

Bayes 1701–1761Laplace 1749–1827InverseProbability

“Bayesian data analysis” isnot the best of names.“Probabilistic modeling”would be better!Bayes 1701–1761Laplace 1749–1827InverseBayesians!ProbabilityFisher 1890–1962

UseR! 2015 prediction competitionhttp://bit.ly/1LuF64m20 minutes

Why use Bayesian dataanalysis? You have great flexibility when buildingmodels, and can focus on that, rather thancomputational issues.Why?

“Marked fish get shy! It is half as likely to catch a markedfish compared to a fish that has not been marked.”

ParametersNo. of Fish1. Mark 20 “fish”2. Sample 20 “fish”3. Count the no. marked fish5 marked fishData

ParametersNo. of Fish1. Mark 20 “fish”2. Sample 20 “fish”, where there is a50% chance to sample a markedfish compared to a unmarked fish.3. Count the no. marked fish5 marked fishData

DemoThe script that was “live coded” can be found here:http://rpubs.com/rasmusab/live coding user 2015 bayes tutorial

Why use Bayesian dataanalysis? You have great flexibility when buildingmodels, and can focus on that, rather thancomputational issues. You can include information sources inaddition to the data, for example, expertopinion.

“There has always been plenty of fish in the lake.Around 200, I would say!”

DemoThe script that was “live coded” can be found here:http://rpubs.com/rasmusab/live coding user 2015 bayes tutorial

“If you’re not using a informative prior, you’releaving money on the table.”- Robert Weiss, UCLA, Los Angeles.

Why use Bayesian dataanalysis? You have great flexibility when buildingmodels, and can focus on that, rather thancomputational issues. You can include information sources inaddition to the data, for example, expertopinion. The result of a Bayesian analysis retains theuncertainty of the estimated parameters,which is very useful in decision analysis.

draw id no fish190216232024985176.

draw id no fishno fish * 10019090002162162003202202004989800517617600.

“If there are less than 50 fish in the lake, they wontlast the season. It will cost 10 000 kr to plant new fishinto the lake!

draw id no fishcatch 80 fishx 100 kr1908000216280003202800049880005717100.

draw id no fishcatch 80 fishx 100 krfishleftrepopulation .

profit - min(no fish, 80) * 100 (no fish - 80 50) * 10000draw id no fishcatch 80 fishx 100 krfishleftrepopulation .

draw id no fishcatch 80 fishx 100 krfishleftrepopulation .

mean(profit)[1] -1013draw id no fishcatch 80 fishx 100 krfishleftrepopulation .

What’s the optimal catch quota?Catch quota: 27 fishExpected profit: 2409 kr

80vs

Why use Bayesian dataanalysis? You have great flexibility when buildingmodels, and can focus on that, rather thancomputational issues. You can include information sources inaddition to the data, for example, expertopinion. The result of a Bayesian analysis retains theuncertainty of the estimated parameters,which is very useful in decision analysis. You probably are already.

t.test(y)

t.test(y1, y2)

lm(y 1 x)

glm(y 1 x, family “poisson”)

Why not use Bayesian dataanalysis? Everything is just working fine as it is. I’m not that interested in uncertainty. It’s too computationally demanding.

Exercise 1Bayesian A/B testing forSwedish Fish Incorporatedhttp://bit.ly/1SSCAaj

How to interpret and perform aBayesian data analysis in R? Interpreting the result of an Bayesian dataanalysis is usually straight forward.How?

With 95% probability the support of the voters lie withinthis band.

How to interpret and perform aBayesian data analysis in R? Interpreting the result of an Bayesian dataanalysis is usually straight forward. But if you scratch the surface there is a lotof Bayesian jargon!

PriorMaximum likelihoodestimatePosterior50 %CredibleIntervallPosterior median

More Bayesian Jargon Priors Objective priors Subjective priors Informative priors Improper priors Conjugate priorsCompletely datadriven modelExpert opinionBayesianmodels

More Bayesian Jargon: Distributions!

More Bayesian Jargon: Distributions! The usual suspects: The Normalx Normal(µ, σ)x - rnorm(n draw, mu, sd)

More Bayesian Jargon: Distributions! The usual suspects: The Binomialx Binomial(p, n)x - rbinom(n draw, size, prob)

More Bayesian Jargon: Distributions! The usual suspects: The Poissonx Poisson(λ)x - rpois(n draw, lambda)

More Bayesian Jargon: Distributions! Less common beasts: The Betax Beta( , )x - rbeta(n draw, shape1, shape2)

More Bayesian Jargon: Distributions! Less common beasts: The Gammax Gamma(k, θ)x - rgamma(n draw, shape, scale)

More Bayesian Jargon: Distributions! Less common beasts: The Hypergeometricn fishpick5o Fisher's noncentral hypergeometricdistributionn fish-pick withshy marked fish5o When it comes to distributions, Wikipedia isyour friend!

More Bayesian Jargon Samples, samples, samples.Prior samples:[63, 30, 167, 30, 164, 222, 225, 42, 122, ]Posterior samples:[167, 135, 148, 90, 162, 88, 98, 110, 176, ] Methods to generate posterior samples: Approximate Bayesian Computation (ABC) Markov Chain Monte Carlo (MCMC) Metropolis-Hastings Gibbs Sampling Hamiltonian monte carlo Other methods Conjugate models Laplace Approximation Etc. Etc. Etc.

Faster Bayesian computation We have been doing approximate Bayesiancomputation, which is the most general andslowest method for fitting a Bayesianmodel. Faster methods have in common that: They require that the likelihood that the generativemodel will generate any given data can becalculated.n fishpick5o

Faster Bayesian computation We have been doing approximate Bayesiancomputation, which is the most general andslowest method for fitting a Bayesianmodel. Faster methods have in common that: They require that the likelihood that the generativemodel will generate any given data can becalculated.5on fishpickP(5 o n )

Faster Bayesian computation We have been doing approximate Bayesiancomputation, which is the most general andslowest method for fitting a Bayesianmodel. Faster methods have in common that: They require that the likelihood that the generativemodel will generate any given data can becalculated. They explore the parameter space in a smarter way. What you get are samples as if you would havedone the analysis using approximate Bayesiancomputation.

MCMC: The Metropolis-Hastingalgorithm The “classic” MCMC algorithm. Performs arandom walk in the parameter space, andwill stay at a parameter value proportionalto its posterior probability.Source: lis-hastings-sampling/ A good R implementation can be found inthe MCMCpack package as the functionMCMCmetrop1R(fun, theta.init, .)

Source: is/

Source: is/

MCMC: Gibbs sampling and JAGS Similar to Metropolis,but moves by oneparameter at a time. Can be much moreefficient, but usuallyrequired custom builtsampling schemes. Unless you use JAGS!