Bayesian SAE Using Complex Survey Data Lecture 4A .

Bayesian SAE using Complex Survey DataLecture 4A: Hierarchical Spatial BayesModelingJon WakefieldDepartments of Statistics and BiostatisticsUniversity of Washington1 / 37

Lecture ContentMotivationSpatial Hierarchical Models for Normal DataSpatial Hierarchical Models for Binomial DataOverview of Spatial Random Effects ModelsNormal and Binomial ExamplesDiscussionTechnical Appendix: The Conditional Spatial Model2 / 37

Motivation3 / 37

SmoothingIn the last lecture, we considered hierarchical models that shrunkestimates towards a central value, with no consideration of thegeography of the areas.In general, we might expect unknown paramaters of interest in areasthat are “close” to be more similar than in areas that are not “close”.We would like to encode this observation in a model, in order tosmooth locally in space, in order to provide more reliable estimates ineach area.This is analogous to the use of a covariate x, in that areas with similarx values are likely to have similar parameters.Unfortunately the modeling of spatial dependence is much moredifficult since spatial location is acting as a surrogate for unobservedcovariates.We need to choose an appropriate spatial model, but do not directlyobserve the covariates whose effect we are trying to mimic.4 / 37

Spatial Hierarchical Models for Normal Data5 / 37

Normal-Normal Spatial ModelPreviously, we examined the non-spatial random effects model:Yik β δ 0 {z }i ik ,Mean of Area iwith δi iid N(0, σδ2 ) – these are the area-specific deviations (therandom effects) from the overall level β0 – and ik iid N(0, σ 2 ), is themeasurement error.We extend this model toYik β0 δi Si ik , {z}Mean of Area iwhere Si are spatial random effects.We are separating the residual variability into:I Unstructured area-level variability δi .I Spatial area-level variability Si .I Measurement error ik .6 / 37

Normal-Normal Spatial ModelWe will not go into detail on prior specification or computation forspatial models, in the accompanying R notes, we show how INLAprovides a means for computing posterior summary measures, withsensible prior choices.For more details on space-time modeling with INLA, see Blangiardoand Cameletti (2015).7 / 37

Spatial Hierarchical Models for Binomial Data8 / 37

Binomial-GLMM Spatial ModelWe first consider the modelYi θi ind Binomial(ni , θi )with logθi1 θi(1) β0 xi β1 Si δi ,(2)whereIthe random effects δi σδ2 iid N(0, σδ2 ) represent non-spatialoverdispersion,ISi are random effects with spatial structure.IWe describe two possible forms for the spatial random effects.9 / 37

Overview of Spatial Random Effects Models10 / 37

Spatial Models OverviewIn general, there have been two approaches to modeling spatialdependence:ILocal conditional modeling: in our context, are usually used forarea data.IGeostatistical modeling: in our context, are usually used for pointdata.11 / 37

Spatial Models OverviewThe local approach, an early reference to which is Besag (1974), isbased on conditional specifications Si S i , whereS i (S1 , . . . , Si 1 , Si 1 , . . . , Sn ).In general, the only variables in S i that are relevant are theneighbors (suitably defined), which we write as Si Sj , j ne(i).In words, what is the distribution of Si , given we know the valuestaken by the neighboring random variables Sj , j ne(i) – known as aMarkov Random Field (MRF) model.12 / 37

Spatial Models OverviewThe geostatistical approach, see for example Stein (1999), is basedon the specification of the full multivariate distribution ofS (S1 , . . . , Sn )Kriging, which is used for prediction in many spatial contexts, may bederived from a multivariate normal geostatistical model.For modeling area-level data, we will concentrate on conditionallyspecified spatial models1 .1 though return to the above model in the last lecture when we consider constructionof a continuous surface13 / 37

A Conditional Spatial ModelWe need to specify a rule for determining the neighbors of each area.In an epidemiological context the areas are not regular in shape.This is in contrast to image processing applications in which the dataare collected on a regular grid.Hence, there is an arbitrariness in specification of the neighborhoodstructure.14 / 37

A Conditional Spatial ModelTo define neighbors, the most common approach is to take theneighborhood scheme to be such that two areas are treated asneighbors if they share a common boundary.This is reasonable if all regions are (at least roughly) of similar sizeand arranged in a regular pattern (as is the case for pixels in imageanalysis where these models originated), but is not particularlyattractive otherwise (but reasonable practical alternatives are notavailable).15 / 37

A Conditional Spatial ModelVarious other neighborhood/weighting schemes are possible:IOne can take the neighborhood structure to depend on thedistance between area centroids and determine the extent of thespatial correlation (i.e. the distance within which regions areconsidered neighbors).IOne could also define neighbors in terms of cultural similarity.In typical applications it is difficult to assess whether the spatial modelchosen is appropriate, which argues for a simple form, and to assessthe sensitivity of conclusions to different choices.16 / 37

A Conditional Spatial ModelA common model, due to Besag et al. (1991), is to assign the spatialrandom effects an intrinsic conditional autorgressive (ICAR) prior.Under this specification it is assumed that the spatial random effect isdrawn from a normal distribution whose mean is the mean of theneighbors’ random effects, with variance proportional to one over thenumber of neighbors (so more neighbors, less variability).Formally, σ2Si Sj , j ne(i) N S i , s ,miwhere ne(i) is the set of neighbors of area i, mi is the number ofneighbours, and1 XSi Sjmij ne(i)is the mean of the spatial random effects of these neighbors.17 / 37

A Conditional Spatial ModelThe parameter σs2 is a conditional variance and its magnitudedetermines the amount of spatial variation.Recall, we split the residual variability asδi Si .The variance parameters σ 2 and σs2 have different interpretations.Both are defined on the same scale, but σ has a marginalinterpretation while σs has a conditional interpretation.Specifically, for area i, the variance of Si is conditional on Sj , j ne(i).Hence the variances are not directly comparable ; the random effects i and Si are comparable, however (so side-by-side maps of thecontributions are useful).Bottom line: Larger values of σs2 are indicative of greater spatialdependence.18 / 37

Normal and Binomial Examples19 / 37

Motivating Example: Normal Data200Spatial smoothing200Non spatial smoothing 190INLAINLA180 180190 170170 170 175 180 185 190 195 200170 175 180 185 190 195 200MLEMLEFigure 1: Comparison of area averages: Posterior medians from non-spatialmodel (described in Lecture 3) versus MLEs (left). Posterior medians fromspatial model versus MLEs (right).The shrinkage is less predictable with the spatial model, which isbecause of the local adaptation.20 / 37

Motivating Example: Normal Datau.spatu.iid5500 5 5Figure 2: Spatial (left) and non-spatial (left) random effects from thespatial IID model.The IID contribution is much smaller than the spatial contribution.21 / 37

Motivating Example: Normal DataNon spatial smoothing random effectsSpatial smoothing structured random effectsu.iid.nonspaceu.spat10105500 5 5Figure 3: Non-spatial random effects δi from the non-spatial model (left) andspatial random effects (right) random effects Si .The non-spatial model random effects are trying to pick up the spatialstructure!22 / 37

Motivating Example: Normal DataMLEmedian.spatial190190180180170170Figure 4: Estimates of area averages of weight via MLE’s (left) and posteriormedians from spatial model (right).The extremes are attenuated under the spatial model.23 / 37

Motivating Example: Normal Datamedianmedian.spatial190190180180170170Figure 5: Posterior median estimates of area averages of weight vianon-spatial hierarchical model with β0 δi (left) and spatial hierarchicalmodel β0 δi Si (right); δi are iid and Si are spatial random effects.Some differences between the estimates, but relatively minor.24 / 37

Motivating Example: Binomial Datau.spatu.iid221100 1 1Figure 6: Spatial (left) and non-spatial (left) random effects from thespatial iid model with logit(pi ) β0 δi Si ; δi are iid and Si are spatialrandom effects.The majority of the between-area variability is spatial.25 / 37

Motivating Example: Binomial DataNon spatial smoothing random effectsSpatial smoothing structured random effectsu.iid.nonspaceu.spat33221100 1 1Figure 7: Non-spatial random effects from the non-spatial model (left) andspatial random effects (right) random effects.The non-spatial model random effects are trying to pick up the spatialstructure!26 / 37

Motivating Example: Binomial re 8: MLEs of area diabetes risk (left) and posterior medians from thespatial hierarchical model (right).27 / 37

Motivating Example: Binomial igure 9: Posterior median estimates of area diabetes risk via non-spatialhierarchical model (left) and spatial hierarchical model (right).Estimates are very similar!28 / 37

Motivating Example: Binary OutcomeSpatial smoothing0.100.10Non spatial smoothing 0.02 Posterior sd 0.00 0.04 0.060.08 0.000.040.06 0.02Posterior sd0.08 80.10se(MLE)Figure 10: Comparison of area averages. Posterior standard deviation versusstandard errors of MLEs on the probability scale, for the non-spatialhierarchical model (left), and the spatial hierarchical model (right).The problem of standard errors being estimated as zero is clearlyalleviated, and the two sets of posterior standard deviations are quitesimilar.29 / 37

0.31020300.10.20 0.00.0 Posterior median bias 0.3 0.2 0.1 0.3 0.2 0.1MLE bias0.10.20.3Motivating Example: Binary Outcome400HRA index1020 3040HRA indexFigure 11: Bias of MLEs, with confidence intervals (left). Bias of posteriormedians, with credible intervals (right).If we calculate,n1Xbi pi , pni 1we get 0.026 (MLE) and 0.018 (Bayes).30 / 37

Discussion31 / 37

DiscussionIf the data are sparse in an area, averages and totals are unstablebecause of the small denominators.More reliable estimates can be obtained by using the totality of datato inform on the distribution, both locally and globally, of the averagesacross the study region.A GLMM can include spatial dependence relatively easily, with theICAR model being particularly popular.32 / 37

DiscussionFour levels of understanding for hierarchical models, in descendingorder of importance:IThe intuition on global and local smoothing.IThe models to achieve this.IHow to specify prior distributions.IThe computation behind the modeling.Overall StrategyIFirst, calculate empirical means and map them. Also look at mapof standard errors and/or confidence intervals.IIFit non-spatial random effects models.Fit the ICAR IID spatial model.IAdd in covariates if available.33 / 37

ReferencesBesag, J. (1974). Spatial interaction and the statistical analysis oflattice systems. Journal of the Royal Statistical Society, Series B,36, 192–236.Besag, J., York, J., and Mollié, A. (1991). Bayesian image restorationwith two applications in spatial statistics. Annals of the Institute ofStatistics and Mathematics, 43, 1–59.Blangiardo, M. and Cameletti, M. (2015). Spatial and Spatio-TemporalBayesian Models with R-INLA. John Wiley and Sons.Fong, Y., Rue, H., and Wakefield, J. (2010). Bayesian inference forgeneralized linear mixed models. Biostatistics, 11, 397–412.Rue, H. and Held, L. (2005). Gaussian Markov random fields: theoryand application. Chapman and Hall/CRC Press, Boca Raton.Stein, M. (1999). Interpolation of Spatial Data: Some Theory forKriging. Springer.34 / 37

Technical Appendix: The Conditional SpatialModel34 / 37