Missing Data In Randomized Studies And The Need For Global .
Missing Data in Randomized Studies andthe Need for Global Sensitivity AnalysisDaniel ScharfsteinJohns Hopkins Universitydscharf@jhsph.eduOctober 1, 2013ScharfsteinIntroduction
Missing Data MattersIMissing outcome data are a widespread problem inrandomized trials, including those used as the basis ofregulatory approval of drugs and devices.ScharfsteinIntroduction
Neuronetics TMS StudyIIIIIIRandomized trial of TMS System vs. sham control.One week no treatment phase, six week acute treatmentphase, three week taper phase.Primary outcome: MADRS; scheduled to be measured atbaseline, weeks 2, 4 and 6 of acute phase and weeks 1, 2and 3 of taper phase.Primary treatment comparison: MADRS at week 4 ofacute phase.Secondary treatment comparison: MADRS at week 6 ofacute phase.TMS: n 155; sham: n 146ScharfsteinIntroduction
Neuronetics StudyTable: On-StudyAcuteWk 2 Wk 4 Wk 6TMS 97% 92% 55%Sham 98% 92% 40%IIITaperWk 1 Wk 2 Wk 341% 38% 35%27% 26% 24%After week 4 of the acute phase, treatmentdiscontinuation was primarily due to lack of efficacy.Primary analysis of acute phase used LOCF.FDA requested alternative analyses: completers-onlyanalysis and multiple imputation analysisScharfsteinIntroduction
What are the Estimands?What are the differences in the mean MADRS scoresat weeks 4 and 6 of the acute phase between TMSvs. sham in the counterfactual world in which thereare no missing data at these visits?ScharfsteinIntroduction
Missing Data MattersIIWhile unbiased estimates of treatment effects can beobtained from trials with no missing data, this is nolonger true when data are missing on some patients.The essential problem is that inference about treatmenteffects relies on unverifiable assumptions about the natureof the mechanism that generates the missing data.ScharfsteinIntroduction
Missing Data MattersIWhile we usually know the reasons for missing data, wedo not know the distribution of outcomes for patientswith missing data, how it compares to that of patientswith observed data and whether differences in thesedistributions can be explained by the observed data.ScharfsteinIntroduction
Robert Temple and Bob O’Neil (FDA)I”During almost 30 years of review experience, the issue ofmissing data in . clinical trials has been a major concernbecause of the potential impact on the inferences thatcan be drawn . when data are missing . the analysisand interpretation of the study pose a challenge and theconclusions become more tenuous as the extent of’missingness’ increases.”ScharfsteinIntroduction
NRC Report and Sensitivity AnalysisIIIIn 2010, the National Research Council (NRC) issued areported entitled ”The Prevention and Treatment ofMissing Data in Clinical Trials.”This report, commissioned by the FDA, provides 18recommendations targeted at (1) trial design and conduct,(2) analysis and (3) directions for future research.Recommendation 15 statesISensitivity analyses should be part of the primaryreporting of findings from clinical trials. Examiningsensitivity to the assumptions about the missing datamechanism should be a mandatory component ofreporting.ScharfsteinIntroduction
ICH, EMEA and Sensitivity AnalysisII1998 International Conference of Harmonization (ICH)Guidance document (E9) entitled ”Statistical Principles inClinical Trials” states: ”it is important to evaluate therobustness of the results to various limitations of the data,assumptions, and analytic approaches to data analysis”European Medicines Agency 2009 draft ”Guideline onMissing Data in Confirmatory Clinical Trials” states ”[i]nall submissions with non-negligible amounts of missingdata sensitivity analyses should be presented as supportto the main analysis.”ScharfsteinIntroduction
Sensitivity AnalysisThe set of possible assumptions about the missing datamechanism is very large and cannot be fully explored. Thereare different approaches to sensitivity on
Ad-hoc Sensitivity AnalysisIIIAnalyzing data using a few different analytic methods,such as last or baseline observation carried forward,complete or available-case analysis, mixed models ormultiple imputation, and evaluate whether the resultinginferences are consistent.The problem with this approach is that the assumptionsthat underlie these methods are very strong and for manyof these methods unreasonable.More importantly, just because the inferences areconsistent does not mean that there are no otherreasonable assumptions under which the inference aboutthe treatment effect is different.ScharfsteinIntroduction
Neuronetics StudyIIILOCF: Valid if patients outcomes don’t change afterdropout.Completers: Valid under Missing Completely at Random;Distribution of outcomes for patients off study at week kis the same as the distribution of outcome for patients onstudy at week k.Multiple Imputation: Valid under Missing at Random; Forpatients on-study at week k 1 and who share the samehistory of observed outcomes through week k 1, thedistribution of outcomes after week k 1 is the same forthose who are last seen at week k 1 and those whoremain on-study at week k.ScharfsteinIntroduction
Local Sensitivity AnalysisIISpecify a reasonable benchmark assumption (e.g., missingat random) and evaluate the robustness of the resultswithin a small neighborhood of this assumption.What if there are assumptions outside the localneighborhood which are plausible?ScharfsteinIntroduction
Global Sensitivity AnalysisIIIEvaluate robustness of results across a much broaderrange of assumptions that include a reasonable benchmarkassumption and a collection of additional assumptionsthat trend toward best and worst case assumptions.Emphasized in Chapter 5 of the NRC report.This approach is substantially more informative because itoperates like ”stress testing” in reliability engineering,where a product is systematically subjected toincreasingly exaggerated forces/conditions in order todetermine its breaking point.ScharfsteinIntroduction
Global Sensitivity AnalysisIIIIn the missing data setting, global sensitivity analysisallows one to see how far one needs to deviate from thebenchmark assumption in order for inferences to change.”Tipping point” analysis (Yan, Lee and Li, 2009;Campbell, Pennello and Yue, 2011)If the assumptions under which the inferences change arejudged to be sufficiently far from the benchmarkassumption, then greater credibility is lent to thebenchmark analysis; if not, the benchmark analysis can beconsidered to be fragile.ScharfsteinIntroduction
Case Study 1: Chronic SchizophreniaIIIMajor breakthroughs have been made in the treatment ofpatients with psychotic symptoms.However, side effects associated with typical and atypicalneuroleptics have limited their usefulness.RIS-INT-3 (Marder and Meibach, 1994, Chouinard et al.,1993) was a multi-center study designed to assess theeffectiveness and adverse experiences of four fixed dosesof risperidone compared to haliperidol and placebo in thetreatment of chronic schizophrenia.ScharfsteinIntroduction
RIS-INT-3IIIIIAt selection, patients were required to have a PANSS(Positive and Negative Syndrome Scale) score between 60and 120.Prior to randomization, there was a single-blind, one-weekwashout phase during which all anti-psychoticmedications were to be discontinued.If acute psychotic symptoms occurred, patients wererandomized to a double-blind treatment phase, scheduledto last 8 weeks.Patients were randomized to one of 6 treatment groups:risperidone 2, 6, 10 or 16 mg, haliperidol 20 mg, orplacebo.Dose titration occurred during the first week of thedouble-blind phase.ScharfsteinIntroduction
RSIP-INT-3IIIPatients scheduled for 5 post-baseline assessements atweeks 1,2,4,6, and 8 of the double-blind phase.Primary efficiacy variable: PANSS score521 patients randomized to receive placebo (n 88),haliperidol 20 mg (n 87), risperidone 2mg (n 87),risperidone 6mg (n 86), risperidone 10 mg (n 86), orrisperidone 16 mg (n 87).ScharfsteinIntroduction
Premature WithdrawalIIIOnly 49% of patients completed the 8 week treatmentperiod.The most common reason for discontinuation was“insufficient response.”Other main reasons included: adverse events,uncooperativeness, and withdrawal of consent.ScharfsteinIntroduction
Premature WithdrawalCompletedWithdrawnLack of EfficacyOtherPlacebo(n 88)2731%6169%5158%1011%Haliperidol(n 87)3641%5159%3641%1517%ScharfsteinRisp 2mg(n 87)3641%5159%4147%1011%IntroductionRisp 6mg(n 86)5362%3338%1214%2124%Risp 10mg(n 86)4856%3844%2529%1315%Risp 16 mg(n 87)5462%3338%1821%1517%
Observed DataScharfsteinIntroduction
Observed DataScharfsteinIntroduction
Central QuestionWhat is the difference in the mean PANSS scores atweek 8 between risperidone at a specified dose levelvs. placebo in the counterfactual world in which allpatients were followed to that week?ScharfsteinIntroduction
Global Sensitivity AnalysisIIRestrict consideration to follow-up randomized studydesigns that prescribe that measurements of an outcomeof interest are to be taken on each study participant atfixed time-points.Consider the case where interest is focused on acomparison of treatment arm means at the last scheduledvisit.ScharfsteinIntroduction
Global Sensitivity AnalysisIIThe missingness mechanism is typically not under thecontrol of the investigatorInference about the treatment arm means requires twotypes of assumptions:(i) unverifiable assumptions about the distribution ofoutcomes among those with missing data and(ii) additional testable assumptions that serve to increasethe efficiency of estimation.ScharfsteinIntroduction
Global Sensitivity AnalysisIIIIType (i) assumptions are necessary to identify thetreatment-specific means.By identification, we mean that we can write it as afunction that depends only on the distribution of theobserved data.When a parameter is identified we can hope to estimate itas precisely as we desire with a sufficiently large samplesize,In the absence of identification, statistical inference isfruitless as we would be unable to learn about the trueparameter value even if the sample size were infinite.ScharfsteinIntroduction
Global Sensitivity AnalysisIIITo address the identifiability issue, it is essential toconduct a sensitivity analysis, whereby the data analysis isrepeated under different type (i) assumptions, so as toinvestigate the extent to which the conclusions of the trialare dependent on these subjective, unverifiableassumptions.The usefulness of a sensitivity analysis ultimately dependson the plausibility of the unverifiable assumptions.It is key that any sensitivity analysis methodology allowthe formulation of these assumptions in a transparent andeasy to communicate manner.ScharfsteinIntroduction
Global Sensitivity AnalysisIIIThere are an infinite number of ways of positing type (i)assumptions.Ultimately, however, these assumptions prescribe howmissing outcomes should be ”imputed.”A reasonable way to posit these assumptions is toIIstratify individuals with missing outcomes according tothe data that we were able to collect on them and theoccasions at which the data were collectedseparately for each stratum, hypothesize a connection(or link) between the distribution of the missing outcomewith the distribution of the outcome among those withthe observed outcome and who share the same recordeddata.ScharfsteinIntroduction
Global Sensitivity AnalysisIIIIIType (i) assumptions will not suffice when the repeatedoutcomes are continuous or categorical with many levels.This is because of data sparsity.For example, the stratum of people who share the samerecorded data will typically be small. As a result, it isnecessary to draw strength across strata by ”smoothing.”Without smoothing, the data analysis will rarely beinformative because the uncertainty concerning thetreatment arm means will often be too large to be ofsubstantive use.As a result, it is necessary to impose type (ii) smoothingassumptions.Type (ii) assumptions should be scrutinized with standardmodel checking techniques.ScharfsteinIntroduction
Global Sensitivity Type(ii)Type(i)Assump(onsTreatment- ‐SpecificMeanScharfsteinIntroduction
ExampleSimulated dropout rate0.4 30.3 40.2 4 253 10.151 2 0.0 0 00.0 Placebo Risperidone 6mg0.10.20.3Observed dropout rateScharfsteinIntroduction0.4
Example5500 090 118023705 5 4 Placebo Risperidone 6mg6060 2 34 708090100Mean of observed PANSS scoreScharfsteinVariance of simulated PANSS scoreMean of simulated PANSS score1004 4505 23 2 3 5 4 10 1 350 0 Placebo Risperidone 6mg250250350450Variance of observed PANSS scoreIntroduction550
ExampleScharfsteinIntroduction
ExampleScharfsteinIntroduction
ExampleScharfsteinIntroduction
ResultsScharfsteinIntroduction
ExampleScharfsteinIntroduction
Siddiqui, Hung and O’NeilIIICompared MMRM (Mixed-Effect Model RepeatedMeasure) to LOCF using simulation and data from 25NDAs.Concluded: ”MMRM analysis appears to be a superiorapproach in controlling Type I error rates and minimizingbiases, as compared to LOCF ANCOVA analysis. In theexploratory analyses of the datasets, no clear evidence ofthe presence of MNAR missingness is found.”This is NOT evidence that one should rely on MMRM.How well does MMRM fit the observed data? How doesone conduct global sensitivity analysis?ScharfsteinIntroduction
LaterI will show how toI draw inference under MAR (actually a weaker version)I evaluate the sensitivity of inferences to deviations fromMAR.I incorporate auxiliary variables into the analysisScharfsteinIntroduction
DiscussionIIIHow can sensitivity analysis be integrated into theregulatory decision process?How can companies be encouraged to minimize missingdata? Will requiring the reporting of global sensitivityanalyses be useful in this regard?What is your perspective on intention-to-treat?ScharfsteinIntroduction
A Sensitivity Analysis Paradigm forRandomized Studies with Missing DataDaniel ScharfsteinJohns Hopkins Universitydscharf@jhsph.eduOctober 1, 2013ScharfsteinParadigm
Sensitivity AnalysisIIIRestrict consideration to follow-up randomized studydesigns that prescribe that measurements of an outcomeof interest are to be taken on each study participant atfixed time-points.Focus on monotone missing data patternConsider the case where interest is focused on acomparison of treatment arm means at the last scheduledvisit.ScharfsteinParadigm
NotationIIIIIK scheduled post-baseline assessments.There are (K 1) patterns representing each of the visitsan individual might last be seen, i.e., 0, . . . , K .The (K 1)st pattern represents individuals whocomplete the study.Let Yk be the outcome scheduled to be measured at visitk, with visit 0 denoting the baseline measure (alwaysobserved).Let Yk (Y0 , . . . , Yk ) and Yk (Yk 1 , . . . , YK ).ScharfsteinParadigm
NotationIIIIIILet Rk be the indicator of being on study at visit k.R0 1; Rk 1 implies Rk 1 1.Let C be the last visit that the patient is on-study:C max{k : Rk 1}.We focus inference separately for each treatment arm.The observed data for an individual is O (C , YC ).We want to estimate µ E [YK ].ScharfsteinParadigm
Sensitivity AnalysisIInference about the treatment arm means requires twotypes of assumptions:(i) unverifiable assumptions about the distribution ofoutcomes among those with missing data and(ii) additional testable assumptions that serve to increasethe efficiency of estimation.ScharfsteinParadigm
Sensitivity AnalysisIIIIType (i) assumptions are necessary to identify thetreatment-specific means.By identification, we mean that we can write it as afunction that depends only on the distribution of theobserved data.When a parameter is identified we can hope to estimate itas precisely as we desire with a sufficiently large samplesize,In the absence of identification, statistical inference isfruitless as we would be unable to learn about the trueparameter value even if the sample size were infinite.ScharfsteinParadigm
Sensitivity AnalysisIIITo address the identifiability issue, it is essential toconduct a sensitivity analysis, whereby the data analysis isrepeated under different type (i) assumptions, so as toinvestigate the extent to which the conclusions of the trialare dependent on these subjective, unverifiableassumptions.The usefulness of a sensitivity analysis ultimately dependson the plausibility of the unverifiable assumptions.It is key that any sensitivity analysis methodology allowthe formulation of these assumptions in a transparent andeasy to communicate manner.ScharfsteinParadigm
Sensitivity AnalysisIIIThere are an infinite number of ways of positing type (i)assumptions.Ultimately, however, these assumptions prescribe howmissing outcomes should be ”imputed.”A reasonable way to posit these assumptions is toIIstratify individuals with missing outcomes according tothe data that we were able to collect on them and theoccasions at which the data were collectedseparately for each stratum, hypothesize a connection(or link) between the distribution of the missing outcomewith the distribution of the outcome among those withthe observed outcome and who share the same recordeddata.ScharfsteinParadigm
Sensitivity AnalysisIIIIIType (i) assumptions will not suffice when the repeatedoutcomes are continuous or categorical with many levels.This is because of data sparsity.For example, the stratum of people who share the samerecorded data will typically be small. As a result, it isnecessary to draw strength across strata by ”smoothing.”Without smoothing, the data analysis will rarely beinformative because the uncertainty concerning thetreatment arm means will often be too large to be ofsubstantive use.As a result, it is necessary to impose type (ii) smoothingassumptions.Type (ii) assumptions should be scrutinized with standardmodel checking techniques.ScharfsteinParadigm
Sensitivity Type(ii)Type(i)Assump(onsTreatment- ‐SpecificMeanScharfsteinParadigm
Example: K 2IIIFull Data: (Y0 , Y1 , Y2 )Observed Data: (C , YC )Estimate µ E [Y2 ].ScharfsteinParadigm
Missing at random (MAR)IIn this setting, MAR postulatesf (Y0 C 0, Y0 ) f (Y0 C 1, Y0 )f (Y1 C 1, Y1 ) f (Y1 C 2, Y1 )orP[C 0 C 0, Y2 ] P[C 0 C 0, Y0 ]P[C 1 C 1, Y2 ] P[C 1 C 1, Y1 ]ScharfsteinParadigm
Missing at random (MAR)IIIIMAR is a type (i) assumption. It is ”unverifiable.”For patients with C c, we cannot learn from theobserved data about the conditional (on observed history)distribution of outcomes after visit c.For patients with C c, any assumption that we wouldmake about the conditional (on observed history)distribution of the outcomes after visit c will beunverifiable from the data available to us.For patients with C c, the assumption that theconditional (on observed history) distribution of outcomesafter visit c is the same as those who remain on-studyafter visit c and have the same observed history isunverifiable.ScharfsteinParadigm
Aside: Math ReviewSuppose X and Y are random variables.Zf (y ) f (y x)dF (x)ZE [Y ] E [E [Y X ]] E [Y X x]dF (x)In the special where X is an indicator variable,f (y ) f (y X 1)P[X 1] f (y X 0)P[X 0]E [Y ] E [Y X 1]P[X 1] E [Y X 0]P[X 0]IIf Y is independent of X , then f (y X ) f (y ) andE [Y X ] E [Y ]ScharfsteinParadigm
Aside: Math ReviewSuppose there is a third variable WZf (y x) f (y w , x)dF (w x)E [Y X x] E [E [Y W , X x] X x]Z E [Y W w , X x]dF (w x)wIIf Y is independent of X given W , thenf (y X , W ) f (y W ) and E [Y X , W ] E [Y W ]ScharfsteinParadigm
Missing at random (MAR)IUnder MAR, µ is identifiedZE [Y2 Y0 y0 ]dF (y0 )µ y0Z{E [Y2 C 0, Y0 y0 ]P[C 0 Y0 y0 ] y0E [Y2 C 1, Y0 y0 ]P[C 1 Y0 y0 ]} dF (y0 )Z{E [Y2 C 1, Y0 y0 ]P[C 0 Y0 y0 ] y0E [Y2 C 1, Y0 y0 ]P[C 1 Y0 y0 ]} dF (y0 )ZE [Y2 C 1, Y0 y0 ]dF (y0 ) y0ScharfsteinParadigm
Missing at random (MAR)ZZE [Y2 C 1, Y1 y1 , Y0 y0 ]dF (y1 C 1, Y0 y0 )dF (y0 ) y0Zy1Z{E [Y2 C 1, Y1 y1 , Y0 y0 ]P[C 1 C 1, Y1 y1 , Y0 y0 ] y0y1E [Y2 C 2, Y1 y1 , Y0 y0 ]P[C 2 C 1, Y1 y1 , Y0 y0 ]}dF (y1 C 1, Y0 y0 )dF (y0 )ZZ{E [Y2 C 2, Y1 y1 , Y0 y0 ]P[C 1 C 1, Y1 y1 , Y0 y0 ] y0y1E [Y2 C 2, Y1 y1 , Y0 y0 ]P[C 2 C 1, Y1 y1 , Y0 y0 ]}dF (y1 C 1, Y0 y0 )dF (y0 )ZZE [Y2 C 2, Y1 y1 , Y0 y0 ]dF (y1 C 1, Y0 y0 )dF (y0 ) y0Iy1µ is written as a function of the distribution of theobserved data.ScharfsteinParadigm
Missing at random (MAR)This identification formula holds under the weaker assumption:f (Y2 C 0, Y0 ) f (Y2 C 1, Y0 )f (Y2 C 1, Y1 ) f (Y2 C 2, Y1 )orP[C 0 C 0, Y2 , Y0 ] P[C 0 C 0, Y0 ]P[C 1 C 1, Y2 ] P[C 1 C 1, Y1 ]More generally,f (YK C k, Yk ) f (YK C k 1, Yk )P[C k C k, YK , Yk ] P[C k C k, Yk ]ScharfsteinParadigm
Missing at random (MAR)III ) (parameters η).Specify models for f (Yk C k, Yk 1Estimate η using maximum likelihood.Estimate µ by repeating the following simulationprocedure:1. Simulate Y0 from its empirical distribution. Set k 1 2. Simulate Yk from f (Yk C k, Yk 1; ηb), Set k k 1.3. If k K then stop; otherwise repeat step 2.IIITake an average of the simulated YK ’sG-computation algorithm.Standard errors using non-parametric bootstrap.ScharfsteinParadigm
Missing at random (MAR)IUnder MAR,"µ E QK 1k 0III (C K )YK#P[C k C k, Yk ] ), one canSo, rather than modeling f (Yk C k, Yk 1 model P[C k C k, Yk ]Suppose we assumelogit{P[C k C k, Yk ]} hk (Yk ; γ)IIEstimate γ by maximum likelihoodEstimate µ by the inverse-weighted estimator"#I (C K )YKµ̃ En QK 1 b]k 0 P[C k C k, Yk ; γScharfsteinParadigm
Missing not at random (MNAR)IThe MAR assumption is not the only one that is (1)unverifiable and (2) admits identification of µ.ScharfsteinParadigm
Missing not at random (MNAR)INon-future dependence:f (Y2 C 0, Y1 ) f (Y2 C 1, Y1 )(1)andf (Y1 C 1, Y0 ) exp{αr (Y1 )}E [exp{αr (Y1 )} C 1, Y0 ]f (Y2 C 2, Y1 ) exp{αr (Y2 )} f (Y2 C 1, Y1 ) E [exp{αr (Y2 )} C 2, Y1 ]f (Y1 C 0, Y0 ) IIIr (y ) is a specified function of yα is a sensitivity analysis parameter that governsdepartures from the MAR assumptionα 0 is MAR, α 6 0 is MNAR.ScharfsteinParadigm(2)
Exponential TiltingIIf [Y1 C 1, Y0 ] N(µ1 (Y0 ), σ12 ) and r (Y1 ) Y1 , then[Y1 C 0, Y0 ] N(µ1 (Y0 ) ασ12 , σ12 )IIf [Y1 C 1, Y0 ] Beta(a1 (Y0 ), b1 (Y0 )) andr (Y1 ) log(Y1 ), then[Y1 C 0, Y0 ] Beta(a1 (Y0 ) α, b1 (Y0 ))α a1 (Y0 )ScharfsteinParadigm
2.0Beta0.51.0f(y L 1)0.0Density1.5f(y L 0)0.00.20.40.6yScharfsteinParadigm0.81.0
Exponential TiltingIIf [Y1 C 1, Y0 ] Gamma(a1 (Y0 ), b1 (Y0 )) andr (Y1 ) log(Y1 ), then[Y1 C 0, Y0 ] Gamma(a1 (Y0 ) α, b1 (Y0 )),Iα a1 Y0 ).If [Y1 C 1, Y0 ]] Gamma(a1 (Y0 ), b1 (Y0 )) andr (Y1 ) Y1 , then[Y1 C 0, Y0 ] Gamma(a1 (Y0 ), b1 (Y0 ) α),α b1 (Y0 ).ScharfsteinParadigm
0.81.0GammaDensity0.6f(y L 1)0.00.20.4f(y L 0)0123yScharfsteinParadigm45
Exponential TiltingIIf [Y1 C 1, Y0 ] Bernoulli(p1 (Y0 )) and r (Y1 ) Y1 ,then [Y1 C 0, Y0 ] BernoulliScharfsteinp1 (Y0 ) exp(α)p1 (Y0 ) exp(α) 1 p1 (Y0 )Paradigm
IdentificationIIf (Y2 C 2, Y1 ) is identifiedBy (2), f (Y2 C 1, Y1 ) is identifiedf (Y2 C 0, Y0 )Z f (Y2 C 0, Y1 y1 , Y0 )dF (y1 C 0, Y0 )y1(1,2)Zf (Y2 C 1, Y1 y1 , Y0 ) y1dF (y1 C 1, Y0 ) exp{αr (y1 )}E [exp{αr (y1 )} C 1, Y0 ]f (Y2 C 1, Y1 y1 , Y0 ) f (Y2 C 1, Y1 y1 , Y0 )P[C 1 C 1, Y1 y1 , Y0 ] f (Y2 C 2, Y1 y1 , Y0 )P[C 2 C 1, Y1 y1 , Y0 ]If (Y2 C 0, Y0 ) is identifiedScharfsteinParadigm
IdentificationµZE [Y2 Y0 y0 ]dF (y0 ) y0Z{E [Y2 C 0, Y0 y0 ]P[C 0 Y0 y0 ] y0E [Y2 C 1, Y0 y0 ]P[C 1 Y0 y0 ]} dF (y0 )Z{E [Y2 C 0, Y0 y0 ]P[C 0 Y0 y0 ] y0)(ZE [Y2 C 1, Y1 y1 , Y0 y0 ]dF (y1 C 1, Y0 y0 )y1ScharfsteinParadigm)P[C 1 Y0 y0 ]dF (y0 )
EstimationIIII ) (params η).Specify models for f (Yk C k, Yk 1Specify models for P[C k C k, Yk ] (params γ).Estimate η and γ using maximum likelihood.Estimate µ by repeating the following simulationprocedure:1. Simulate Y0 from its empirical distribution.2. Draw from P[C 0 C 0, Y0 ; γb].3. If C 0, then draw from f (Y2 C 0, Y0 ; γb, ηb; α) andstop.4. If C 6 0, the draw Y1 from f (Y1 C 1, Y0 ; ηb).5. Draw from P[C 1 C 1, Y1 ; γb]6. If C 1, then draw from f (Y2 C 1, Y1 ; ηb; α) andstop7. If C 2 then draw from f (Y2 C 2, Y1 ; ηb).ScharfsteinParadigm
EstimationIIITake an average of the simulated Y2 ’sGeneralization of G-computation algorithm.Standard errors using non-parametric bootstrap.ScharfsteinParadigm
EstimationTo draw from f (Y2 C 1, Y0 ; ηb; α) in Step 6, draw fromf (Y2 C 2, Y1 ; ηb) exp{αr (Y2 )}E [exp{αr (Y2 )} C 2, Y1 ; ηb]ScharfsteinParadigm
EstimationTo draw from f (Y2 C 0, Y0 ; γb, ηb; α) in Step 3, draw from1. Draw Y1 fromf (Y1 C 1, Y0 ; ηb) exp{αr (Y1 )}E [exp{αr (Y1 )} C 1, Y0 ; ηb]2. Draw from P[C 1 C 1, Y1 ; γb]3. If C 1, then draw from f (Y2 C 1, Y1 ; ηb; α) (seeprevious slide) and stop4. If C 2 then draw from f (Y2 C 2, Y1 ; ηb).Recursive algorithm.ScharfsteinParadigm
Missing not at random (MNAR)Assumption (1) and (2) are equivalent to ]} hk (Yk ) αr (Yk 1 )logit{P[C k C k, YK , Yk 1wherehk (Yk ) logit{P[C k C k, Yk ]} log{E [exp{αr (Yk 1 )} C k, Yk ]}IIα is the conditional log odds ratio of last being seen atvisit k between patients who differ by one unit in r (Yk 1 ).Assuming that r (y ) is monotonically increasing, α 0implies that patients with higher values of Yk 1 are morelikely to withdraw than those who remain on study.ScharfsteinParadigm
Missing not at random (MNAR)"µ E QK 1k 0II (C K )YK#{1 expit(hk (Yk ) αr (Yk 1 ))}(Indirectly) Estimate hk (Yk ) byb, ηb; α)h(Yk ; γ logit{P[C k C k, Yk ; γb]} log{E [exp{αr (Yk 1 )} C k, Yk ; ηb]}IEstimate µ by the inverse-weighted estimator"µ̃ En QK 1k 0I (C K )YK{1 expit(hk (Yk ; γb, ηb; α) αr (Yk 1 ))}ScharfsteinParadigm#
Notes on G-Computation vs. Inverse-WeightedEstimatorIIIUnder correct model specification, G-computationestimator is more efficient.Inverse-weighted estimator does not extrapolate.Can also directly model hk (Yk ). Model checking harder.ScharfsteinParadigm
Incorporating Auxiliary VariablesIIILetVk denote auxiliary variables scheduled to be collectedat assessment kLet Wk (Yk , Vk )MARf (YK C k, Wk ) f (YK C k 1, Wk )P[C k C k, YK , Wk ] P[C k C k, Wk ]INon-Future Dependencef (YK C k, Wk , Yk 1 ) f (YK C k 1, Wk , Yk 1 )P[C k C k, YK , Yk 1 , Wk ] P[C k C k, Yk 1 , Wk ]ScharfsteinParadigm
Incorporating Auxiliary VariablesISensitivity Analysis Modelsf (Yk 1 C k, Wk )f (Yk 1 C k 1, Wk ) exp(αr (Yk 1 )) E [exp(αr (Yk 1 ) C k 1, Wk ]logit{P[C k C k, YK , Wk , Yk 1 ]} h(Wk ) αr (Yk 1 )whereh(Wk ) logit{P[C k C k, Wk ]} log{E [exp{αr (Yk 1 )} C k, Wk ]}ScharfsteinParadigm
Incorporating Auxiliary VariablesIINeed a model for f (Vk C k, Yk , Wk )Can extend G-computation and Inverse-weightedestimation procedureScharfsteinParadigm
Main IdeaIILink non-identifiable to identifiable distributions usingsensitivity analysis parametersModel the distribution of the observed dataScharfsteinParadigm
DiscussionIMethods made seem complicated, but so are thoseunderlying other statistical procedures such as multipleimputation and MMRM.ScharfsteinParadigm
A Sensitivity Analysis Paradigm forRandomized Studies with Missing DataDaniel ScharfsteinJohns Hopkins Universitydscharf@jhsph.eduOctober 1, 2013ScharfsteinCase Study
Case Study: Chronic SchizophreniaIIIMajor breakthroughs have been made in the treatment ofpatients with psychotic symptoms.However, side effects associated with typical and atypicalneuroleptics have limited their usefulness.RIS-INT-3 (Marder and Meibach, 1994, Chouinard et al.,1993) was a multi-center study designed to assess theeffectiveness and adverse experiences of four fixed dosesof risperidone compared to haliperidol and placebo in thetreatment of chronic schizophrenia.ScharfsteinCase Study
RIS-INT-3IIIIIAt selection, patients were required to have a PANSS(Positive and Negative Syndrome Scale) score between 60and 120.Prior to randomization, there was a single-blind, one-weekwashout phase during which all anti-psychoticmedications were to be discontinued.If acute psychotic symptoms occurred, patients wererandomized to a double-blind treatment phase, scheduledto last 8 weeks.Patients were randomized to one of 6 treatment groups:risperidone 2, 6, 10 or 16 mg, haliperidol 20 mg, orplacebo.Dose titration occurred during the first week of thedouble-blind phase.ScharfsteinCase Study
RSIP-INT-3IIIPatients scheduled for 5 post-baseline assessements atweeks 1,2,4,6, and 8 of the double-blind phase.Primary efficiacy variable: PANSS score521 patients randomized to receive placebo (n 88),haliperidol 20 mg (n 87), risperidone 2mg (n 87),risperidone 6mg (n 86), risperidone 10 mg (n 86), orrisperidone 16 mg (n 87).ScharfsteinCase Study
Premature WithdrawalIIIOnly 49% of patients completed the 8 week treatmentperiod.The most common reason for discontinuation was“insufficient response.”Other main reasons included: adverse events,uncooperativeness, and withdrawal of consent.ScharfsteinCase Study
Premature WithdrawalCompletedW
Robert Temple and Bob O’Neil (FDA) I "During almost 30 years of review experience, the issue of missing data in . clinical trials has been a major concern because of the potential impact on the inferences that can be drawn . when data are missing . the analysis and interpretation of the study pose a challenge and the
resulting inferences are generally conditional on the observed pattern of missing data. Further, ignoring the process that causes missing data when making direct-likelihood or Bayesian inferences about 6 is appropriate if the missing data are missing at random and q is distinct from 0.
Robert Temple and Bob O’Neil (FDA) I "During almost 30 years of review experience, the issue of missing data in . clinical trials has been a major concern because of the potential impact on the inferences that can be drawn . when data are missing . the analysis and interpretation of the study pose a challenge and the
2 Click Quick Actions, and then click Add Missing Punch. 3 Click the field with the missing punch, which is indicated by solid red. Note: You can click multiple missing punch fields if necessary. 4 To turn off the Missing Punch action, click Add Missing Punch. 5 Click Save.
Missing Data Using Stata Paul D. Allison, Ph.D. February 2016 www.StatisticalHorizons.com 1 Basics Definition: Data are missing on some variables for some observations Problem: How to do statistical analysis when data are missing? Three goals: Minimize bias Maximize use of available information Get good estimates of uncertainty
Review useful commands in Stata for missing data. General Steps for Analysis with Missing . Some MAR analysis methods using MNAR data are still pretty good. . 12 grade math score F 45 . M . 99 F 55 86 F 85 88 F 80 75. 81 82 F 75 80 M 95 . M 86 90 F 70 75
machine learning and statistical data analysis. This thesis focuses on the problems of collab-orative prediction with non-random missing data and classi cation with missing features. We begin by presenting and elaborating on the theory of missing data due to Little and Rubin. We
Where's My Data? Evaluating Visualizations with Missing Data Hayeong Song & Danielle Albers Szafir Visualizations with High Data Quality Visualizations with Low Data Quality Fig. 1: We measured factors influencing response accuracy, data quality, and confidence in interpretation for time series data with missing values.
Before accepting any instruction, the expert witness must ensure that the services required of them and all terms and conditions are clearly identified in writing and that they are aware of and compliant with the following: Protocol for the Instruction of Experts to give Evidence in Civil Claims (‘Protocol’); Civil Procedure Rules Part 35 Experts and Assessors (‘PR35 .
