First Steps In Developing A Risk-benefit Analysis .

First steps in developing a risk-benefitanalysis framework: a systematic review andcritical evaluation of risk-benefit analysismethodsLarry Lynd,LL d PhDMegan Coombes, MScAmir Adel RishidiRishidi, MD,MD MHAMark Sculpher, PhDAndrew Willan,, PhDFaculty of Pharmaceutical Sciences, University of British Columbiapgy and EvaluationCentre for Clinical EpidemiologyVancouver General HospitalVancouver, BC, CanadaCentre for Health Economics,, Universityy of York,, UK

Background to RiskRisk-Benefit Evaluation Historically in risk-benefitrisk benefit analysis,analysis only benefitwas deemed important Current paradigm – Frequentist– Independent evaluation of risks and benefits– Arbitrary threshold of p 0.05

Li it tiLimitationsoff currentt paradigmdi R-BRB ratioti mostt oftenft didiscussedd ini absoluteb l t terms,tif att allllOften based on RCT data – limited precision in estimatingdifferences in riskDoes not consider the valuation of the risks and benefits Fails to consider:1. The nature of the risks or benefits2. The precision or uncertainty of the incremental risksand benefits3. Risk preferencesp4. Risks and benefits concurrently

The way of the future Regulatory bodies increasingly requiring explicit R-BevaluationQuantitative methods for concurrently evaluating risks andbenefits Æ EVIDENCE BASED DECISIONSEvaluating multiple risks and multiple benefitsIncorporate:– Relevant preferences– Uncertainty– Different patient characteristics (risk) PROCESS NEEDS TO BECOME SYSTEMATIC & EXPLICIT

Change in Nomenclature Traditionally referred to ‘risk-benefit’ analysis ‘Risk’‘Ri k’ refersftot bothb th “BENEFITS”,“BENEFITS” andd “ADVERSEEVENTS” Rather,Rather we are comparing ‘harms’ and ‘benefits’ Therefore, appropriate nomenclature:HARM-BENEFIT ANALYSIS

Objectives: Identifyy and establish criteria necessaryy for apractical, applied HBA methodology Perform a systematic review to identify allcurrently proposed HBA methods Propose a methodologic framework that bestmeets the proposed criteria

Criteria1. Universal All interventions and health states2. Inclusive Multiple benefits and multiple harms3. Comprehensive Objective and subjective harms and benefits4. Patient-sensitive Stratified risk analysis5. Easily interpreted By all potential stakeholders/perspective

Criteria6 Explicit preferences6. For both harms and benefits7 Threshold7. Inherently defined H-B threshold8 Incorporates uncertainty8. Quality and source of data, and in the finalmetric9. Flexible/Adaptable Rapid,Rapid efficientefficient, incorporate new knowledge10. Integrate with Economic Evaluations

Results 10 metrics / methods identified Not all are HB methods– Some onlyy evaluate benefits– Chronological progression Complexity Increasingly satisfy more criteria

Methods in Chronology NNT , NNE / NNH IntegrationI tti off BenefitBfit andd HarmH– Unqualified Success/Unmitigated Failures– R1 & R2 (Chuang(ChSt i )Stein) Preference / Threshold based– NNTT, NNTU&T, NNTMCE– Risk and preference adjusted surplus efficacy

Methods in Chronology Risk Benefit Contour Q-TWiSTQ TWiST ‘Net benefit’ – decision analytic methods

Examplep of ‘net benefit’/decision analysis:yHB analysis of HRT postpost-menopause Probabilistic clinical decision model Benefits:– Improved sxsx, rate of hip fracture,fracture risk ofcolorectal and endometrial CA Harm:– Breast CA, coronary heart disease, stroke, PEMinelli, C. et al. BMJ 2004;328:371

Minelli, C. et al. BMJ 2004;328:371

AsymptomaticWomenSymptomaticWomenMinelli, C. et al. BMJ 2004;328:371

Fig 3: Probability of net harm (%) associated with HRT use for five years according toutility attributed to menopausal symptoms by individual women and their baseline risksof breast cancer. Isolines define combinations of utility and baseline risk with sameprobability of net harmMinelli CMinelli,C. et alal. BMJ 2004;328:371Copyright 2004 BMJ Publishing Group Ltd.

Recommendations and Conclusions Net benefit (decision analysis) meets all a prioriestablished criteria– Universal All interventions, health states & scales– Inclusive Multiple benefits and multiple harms– Comprehensive Objective and subjective harms andbenefits– Patient-sensitive Stratified risk analysis– Easily interpreted (we think) By all potential stakeholders/perspective

Recommendations and Conclusions– Explicit preferences For both harms and benefits– Threshold Naturally zero (net health benefit)– Incorporates uncertainty QualityQand source off data, finalfmetric,decision– Flexible/Adaptable Rapid, efficient, incorporate newknowledgeg Facilitates modeling, when necessary– Integrates with Economic Evaluations

Implementation issues Data synthesis Availability of preferences– Utilities– Contingent valuation– Conjoint analysis or Discrete ChoiceExperimentation Perspective Acceptance of methods/results by decisionmakers

Background to RiskBackground to Risk--Benefit EvaluationBenefit Evaluation Historically in riskHistorically in risk -benefit analysis only benefitbenefit analysis , only benefit was deemed important Current paradigm – Frequentist – Independent evaluation of risks and benefitsIndependent evaluat