Some Problems Connected With Statistical Inference
SOME PROBLEMS CONNECTED WITH STATISTICAL INFERENCEBY D. R. CoxofLondon'BirkbeckCollege,University1. Introduction.This paper is based on an invited address given to a jointmeetingof the Instituteof Mathematical Statisticsand the BiometricSocietyat Princeton,N. J., 20th April,1956. It consistsof somegeneralcomments,fewof themnew,about statisticalinference.Since the address was given publicationsby Fisher [11], [12], [13], have produced a spiriteddiscussion[7], [21], [24], [31] on the generalnatureof statisticalmethods.I have not attemptedto revisethe paper so as to commentpoint byalthoughI have, of course,pointon the specificissuesraised in this controversy,checkedthat the literatureof the controversydoes -notlead me to change theopinionsexpressedin the finalformof the paper. Parts of the paper are controversial;these are not put forwardin any dogmatic spirit.2. Inferencesand decisions. A statistical inferencewill be definedfor thepurposesofthepresentpaperto be a statementabout statisticalpopulationsmadefromgiven observationswith measureduncertainty.An inferencein general isan uncertain conclusion. Two things mark out statistical inferences.First,on whichtheyare based is statistical,i.e. consistsofobservationsthe informationsubject to randomfluctuations.Secondly,we explicitlyrecognisethat our conclusionis uncertain,and attemptto measure,as objectivelyas possible,the uncertaintyinvolved. Fisher uses the expression'the rigorousmeasurementofuncertainty'.A statisticalinferencecarriesus fromobservationsto conclusionsabout thepopulationssampled. A scientificinferencein the broadersense is usually concerned with arguingfromdescriptivefacts about populationsto some deeperofthesystemunderinvestigation.Of course,the morethe statistiunderstandingcal inferencehelps us with this latterprocess,the better.For example,consideran experimenton the effectof varioustreatmentson the macrQscopicpropertiesof a polymer.The statisticalinferenceis concernedwith what can be inferredfromthe experimentalresultsabout the true treatmenteffects.The scientificinferencemight concern the implicationsof these effectsfor the molecularstructureof the polymer;the statisticaluncertaintyis only a part, sometimessmall, of the uncertaintyof the finalinference.in the sensemeanthere,involvethe data, a specificationStatisticalinferences,of the set of possible populationssampled and a question concerningthe truepopulations.No considerationof losses is usually involved directlyin the inference,althoughthesemay affectthe questionasked. If the populationsampledReceived October7, 1957; revisedFebruary10, 1958.1Workdoneat theDepartmentof Biostatistics,School of Public Health, UniversityofNorth Carolina.357This content downloaded from 128.173.127.127 on Tue, 12 Nov 2013 12:54:09 PMAll use subject to JSTOR Terms and Conditions
358D. R. COXhas itselfbeeinselectedby a randomprocedurewithknownpriorprobabilities,it seems to be generallyagreed that inferenceshould be made using Bayes'stheorem. Otherwise,prior informationconcerningthe parameter of directinterest2will not be involved in a statisticalinference.The place of priorinformationis discussedsome morewhenwe come to talk about decisions,but thethat is not statisticalcannotbe includedgeneralpoint is that priorinformationthatwithoutabandoningthe frequencytheoryof probability,and informationis derivedfromotherstatisticaldata can be handled by methodsforthe combinationof data.The theoryof statisticaldecisiondeals withthe action to take on the basis ofDecisions are based on not onlythe considerationslistedstatisticalinformation.for inferences,but also on an assessmentof the losses resultingfromwrongas well as, ofcourse,on a specificationofthedecisions,and on priorinformation,set ofpossibledecisions.Currenttheoriesofdecisiondo not give a directmeasureof the uncertaintyinvolvedin makingthe decision; as explainedabove, a statisticalinferenceis regardedhere as having an explicitlymeasureduncertainty,and this is to be thoughtof as an essential distinctionbetweenstatisticaldecisionsand statisticalinferences.Thus, significancetests and confidenceintervals,if looked at in the way explained below, are das amethod for classifyingindividualsinto one of two groups,is a decision procedure; consideredas a tool forassigninga score to an individualto say howreasonableit is that the individualcomesfromone groupratherthan the other,it.is an inferenceprocedure.Strictpoint estimationrepresentsa decision; estimation by point estimateand standarderroris a condensedand approximateformof interval estimationand is an inferenceprocedure.Estimation by aposteriordistributionderivedfroman agreed priordistributionis an inferenceprocedure.A test of a hypothesis,consideredin the literal Neyman-Pearsonsense as a rule for taking one of two decisions concerninga statistical hypothesis,is a decisionprocedure,in whichpriorknowledgeand losses enterimplicitly.The readermay findit helpfulto considertheextentto whichthespecification, implicitlyor explicitly,of losses and prior knowledgeis essential forsolutionof the problemsjust listedas ones of decision.For example, consider the analysis of an experimentto compare two industrialprocesses,A and B. The statisticalinferencemightbe that, undercertain assumptionsabout the populations,processA givesa yieldhigherthan thatof processB, the differencepast the 1/1000level,being statisticallysignificant90, 95 and 99 per cent confidenceintervalsforthe amountof the truedifferencebeingsuch and such. The decisionmightbe that havingregardto the differencesin yield of practicalimportance,and our priorknowledge,we will considerthatthe experimenthas established,underthe conditionsexamined,that processAhas a higheryield than B and will take futureaction accordingly.2 i.e. relevantinformationabout the parameterof interest,otherthan that containedin the data and in the specificationof the set of possible parametervaluies.This content downloaded from 128.173.127.127 on Tue, 12 Nov 2013 12:54:09 PMAll use subject to JSTOR Terms and Conditions
STATISTICAL INFERENCE359An inferencewithouta priordistributioncan be consideredas answeringthequestion: 'What do these data entitleus to say about a particularaspect of thepopulationsthat interestus?' It is, however,irrationalto take action, scientificor technological,withoutconsideringboth all available relevant information,explanationsof a setincludingforexamplethe priorreasonablenessof differentof data, and also the consequencesof doing the wrongthing.Why then,do wewhichgo, as it were,onlypart of theway towardsthe finalbotherwithinferencesdecision?Even in problemswherea clear-cutdecisionis the main object, it very oftenhappens that the assessmentof losses and prior informationis subjective,sothat it will help to get clear firstthe relativelyobjective matterof what thedata say, beforeembarkingon the more controversialissues. In particular,itmay happen eitherthat the data are littleaid in decidingthe point at issue, orthat the data suggestone conclusionso stronglythat the only people in doubtabout what to do are those with priorbeliefs,or opinionsabout losses, heavilybiased in one direction.In some fields,too, it may be arguedthat one ofthe maincalls for probabilisticstatisticalmethodsarises fromthe need to have agreedrulesforassessingstrengthof evidence.A fulldiscussionof this distinctionbetweeninferencesand decisionswill notbe attempted here. Three more points are, however,worth making briefly.First, some people have suggestedthat what is here called inferenceshould beconsideredas 'summarizationof data'. This choice of wordsseems not to recognise that an essentialelementis the uncertaintyinvolved in passing fromtheobservationsto the underlyingpopulations.3Secondly, the distinctiondrawnhere is betweenthe applied problem of inferenceand the applied problemofit is possiblethat a satisfactoryset of techniquesforinferencedecision-making;could be constructedfroma mathematicalstructureverysimilarto that used indecisiontheory.Finally, it mightbe argued that in making an inferencewe are 'deciding'to make a statementof a certaintype about the populationsand that therefore,providedthat the word decision is not interpretedtoo narrowly,the study ofstatisticaldecisionsembracesthat of inference.The pointhereis that one of themain generalproblemsof statisticalinferenceconsistsin decidingwhat typesofstatementcan usefullybe made and exactly what they mean. In statisticaldecisiontheory,on the otherhand, the possibledecisionsare consideredas alreadyspecified.the observations3. The sample space. Statisticalmethodsworkby referringS to a sample space z of observationsthat mighthave been obtained. Over 2:one ormoreprobabilitymeasuresare definedand calculationsin theseprobabilitydistributionsgive our significancelimits,confidenceintervals,etc. 2 is usuallytaken to be the set of all possible samples having the same size and structureas the observations.3Aofevidence,'whichseemsa good one.refereehas suggestedtheterm'summarizationThis content downloaded from 128.173.127.127 on Tue, 12 Nov 2013 12:54:09 PMAll use subject to JSTOR Terms and Conditions
360D. R. COXFisher (see, forexample, [11]) and Barnard [41have pointed out that z mayhave no direct counterpartin indefiniterepetitionof the experiment.For example, if the experimentwere repeated,it may be that the sample size wouldchange. Thereforewhat happens when the experimentis repeated is not sufficientto determine2, and the correctchoiceof 2 may need carefulconsideration.As a commenton this point,it may be helpfulto see an example wherethesample size is fixed,wherea definitespace 2 is determinedby repetitionof theexperimentand yet whereprobabilitycalculationsover 2 do not seem relevantto statisticalinference.Suppose that we are interestedin the mean 0 ofa normalpopulationand that,by an objective randomizationdevice, we draw either (i) with probability2,one observation,x, froma normalpopulationof mean 0 and variance a, or (ii)withprobability2, one observationx, froma normalpopulationof mean 0 andvariancev2, whereo-, 02 are known,O2 02 and wherewe knowin any particularinstancewhichpopulationhas been sampled.More realisticexamplescan be given,forinstancein termsofregressionproblems in whichthe frequencydistributionof the independentvariable is known.However,the presentexampleillustratesthe pointat issue in the simplestterms.(A similarexample has been discussedfroma ratherdifferentpoint of view in[6], [29]).The sample space formedby indefiniterepetitionof the experimentis clearlydefinedand consists of two real lines I, ,2,each having probability2, andconditionallyon Li there is a normal distributionof mean 0 and variance ai.Now suppose that we ask, acceptingforthe momentthe conventionalformulation,fora test of the null hypothesis0 0, withsize say 0.05, and withmaximum poweragainst the alternative0', where0'0-1 02Considertwo tests. First, thereis what we may call the conditionaltest, inwhichcalculationsofpowerand size are made conditionallywithinthe particulardistributionthat is known to have been sampled. This leads to the criticalregionsx 1.64 u- or x 1.64 o-2, dependingon whichdistributionhas beensampled.This is mplespace.An applicationof the Neyman-Pearsonlemma showsthat the best test dependsslightlyon 0', 0-1 , 92, but is verynearlyofthe followingform.Take as the criticalregionx 1.28o-,x 50-2,if the firstpopulationhas been sampled;if the second populationhas been sampled.Qualitatively,we can achieve almost completediscriminationbetween 0 0and 0 0' when our observationis from12, and thereforewe can allow theerrorrate to rise to very nearly 10 % under 2X. It is intuitivelyclear, and caneasily be verifiedby calculation,that this increasesthe power,in the regionofinterest,as comparedwiththe conditionaltest.Now if the object of the analysisis to make statementsby a rule withcertainThis content downloaded from 128.173.127.127 on Tue, 12 Nov 2013 12:54:09 PMAll use subject to JSTOR Terms and Conditions
,the unconditionaltest just given is in order,althoughit may be doubted whetherthe specificationof desiredpropertiesis inthis case verysensible.If, however,our object is to say 'what we can learnfromthe data that we have', the unconditionaltest is surelyno good. Suppose thatwe knowwe have an observationfrom24 . The unconditionaltest says that wecan assign this a higherlevel of significancethan we ordinarilydo, because ifdistriwe were to repeat the experiment,we mightsample some quite differentof an observationbution. But this fact seems irrelevantto the interpretationwhichwe knowcame froma distributionwithvariancea2 . That is, our calculations of power,etc. should be made conditionallywithinthe distributionknownto have have been sampled,i.e. if we are using tests of the conventionaltype,the conditionaltest should be chosen.To sum up, ifwe are to use statisticalinferencesof the conventionaltype,thesample space z mustnot be determinedsolelyby considerationsofpower,or byIf difficultieswhat would happen if the experimentwere repeated indefinitely.of the sortjust explainedare to be avoided, 2 shouldbe taken to consist,so faras is possible,of observationssimilarto the observedset S, in all respectswhichbetweenthepossiblevalues oftheunknowndo not givea basis fordiscriminationas to whetherit wasparameter0 of interest.Thus, in the example,information21 or Z2 that we sampled tells us nothingabout 0, and hence we make our inferenceconditionallyon 2, or 22 .Fisher has formalizedthis notion in his concept of ancillarystatistics[10],[23], [27]. His definitionsdeal with the situationwithoutnuisance parametersand beforeoutliningan extensionthat attempts to cope with nuisance paof the originaldefinitions.rameters,it is convenientto statea forofsufficientmaminimalsetbeLetthe distributionof awheremwrittenbethatcan(t,a),interest,0, and supposebe extractedfromtcanfurthernothatcomponentsis independentof 0, andtheifinwea.Thatspace of m into setsdivide, possible,and medhereeach similar the sample space,an ancillarystatisticiscalledThenatoconditions.to be unique subject allyand we agree(i) In the example of section 3, a minimal set consists of theEXAMPLES.observation,x, and an indicatorvariable to show which populationhas beensampled. The lattersatisfiesthe conditionsforbeing an ancillarystatistic.Provided that the possiblevalues of the mean 0 includean interval,thereis no setof x values withthe same probabilityforall 0.(ii) Under the ordinaryassumptionsof normallinear regressiontheory,plusthe assumptionthat the independentvariablehas any knowndistribution(without unknown parameters),the values of the independentvariable form anancillarystatistic.(iii) The followingexample is derived fromone put forwardby a referee.statisticswithmorecom4The termsused by Fisherare that a minimalset ofsufficientponentsthan thereare parametersis called exhaustiveand a minimalset with the samenumberof componentsas thereare parametersis called sufficient.This content downloaded from 128.173.127.127 on Tue, 12 Nov 2013 12:54:09 PMAll use subject to JSTOR Terms and Conditions
362R. COXD.-1 x 1.Let x be a singleobservationwithdensity1 20x,Then we can writex [sgnx, lxl]and lxlhas the same densityforall 0. Hencewe argue conditionallyon the observed value of lxl. For example in testingo 0 against 0 0, the possibleP values (see section5) are 1 and -. This mayseem a curious result but is, I think,reasonable if one regardsa significancetest as concernedwiththe extentto whichthe data are consistentwiththe nullhypothesis.Suppose now that thereare nuisance parameters4). Let m be a minimalsetstatisticsforestimating(0, 4))and supposethat m can be partitionedofsufficientinto [t,s, a] in such a way that(i) functionsoft and 0, so-calledpivotal quantities,exist with a distributionconditionallyon a that is independentof P). If any componentof s is added tot or a, thisindependencefrom4)no longerholds. Further,no componentscan beextractedfromt and incorporatedin a;about 0 in the sense to be(ii) the values of a and s give no directinformationdefinedbelow. Then we agree to make inferencesabout 0 fromthe conditionaldistributionof (i).We need then to definewhat is meant by saying that a quantityy gives noabout 0, when nuisanceparameters4) are present.One condirectinformationdition that mightbe consideredis that the densitypry; 0, 4) should be independentof 0. This seemstoo strong,as does also the requirementthat foreverydifferentpair 0,, 02 and foreveryy, p(y; 01, 4) / p(y; 02, 4))shouldrunthroughall positivereal values as 4) varies. An appropriateconditionseems to be thatgivenadmissiblevalues y, 01, 02, 4),thereexistadmissible0, 41 , 4)2, such that01,7 ) p(Y;(1)p(y; 02,4)) PtY; 0, 4 1)p(y; 0,42)The importof the conditionis that any contemplateddistinctionbetweentwovalues of 0 mightjust as well be regardedas a distinctionbetweentwo valuesof4).For example,suppose that x is a singleobservationfroma normaldistributionabout 0of unknownmean 4 and variance 0. Then x gives no directinformationin the sense of (1), providedthat 4 is completelyunknown.Anotherexampleisnormal regressiontheorywith the independentvariable ha-vingan arbitraryunknowndistribution,not involvingthe regressionparametersof interest[10].Here a is the set of values of the independentvariable and s is the sum squaresabout the regressionline, assuming that the residual variance about the regressionline,4, is a nuisanceparameter.For a thirdexample,let ri, r2be randomlydrawnfromPoisson distributionsof means Al, ,2 and let A2 IIAI 0 be the parameterof interest;that is writethe means as X, 40, where 40is a nuisance parameter.The likelihoodof r1, r2can be written01a!e "(1 e)[e(l O)Iav*,,)(This content downloaded from 128.173.127.127 on Tue, 12 Nov 2013 12:54:09 PMAll use subject to JSTOR Terms and Conditions
STATISTICALINFERENCE'363wheret r1, a ri r2and withs null. The equation (1) is satisfied,tellingus that a gives us no direct informationabout 0. Thereforesignificanceandconfidencecalculationsare to be made conditionallyon the observedvalue of a,as is the conventionalprocedure[25].To apply the definitionswe have to regardour observationsas generatedby arandomprocess;the idea of ancillarystatisticssimplytells us how to cut downthe samplespace to thosepointsrelevantto theinterpretationofthe observationswe have.In the problemswithoutnuisance parameters,it is known that methodsofinference[5], that use only observedvalues of likelihoodratios, and not tailareas, avoid the difficultiesdiscussed above, since the likelihoodratio is thesame whetherwe argue conditionallyor not. Lindley,usingconceptsfrom[18],has recentlyshownthat fora broad class ofproblemswithnuisanceparameters,the conditionalmethodsare optimumin the edwith the choice of the sample space,not discussedhere,concernsthe possibilityand desirabilityof makinginferenceswithin finitesample spaces obtained by permutingthe observations;see, forexample,[16].4. Interval estimation. Much controversyhas centred on the distinctionbetweenfiducialand confidenceestimation.Here followfiveremarks,not aboutthe mathematics,but about the generalaims of the two methods.(i) The fiducialapproachleads to a distributionforthe unknownparameter,whereas the method of confidenceintervals,as usually formulated,gives onlyone intervalat some preselectedlevel of probability.This seems at firstsightadistinctpoint in favour of the fiducialmethod. For when we writedown theconfidenceinterval(x - 1.96 ac/\/n,x 1.96 o/V/n) fora completelyunknownnormalmean,thereis certainlya sensein whichthe unknownmean 0 is likelytolie near the centreof the interval,and ratherunlik
A statistical inference carries us from observations to conclusions about the populations sampled. A scientific inference in the broader sense is usually con- cerned with arguing from descriptive facts about populations to some deeper understanding of the system under investigation. Of course, the more the statisti-
1 Problems: What is Linear Algebra 3 2 Problems: Gaussian Elimination 7 3 Problems: Elementary Row Operations 12 4 Problems: Solution Sets for Systems of Linear Equations 15 5 Problems: Vectors in Space, n-Vectors 20 6 Problems: Vector Spaces 23 7 Problems: Linear Transformations 28 8 Problems: Matrices 31 9 Problems: Properties of Matrices 37
CHEMICAL KINETICS & NUCLEAR CHEMISTRY 1. Theory 2. Solved Problems (i) Subjective Type Problems (ii) Single Choice Problems (iii) Multiple Choice Problems (iv) Miscellaneous Problems Comprehension Type Problems Matching Type Problems Assertion-Reason Type Problems 3. Assignments (i) Subjective Questions (ii) Single Choice Questions
agree with Josef Honerkamp who in his book Statistical Physics notes that statistical physics is much more than statistical mechanics. A similar notion is expressed by James Sethna in his book Entropy, Order Parameters, and Complexity. Indeed statistical physics teaches us how to think about
Module 5: Statistical Analysis. Statistical Analysis To answer more complex questions using your data, or in statistical terms, to test your hypothesis, you need to use more advanced statistical tests. This module revi
Lesson 1: Posing Statistical Questions Student Outcomes Students distinguish between statistical questions and those that are not statistical. Students formulate a statistical question and explain what data could be collected to answer the question. Students distingui
to calculate the observables. The term statistical mechanics means the same as statistical physics. One can call it statistical thermodynamics as well. The formalism of statistical thermodynamics can be developed for both classical and quantum systems. The resulting energy distribution and calculating observables is simpler in the classical case.
Statistical Methods in Particle Physics WS 2017/18 K. Reygers 1. Basic Concepts Useful Reading Material G. Cowan, Statistical Data Analysis L. Lista, Statistical Methods for Data Analysis in Particle Physics Behnke, Kroeninger, Schott, Schoerner-Sadenius: Data Analysis in High Energy Physics: A Practical Guide to Statistical Methods
In addition to the many applications of statistical graphics, there is also a large and rapidly growing research literature on statistical methods that use graphics. Recent years have seen statistical graphics discussed in complete books (for example, Chambers et al. 1983; Cleveland 1985,1991) and in collections of papers (Tukey 1988; Cleveland
