ADOA099 507 STANFORD UNIV CA DEPT OF PSYCHOLOGY

PAGEactivein thefeminist movement.statistically naivestatement to beIn aundergraduates,more probable.graduate students, only 5086%In alarge samplejudged10ofthe secondsample of psychologycommitted thiserror.However,the difference between statistically naive and sophisticatedrespondentsembeddedLinda.vanishedin awhenlist ofOver 80%effect.Similardesign,in whichoftheeightused incriticalresults were obtainedthecriticalrule-endorsementa between-subjectcompared(1982).an effort to determine whether people understand andstatements,which theyThe statement:First, werespondents.probablewith severalwere to classify asofXandY"For comparison,than B.wasrule-liketrue or false.then theythe endorsementisendorsedonly 6X endorsedcannotresults indicate some understandingalthoughpresented a group of"The probability of X is always greater thanprobabilityofwereofA is morethe conjunctionrule,perhaps becauseformulation.given the-"If81XTheseAn argument-endorsement procedure wasrespondentsbyboth occur".not unanimous,the abstract and unfamiliarwhichaboutthe conjunctioncategories werestatistically naive college studentsofinwereand argument-endorsement wereaccept the conjunction rule.theitemscomparable statementsboth groups exhibitedindirectly (Tversky C KahnemanTests oftwo also employed,description-of.--inLinda,

PAGE 11followedcheckbystatements Awhichofand8 above,the followingand wereargumentstheyaskedtoconsideredcorrect:(i)A is moreprobable than 8 because the probabilitythatLinda isboth a bank teller and an activemustbe smaller than the probability that shefeministis a bankteller.(ii)B is more probable than A because Linda resemblesa bankteller whomore than sheArgumentby 83%ofofthe(i)isactive inresembles a bankfavoringthefeminist movementteller.the conjunction rule wasthe psychology graduate ern.immediatelyconjunctionwere muchExtensivethisrespondentson the other hand,normative arguments,but only by 43%undergraduates.respondentsthe mpressedand many remained committedthatwereinconsistentHaivebyto theirwiththeconjunction rule.Muchtoour surprise,solid grasp ofitthe conjunctionin the abstractimpressionofnaive subjectsbut notrule;whensubjectshave athey tended to endorseit conflicted with a strongrepresentativeness.statistically traineddid notOnrecognizedtheotherhand,the validityof

PAGErule,theandtransparentdid hereffect inhowever,less transparentIn termsofthepresentofthisissueseeTverskyand(1982).a. attemptJepson & Fongto describelevelsconclusions andThestories.the statisticalof sophistication, Nisbett,requiredtoinferences attributed to characters ples,which were highly correlatedtraining.Naturally,vary with intelligence,oftenstatisticalchange ofknowledge,non-intuitiveSimon &McDermott,intuitive ermore,the gambler'sand1980).ofindividualstatisticalthe levelstatisticalofintuitionsAseducation.novice gfor studentswithbriefis intuitive for the ms ofKrantz,justify certainevaluate anddifferencesstatisticalintuitions ofused an elicitation procedure, in which(1982)respondents wereisespeciallythe more sophisticated subjects.least forpeople at variousotherin ansophistication,same problem.application,Initthe conjunction effect appears to be an error oftreatment,Kahnemanapplythe conjunctionpreventversior&able meor thecounter-theory (Feller,is some evidence thatwhichare commonly

IrPAGE 13committedbynaive respondentscan alsostatistically sophisticated ones,subtlety (Tversky & Kahneman,Evansintuitions1966).andinMason,theIn thealsowell knownover to testversionhasto identify thethe rule "ifthevowelon thefailure ofanodd number(Mason,problem,T,side ofThe correct responseisof whether or not tothe argumentsby whichofbecausefirst card4.thisgive reasonsor aIn a strikingshowing A andtheir decisionseach of the four cards.be turneditinvestigated different versionstoandon one side,reasoning, most subjects electthe cardsthe4 and 7,that shouldon thesubjectsto look atWason andproblem,andor argumentsforlook atthe hidden sideThe investigators concluded thatsubjectswere mere rationalizations,thatproblem7 should be examined,required theiroflogicalsecond would refute the rule.logicalthe hiddenEvansofcardsother".the cards showing A andobservationshowing A.Evansofof thisa card has a vowelan even number on thethatin studiesfour-cardexperimenter displays four cardsasks subjectsof greaterused by Mason and1976)standardproblemsfrom1971).The elicitation method was(1975;withbe elicitedjustifiedtheir responsesrather than statements ofrulesactually guided their decisions.Other evidencethe rules offor people'sverification wasinadequate understanding ofreportedby Mason(1969)and

PAGE 14by Wasonand Johnson-Laird"therapy",oftheirattention to theirlittle effectcomprehension,theoftoin thethat we have considered sorulesorrule becausewe may wantrules thatargumentsandWilsonsuch athethe elicitationofWetestis oftenhave notstate theunreasonablyto credit people with understandingproceduresforestablishing anofpeople'sisalsopossibleand Hisbett(1980)showedSlovic C Tversky,confirmin other research designs.were toldhumane or quitethat theForanbrutal)were typicalofoferrorofexample. Hamill,subjectsopinionsto aa relevant rule or1978;toerror ofresponsesaninterview allegedly conducted with a prison guard.subjectspeople'sfailure ofrespondents to(McClelland C Rohrbaugh,applicationtask.far involvedparticular case with their judgment aboutItthatresponse.of askingapplication require a comparison1974).samethey cannot articulate properly.The preferredargumentsubjects*application.discussed the procedureoftheThis procedure hadsuggestjustify a particulardemanding:called theperformanceresultsprovidesubjects withverification task reflect anot ofThe examplesendorsementjudgments andon subsequentdifficulties in therelevantorder toinconsistent answers.together,argumentsInthese investigators e guardthe(veryprison personnel,

PAGEwhiletheothersubjectsattitudes were atypical,weretoldand thathe wasmuch less humane than most ofthatissues.the opinionsofThe surprisingexpressedbywronginmember ofthisofnotbeeninfluenced bytypicalityotherthewould(Hisbett C Wilson,thatan errorofbetween-group comparisonguardhad d'sthis case andappears reasonableapplicationyields aobviouslyrealize that they had1977).itonjudgment as erroneous,informationbetween-subject studies,concludeprison personneljudgments,many subjectsThe subjectsSomething isimpossible to describe any particularand unlikely thateither much more oran atypicalthe group.patternguard'sresult of the study wasas much impact on generalizations as didto a typicalthehis colleagues.then estimated the typical attitudesa variety oxthat15wasmaderesult thatifintothemost peoplewould consider untenable.We havethatdefined anviolates aand accepts.thenature ofvaliderrorapplication asrule that theHowever,itan error,isoften difficult to determineof a rule mayFurthermore, the same rule mayproblem context and notinanother.provides a striking example:verify the rule "ifa card hasa responseindividual understandsbecause differentunderstanding and acceptanceresults.oftestsoftheyield differentbe violatedin oneThe verification tasksubjects who did not correctlya vowel on one side,it has

PAGEan even number on the other"a formally equivalentrule:"ifis sealeda letter1972;Sonino-Legrenzi,in verifyingno difficulty(see Wason G Shapiro,five cent stamp"Legrenzi &had1971;16it has aJohnson-Laird,& Wason,Johnson-Laird1977).These results illustrate a typicalofIt appears that people do not possess a validreasoning.generalrule for the verification ofthey wouldare not--ifblindto the correctthansinceis in factSeveraltakento meanthissuggest ait couldpeople do notthatstrictly speaking, correctOn the other hand,followed.misleadingisstatement may bemore generalofconclusionsearlythat manyof baseprinciples ofthecaserelevant willratesinBayesianeveryproblem tobe answeredcannot appear compellinglaw ofwhichincorrectly,ofitIt hasgenerallylarge numbers,inference,Butregressive prediction.thatdeficitstudiesnot haveadults dointuitions corresponding to theroleisobserved.been demonstratedtheitthatrepresentativeness appear to have a similar status.validtheywould alsorule or else theythe correct intuition is,a ruleor elseOn the other hand,The statementproblem.possession ofalwaysif-statements.solve the card problem.fail the stamppossessin the studypatternortheis simply nottheseor thatin particular contexts.rulesarethe rules

IPAGEThepropertieseasy orhardmodels,orthat maketo solveformally equivalentappear toschemas,be relatedwhich the problemsto17problemsthe mentalevoke (Rumelhart,1979).For example, it seems easier to see the relevance of"not-q"tothe implicationcontrol schemathanin aconclusionappears(didimply theofnegationthat differentConsequently,terms ofiscontextsqualityletter?)oftheItschema-bound ororinferential(HayesC Simon,human reasoning cannot be adequatelycontent-independent formalThe problem of mapping statisticalorconclusionsbya seriesIt was this method thatrules.logical intuitionsis further complicated by the possibility ofsteps.anegationoperationsavailable in differentunexpectedinthe hypothesis?).the actual reasoning processdescribed inq"they forget to stamp the sealed(does thecontent-bound so1978).impliesconfirmation schemathatrules are"pofreachinghighlyhighlyintuitiveSocrates employed with greatsuccess to convince his naive disciples that they had alwaysknowntruths,Should anywhich he wasonly then making them discover.conclusions that can beintuitivesteps e?theby aBrainecontextreasoning, and he proposedimmediacy as a test:is intuitivetruthandifitonlyif itsis defendedisseries ofof(1978)deductiveA statementimmediately compelling,in a single step.

PAGETheissue oftreatedtherein theareSocratic hintscontext ofnorulesSocraticdistinguishinstruction on theto sriddles on the one hand, andhow Socrates might have taught aanswernot beenjudgment underthatintuitions, from contrivedhas18offromfor example,student to givethe properfollowing question:"Which hospitalmore often record--a large ordaysa small oneon which over 60%of--willthe babiesborn uere boys?".Thisisa difficult question for(Kahneman & Tversky,be elicited1972,in a series of"Wouldyou notp.441),Stanfordundergraduatesbut a correcteasy steps, perhaps asagreethat thebabiesparticular hospital on a particular day cananswer canfollows:borninabe viewed asa sample?""Quite right.confidence insmallnow,the results of awouldyou havelarge sample,the sameorofaone?""Indeed.confidence isbeAndin error?OAndwouldyounotgreater in a sample thatagreeis lessthatyourlikely to

PAGE"Ofcourse youtell me whatofishad always known that.the proportion ofbabies which you considertheboysWould19you nouin a collectionclosest to an idealoftruth?""We agree again.day onDoeswhich more thandeparture from that"Andso,60%ofhavesample to revealrespondent to a desiredofachieving the sameLichtenstein (1979)base ratesencounterandto thecritical variables.design,(1977)effect ofsuchandheavy-handed way ofresponse,goal.Fischhoff,Slovic andbecome sensitive toreliability of evidence,problemsAlthoughthateffects haveby Bar-Hillelvaryonlythese investigatorssample sizeeven inbeen obtained(1979)leadingbut there are subtlershowed that subjectssuccessiveobtain antruth ratherEtc.The Socratic procedure is awaysa gravegreat confidence in a sample,should you not expect thatthebabies born isthat aideal?"if youthan error?".that not mean, then,whenintheythesedid nota within-subjectby Evans and Dusoirwith amore transparentformulation and more extreme sample outcomes.The hint providedto assign weightby parallel problems may lead subjectsto a variable thatis actually irrelevant

PAGE 20to thecorrectresponse:demonstrated thatbase-rateresearchwasthatany featureis relevantdesignsareinterpretation(Poulton,are liable tothatproblems.systematically variedproblems ofifset ofto believeWithin-subjectandBar-Hillelrespondents were sensitiveinformation,distinguishing aproneFischhoffonlyIndeed,to thevariablesubjects arethedatathatiscorrect response.associatedwithin several areas1975).to irrelevanttheof(1980)In studies ofofsignificantpsychologicalintuitions,induce the effect which theytheyare intendedtotest.OH THE LIMITATIOHS OF THE QUETION-AHSWERIHGInthe preceding section wewithin-subjectintuitionsbroader.designsunderand Socraticstudy.Most researchinductive inferenceTh,&hasandhints couldproblembeen conducted into answer questionsor in writing.the possibility thatispromptactuallythemuchon judgment under uncertainty andparadigm in which the subjectis asked.isedPARADIGMis exposedona conversationalto information andor to estimate values,orallyIn this section we discuss some answeringparadigm.The useof shortmotivated subjectsisquestionnaires completedoften criticized on theby casuallygrounds that

PAGEsubjects wouldact differentlymore seriously.ofreasoningwithandchoice thatofSlovic,1973;aresubstantialGrether,& Kahneman,not1979;payoffs anddisappear inerrorsresponses.the presence ofpayoffs.nor s when the stakes are high.Mann,1977;Jervis,Perhaps a moreansweringandpoliticalviewserious concernthat weconversations"cannotFurthermore,from normalrationalityJanis Cinin theirwhichnormalAlthough some Judgmentstoexplicitconversationalthe question-safely assumesubjectsmakein responsemilitary1972;regardingthatmadenotthe dailyandthat(Janis,Rather,choice doNeitheranswer questions will simulateenvironment.areThat is not to saymessages andpeople1979;1975).paradigm is"experimental&they have noaffect judgment.reasoningthehow they wouldand uhenofnewspaperbyHypothetical questionsincentives do notmaintain thatestablished(Lichtensteinare able to predictincentive to lie about theirthat errorsGrether & Plott,respond in a more realistic setting,wethe situationeliminatedincentives1981).appropriate when peoplethattookwere originallyquestionsintroductionTverskytheyHowever, the evidence tions,receiveinferenceswiththeeveryday life aremanyexperiments differsocial interaction.thatarenot.in many ways

PAGE 22Ininterpretingtemptedthe subjects'to assume l o be correct?abouttheselection ofitobservesthisThecluster ofmessageis onlysubjectsanswersuse toin eby thegiven?thattheIs some ofor. isit allthe experimentersubject's answers toAnd thesourcesboth theprovide oncovertandthethatovert(Orne, 1973).Following Grice's William James1975),Second,questionsthe questionquestions.oftheIs an obvious answer atanswer?single overtonetheinto simulate.question?it?included justis determinedinariseasking,Doesexpectedandthethe question that thethisthe information thatirrelevantrelevant?"oftoexperimenter expect me to findlikelyincludednot spontaneouslynever thoughtcorrecthavethat allnormally concerned with manythe experimenterall(ii)isthe experiment is meantthe subject isaandthat wouldThe situation is quite different fromof view.therethoughtssubjectthe subject's pointsituation thatofspontaneously,givenexperimenters arethat the questions merely elicit fromexpressionto themanswers,a large body oflecturesinliterature in philosophy,1967 (Grice,linguisitcsand psycholinguistics has dealt with the contribution ofthe

PAGE 23cooperativeness principle to thereferences,see Clark C Clark,the listener1977).in a conversationthe speaker isand clear"meaning of utterancesisBy this principle,entitled toassume thattrying to be "informative, truthful,(Clark C Clark.p.560).(forrelevantGrice listed severalmaxims that a cooperative speaker will normally follow.example,the maximsaying thingsreadilythattheinfer frommessage.toof quantity prohibits thelistener alreadythe contextIt is by this maximcleantheunsuccessful:house"theorlistener canattempt would havespeaker fromknows,from theorcouldrest ofthat the statement "Johnconveysthatthethetriedattemptassume thatbeen described by theForuasa successfulsimpler sentence:"John cleaned the house".Subjects come to the experiment perienceThey willgenerallycooperative experimenter,althoughin conversation.encounter aexpectationlifelongisoftenhas manywrong.subtle effectsinterpretation of the informationIn particular,experimenterinformation.it makestoinformation violateslikely to seekexample,thetherulesrelevance inTaylor andassumptionon theto which they are nally difficultitstudyBecauseThefor y experimental message.Crocker (1979)commentedonareForthe fact

PAGE 24thatsubjects'statementswith hisimpressionsthat aretrue ofprofessors".is unusuallywouldin other studiesillustratedspeeda studyit smashedwhen askedthe car going whenofwQrdquestioner,whollySimilarC Tversky,investigated the"smash"thanwheninformation.(1974),estimate ofthecarcooperative,whothegoing whenis "howcar?".impliesThe usethatthebelievesthatthean inferencecanbefast.the car?the questionanswer to (i)question wasthe questionquestionofby the question inofaPalmerit hit the otherin thenormative analysisbe affectedinhighersuchdivided into two separate problems.bywhich"how fast wasif sincere andgoingthe speedthat aaKahnemanandgive athe other car?"fast wasThe1981)by Loftuseye-witnessesof a carcar wasincludepresuppositions embeddedinthatthethe beliefirrelevant or worthless information.The role ofshowedinference that Marknot(e.g.,by"Mark is shypersonality description.Nisbett, Zukier & Lemley,impact ofaffectede.g.,be justified byredundant statement in a1973;person areeverybody,experimenterissues ariseaBut the subjects'shy couldcooperativeofinmust(ii)Should the witness be affectedbe positivetothe witnessforming a private opinion offormulatingThe answer(i) should(ii)apublic estimate?Theifthe question conveys newisless clear.Onthe one

PAGE 25hand,itto echohand,appears inappropriate forinformationin the question.the cooperative w

adoa099 507 stanford univ ca dept of psychology f/s 5/10 on the study of statistical intui