Why You Can't Make a Computer That Feels PainAuthor(s): Daniel C. DennettSource: Synthese, Vol. 38, No. 3, Automaton-Theoretical Foundations of Psychology andBiology, Part I (Jul., 1978), pp. 415-456Published by: SpringerStable URL: http://www.jstor.org/stable/20115302Accessed: 12/01/2010 12:08Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available rms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained herCode springer.Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact support@jstor.org.Springer is collaborating with JSTOR to digitize, preserve and extend access to Synthese.http://www.jstor.org
DANIEL C. DENNETTWHY YOU CAN'T MAKE A COMPUTER THAT FEELS PAINimportant to many people to claim that computers cannot inSuch apriprinciple duplicate various human feats, activities, happenings.oristic claims, we have learned, have an embarrassing history of subsequentfalsification. Contrary to recently held opinion, for instance, computers canIt has seemedsuperb checkers and good chess, can produce novel and unexpectedinproofs of nontrivial theorems, can conduct sophisticated rycomputerphiletightlyEnglish.who grounds an uncomplicatedoptimisim in this ungraceful retreat of theplayskeptics, however, is in danger of installing conceptual confusion in the worstplace, in the foundations of his own ascendant view of the mind. The triumphsof Artificial Intelligence have been balanced by failures and false starts. Somehave asked if there is a pattern to be discerned here. Keith Gunderson haspointed out that the successes have been with task-oriented, sapient featuresof mentality,the failures and false starts with sentient features of mentality,a distinctionand has developedresistantfeaturesof mentality.1betweenGunderson'sand programprogram-receptivepointis notwhatsomehaveSome have hoped he had found a fall-back position for them: viz.,can think but they can't feel. His point is rather that themachinesmaybeto feel is a very different task from getting it totask of getting a machineit is not a task that invites solutionthink; in particularsimply nsThishardware.in programming,goessomewaybut rather, if at all, by devisingto explainingtherecalcitranceoffeaturesofcomputersto conductlike pain to computer simulation, but not far enough. Sincethe discreditedofaprioristicthinking about the limitationscan be seen in retrospect to have stumbled over details, I proposea morethan usual philosophicthought experiment. Lettotheproveimagineskeptic wrong about pain by actuallya pain program, or designing a pain-feelingrobot. I think tionsusdetailedsetting outresearch strategy of computer simulation has often been misconstruedby philosophers. Contrary to the misapprehensionsinnocently engendered byTheSyntheseCopyrightAll Rights38(1978)415-456.? 1978 by D. Reidel PublishingReserved.Company,Dordrecht,Holland.
416DANIEL C. it is neverIntelligence,'2tothe point in computer simulation that one's model be indistinguishable fromfor instance, a good computerthe modelled.simulation of aConsider,asbedevisedOnewould not expect tohurricane,by meteorologists.mightor wind-blownget wetakintouse-mentionin its presence.likeerror,simulationThatcoweringof a hurricaneludicrous expectationthebeforeis a program which,bewould'lion.'wordAgoodyou feed innewofconditions, gives you back descriptions ofdescriptionsmeteorologicalsubsequent hurricane behavior. The descriptions might be in roughly ordinaryEnglish, dealing with clouds, waves and tides, or in some arbitrary notation,computerwhendealing with barometric pressure, wind velocities, and yet more esoteric (butfeatures of hurricanes. The goal is to devise a program that willmeasurable)give you good 'predictions' of what a hurricane will do under a great varietyof highly complex conditions. Such a program is tantamount to an immense'if conditions A, B, C,. . . obtain,conditionals:conjunction of complicatedthen R will result; and if conditions D, E, F, . . obtain, S will result;'the only way to populate that conjunction reliably is byand . . . Obviouslythefrom general covering laws, all properlyparticular conditionalsderivingmeshedandcoordinated.Soin orderto writeasimulationgoodprogramonehave a theory of hurricane behavior, and itmust be a good theory. Butif one must have a theory in the first place, why bother incorporating it into ndsor unansweredprogramquestionsit keeps the theoretician honest. Second, once a theory is thusinto a working,incorporated'debugged' program, its implications can bedeterminedandA simulation can be an 'experienceassessed.quicklyin a a simulationprogramcouldgenerate thousands of different storm histories to scrutinize for implausibilityor worse. Also, of course, such a program could be used in high-speed realtime prediction of current weather. The fact that such a simulation programthat someonly a high speed generator of the itionsby theassignsis ultimatelytheorymode of presentation of the input and output. It is often useful, convenient,or just plain exciting to use the output to drive a visual display, a raster or TVscreen on whichCoast,butappears, say, a swirling vortex moving up amap of the Eastthe tail thatthat swirling vortex is a sort of epiphenomenon,
WHY YOU CAN'T MAKE A COMPUTER THAT FEELS PAINthe dog. The theory incorporated into the program directs thethe presentation,and does not read off the behavior of thewhich itself plays no role in the simulation beyond its role as adoesn't wagofbehaviorpresentation,convenientNowdisplay.a similarly inspired computer simulation of humana program, based on our theory of pain, such that when welet us considerpain. We writetype417in descriptionsofconditions,e.g.,'An anvil drops from a height of two feet onto S's unanesthetizedleft foot,'thecomputertypesback'S jumpsWeinputofdescriptionsonaboutresults,a tearfoot,righte.g.,in hiseye,test the program by varying how we fill in the blanksformulae"A is(e.g.,fromdroppedaheightscreaming.'in our permissibleof onS's ")and checking the resulting outputs for plausible variety and dependence onthe input. What is unsatisfying about this computer simulation of pain? Theskeptic might reply that it is a simulationconsider our hurricane simulation: whatat best only of pain behavior, butelse is there to simulate but thebehavior? A better reply is that we have so far only attempted tosimulate external pain behavior. This defect is easily remedied. Revised, ourprogram will yield such outputs ashurricane'sare'S's C-fibresattentiononrightstimulated,is distracted;foot,a tear.a pain-memoryST s heart-ratein theeye,increasesis laid down,. . .SjumpsS'saboutscreaming.'about the internal 'behavior' or effects(We can be sketchy, for the moment,alluded to in the program.) Suppose,then, that we pack our outputwithdescriptionsdescription or even mentalisticpsychoneurophysiologicalabout effects on memory,belief, desire, etc. Still, thelogical description? indeed? ofskeptic may insist we have left somethingimporteverythingance out. We have simulated, perhaps, the internal and external causes andeffects of pain, but not the pain itself. Some identity theorists may wish toretort to this that C-fibre stimulation just is the pain,4 but we needn't take astand on that point,since there are further waysof obligingthe skeptic. We
418canDANIEL C.DENNETTrewriteourso it yieldsprogramsuchasoutputs'There is a pain, P, of the in-the-left-footarestimulatedvariety,'. .in S; S's C-fibresNow we have explicitly included the pain. But, says the skeptic, the programstill leaves out the quality of the pain. Very well. We expand our theory, andour program,;'to yield detailed descriptionsabout even this.concomitantlyAgain we feed in'An anvil is dropped from a height of two feet on *S"sleft foot,'and this time we get back:variety in S; P begins as aThere is a pain, P, of the sure,commencestothrob;P increases in intensity until it explodes into shimmering hotflashes of stabbing stilettoes of excruciating anguish [or wordsthat. .effect]. ; iS"s C-fibresarestimulatedto.I see no reason why our program could not be enlarged to incorporate all this;the biggest problem would seem to be discovering sufficient uniformity andeffects as reported by sufferers toin such 'phenomenological'courseifOfdata we collect suggest a randommuchtheprediction.permitdistribution of these effects within certain boundaries that is easy enough tolawfulnessincorporateintoourI do not expectas well.5programsatisfy the skeptic. He might try to express histhere is nothing pain-like going on in theby pointingitoutwhenthesechurnscomputerreports. But of course not. Nor does thethis wouldoutdoubtsthatcomputer hurricane generate an internal low barometric pressure behind itssteely facade. At this point is should dawn on the skeptic that he has beenbarking up the wrong tree. He has no pressing quarrel with this researchsince its guidingstrategy when it is directed to psychologicalphenomena,is not that men are computers (any more than hurricanes are)presuppositionbutsimplymaterialist,that onedualist,can haveepiphenomenalist,a rigorousortheorywhatever.of humanIsn'tpsychology,there,however,research strategy that differs significantly from the one we've beenis to do, not describe! Forconsidering, where the aim of the computeranotherinstance,'recognize''Shakey'at StanfordsimplewithobjectsResearchInstituteits televisioneyes;is arobotit pushesthat cancubesand
419WHY YOU CAN'T MAKE A COMPUTER THAT FEELS erformancemodels,' one might say, really do things; they do not so much incorporatetheories (as do simulations) as instantiate theories.6 The skeptic's challenge isnowforusto makesucha feelera robot,ofpain,nota mechanizedtheoryfeelers of pain. So let us try to design such a robot. Of course ourefforts in this task will be as much guided by our theory of pain as were ourearlier simulation efforts, and we might ask the skeptic if he had any quarrelsaboutour earlier, programmed theory as a theory of pain. If the skeptic makesno objectionsto it, or if we are able to revise our theory to satisfy hisweare home free, for it is a relatively straightforwardtask toobjections,the help of our earlier 'describing' program. Thebuild the robot withwiththe control system for our new robot.our original program yielded outputs likedescribing program simply becomesHere is how it is done. Supposea tear'S trembles,in hisandeye,says"Ouch!My'hurts."thumbFirst,werewrite all outputs in the first person: T tremble, a tear inmy eye, and I say'"Ouch! My thumb hurts."Then we drop the redundant 'I say' wherever inal,'protocol'which willthen print only 'Ouch! My thumb hurts.' The rest of the output isreprogrammed to drive a display of flashing lights with labels. The 'tremble'light goes on, the 'tear in the eye' light, and so forth. Then we replace thesentencesinputrepresentingheights,anda akeup magnetizedplastictokens?anvils, knives, olivesfalling from e,' etc., so that dropping the anvil token into the thumb slot simulatesdropping the anvil on the thumb. Of course that's not very realistic, but wecanimproveeccentricit easily.Forinstance,that makeswethe wholecanreplacethe'tremble'light withanit is turning;for us by the toyvibrate whenflywheelcomputertear in the eye problem has already been solvedand the other details of verisimilitude are either obviouslymanufacturers,irrelevant or can be solved by the Disney studios given six months and enoughthegrant money. The result will be a robot that really does things; ittrembles and reels and whimpers; it says just where the pain is; it attempts toduck falling objects ? perhaps it even kicks back ifwe kick it.7But what about the rest of our earlier simulation? What happens to the hotFederalin our descriptive progam? These parts ofthe output we transform into labeled flashing lights and leave them that way:sometimesthe 'dull throb' light is on (blinking slowly if you like) andflashes and dull throbs mentioned
420DANIEL C.DENNETTsometimeslight is on. If the skeptic insists on morecanhewhatbe asking for? Remember that these lights arehere,randomly. The 'dull throb' light goes on only at appropriatetheverisimilitude'hot flash'not blinkingtimes, the robot can then say 'there is a dull, throbbing pain' and the otherapposite side effects of dull, throbbing pains are presumed to be arranged tocoincide as well. But, the skeptic persists, no amount of side effect can turnwhat is not a dull, throbbing pain into a dull, throbbing pain, and, obviously,this event a dull, throbbing pain does not make it one either. Thisis unfair as it stands. The skeptic, we mustfor all its plausibility,objection,callingto settling for an IBM typewriter as the 'speech'assume, had no objectionelement in this robot, and surely typing is not talking ? and calling typingtalking would not make it talking. Since he has not challenged us to make aof the species homo sapiens out of whatever bits andbona fide memberpieces are on the shelves at IBM, he must be permitting us to use somesubstitutes ? the legs can be titianium, not flesh and bones ? and since ourfeatures he hasflashing light (or whatever turns it on) has all the functionaldemanded of pain, why is he now changing the game? Calling the robot athe robot a human being either, but that wasto appear that what the skeptic was after allan analogue of pain, but the synthesis of realby Wohler in 1828 that marked the unificationof organic and inorganic chemistry. The synthesis of real pain in a machinewould tend to confirm that we human beings are just fancy soft machines, ashuman being would not makenever set as a goal. It beginsalong was not a simulation orpain, like the synthesis of ureathe materialistcontends.That we might reconstrue our task in this way highlights a peculiarity inour ordinary concept of pain. The word 'pain' has both a sortal grammar ('Ihave a pain,' 'pains shooting up my arm') and amass noun grammar ('there ismorepermitsgicalpainuscausenounsomeThe mass'it willyounow,'pain').?evenusas a sort of biologicalto view paininvitessubstance,ratherthana sycholoinstance,that can be safely administered depends on the amountof pain it has to kill. For excruciating pain (e.g., that of coronary thrombosis)two to four times the usual therapeutic dose may be given without danger.the amount of morphinein cases of severe pains that can quickly and oncoronarybiliary colic)dangerous,(e.g.,since if the pain disappears suddenly the patient may show signs of morphineBut
WHY YOU CAN'T MAKE A COMPUTER THAT FEELS PAIN421to happen, one would do well to punch or slap thepatient, since, as Steams observed in 1883, "pain is the antidote for morphineto neutralize, and thuspoisoning." One creates more pain for the morphineIf such werepoisoning.excessthepreventsspecificityto morphinerobotto pass.painof morphinefrom poisoning.8This suggests thatas an antagonist would be a legitimate test for anyseem, however, to harbor a conceptualoneself in speaking of the synthesis ofas any to be found,urea is urea, as genuinereal pain? Syntheticnotbut syntheticrubber isrubber.9 Is artificialgenuineintelligenceThis reconstrual of the task mightDoesconfusion.one not contradictcoloring is perfectly genuine coloring, but artificialintelligence? Artificialflowers are not flowers. The field of artificial intelligence trades on thisambiguity. Successes are often heralded as examples of genuine intelligencecreated by artifice, but in the face of objections this claim can be adjusted;artificial intelligence works just as well as, or is a useful and even theoreticallyI do not believe the terminteresting substitute for, genuine intelligence.'artificial intelligence' is objectionable on these grounds, for I do not believein the distinction we are invited to makein this instance. Suppose theintelligence of some artifacts does function just as well as human intelligence(an immense supposition, of course); then, since intelligence, like respiration,is a purely functional notion, artificial intelligence, like artificial respiration,is no less genuine for being obtained by artifice. It may not be just likenatural,humanintelligence,butis genuineasas genuineintelligence,(wecanimagine)intelligence of extra-galactic creatures might be.10 Butorwhat of artificialsynthetic pain? Is pain like rubber and flowers, or likecoloring, respiration and intelligence? Whatever answer we might agree onone lesson is(and agreement is both unlikely and ultimately unimportant),the alienclear: //pain is deemed to be essentially a biological phenomenon, essentiallybound up with birth, death, the reproduction of species, and even (in the casethen the computerof human pain) social interactions and interrelations,scientist attemptingcanno morematerials,canto synthesizea mastersucceedthansucceedin making,real pain in a robot is on a fool's errand. Hecabinetmaker,today,a etic pain would be real pain may seemisitbutoverly precious,important to bring them into the open, for severalareasons. First,of the notion of robotgreat deal of the counterintuitivenessReservationsabout whether
422DANIEL C. DENNETTpain no doubt derives from a dim appreciation of this side of our notion ofpain. Real pain is bound up with the struggle to survive, with the realprospect of death, with the afflictions of our soft and fragile and warmflesh.11 With our concept of pain, as with many others,real Chateau Latour has to have been madeparochiality:there is a tug towardin a particular place,an artificially concoctedfluidin a particularway, by particular people:to both chemists and connoisseursfrom Chateau Latourindistinguishablewould still not be real Chateau Latour. (Real vodka, on the other hand, canbe made from just about anything, anywhere, by anybody.) The parochialityoftheofconceptpain,is, sly so, for it has a role to play in defining our moral community. Therecan be no denying (though many have ignored it) that our concept of pain isinextricably bound up with (which may mean something less strong thanwith) our ethical intuitions, our senses of suffering,It will not do to suppose that an assessment of anyofindependentlysynthesis of pain can be conductedessentiallyconnectedobligation,and evil.12atattemptrobotto this robot might be. Onequestions about what our moral obligationsreason, then, why you can't make a co
WHY YOU CAN'T MAKE A COMPUTER THAT FEELS PAIN 417 doesn't wag the dog. The theory incorporated into the program directs the behavior of the presentation, and does not read off the behavior of the presentation, which itself plays no role in the simulation beyond its role as a convenient display.
6. Why Table 1 A 5-Why Analysis Question Table Build the Why Tree One Cause Level at a Time Many people start into a 5-Why analysis by using the 5-Why Table. With each Why question they put in an answer and then ask the next Why question. This question-and-answer tic-tac-toe
13. Why I Believe Jesus Is the Son of God Peter Kreeft PART 6 WHY I HAVE CHOSEN TO FOLLOW CHRIST 14. Why I Still Believe in Christ, in Spite of Evil and Suffering John S. Feinberg 15. Why I Have Made Jesus Christ Lord of My Life J. P. Moreland 16. Why I Believe Jesus Christ Is the Ultimate Source for Meaning Ravi Zacharias Afterword Josh McDowell
Why Men Want Women to Dress Like Tarts (But Never in Public) Why Men Are Three-Minute Wonders The Ball Game Balls Have Brains Too Men and Ogling What Men Need to Do What We Really Want, Long-Term Why Men Want 'Just One Thing' Why Sex Suddenly Stops What Men Want From Sex What Women Want From Sex Why Men Don't Talk During Sex
Have you ever wondered why social media is addictive, why you can’t seem to drag yourself away from scrolling through Facebook? Or maybe why you get that feeling when you get a notification? Do you ever check your news feed for “five minutes”, . Here are some questions
you think like a Freak? Why or why not? Would you encourage others to think like a Freak? Why or why not? 2. If ÒThinking Like a FreakÓ is so easy and simple, why do so few people think like a Freak? 3. Using the soccer example from Chapter 1, if you were taking a
Start reading the book Superfudge You have 12 days to read this book. Started reading at Finished reading at I read up to page _ today. 3 new words I learned today: Here is what happened today: Describe a connection you can make to this chapter. Did the first chapter of the book draw your attention? Why or why not?
Illuminati on my heels so I got to run And I’ll never stop until that hill is what my mom is on Why did Aliyah take that plane? Why did Left Eye get in that lane? Why did Tupac go to the Veg[as]? Why did Big go back to L.A.? Damn, damn/Why did Pimp C pick up that drink? When Mike Jack died I
Why don’t you and your friends want to come to the game? They aren’t very interested in football,. 11. Why were you and your wife so nervous during the flight? I had never flown before today, . 12. Why have you and your friends stopped shopping at my store? I can’t afford your prices, . 13. Why don’t you and your sister want me