On Optimal Decision-making In Brains And Social Insect .
Downloaded from rsif.royalsocietypublishing.org on 14 April 2009On optimal decision-making in brains and social insectcoloniesJames A.R Marshall, Rafal Bogacz, Anna Dornhaus, Robert Planqué, Tim Kovacs and Nigel RFranksJ. R. Soc. Interface published online 25 February 2009doi: 10.1098/rsif.2008.0511Supplementary data"Data ferencesThis article cites 61 articles, 7 of which can be accessed freeP PPublished online 25 February 2009 in advance of the print journal.Subject collectionsArticles on similar topics can be found in the following ref-list-1biocomplexity (6 articles)biomathematics (39 articles)computational biology (33 articles)Email alerting serviceReceive free email alerts when new articles cite this article - sign up in the box at the topright-hand corner of the article or click hereAdvance online articles have been peer reviewed and accepted for publication but have not yet appeared inthe paper journal (edited, typeset versions may be posted when available prior to final publication). Advanceonline articles are citable and establish publication priority; they are indexed by PubMed from initial publication.Citations to Advance online articles must include the digital object identifier (DOIs) and date of initialpublication.To subscribe to J. R. Soc. Interface go to: nsThis journal is 2009 The Royal Society
Downloaded from rsif.royalsocietypublishing.org on 14 April 2009J. R. Soc. Interfacedoi:10.1098/rsif.2008.0511Published onlineOn optimal decision-making in brains andsocial insect coloniesJames A. R. Marshall1,*, Rafal Bogacz1, Anna Dornhaus2, Robert Planqué3,Tim Kovacs1 and Nigel R. Franks41Department of Computer Science, University of Bristol, Woodland Road,Bristol BS8 1UB, UK2Department of Ecology and Evolutionary Biology, University of Arizona,PO Box 210088, Tucson, AZ 85721, USA3Department of Mathematics, VU University Amsterdam, De Boelelaan 1081a,1081 HV Amsterdam, The Netherlands4School of Biological Sciences, University of Bristol, Woodland Road, Bristol BS8 1UG, UKThe problem of how to compromise between speed and accuracy in decision-making facesorganisms at many levels of biological complexity. Striking parallels are evident betweendecision-making in primate brains and collective decision-making in social insect colonies: inboth systems, separate populations accumulate evidence for alternative choices; when onepopulation reaches a threshold, a decision is made for the corresponding alternative, and thisthreshold may be varied to compromise between the speed and the accuracy of decisionmaking. In primate decision-making, simple models of these processes have been shown,under certain parametrizations, to implement the statistically optimal procedure thatminimizes decision time for any given error rate. In this paper, we adapt these same analysistechniques and apply them to new models of collective decision-making in social insectcolonies. We show that social insect colonies may also be able to achieve statistically optimalcollective decision-making in a very similar way to primate brains, via direct competitionbetween evidence-accumulating populations. This optimality result makes testable predictions for how collective decision-making in social insects should be organized. Our approachalso represents the first attempt to identify a common theoretical framework for the study ofdecision-making in diverse biological systems.Keywords: decision-making; diffusion model; optimality; neurons; social insects;sequential probability ratio testbrain (Usher & McClelland 2001) and compare it withthree new models of collective decision-making duringhouse-hunting by social insect colonies. These modelsare based on a proposed model for emigration in therock ant Temnothorax albipennis (Pratt et al. 2002),and two models proposed for nest-site selection in thehoneybee Apis mellifera (Britton et al. 2002). Thesimilarities are striking: both systems are modelledwith mutually interacting populations; in both systems,a decision is made when one population exceeds somethreshold; and in both systems, this threshold can bevaried to mediate between the speed and the accuracyof the decision-making process. As well as examiningthe structural similarities and differences between theneuron model and social insect models, we examineoptimality criteria for decision-making in the socialinsect models. Bogacz et al. (2006) showed how themodel of decision-making in the brain proposed byUsher & McClelland (2001) can be parametrized toimplement the statistically optimal strategy for choosing between two alternatives. Here, we analyse to whatextent each of the social insect models can implement or1. INTRODUCTIONAnimals constantly make decisions. Habitat selection,mate selection and foraging require investigation of,and choice between, alternatives that may determinean animal’s reproductive success. For example, manyanimals invest considerable time and energy in finding asafe home (Hazlett 1981; Seeley 1982; Hansell 1984;Franks et al. 2002). Similarly, an animal may frequentlyhave to deal with ambiguous sensory information indeciding whether a predator is present or not (Trimmeret al. 2008).There has been ongoing speculation as to whetherdecision-making mechanisms in brains and in coloniesof social insects might be closely related to each other,beginning at least with Hofstadter (1979) and generating continued interest (Seeley & Buhrman 2001;Visscher 2007; Passino et al. 2008). In this paper, weexamine a model of decision-making in the primate*Author for correspondence (James.Marshall@bristol.ac.uk).Electronic supplementary material is available at http://dx.doi.org/10.1098/rsif.2008.0511 or via http://rsif.royalsocietypublishing.org.Received 3 December 2008Accepted 15 January 20091This journal is q 2009 The Royal Society
Downloaded from rsif.royalsocietypublishing.org on 14 April 20092 On optimal decision-makingJ. A. R. Marshall et al.approximate this statistically optimal strategy. Thisgives testable predictions for how social insects shouldbehave when house-hunting in order to optimize theirdecision-making. The analysis we present representsthe first step in establishing a common theoreticalframework for the study of decision-making in biological systems, i.e. based on the interactions betweentheir components rather than the details of thecomponents themselves. Hence, this framework shouldprove applicable to diverse biological systems at manylevels of biological complexity.2. OPTIMAL DECISION-MAKINGDecision-making is a process in which uncertaininformation must be processed in order to make achoice between two or more alternatives. We canillustrate decision-making with a simple perceptualchoice task, in which a primate subject is presentedwith a display filled with moving dots. The subject isrequired to decide whether the majority of dots move tothe left or the right and to make an eye movement in thesame direction. The proportion of the displayed dotsthat move in a coherent direction can be varied to makethe decision task easier or harder, and the rewards forcorrect choices can be modified to vary the optimalcompromise between the speed and the accuracy ofthe decision.The above description is just one example ofa decision-making problem, but diverse organismsface a wide variety of decision problems exhibiting thekey features of variable difficulty, and a dynamictension between the speed and the accuracy of thedecision-making process (Edwards 1965; Chittka et al.2003). Based on the analysis of human reaction-timedistributions in decision tasks, psychologists proposedthe ‘diffusion model’ of decision-making (Stone 1960;Ratcliff 1978), which represents the process abstractlyas Brownian motion on a line representing relativeevidence for the two alternatives, with constant drifttowards the correct hypothesis (figure 1).The diffusion model of decision-making is, in fact,a special case of the more general sequential probabilityratio test (SPRT).1 The SPRT provably achievesoptimal decision-making over two alternatives (Wald &Wolfowitz 1948), as it makes use of the Neyman–Pearson(1933) lemma familiar to statisticians and scientists.The SPRT works by continuing to gather evidence forthe two alternative hypotheses until the log of theirlikelihood ratio exceeds a positive or negative threshold;this is the test that, among all possible tests, minimizesdecision time for any desired decision error rate. Throughan adjustment of this threshold, the test can achievethe optimal trade-off between decision accuracy anddecision speed.The diffusion model of decision-making has recentlybeen shown to fit reaction-time data better than themodels that do not implement statistically optimaldecision-making (Ratcliff & Smith 2004). Moreover,1This special case is obtained when information gain over timebecomes continuous: the standard SPRT works with discreteevidence samples.J. R. Soc. Interface1.00.50–0.5–1.00501000Figure 1. The diffusion model of decision-making can bethought of as a random walk with normally distributed stepsize (Wiener process or Brownian motion) along a line wherethe positive direction corresponds to increasing evidence forone of the available alternatives, and the negative direction toincreasing evidence for the other alternative. The randomwalk is subject to a constant drift A, a tendency to move alongthe line towards the better alternative, whose strength is thedifference between the expectations of the incoming data onthe available alternatives. The variance in the random walk(proportional to s2) represents the uncertainty in theseincoming data. The diffusion model of decision-makingimplements the statistically optimal sequential probabilityratio test, and by varying the decision threshold z cancompromise between speed and accuracy of decision-making.Inset. A sample trace of the diffusion model: at time tZ0 whendecision-making starts, there is no evidence in favour of eitheralternative. As time passes evidence accumulated so far variesstochastically, but consistently tends to increase in favour ofone of the two alternatives. At approximately time tZ90, thepositive decision threshold is reached and the correspondingalternative is chosen.neural recordings from cortical regions in monkeysundertaking the moving-dots decision task are betterdescribed by the diffusion model than by other, nonoptimal models (Ratcliff et al. 2003). This suggests thatneural decision networks can be parametrized in a waythat allows optimal decision-making, as we shall discussin §§3 and 4.3. DECISION-MAKING IN THE CORTEXThe neural bases of decision-making are typicallystudied in the context of the moving-dots experimentdescribed in §2. Neuronal activity recordings fromsingle cells in the monkey cortex suggest that decisionmaking during this task involves two main brain areas.First, the neurons in the medial temporal (MT) areaprocess the motion present in the visual field. Each ofthe MT neurons responds proportionally to themagnitude of motion in a particular direction (Brittenet al. 1993). Hence, the neurons in the MT area that areselective for motion in different directions providesensory evidence supporting the corresponding alternatives. However, this sensory evidence is uncertainowing to the noise present in the stimulus and theneural representation itself.Second, the neurons in the lateral intraparietal (LIP)area and the frontal eye field are concerned withcontrolling eye movement. These neurons are selectivefor the direction of eye movement. During the motiondiscrimination task, it has been observed that theneurons corresponding to the correct alternative
Downloaded from rsif.royalsocietypublishing.org on 14 April 2009On optimal decision-makinggradually increase their firing rate (Schall 2001;Shadlen & Newsome 2001; Roitman & Shadlen 2002).Detailed studies of their activity provide strongevidence that the LIP neurons integrate input from thecorresponding MT neurons over time (Huk & Shadlen2005; Hanks et al. 2006). Hence, as time progresses in thetask, the sensory evidence accumulated in the LIPneurons becomes more and more accurate.It has been observed that when the activity of theLIP neurons exceeds a certain threshold, the decision ismade and an eye movement in the correspondingdirection is initiated (Schall 2001; Shadlen & Newsome2001; Roitman & Shadlen 2002). This arrangementof neural populations with decision thresholds lendsitself to representation through a simple model, asdescribed in §4.J. A. R. Marshall et al. 3excitationinhibitionlossFigure 2. In the Usher–McClelland model of decision-makingin the primate visual cortex, neural populations representaccumulated evidence for each of the alternatives. Thesepopulations y1 and y2 integrate noisy inputs I1 and I2, but leakaccumulated evidence at rate k. Each population also inhibitsthe other in proportion to its own activation level, at rate w.When wZk and both are large, the Usher–McClelland modelreduces to the diffusion model of decision-making (figure 3).16144. THE USHER–MCCLELLAND MODEL12The Usher–McClelland model represents decisionmaking using neural populations that act as mutuallyinhibitory, leaky integrators of incoming evidence(figure 2). In the moving-dots decision task describedabove, these integrator populations would represent theLIP neural populations corresponding to the differentpossible eye movement decisions. Each population ofintegrator neurons receives a noisy input signal that itintegrates, subject to some constant loss. Each population also inhibits the activation of the other to adegree proportional to its own activation. So, as onepopulation becomes highly activated, the suppression itexerts on the activation of the other grows stronger. Fora binary choice, the linearized version of the system isformally described as a pair of coupled stochasticordinary differential equations(y 1 Z I1 C ch1 Ky1 k Ky2 w;ð4:1Þy 2 Z I2 C ch2 Ky2 k Ky1 w;10where yi is the activity of population i; y 1 is the changein that activity over time; Ii is the strength of the inputsignal for alternative i; chi is the noise in that inputsignal described as a Wiener process with mean zeroand standard deviation proportional to c; w is the rateat which one population inhibits the activation of theother; and k is the rate at which a population’sactivation level decays. The decision is made by themodel if the activity of either of the populations reachesa threshold value.pffiffiffiChoosing differentpffiffiffi coordinates, x1 Z ðy1 Ky2 Þ 2and x 2 Z ðy1 C y2 Þ 2, the model can be decoupledinto two independent random processes8I KI x 1 Z ðwKkÞx1 C 1pffiffiffi 2 C ch1 0 ; 2ð4:2ÞI1 C I2 x 2 Z ðKk KwÞx 2 C pffiffiffi C ch2 0 ; :2pffiffiffiwhere h10 Z ðch1 Kch2 Þ 2, and similarly for c h2 0 . IfwZk, then x1 simply undergoes a biased random walk.Moreover, taking both these parameters to be large,x 2 quickly converges to a limiting value (figure 3).J. R. Soc. Interface86420246810121416Figure 3. The expected dynamics of the Usher–McClellandmodel, plotted as the activation of population y1 against theactivation of population y2. When decay equals inhibition(wZk), the system converges to a line (bold arrow) anddiffuses along it, until a movable decision threshold is reached(dashed lines). Along the attracting line, the Usher–McClellandmodel is equivalent to the optimal diffusion model of decisionmaking (figure 1).Since x 2 quickly converges, the decision is madewhen the difference between integrated evidence, x 1,exceeds a positive or negative threshold (Bogacz et al.2006). This corresponds to the statistically optimaldiffusion model of decision-making (figure 1). Thus,when wZk and both of these parameters are relativelyhigh, the Usher–McClelland model approximatesoptimal decision-making.5. DECISION-MAKING IN SOCIAL INSECTCOLONIESThe rock ant T. albipennis lives in colonies of up to afew hundred individuals, inhabiting small enclosedcavities such as rock crevices, which break down overtime. Colonies of the honeybee A. mellifera aresubstantially larger, often comprising more than50 000 workers nesting in large cavities in hollowtrees. For both species, the need to hunt for a newnest site arises when the nest degrades or is destroyed,or when the colony propagates. To minimize exposure,colonies must minimize the duration of the emigration,while still gathering substantial information aboutpotential nest sites. ‘Scout’ individuals leave the old
Downloaded from rsif.royalsocietypublishing.org on 14 April 20094 On optimal decision-makingJ. A. R. Marshall et al.nest (or the bee swarm) to search for new suitable sites.When an individual has located a new site, this isthoroughly inspected according to multiple criteria(Mallon & Franks 2000; Franks et al. 2003b, 2005,2006b, 2007a).A unanimous decision among the discovered nestsites is required, and the highest quality site should beidentified to maximize future success. To achieve all ofthis, both rock ants and honeybees use a process ofquality-dependent recruitment to nest sites, causingpositive feedback, coupled with quorum sensing,ultimately leading to a collective decision.In T. albipennis, scouts recruit nest-mates to a sitethey have discovered by tandem running, teachingothers the route (Möglich & Hölldobler 1974; Möglich1978; Franks & Richardson 2006; Richardson et al.2007). The recruiters pause for longer before recruitingto poor nests than they do for good nests (Mallon et al.2001). Recruits inspect the new nest and also startrecruiting, leading to a positive feedback, with moreants arriving faster at higher quality nest sites. Whena certain number of ants, a quorum, have accumulated in the new site, all ants that are committed to itswitch from tandem running to carrying nest-mates.This represents a switch to decision implementation, asbrood items and passive ants are carried to the chosennest site. The colony’s collective decision for a new nestis thus usually for the site that first attracts the quorumnumber of ants. By adjusting this number, the ants optfor a quick but error-prone, or a slower but moreaccurate decision, depending on their group size andexternal conditions (Franks et al. 2003a; Dornhauset al. 2004; Dornhaus & Franks 2006).In A. mellifera, scouts similarly start recruitingto discovered nest sites, here using the honeybeewaggle dance (Lindauer 1955; von Frisch 1967; Seeley1982). The probability of performing waggle dances, aswell as their duration, depends on the quality of thediscovered site. Positive feedback thus arises only ifrecruits become recruiters, which happens only forsuperior sites (Seeley & Buhrman 1999, 2001). Eventually all recruitment is for a single site, which isthen chosen by the honeybee swarm (Seeley 2003).Honeybees may also use a quorum at the new siteto determine whether this decision has been made(Seeley & Visscher 2004b).Both of these decision-making mechanisms operatewithout central control, and individuals use only localinformation (Mallon et al. 2001). Both species can selectthe best nest site even if news of it arrives late in thedecision-making process (Britton et al. 2002; Frankset al. 2007b). However, there are some differencesbetween the two collective decision-making systems. Inthe bees, the decision-making process is separatedfrom the execution of the decision: the flight of theswarm to the new site occurs after the decision has beenmade, and is guided by a small number of informedindividuals (Beekman et al. 2006; see also Couzin et al.2005). In the ants, these processes are integrated,introducing a logistics problem into the decisionproblem (Planqué et al. 2007). Note that the decisionproblem solved by colonies during emigration is oneof optimal consensus decision-making, and thus differsJ
2. OPTIMAL DECISION-MAKING Decision-making is a process in which uncertain information must be processed in order to make a choice between two or more alternatives. We can illustrate decision-making with a simple perceptual choice task, in which a primate subject is presented with a display filled with moving dots. The subject is
process, and collective decision-making process in social insect colonies. I The direct switching bee model (A. mellifera) is the only model that plausibly approximates statistically optimal decision making. I Hypothesis: Social insect colonies need to apply direct switching with recruitment to have an optimal decision making strategy.
Decision making is an intellectual process of selecting optimal and best option among many alternative choices. It results in an outcome which can be in form of action or an opinion [1]. Decision making process can be categorized into four processes:- Group decision making process Individual decision making process
Testing Optimal Timing in Value-Linked Decision Making Rahul Bhui1 # Society for Mathematical Psychology 2019 Abstract Influential theories of the decision making process hold that a choice is made once the cumulative weight of noisily sampled information reaches a desired level. While these theories were originally motivated as optimal .
required for optimal decision making when biologically realistic limits on synaptic weights are introduced. We also describe the model's predic-tions concerning reaction times and neural responses during learning, and we discuss directions required for further integration of reinforce-ment learning and optimal decision-making theories. 1 .
The farmer and decision-making Decision-making is central to farm management. Each decision has an impact on the farm and on the farm household. Even deciding to do nothing is a decision and has an impact. The more a farmer is aware of the decision-making processes that
Decision-making is a problem-solving process which ends when a satisfying solution is reached. Therefore, decision- . decision-making and reduce the costs of decision-making mistakes. Keywords: Decision, . stimulus may occur after reward presentation and might in-volve some varied regions of the brain in deciding about
4 Supporting decision making — A quick reference guide for disability support workers Where it fits in the decision making spectrum Best interests decision made on behalf of person Person makes the decision Autonomous decision making talking to family, friends, other participants, experts etc researching via brochure, media, internet .
Easily build API composition across connectors SAP Cloud Platform Integration SAP Cloud Platform API Management SAP Workflow Services SAP Data Hub SAP Cloud Platform Open Connectors Simplifying integration and innovation with API-first approach in partnership with Cloud Elements
