Similarities And Differences In Spatial And Non-spatial .
PLOS COMPUTATIONAL BIOLOGYRESEARCH ARTICLESimilarities and differences in spatial and nonspatial cognitive mapsCharley M. Wu ID1,2*, Eric Schulz3, Mona M. Garvert4,5,6, Björn Meder ID2,7,8, NicolasW. Schuck 11111111111 Department of Psychology, Harvard University, Cambridge, Massachusetts, United States of America,2 Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany, 3 MaxPlanck Research Group Computational Principles of Intelligence, Max Planck Institute for BiologicalCybernetics, Tübingen, Germany, 4 Department of Psychology, Max Planck Institute for Human Cognitive andBrain Sciences, Leipzig, Germany, 5 Max Planck Research Group NeuroCode, Max Planck Institute forHuman Development, Berlin, Germany, 6 Wellcome Centre for Integrative Neuroimaging, University of Oxford,John Radcliffe Hospital, Oxford, United Kingdom, 7 Max Planck Research Group iSearch, Max Planck Institutefor Human Development, Berlin, Germany, 8 Department of Psychology, University of Erfurt, Erfurt, Germany,9 Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, Germany* charleywu@fas.harvard.eduOPEN ACCESSCitation: Wu CM, Schulz E, Garvert MM, Meder B,Schuck NW (2020) Similarities and differences inspatial and non-spatial cognitive maps. PLoSComput Biol 16(9): e1008149. : Daniele Marinazzo, Ghent University,BELGIUMReceived: February 5, 2020Accepted: July 13, 2020Published: September 9, 2020Peer Review History: PLOS recognizes thebenefits of transparency in the peer reviewprocess; therefore, we enable the publication ofall of the content of peer review and authorresponses alongside final, published articles. Theeditorial history of this article is available opyright: 2020 Wu et al. This is an open accessarticle distributed under the terms of the CreativeCommons Attribution License, which permitsunrestricted use, distribution, and reproduction inany medium, provided the original author andsource are credited.Data Availability Statement: All data and analysiscode is available from tLearning and generalization in spatial domains is often thought to rely on a “cognitive map”,representing relationships between spatial locations. Recent research suggests that thissame neural machinery is also recruited for reasoning about more abstract, conceptualforms of knowledge. Yet, to what extent do spatial and conceptual reasoning share commoncomputational principles, and what are the implications for behavior? Using a within-subjectdesign we studied how participants used spatial or conceptual distances to generalize andsearch for correlated rewards in successive multi-armed bandit tasks. Participant behaviorindicated sensitivity to both spatial and conceptual distance, and was best captured using aBayesian model of generalization that formalized distance-dependent generalization anduncertainty-guided exploration as a Gaussian Process regression with a radial basis functionkernel. The same Gaussian Process model best captured human search decisions and judgments in both domains, and could simulate realistic learning curves, where we found equivalent levels of generalization in spatial and conceptual tasks. At the same time, we also findcharacteristic differences between domains. Relative to the spatial domain, participantsshowed reduced levels of uncertainty-directed exploration and increased levels of randomexploration in the conceptual domain. Participants also displayed a one-directional transfereffect, where experience in the spatial task boosted performance in the conceptual task, butnot vice versa. While confidence judgments indicated that participants were sensitive to theuncertainty of their knowledge in both tasks, they did not or could not leverage their estimatesof uncertainty to guide exploration in the conceptual task. These results support the notionthat value-guided learning and generalization recruit cognitive-map dependent computationalmechanisms in spatial and conceptual domains. Yet both behavioral and model-based analyses suggest domain specific differences in how these representations map onto actions.Funding: ES is supported by the Max PlanckSociety and the Jacobs Foundation. The fundersPLOS Computational Biology https://doi.org/10.1371/journal.pcbi.1008149 September 9, 20201 / 28
PLOS COMPUTATIONAL BIOLOGYhad no role in study design, data collection andanalysis, decision to publish, or preparation of themanuscript.Competing interests: The authors have declaredthat no competing interests exist.Similarities and differences in spatial and non-spatial cognitive mapsAuthor summaryThere is a resurgence of interest in “cognitive maps’’ based on recent evidence that thehippocampal-entorhinal system encodes both spatial and non-spatial relational information, with far-reaching implications for human behavior. Yet little is known about thecommonalities and differences in the computational principles underlying human learning and decision making in spatial and non-spatial domains. We use a within-subjectdesign to examine how humans search for either spatially or conceptually correlatedrewards. Using a Bayesian learning model, we find evidence for the same computationalmechanisms of generalization across domains. While participants were sensitive toexpected rewards and uncertainty in both tasks, how they leveraged this knowledge toguide exploration was different: participants displayed less uncertainty-directed and morerandom exploration in the conceptual domain. Moreover, experience with the spatial taskimproved conceptual performance, but not vice versa. These results provide importantinsights about the degree of overlap between spatial and conceptual cognition.IntroductionThinking spatially is intuitive. We remember things in terms of places [1–3], describe theworld using spatial metaphors [4, 5], and commonly use concepts like “space” or “distance” inmathematical descriptions of abstract phenomena. In line with these observations, previoustheories have argued that reasoning about abstract conceptual information follows the samecomputational principles as spatial reasoning [6–8]. This has recently gained new supportfrom neuroscientific evidence suggesting that common neural substrates are the basis forknowledge representation across domains [9–13].One important implication of these accounts is that reinforcement learning [14] in nonspatial domains may rely on a map-like organization of information, supported by the computation of distances or similarities between experiences. These representations of distance facilitate generalization, allowing for predictions about novel stimuli based on their similarity toprevious experiences. Here, we ask to what extent does the search for rewards depend on thesame distance-dependent generalization across two different domains—one defined by spatiallocation and another by abstract features of a Gabor patch—despite potential differences inhow the stimuli and their similarities may be processed?We formalize a computational model that incorporates distance-dependent generalizationand test it in a within-subject experiment, where either spatial features or abstract conceptualfeatures are predictive of rewards. This allows us to study the extent to which the same organizational structure of cognitive representations is used in both domains, based on examiningthe downstream behavioral implications for learning, decision making, and exploration.Whereas early psychological theories described reinforcement learning as merely developing an association between stimuli, responses, and rewards [15–17], more recent studies haverecognized that the structure of representations plays an important role in making value-baseddecisions [11, 18] and is particularly important for knowing how to generalize from limiteddata to novel situations [19, 20]. This idea dates back to Tolman, who famously argued thatboth rats and humans extract a “cognitive map” of the environment [21]. This cognitive mapencodes relationships between experiences or options, such as the distances between locationsin space [22], and—crucially—facilitates flexible planning and generalization. While cognitivemaps were first identified as representations of physical spaces, Tolman hypothesized thatPLOS Computational Biology https://doi.org/10.1371/journal.pcbi.1008149 September 9, 20202 / 28
PLOS COMPUTATIONAL BIOLOGYSimilarities and differences in spatial and non-spatial cognitive mapssimilar principles may underlie the organization of knowledge in broader and more complexcognitive domains [21].As was the case with Tolman, neuroscientific evidence for a cognitive map was initiallyfound in the spatial domain, in particular, with the discovery of spatially selective place cells inthe hippocampus [23, 24] and entorhinal grid cells that fire along a spatial hexagonal lattice[25]. Together with a variety of other specialized cell types that encode spatial orientation [26,27], boundaries [28, 29], and distances to objects [30], this hippocampal-entorhinal machineryis often considered to provide a cognitive map facilitating navigation and self-location. Yetmore recent evidence has shown that the same neural mechanisms are also active when reasoning about more abstract, conceptual relationships [31–36], characterized by arbitrary feature dimensions [37] or temporal relationships [38, 39]. For example, using a techniquedeveloped to detect spatial hexagonal grid-like codes in fMRI signals [40], Constantinescuet al. found that human participants displayed a pattern of activity in the entorhinal cortexconsistent with mental travel through a 2D coordinate system defined by the length of a bird’slegs and neck [9]. Similarly, the same entorhinal-hippocampal system has also been found toreflect the graph structure underlying sequences of stimuli [10] or the structure of social networks [41], and even to replay non-spatial representations in the sequential order that characterized a previous decision-making task [42]. At the same time, much evidence indicates thatcognitive map-related representations are not limited to medial temporal areas, but alsoinclude ventral and orbital medial prefrontal areas [9, 11, 40, 43–45]. Relatedly, a study byKahnt and Tobler [46] using uni-dimensional variations of Gabor stimuli showed that the generalization of rewards was modulated by dopaminergic activity in the hippocampus, indicatinga role of non-spatial distance representations in reinforcement learning.Based on these findings, we asked whether learning and searching for rewards in spatialand conceptual domains is governed by similar computational principles. Using a within-subject design comparing spatial and non-spatial reward learning, we tested whether participantsused perceptual similarities in the same way as spatial distances to generalize from previousexperiences and inform the exploration of novel options. To ensure commensurate stimulidiscriminability between domains, participants completed a training phase where they wererequired to reach the same level of proficiency in correctly matching a series of target stimuli(see Methods; Fig 1c). In both domains, rewards were correlated (see S2 Fig), such that nearbyor similar options tended to yield similar rewards. To model how participants generalize andexplore using either perceptual similarities or spatial distances, we used Gaussian Process (GP)regression [47, 48] as a Bayesian model of generalization based on the principle of functionlearning. The Bayesian predictions of the GP model generalize about novel options using acommon notion of similarity across domains, and provide estimates of expected reward anduncertainty. We tested out-of-sample predictions of the GP model against a Bayesian learnerthat incorporates uncertainty-guided exploration but without generalization, and investigateddifferences in parameters governing value-based decision making and uncertainty-directedexploration [49–51].Participant performance was correlated across tasks and was best captured by the GPmodel in both domains. We were also able to reliably predict participant judgments aboutunobserved options based on parameters estimated from the bandit task. Whereas the modelparameters indicated similar levels of generalization in both domains, we found lower levels ofdirected exploration in the conceptual domain, where participants instead showed increasedlevels of random exploration. Moreover, we also observed an asymmetric task order effect,where performing the spatial task first boosted performance on the conceptual task but notvice versa. These findings provide a clearer picture of both the commonalities and differencesPLOS Computational Biology https://doi.org/10.1371/journal.pcbi.1008149 September 9, 20203 / 28
PLOS COMPUTATIONAL BIOLOGYSimilarities and differences in spatial and non-spatial cognitive mapsFig 1. Experiment design. a) In the spatial task, options were defined as a highlighted square in a 8 8 grid, where thearrow keys were used to move the highlighted location. b) In the conceptual task, each option was represented as a Gaborpatch, where the arrow keys changed the tilt and the number of stripes (S1 Fig). Both tasks corresponded to correlatedreward distributions, where choices in similar locations or having similar Gabor features predicted similar rewards (S2Fig). c) The same design was used in both tasks. Participants first completed a training phase where they were asked tomatch a series of target stimuli. This used the same inputs and stimuli as the main task, where the arrow keys modifiedeither the spatial or conceptual features, and the spacebar was used to make a selection. After reaching the learningcriterion of at least 32 training trials and a run of 9 out of 10 correct, participants were shown instructions for the maintask and asked to complete a comprehension check. The main task was 10 rounds long, where participants were given 20selections in each round to maximize their cumulative reward (shown in panels a and b). The 10th round was a “bonusround” where after 15 selections participants were asked to make 10 judgments about the expected reward and associateduncertainty for unobserved stimuli from that round. After judgments were made, participants selected one of theoptions, observed the reward, and continued the round as .g001in how people reason about and represent both spatial and abstract phenomena in complexreinforcement learning tasks.Results129 participants searched for rewards in two successive multi-armed bandit tasks (Fig 1). Thespatial task was represented as an 8 8 grid, where participants used the arrow keys to move ahighlighted square to one of the 64 locations, with each location representing one option (i.e.,PLOS Computational Biology https://doi.org/10.1371/journal.pcbi.1008149 September 9, 20204 / 28
PLOS COMPUTATIONAL BIOLOGYSimilarities and differences in spatial and non-spatial cognitive mapsarm of the bandit). The conceptual task was represented using Gabor patches, where a singlepatch was displayed on the screen and the arrow keys changed the tilt and stripe frequency(each having 8 discrete values; see S1 Fig), providing a non-spatial domain where similaritiesare relatively well defined. Each of the 64 options in both tasks produced normally distributedrewards, where the means of each option were correlated, such that similar locations or Gaborpatches with similar stripes and tilts yielded similar rewards (S2 Fig), thus providing tractionfor similarity-guided generalization and search. The strength of reward correlations weremanipulated between subjects, with one half assigned to smooth environments (with higherreward correlations) and the other assigned to rough environments (with lower reward correlations). Importantly, both classes of environments had the same expectation of rewards acrossoptions.The spatial and conceptual tasks were performed in counter-balanced order, with each taskconsisting of an initial training phase (see Methods) and then 10 rounds of bandits. Eachround had a different reward distribution (drawn without replacement from the assigned classof environments), and participants were given 20 choices to acquire as many points as possible(later converted to monetary rewards). The search horizon was much smaller than the totalnumber of options and therefore induced an explore-exploit dilemma and motivated the needfor generalization and efficient exploration. The last round of each task was a “bonus round”,where after 15 choices, participants were shown 10 unobserved options (selected at random)and asked to make judgments about the expected reward and their level of confidence (i.e.,uncertainty about the expected rewards). These judgments were used to validate the internalbelief representations of our models. All data and code, including interactive notebooks containing all analyses in the paper, is publicly available at tional models of learning, generalization, and searchMulti-armed bandit problems [52, 53] are a prominent framework for studying learning,where various reinforcement learning (RL) models [14] are used to model the learning ofreward valuations and to predict behavior. A common element of most RL models is someform of prediction-error learning [54, 55], where model predictions are updated based on thedifference between the predicted and experienced outcome. One classic example of learningfrom prediction errors is the Rescorla-Wagner [55] model, in which the expected reward V( )of each bandit is described as a linear combination of weights wt and a one-hot stimuli vectorxt representing the current state st:Vðxt Þ ¼ w t xtð1Þwtþ1 ¼ wt þ Zdt xtð2ÞLearning occurs by updating the weights w as a function of the prediction error δt rt V(xt), where rt is the observed reward, V(xt) is the reward expectation, and 0 η 1 is thelearning rate parameter. In our task, we used a Bayesian Mean Tracker (BMT) as a Bayesianvariant of the Rescorla-Wagner model [55, 56]. Rather than making point estimates of reward,the BMT makes independent and normally distributed predictions Vðsi;t Þ N ðmi;t ; vi;t Þ foreach state si,t, which are characterized by a mean m and variance v and updated on each trial tvia the delta rule (see Methods for details).Generalization using Gaussian process regression. Yet, an essential aspect of humancognition is the ability to generalize from limited experiences to novel options. Rather thanlearning independent reward representations for each state, we adopt a function learningPLOS Computational Biology https://doi.org/10.1371/journal.pcbi.1008149 September 9, 20205 / 28
PLOS COMPUTATIONAL BIOLOGYSimilarities and differences in spatial and non-spatial cognitive mapsapproach to generalization [19, 57], where continuous functions represent candidate hypotheses about the world, mapping the space of possible options to some outcome value. For example, a function can map how pressure on the gas pedal is related to the acceleration of a car, orhow different amounts of water and fertilizer influence the growth rate of a plant. Crucially,the learned mapping provides estimates even for outcomes that have not been observed, byinterpolating or extrapolating from previous experiences.While the literature on how humans explicitly learn functions extends back to the 1960s[58], more recent approaches have proposed Gaussian Process (GP) regression [47] as a candidate model of human function learning [59–61]. GPs unite previous proposals of rule-based[62, i.e., learning the weights of a particular parametric function] and exemplar-based theories[63, i.e., neural networks pr
RESEARCH ARTICLE Similarities and differences in spatial and non-spatial cognitive maps Charley M. Wu ID 1,2*, Eric Schulz3, Mona M. Garvert4,5,6, Bjo rn Meder ID 2,7,8, Nicolas W. Schuck ID 5,9 1 Department of Psychology, Harvard University, Cambridge, Massachusetts, United States of America, 2 Center for Adaptive Rationality, Max Planc
Accounting Differences There are no differences. System Management Differences There are no differences. Execution/Call Processing Differences There are no differences. Client Application Differences There are no differences. Deployment/Operational Differences There are no differences. System Engineering Differences There are no differences.
similarities and differences, with the goal of identifying which psychological attributes show large gender differences, which show small differences, and which show no differences. The gender similarities hypothesis states that males and females are similar o
The term spatial intelligence covers five fundamental skills: Spatial visualization, mental rotation, spatial perception, spatial relationship, and spatial orientation [14]. Spatial visualization [15] denotes the ability to perceive and mentally recreate two- and three-dimensional objects or models. Several authors [16,17] use the term spatial vis-
Differences and Similarities Key stage 1 Key stage 2 Learning Intention: To recognise that human differences and similarities arise from a number of factors including cultural, ethnic, racial and religious diversity, gender and d
Celebrating Differences and Similarities: Exploring Identity Kindergarten Mini-Unit By Michelle White . 2 Unit Rationale and Link to Standards: This unit was created as a back to school unit for building community in the classroom and celebrating one another’s differences and similarities
Identifying Similarities and Differences Seeing similarities and differences is a fundamental cognitive process (Gentner & Markman, 1994; Medin, Goldstone, & Markman, 1995). As an instructional strategy, it includes various activities that help learners see patterns and make connections.
HABITAT SIMILARITIES & DIFFERENCES Students compare similarities and differences in habitats. LEARNING OBJECTIVES Observe how habitat features in nature compare with habitat features in a salmon tank. Understand how plants and animals live in different conditions. Cultivate observat
Institute Publication (ANSI) A300 and the most recent edition of the companion publication “Best Management Practices – Tree Pruning”, published by the International Society of Arboriculture; POLICY FOR THE MANAGEMENT OF TREES ON CITY page OWNED OR OCCUPIED LAND 2 “Director of Engineering & Public Works” means the person designated to manage the City’s parks and boulevards; “drip .
