From Deep Learning To Deep Reasoning
Tutorial at KDD, August 14th 2021From Deep Learning to Deep ReasoningPart C: Memory Data efficiency Recursive reasoningTruyen Tran, Vuong Le, Hung Le and Thao u.auhttps://bit.ly/37DYQn714/08/20211
Agenda Reasoning with external memories Memory of entities – memory-augmented neural networks Memory of relations with tensors and graphs Memory of programs & neural program construction. Learning to reason with less labels Data augmentation with analogical and counterfactual examplesQuestion generationSelf-supervised learning for question answeringLearning with external knowledge graphs Recursive reasoning with neural theory of mind.2
Agenda Reasoning with external memories Memory of entities – memory-augmented neural networks Memory of relations with tensors and graphs Memory of programs & neural program construction. Learning to reason with less labels: Data augmentation with analogical and counterfactual examplesQuestion generationSelf-supervised learning for question answeringLearning with external knowledge graphs Recursive reasoning with neural theory of mind.3
Introduction4
Memory is part of intelligence Memory is the ability tostore, retain and recallinformation Brain memory storesitems, events and highlevel structures Computer memorystores data andtemporary variables5
Memory-reasoning analogy 2 processes: fast-slowo Memory: familiarityrecollection Cognitive test:o Corresponding reasoning andmemorization performanceo Increasing # premises,inductive/deductivereasoning is affectedHeit, Evan, and Brett K. Hayes. "Predicting reasoning from memory." Journal of Experimental Psychology: General 140, no. 1 (2011): 76.6
Common memory activities Encode: write information tothe memory, often requiringcompression capability Retain: keep the informationovertime. This is often assumedin machinery memory Retrieve: read information fromthe memory to solve the task athandEncodeRetainRetrieve7
Memory taxonomy based on memory contentItemMemory Objects, events, items,variables, entitiesRelationalMemory Relationships, structures,graphsProgramMemory Programs, functions,procedures, how-to knowledge8
Item memoryAssociative memoryRAM-like memoryIndependent memory9
Distributed item memory asassociative memoryLanguage"Green" means"go," but whatdoes "red" mean?SemanticmemoryTimeObjectbirthday party on30th JanWhere is my pen?What is tormemory10
Associate memory can be implemented asHopfield network“Fast-weight 𝑀𝑀Correlation matrix memoryEncodeHopfield entretrieval11
Rule-based reasoning with associativememory Encode a set of rules:“pre-conditions post-conditions” Support variablebinding, rule-conflicthandling and partialrule input Example of encodingrule “A:1,B:3,C:4 X”Outer productfor bindingAustin, Jim. "Distributed associative memories for high-speed symbolic reasoning." Fuzzy Sets and Systems 82, no. 2 (1996): 223-233.12
Memory-augmented neural networks:computation-storage separationRAMRNN Symposium 2016: Alex Graves - Differentiable Neural Computer13
Neural Turing Machine (NTM) Memory is a 2d matrix Controller is a neuralnetwork The controllerread/writes to memoryat certain addresses. Trained end-to-end,differentiable Simulate Turing Machine support symbolicreasoning, algorithmsolvingGraves, Alex, Greg Wayne, and Ivo Danihelka. "Neural turing machines." arXiv preprint arXiv:1410.5401 (2014).14
Addressing mechanism in NTMInputMemory writing𝑒𝑒𝑡𝑡 , 𝑎𝑎𝑡𝑡Memory reading
Algorithmic reasoningAssociativerecallCopyPriority sort16
Optimal memory writing formemorization Simple finding: writing too oftendeteriorates memory content (notretainable) Given input sequence of length Tand only D writes, when should wewrite to the memory?Uniform writing is optimal formemorizationLe, Hung, Truyen Tran, and Svetha Venkatesh. "Learning to Remember More with Less Memorization." In International Conference on Learning Representations. 2018.17
Better memorization means better algorithmic reasoningT 50, D 5RegularUniform (cached)18
Memory of independent entitiesWeston, Jason, Bordes, Antoine, Chopra, Sumit, and Mikolov, Tomas.Towards ai-complete question answering: A set of prerequisite toy tasks. CoRR, abs/1502.05698, 2015. Each slot store one or some entities Memory writing is done separately foreach memory slot each slot maintains the life of one ormore entities The memory is a set of N parallel RNNsRNN 1John AppleJohn Apple OfficeJohn AppleKitchenRNN 2Apple JohnApple JohnApple JohnKitchen TimeOffice19
Recurrent entity networkGardenHenaff, Mikael, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun."Tracking the world state with recurrent entity networks."In 5th International Conference on Learning Representations, ICLR 2017. 2017.20
Recurrent Independent MechanismsGoyal, Anirudh, Alex Lamb, Jordan Hoffmann, Shagun Sodhani, Sergey Levine, Yoshua Bengio, and Bernhard Schölkopf. "Recurrent independent mechanisms.“ ICLR21.21
Reasoning with independentdynamicsCopyBalldynamics22
Relational memoryGraph memoryTensor memory23
Why relational memory? Item memoryis weak at recognizing relationshipsItemMemory Store and retrieve individual items Relate pair of items of the same time step Fail to relate temporally distant items24
Dual process in memoryItemMemory Store items Simple, low-order System 1RelationalMemory Store relationships between items Complicated, high-order System 2Howard Eichenbaum, Memory, amnesia, and the hippocampal system (MIT press, 1993).Alex Konkel and Neal J Cohen, "Relational memory and the hippocampus: representations and methods", Frontiers in neuroscience 3 (2009).25
Memory as graph Memory is a static graph withfixed nodes and edges Relationship is somehowknown Each memory node storesthe state of the graph’s node Write to node via messagepassing Read from node via MLPPalm, Rasmus Berg, Ulrich Paquet, and Ole Winther. "Recurrent Relational Networks." In NeurIPS. 2018.26
CLEVERbAbINodeFact 1EdgeQuestionNodeAnswer(colour, shape. position)Edge(distance)Fact 2Fact 327
Memory of graphs access conditioned on query Encode multiple graphs, eachgraph is stored in a set ofmemory row For each graph, the controllerread/write to the memory: Read uses content-basedattention Write use message passing Aggregate read vectors fromall graphs to create outputPham, Trang, Truyen Tran, and Svetha Venkatesh. "Relational dynamic memory networks." arXiv preprint arXiv:1808.04247 (2018).28
Capturing relationship can be done viamemory slot interactions using attention Graph memory needs customization to an explicit design of nodes andedges Can we automatically learns structure with a 2d tensor memory? Capture relationship: each slot interacts with all other slots (selfattention)Santoro, Adam, Ryan Faulkner, David Raposo, Jack Rae, Mike Chrzanowski, Théophane Weber, Daan Wierstra, Oriol Vinyals, Razvan Pascanu, and Timothy Lillicrap."Relational recurrent neural networks." In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 7310-7321. 2018.29
Relational Memory Core (RMC) operationRNN-likeInterface30
Allowing pair-wise interactions can answerquestions on temporal relationship31
Dot product attention works forsimple relationship, but What ismostsimilar tome?What is mostsimilar to mebut differentfrom tiger?0.70.9-0.10.4For hard relationship, scalarrepresentation is limited32
Complicated relationship needs highorder relational memoryItemmemoryRelationalmemoryExtract items 3d relationaltensorLe, Hung, Truyen Tran, and Svetha Venkatesh. "Selfattentive associative memory." In International Conferenceon Machine Learning, pp. 5682-5691. PMLR, 2020.Associate every pairs of them33
Program memoryModule memoryStored-program memory34
Predefining program for subtask A program designed for atask becomes a module Parse a question to modulelayout (order of programexecution) Learn the weight of eachmodule to master the taskAndreas, Jacob, Marcus Rohrbach, Trevor Darrell, and Dan Klein. "Neural module networks." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 39-48. 2016.35
Program selection is based onparser, others are end2end trained5 moduletemplates1324Parsing536
The most powerful memory is one that storesboth program and data Computer architecture:Universal TuringMachines/Harvard/VNM Stored-program principle Break a big task into subtasks,each can be handled by aTM/single purposed programstored in a program memoryhttps://en.wikipedia.org/37
NUTM: Learn to select program (neural weight)via program attention Neural stored-program memory(NSM) stores key (the address)and values (the weight) The weight is selected andloaded to the controller of NTM The stored NTM weights andthe weight of the NUTM islearnt end-to-end bybackpropagationLe, Hung, Truyen Tran, and Svetha Venkatesh. "Neural Stored-program Memory."In International Conference on Learning Representations. 2019.38
Scaling with memory of mini-programs Prior, 1 program 1 neuralnetwork (millions ofparameters) Parameter inefficiency sincethe programs do not sharecommon parameters Solution: store sharablemini-programs to composeinfinite number of programsit is analogous to building Lego structurescorresponding to inputs from basic Lego bricks.39
Recurrent program attention to retrievesingular components of a programLe, Hung, and Svetha Venkatesh. "Neurocoder: Learning General-Purpose Computation Using Stored Neural Programs." arXiv preprint arXiv:2009.11443 (2020).40
Program attention is equivalent tobinary decision tree reasoningRecurrent program attention autodetects task boundary41
Agenda Reasoning with external memories Memory of entities – memory-augmented neural networks Memory of relations with tensors and graphs Memory of programs & neural program construction. Learning to reason with less labels: Data augmentation with analogical and counterfactual examplesQuestion generationSelf-supervised learning for question answeringLearning with external knowledge graphs Recursive reasoning with neural theory of mind.42
Data Augmentation with Analogical andCounterfactual Examples Poor generalization when training under independentand identically distributed assumption. Intuition: augmenting counterfactual samples to allowmachines to understand the critical changes in theinput that lead to changes in the answer space. Perceptually similar, yet Semantically dissimilar realistic samplesVisual counterfactual exampleGokhale, Tejas, et al. "Mutant: A training paradigm for out-of-distributiongeneralization in visual question answering." EMNLP’20.Language counterfactual examples43
Question GenerationsKrishna, Ranjay, Michael Bernstein, and Li Fei-Fei. "Information maximizing visual questiongeneration." CVPR’19. Question answering is a zero-shotlearning problem. Questiongeneration helps cover a widerrange of concepts. Question generation can be donewith either supervised andunsupervised learning.Li, Yikang, et al. "Visual question generation as dual task of visual question answering." CVPR’18.44
BERT: Transformer That Predicts Its OwnMasked PartsBERT is like parallelapproximate pseudolikelihood Maximizing theconditional likelihood ofsome variables given therest. When the number ofvariables is large, thisconverses to MLE(maximum r-nlp-f8b21a9b6270[Slide credit: Truyen Tran]46
Visual QA as a Down-stream Task of VisualLanguage BERT Pre-trained ModelsNumerous pre-trained visual language models during 2019-2021.Single-stream modelVisualBERT (Li, Liunian Harold, et al., 2019)VL-BERT (Su, Weijie, et al., 2019)UNITER (Chen, Yen-Chun, et al., 2019)12-in-1 (Lu, Jiasen, et al., 2020)Pixel-BERT (Huang, Zhicheng, et al., 2019)OSCAR (Li, Xiujun, et al., 2020)Two-stream modelViLBERT (Lu, Jiasen, et al. , 2019)LXMERT (Tan, Hao, and Mohit Bansal, 2019)[Slide credit: Licheng Yu et al.]47
Learning with External KnowledgeWhy external knowledgefor reasoning? Questions can be beyondvisual recognition (e.g.firetrucks usually use a firehydrant). Human’s prior knowledge forcognition-level reasoning (e.g.human’s goals, intents etc.)Q: What sort of vehicle uses this item?A: firetruckQ: What is the sports position of theman in the orange shirt?A: goalie/goalkeeperMarino, Kenneth, et al. "Ok-vqa: A visual questionanswering benchmark requiring externalknowledge." CVPR’19.Zellers, Rowan, et al. "From recognition to cognition: Visual commonsense reasoning." CVPR’19.48
Learning with External KnowledgeRetrieved by Wikipedia search APIShah, Sanket, et al. "Kvqa: Knowledge-aware visual questionanswering." AAAI’19.Marino, Kenneth, et al. "Ok-vqa: A visual questionanswering benchmark requiring externalknowledge." CVPR’19.49
Agenda Reasoning with external memories Memory of entities – memory-augmented neural networks Memory of relations with tensors and graphs Memory of programs & neural program construction. Learning to reason with less labels: Data augmentation with analogical and counterfactual examplesQuestion generationSelf-supervised learning for question answeringLearning with external knowledge graphs Recursive reasoning with neural theory of mind.50
Core AI faculty:Theory of mindSource: religious studies project
Theory of mindRecursive reasoningMemoryFactsSemanticsEvents and relationsWorking spaceSystem1:System1: 1:SystemIntuitiveIntuitiveIntuitivePerception FastImplicit/automaticPattern recognitionMultipleSingleSystem 2:Analytical SlowDeliberate/rationalCareful analysisSingle, sequentialWhere would ToM fit in?Image credit: VectorStock Wikimedia
Contextualized recursive reasoning Thus far, QA tasks are straightforward and objective: Questioner: I will ask about what I don’t know. Answerer: I will answer what I know. Real life can be tricky, more subjective: Questioner: I will ask only questions I think they cananswer. Answerer 1: This is what I think they want from an answer. Answerer 2: I will answer only what I think they think I can.14/08/2021 We need Theory of Mind to function socially.53
Social dilemma: Stag Hunt games Difficult decision: individual outcomes (selfish)or group outcomes (cooperative). Together hunt Stag (both are cooperative): Both have moremeat. Solely hunt Hare (both are selfish): Both have less meat. One hunts Stag (cooperative), other hunts Hare (selfish): Onlyone hunts hare has meat. Human evidence: Self-interested butconsiderate of others (cultures vary). Idea: Belief-based guilt-aversion One experiences loss if it lets other down. Necessitates Theory of Mind: reasoning about other’s mind.
Theory of Mind Agent with Guilt Aversion (ToMAGA)Update Theory of Mind Predict whether other’s behaviour arecooperative or uncooperative Updated the zero-order belief (whatother will do) Update the first-order belief (what otherthink about me)Guilt Aversion Compute the expected material rewardof other based on Theory of Mind Compute the psychological rewards, i.e.“feeling guilty” Reward shaping: subtract the expectedloss of the other.[Slide credit: Dung Nguyen]Nguyen, Dung, et al. "Theory of Mind with Guilt Aversion FacilitatesCooperative Reinforcement Learning." Asian Conference on MachineLearning. PMLR, 2020.
Rabinowitz, Neil, et al. "Machine theory of mind." International conference on machine learning. PMLR, 2018.Machine Theory of Mind Architecture (inside the Observer)next-step actionprobability[Slide credit: Dung Nguyen]goalSuccessorrepresentations
A ToMarchitecture Observer maintains memory ofprevious episodes of the agent. It theorizes the “traits” of theagent. Implemented as Hyper Networks. Given the current episode, theobserver tries to infer goal,intention, action, etc of theagent. Implemented as memory retrievalthrough attention mechanisms.14/08/202157
Wrapping up58
Wrapping up Reasoning as the next challenge for deep neural networks Part A: Learning-to-reason framework Reasoning as a prediction skill that can be learnt from data Dynamic neural networks are capable Combinatorics reasoning Part B: Reasoning over unstructured and structured data Reasoning over unstructured sets Relational reasoning over structured data Part C: Memory Data efficiency Recursive reasoning Memories of items, relations and programs Learning with less labels Theory of mind14/08/202159
Theory of mindRecursive reasoningMemoryFactsSemanticsEvents and relationsWorking spaceSystem1:System1: 1:SystemIntuitiveIntuitiveIntuitivePerception FastImplicit/automaticPattern recognitionMultipleSingleSystem 2:Analytical SlowDeliberate/rationalCareful analysisSingle, sequentialA possible framework for learning and reasoningwith deep neural networksImage credit: VectorStock Wikimedia
QAhttps://bit.ly/37DYQn714/08/202161
when training under independent . Krishna, Ranjay, Michael Bernstein, and Li Fei-Fei. "Information maximizing visual question . Visual QA as a Down-stream Task of Visual-Language BERT Pre-trained Models 47 Nume
Deep Learning: Top 7 Ways to Get Started with MATLAB Deep Learning with MATLAB: Quick-Start Videos Start Deep Learning Faster Using Transfer Learning Transfer Learning Using AlexNet Introduction to Convolutional Neural Networks Create a Simple Deep Learning Network for Classification Deep Learning for Computer Vision with MATLAB
2.3 Deep Reinforcement Learning: Deep Q-Network 7 that the output computed is consistent with the training labels in the training set for a given image. [1] 2.3 Deep Reinforcement Learning: Deep Q-Network Deep Reinforcement Learning are implementations of Reinforcement Learning methods that use Deep Neural Networks to calculate the optimal policy.
Deep Learning Personal assistant Personalised learning Recommendations Réponse automatique Deep learning and Big data for cardiology. 4 2017 Deep Learning. 5 2017 Overview Machine Learning Deep Learning DeLTA. 6 2017 AI The science and engineering of making intelligent machines.
English teaching and Learning in Senior High, hoping to provide some fresh thoughts of deep learning in English of Senior High. 2. Deep learning . 2.1 The concept of deep learning . Deep learning was put forward in a paper namedon Qualitative Differences in Learning: I -
work/products (Beading, Candles, Carving, Food Products, Soap, Weaving, etc.) ⃝I understand that if my work contains Indigenous visual representation that it is a reflection of the Indigenous culture of my native region. ⃝To the best of my knowledge, my work/products fall within Craft Council standards and expectations with respect to
-The Past, Present, and Future of Deep Learning -What are Deep Neural Networks? -Diverse Applications of Deep Learning -Deep Learning Frameworks Overview of Execution Environments Parallel and Distributed DNN Training Latest Trends in HPC Technologies Challenges in Exploiting HPC Technologies for Deep Learning
Artificial Intelligence, Machine Learning, and Deep Learning (AI/ML/DL) F(x) Deep Learning Artificial Intelligence Machine Learning Artificial Intelligence Technique where computer can mimic human behavior Machine Learning Subset of AI techniques which use algorithms to enable machines to learn from data Deep Learning
side of deep learning), deep learning's computational demands are particularly a challenge, but deep learning's specific internal structure can be exploited to address this challenge (see [12]-[14]). Compared to the growing body of work on deep learning for resource-constrained devices, edge computing has additional challenges relat-
