Machine Common Sense - DARPA

2y ago
10 Views
1 Downloads
3.32 MB
43 Pages
Last View : 1d ago
Last Download : 2m ago
Upload by : Abram Andresen
Transcription

Machine Common SenseDavid GunningDARPA/I2OProposers DayOctober 18, 2018Source: cacm.acm.orgErnest Davis and Gary Marcus. 2015. Commonsense reasoning and commonsense knowledge inartificial intelligence. Commun. ACM 58, 9 (August 2015), 92-103.Approved for Public Release, Distribution Unlimited

AgendaStartEnd8:00 AM9:00AMRegistration9:00 AM9:05 AMSecurityLeon Kates, Program Security Representative, DARPA SID9:05 AM10:10 AMMachine Common Sense (MCS)Dave Gunning, Program Manager, DARPA I2O10:10 AM 10:30 AMItemContractsMark Jones, Contracting Officer, DARPA CMO10:30 AM 11:30 AM Break11:30 AM1:00 PMQ&A Session (in-person and webcast)Please email your questions to mcs@darpa.milApproved for Public Release, Distribution Unlimited2

MCS BAA OutlineFunding Opportunity DescriptionA. Introduction/BackgroundB. Program Description/ScopeC. Technical Areas (TAs)TA1: Foundations of Human Common SenseTA2: Test Environment for the Foundations of Human Common SenseTA3: Broad Common KnowledgeD. Schedule/MilestonesE. TA-specific DeliverablesF. Government-furnished Property/Equipment/InformationG. Intellectual PropertyApproved for Public Release, Distribution Unlimited3

DoD Funding CategoriesMCSCategoryDefinitionBasic Research(6.1)Systematic study directed toward greater knowledge orunderstanding of the fundamental aspects of phenomenaand/or observable facts without specific applications inmind.Applied Research(6.2)Systematic study to gain knowledge or understandingnecessary to determine the means by which a recognizedand specific need may be met.Technology Development Includes all efforts that have moved into the development(6.3)and integration of hardware (and software) for fieldexperiments and tests.Approved for Public Release, Distribution Unlimited4

What are we trying to do?TODAYNarrowArtificial IntelligenceTOMORROWMachineCommon SenseFUTUREGeneralArtificial IntelligenceAI Application(Robot, Assistant,Analytic)Whereshould Isit to sawoff thelimb ofthis tree?PerceivingLearningAbstractingReasoningSource: cacm.acm.orgNarrow AICarefully train orprogram the systemfor every possiblesituationSource: magazine.owen.vanderbilt.eduSource: art.comHuman-LevelAIThe elephant in the roomApproved for Public Release, Distribution Unlimited5

What are we trying to do?TODAYTOMORROWAI ApplicationAI ApplicationNarrowArtificial Intelligence(Robot, Assistant,Analytic)Whereshould Isit to sawoff thelimb ofthis tree?Narrow AICarefully train orprogram the systemfor every possiblesituationGeneralArtificial Intelligence(Robot, Assistant,Analytic)?Source: cacm.acm.orgMachineCommon SenseFUTUREWhereshould Isit to sawoff thelimb ofthis tree?Sitbetweenthe trunkand thecut pointModified from: cacm.acm.orgCommonsenseServiceApproved for Public Release, Distribution rce: art.comHuman-LevelAI6

What are we trying to do?TODAYTOMORROWAI ApplicationAI ApplicationNarrowArtificial Intelligence(Robot, Assistant,Analytic)Whereshould Isit to sawoff thelimb ofthis tree?GeneralArtificial Intelligence(Robot, Assistant,Analytic)?Source: cacm.acm.orgMachineCommon SenseFUTUREWhereshould Isit to sawoff thelimb ofthis tree?Sitbetweenthe trunkand thecut pointModified from: Source: art.comNarrowAI computing foundations needed to develop machine commonsense services toMCS lenable programAI applicationsthe system to understand new situations, monitor the reasonableness of their actions,ServiceAIfor every possible more effectively with people,communicateand transfer learning to new domains.situationApproved for Public Release, Distribution Unlimited7

What is Common Sense? Wikipedia:Which of these would fitthrough a doorway?Source: AI2 If I put my socks in thedrawer, will they still be theretomorrow?Which object is flying andwhich is stationary in thissentence?I saw the Grand Canyonflying to Los Angeles.The basic ability to perceive, understand, and judge things that areshared by ("common to") nearly all people and can reasonably beexpected of nearly all people without need for debate.John McCarthy (Stanford, circa 1960): a. Name(a) ANY-FOOL 𝑘. Knows(ANY-FOOL, k) 𝑝 Persons. Knows(p, k) 𝑘. Commonsense(k) Knows(ANY-FOOL, k)Source: amturing.acm.orgCore Domains of Human Cognition:CommonFactsIntuitive IntuitivePhysics PsychologyApproved for Public Release, Distribution UnlimitedSource: scholar.harvard.eduExamples:Elizabeth Spelke(Harvard) ObjectsAgentsPlacesNumberFormsSocial Beings8

How is it done today?Taxonomy of Approaches toCommonsense ReasoningWeb ItAllCrowd ge-scaleSituation calculus,Region connection calculus,Qualitative process theoryScripts,Frames,Case-based reasoningCYCErnest Davis and Gary Marcus. 2015. Commonsense reasoning and commonsense knowledge in artificial intelligence.Communications of the ACM 58, 9 (August 2015), 92-103. DOI: https://doi.org/10.1145/2701413Approved for Public Release, Distribution Unlimited9

How is it done today?CycCommonsenseKnowledge BaseThe Cyc KB is a formalized representation(in First Order and Higher Order Logic) of avast quantity of human knowledgeCyc’s ontology contains: 42,000 Predicates 500,000 Collection types 1,500,000 General concepts 24,500,000 Assertions(1985-Today)Source: Dr. Doug Lenat, Cycorp, cyc.comApproved for Public Release, Distribution Unlimited10

What is new in your approach?Learning Grounded RepresentationsLearning Predictive Models from ExperiencePredictionSource: medium.comSource: colah.github.ioVondrick et al., 2016. Anticipating visual representations from unlabeled video(S-O) Car is found in Raceway.(O-O) Corolla is a kind of/looks similar to Car.Never Ending LanguageLearning (NELL)rowdednd shapeunding Boxes and Visual Subcategories)Never Ending ImageLearning (NEIL)Source: Dr. Abhinav Gupta, CMUCar(O-O) Wheel is a part of Car.WheelMitchell et al., 2018. Never-Ending LearningLearning Commonsense Knowledge from the WebUnderstanding & Modeling Childhood CognitionSource: scholar.harvard.eduVector-based “embeddings”extracted from hidden layersElizabeth Spelke(Harvard)Approved for Public Release, Distribution Unlimited ObjectsAgentsPlacesNumberFormsSocial BeingsCore Domains of Child Cognition11

Program ApproachTA2: TestEnvironmentVideo &SimulationExperiencesPredictions,Natural LanguageExpectations, && Image-basedSimulation ActionsQuestionsTrue/False &Multiple ChoiceAnswersSimulated CommonsenseAgentBroad CommonsenseQA ServiceCommonFactsCommonFactsAI BeginnerLearns thefoundations ofcommon sensefrom simulatedexperiencesAI2 Benchmarks forCommon itive IntuitivePhysics PsychologyObjects AgentsTA1: Foundations ofHuman Common SenseAI LibrarianLearns commonsense by readingand extractingknowledge from theWebTA3: Broad CommonKnowledgeApproved for Public Release, Distribution Unlimited12

Did the Wright Flyer need to fly like a bird?Source: Bec Green, youtube.comStork in FlightSource: youtube.comWright Flyer, 1903Approved for Public Release, Distribution Unlimited13

Otto LilienthalStork FlightSource: wikipedia.org, Der Vogelflug als Grundlage der FliegekunstDer Vogelflug als Grundlageder Fliegekunst (Bird flight asthe basis for flying art), 1882Source: wikipedia.orgAt left, 1901 glider flown by Wilbur andOrville (using Lilienthal’s original lift anddrag tables) exhibiting a steep angle ofattack due to poor lift and high drag. Atright, 1902 glider (after correctingLilienthal’s coefficients) showing dramaticimprovement in performance.Source: airandspace.si.eduLilienthal’s tablesof lift and dragSource: wikipedia.orgSource: wikipedia.orgOtto Lilienthal in mid-flight(first successful glider, 1895)"Lilienthal was without question thegreatest of the precursors, and theworld owes to him a great debt.” –Wilbur Wright, 1912Approved for Public Release, Distribution Unlimited14

Core Domains of Child CognitionSource: scholar.harvard.eduElizabeth Spelke(Harvard)Director of the HarvardLaboratory for DevelopmentalStudies. Since the 1980s, shehas carried out experimentsto test the cognitive facultiesof children and formulate hertheories of child cognition.DomainDescriptionObjectssupports reasoning about objects and the laws ofphysics that govern themAgentssupports reasoning about agents that actautonomously to pursue goalsPlacessupports navigation and spatial reasoning aroundan environmentNumbersupports reasoning about quantity and howmany things are presentFormssupports representation of shapes and theiraffordancesSocial Beingssupports reasoning about Theory of Mind andsocial interactionsApproved for Public Release, Distribution Unlimited15

Understanding the Foundations of Human CognitionLookit: the online child lab, MIT Early Childhood Cognition LabStimuliResponse“Your baby, the physicist” study: abd1-47534d6c4dd2/Approved for Public Release, Distribution Unlimited16

Cognitive Development Milestones (for children 0-18 months old)Approved for Public Release,Distribution Unlimited17

Foundations of Human Common SenseObjectsAgentsSource: medium.comSource: medium.comInfant cognition for Objects and Agents. These core domains likely form the fundamental building blocks ofhuman intelligence and common sense, especially the core domains of objects (intuitive physics), agents(intentional actors), and places (spatial navigation). For example, the core domain of objects not only provides thefundamental concepts for understanding the physical world, but also provides the foundation for understandingcausality. The core domain of agents not only provides the fundamental concepts for understanding intentionalactors and Theory of Mind (TOM), but also provides the foundation for dealing with the “frame problem” in AI(i.e., knowing that objects in a scene only change if acted on by an agent).Approved for Public Release, Distribution Unlimited18

Approved for Public Release, Distribution UnlimitedCourtesy: Dave Gunning19

Approved for Public Release, Distribution UnlimitedCourtesy: Dave Gunning20

TA2 Foundations Test Environment atio-temporalpermanencecontinuityCore principleMilestoneObjects don’t pop in and out ofexistence5 monthsObject trajectories arecontinuous4 monthsObjects keep their shapes10 monthsCore principleMilestoneActive, self-guided locomotion10 monthsLearn environment layout10 monthsEncode distances/directions ofstable surfaces to navigate10 monthsSource: coml.lscp.ens.fr/intphys/ (Emmanuel Dupoux, Mathieu Bernard, Ronan Riochet)Source: AI2Approved for Public Release, Distribution Unlimited21

TA2 Foundations Tests: Levels of PerformancePrediction/ExpectationExperience LearningProblem Solving The test environment willpresent the TA1 models withvideos and simulationexperiences of the type used totest child cognition for eachcognitive milestone. The models will produce aexpectation output (ameasurable Violation ofExpectation (VOE) signal) thatwill be used to determine if themodel matches human cognitiveperformance by comparison tothe VOE results observed inchildren. The test environment willpresent TA1 models with videosand simulation experiences inwhich a new object, agent, orplace is introduced. The models will be tested todetermine that they are able tolearn the properties of thenewly introduced item in a waythat matches human cognitiveperformance. The test environment willpresent the TA1 models withvideos and simulationexperiences in which a problemsolving task is introduced. The models will be tested todetermine that they solve theproblem in a way that matcheshuman cognitive performance.Approved for Public Release, Distribution Unlimited22

Examples of Developmental Psychology Research & ScorecardVOELearnSolve Approved forPublic Release,DistributionUnlimited23

TA1 Schedule and Scorecard Targets (Estimates)Year 1Year 2Year 3Year %30%50%50%30%50%OBJECTSViolation of ExpectationsExperience LearningProblem SolvingAGENTSViolation of Expectations30%Experience LearningProblem SolvingPLACESViolation of ExpectationsExperience Learning30%Problem SolvingApproved for Public Release, Distribution Unlimited24

Learning Commonsense Knowledge from the Web(S-O) Car is found in Raceway.(O-O) Corolla is a kind of/looks similar to Car.(O-O) Wheel is a part of Car.Relationships Extracted by NEILWheelHero cyclesVisual Instances Labeled by NEILWheel is/has Round shape.Alley is/has Narrow.Bamboo forest is/has Vertical lines.Sunflower is/has Yellow.Source: Dr. Abhinav Gupta, CMU(O-A)(S-A)(S-A)(O-A)(c) AttributesCrowdedRound shapeCrowdedRound shapeCorollaWheelRacewayParking lot(S-O) Car is found in Raceway.Egypt.found inknowledge(S-O) PyramidFigure 1. NEIL isacomputer program that runs 24 hours a day and 7 days aweekto gatherisvisualfrom the Internet. Specifically,it simultaneously labels the data and extracts common sense relationships betweencategories.is/has Round shape.(O-A) WheelSource: Dr. Abhinav Gupta, CMURaceway(S-O) Pyramid is found in Egypt.(O-O) Corolla is a kind of/looks similar to Car.(S-A) Alley is/has Narrow.CrowdedRound shapeRacewayParking lot(a) ObjectsBoxes Verticaland VisualSubcategories)relationships or learns relationships in a supervised setting.proachesto(w/Boundingbuildis usingmanual annotations by(O-O) Corolla is a kind of/looks similar to Car.lines.is/hasforestdatasetsBamboo(S-A)(c) AttributesYellow. [31] or the power of crowds [8,(O-A) SunflowerOur key insight is that at a large scale one can simultanemotivatedteamsis/hasof peopleouslyreadlabel thevisualinstancesminimize human effort, recent works have also foNELL has been learningtotheWeb24and extract common sense 41]. ToRelationshipsNEIL selects label reby whichExtractedNEILInstances LabeledVisualrelationshipsin ajoint bysemi-supervisedlearning framework.cused on active learning[38, 40]hoursadaysinceJanuary2010.Sofar,NELLqueststhat are mostboth of these(c) Semanticallyivenaknowledgeacquisition:Weuse visual(S-O) Pyramid is found in Egypt.Specifically,the Internet. However,frominformative.knowledgegatheraweek today and 7 dayshoursthat runs 24 drFigure 1. NEIL isa computer reexpen(O-A) Wheel is/has Round xtractsanddatathelabelssimultaneouslyithas acquired a knowledge base with 120(S-A) Alley is/has Narrow.sive, prone to errors, biased and do not scale.group visual data based on semantic categories and develop(S-A) Bamboo forest is/has Vertical lines.(b) Scenes(c) Attributes(O-A) Sunflower is/has nnotationsis usingapproachbuild datasetsto ussetting.in a supervisedrelationshipsor anticcategories.proachesThis allowsAn alternativeto use hipsExtractedby NEIL knowledgefor extractingdatasets automatically from the Inter- ExtractNELL runs continuouslytotoextractnewExtractvisual anprogramTo minimize41].Figureously label the visual instances and extract common sense1. NEIL isa computerruns 24workshours a dayandalso7 daysfoaweek to gather visual knowledge from the Internet. Specifically,instances of categoriesandrelations.theWebContributions:Our main contributions are: it(a)We pro- knowledgea datasetisandto fromusesearchresultsand rerankthem categories.viaof actions and objects from the Webthe labelselectswhich40]imagelearningactive ionships in ajoint semi-supervisedpose a never ending learning algorithm for gathering visualvisual classifiers [14] or some form of joint-clustering inrelationshipslearnsinformative.relationships in asupervised bothsetting.of theseproaches to build datasets is using manual annotations byHowever,areormostthat hasquestsuseWeviaacquisition:(c) Semantically dr iven knowledgeknowledgefrom the Internetmacro-vision.NEILtext and visual space [2, 35]. Another approach is to motivated teams of people [31] or the power of crowds onsbeencontinuouslyrunningfor2.5monthsona200corea semantic representation for visual knowledge; that is, wea semi-supervisedframework[43]. Here, a small elvisual Release,instances andextract commonsense41]. To minimize human effort, recent works have also focluster; (b) We are automatically o errors,pronesive,of labeleddatausedconjunction with a large amount(b) Scenes(Forbes & Choi, 2017). VERB PHYSICS: Relative Physical Knowledge of Actions and Objects(S-O) Car is found in Raceway.Car(Mitchell et al. 2018) Never-Ending LearningCarHero cycles(O-O) Wheel is a part of Car.WheelCorollaCorollaHero cycles(b) ScenesVerb Physics(O-O) Wheel is a part of Car.(a) Objects (w/Bounding Boxes and Visual Subcategories)(a) Objects (w/Bounding Boxes and Visual Subcategories)Parking lotCarNever Ending Image Learning(NEIL)Source: Dr. Abhinav Gupta, CMUNever Ending Language Learning(NELL)25

Allen Institute for Artificial Intelligence (AI2)Benchmarks for Common SenseDeveloper’s EnvironmentImplement ModelUploadCodeTrain Model(Public Train Set)Train Model(Public Train Set)Debug &IterateAI2’s Project Common Sense isdeveloping a suite of standardmeasurements for the commonsense abilities of an AI system.The initial test set and leaderboardwill be available in OCT 2018.UploadModelAI2’s CommonsenseTest Sets Commonsense NaturalLanguage Inference (NLI) Commonsense NLI withVisionEvaluate Model(Dev Set) Abductive NLIEvaluate Model(Blind Test Set)SequesteredTestEnvironmentExtract metrics Physical InteractionQuestion Answering (QA) Social Interaction QAPublish resultsto LeaderboardApproved for Public Release, Distribution Unlimited26

TA3 Schedule and Target MilestonesAI2 Common Sense Benchmark Data SetYear 1Year 2Year 3Year 4Commonsense Natural Language Inference (NLI)50%60%70%80%Commonsense NLI with Vision50%60%70%80%Abductive NLI50%60%70%80%Physical Interaction Question Answering (QA)50%60%70%80%Social Interaction QA50%60%70%80%Approved for Public Release, Distribution Unlimited27

Technical AreasTA2: TestEnvironmentVideo &SimulationExperiencesPredictions,Natural LanguageExpectations, && Image-basedSimulation ActionsQuestionsTrue/False &Multiple ChoiceAnswersSimulated CommonsenseAgentBroad CommonsenseQA ServiceCommonFactsCommonFactsAI BeginnerLearns thefoundations ofcommon sensefrom simulatedexperiencesAI2 Benchmarks forCommon itive IntuitivePhysics PsychologyObjects AgentsTA1: Foundations ofHuman Common SenseAI LibrarianLearns commonsense by readingand extractingknowledge from theWebTA3: Broad CommonKnowledgeApproved for Public Release, Distribution Unlimited28

TA1: Foundations of Human Common SenseGoal: develop computational models that mimic the core cognitive capabilities ofchildren, 0-18 months old Multiple TA1 development teams will be selected to construct the computational models.The TA1 teams may propose a variety of development strategies, ranging from pre-buildinginitial models, to learning everything from scratch using any combination of symbolic,probabilistic, or deep learning techniques.The TA1 teams are expected to include both AI and developmental psychology expertise, toproduce both computational models and refined psychological theories of cognition.Although the primary goal of TA1 is to develop computational models, a secondary goal is toconsolidate, refine, and extend the psychological theories of child cognition needed to guidemodel development, and to test, through research, key predictions made by the computationalmodels.The TA1 teams may also propose optional companion research experiments in developmentalpsychology to refine their theories of cognition, where needed, to answer critical designquestions relevant to their computational models.Note that, although TA2 will provide sample test problems, each TA1 team is responsible fordesigning and providing their own development strategy, training regimen, and any necessarydatasets.Approved for Public Release, Distribution Unlimited29

TA1: Foundations of Human Common SenseTA1 proposals should include a detailed discussion of the technical plan to: Design and develop computational models that mimic the foundations of human commonsense for the core domains of objects, agents, and places;Consolidate, refine, and extend the psychological theories of child cognition needed to guidemodel development; and test key predictions made by the computational models;Sequence the development and evaluation of the computational models over the four-yearprogram, including any optional companion research experiments in developmentalpsychology to refine relevant theories of cognition;Perform the evaluation tasks, including the three levels of performance: prediction/expectation, experience learning, and problem solving;Achieve the target milestones and metrics identified in the Schedule/Milestone section of theBAA; andPublish, share, and disseminate the results of research and development to the broader AIand Developmental Psychology communities.Approved for Public Release, Distribution Unlimited30

TA2: Test Environment for the Foundations of Human Common SenseGoal: provide the test and evaluation environment for evaluating the TA1models against cognitive development milestones as evidenced in developmentalpsychology research with children from 0 to 18-months old The existing body of research will be used as an initial starting point for the TA2 team toconstruct the test environment and develop specific test problems for each milestone in orderto evaluate the TA1 computational models at three levels of performance: prediction/expectation, experience learning, and problem solving.Approved for Public Release, Distribution Unlimited31

TA2: Test Environment for the Foundations of Human Common SenseTA2 proposals should include a detailed discussion of the technical plan to: Refine and expand the cognitive milestones for the core domains of objects, agents, and places(including combinations of the three domains);Devise a set of specific test problems for each cognitive milestone to assess computational modelsat the required three levels of performance (prediction/expectation, experience learning, problemsolving);Select, modify, or construct the video and 3D simulation infrastructure needed to conduct the TA1evaluations;Provide the training and testing infrastructure, with sample test problems, to support TA1computational model development. TA2 is not expected to provide all of the training data that may be needed by the TA1 teams.TA1 teams are responsible for designing and providing their own development strategy andtraining regimen.Develop and provide all of the documentation (e.g., user guides, test environment specifications,etc.) and APIs for testing TA1 computational models;Conduct formal evaluations of TA1 computational models every six months; andProvide written test reports that document the performance of the TA1 models for each 6-monthevaluation.Approved for Public Release, Distribution Unlimited32

Technical AreasTA2: TestEnvironmentVideo &SimulationExperiencesPredictions,Natural LanguageExpectations, && Image-basedSimulation ActionsQuestionsTrue/False &Multiple ChoiceAnswersSimulated CommonsenseAgentBroad CommonsenseQA ServiceCommonFactsCommonFactsAI BeginnerLearns thefoundations ofcommon sensefrom simulatedexperiencesAI2 Benchmarks forCommon itive IntuitivePhysics PsychologyObjects AgentsTA1: Foundations ofHuman Common SenseAI LibrarianLearns commonsense by readingand extractingknowledge from theWebTA3: Broad CommonKnowledgeApproved for Public Release, Distribution Unlimited33

TA3: Broad Common KnowledgeGoal: learn/extract/construct a commonsense knowledge repository capable of answeringnatural language and image-based questions about commonsense phenomena from the AI2Benchmarks for Common Sense. Multiple development teams will be selected to develop the TA3 commonsenserepositories/question answering services. TA3 teams may propose any combination of manual construction, information extraction,machine learning, and crowdsourcing techniques to construct a repository of broadcommonsense knowledge. TA3 teams are not required to include personnel with expertise in psychology and are free to usewhatever techniques they prefer, whether artificial or biologically inspired. The TA3 teams are expected to submit their system for testing on the blind evaluationdatasets (i.e., all five commonsense question datasets identified above, ifcompleted/available) every six months. Additional datasets may be developed over the course of the program, as needed, to align with theevolution of the TA3-developed capabilities. After the first year, TA3 teams may propose their own question datasets for inclusion in the AI2benchmarks, or propose suggestions for the development of additional datasets by AI2, for testingby all of the TA3 teams.Approved for Public Release, Distribution Unlimited34

TA3: Broad Common KnowledgeTA3 proposals should include a detailed discussion of the technical approach to: Design and develop the broad commonsense knowledge service;Sequence the development and evaluation of the broad commonsense knowledge serviceover the four-year program;Perform the evaluation tasks for the AI2 Benchmarks for Common Sense;Achieve the target milestones and metrics identified in Schedule/Milestone section of theBAA; andPublish, share, and disseminate the results of research and development to the broader AIcommunity.Approved for Public Release, Distribution Unlimited35

AI2 Benchmarks Initially, five commonsense question datasets will be developed and available for testing of TA3developed services:1. Commonsense Natural Language Inference (NLI): multiple choice, natural languagebased questions about commonsense events derived from captions in the ActivityNet Captionsand Large Scale Movie Description Challenge (LSMDC) datasets.2. Commonsense NLI with Vision: multiple choice, image-based questions aboutcommonsense events selected from the same ActivityNet and LSMDC datasets.3. Abductive NLI: questions about inferring the most likely hypothesis for a given set ofobservations.4. Physical Interaction Question Answering (QA): natural language questions (initially) andimage-based questions (in later years) about everyday objects and actions.5. Social Interaction QA: questions about human social behavior and the causal effects ofeveryday events.The development of the first dataset, Commonsense NLI, is completed and is described further inZellers, R., et al. (2018).The remaining four datasets are currently in development and will be completed by the start of theprogram.More information about the AI2 commonsense question datasets and leaderboard will be available athttps://leaderboard.allenai.org/ (which requires Chrome).Approved for Public Release, Distribution Unlimited36

Schedule DARPA anticipates a June 2019 start date for the MCS program that will run for a duration of48 months.The following PI Meetings will take place: An in-person Kickoff Meeting at program start. For planning purposes, assume a 3-day meeting inArlington, VA; Four (4) web-based PI meetings held at six (6) months into each year of the program to reviewtechnical progress. For planning purposes, assume the government as host for these 2-day virtualmeetings; and Four (4) in-person PI meetings held at the end of each year of the program to review technicalprogress, conduct demonstrations, and provide opportunities for face-to-face collaboration. Forplanning purposes, assume 3-day meetings, alternating between a west coast and east coastlocation. In addition to the PI Meetings above, each team should expect to: Host an onsite visit from the PM (and potentially other government personnel) at least once a year;and Make two additional trips to the Washington, D.C. area in the last two years of the program forpossible demonstrations and technology transition meetings.Approved for Public Release, Distribution Unlimited37

Milestones The target milestones and metrics identified have been established to assess technicalprogress over the course of the program.The targets are not “go/no-go” criteria and it is not DARPA’s intention to use the targets asthe basis for down-selects or as the primary reason for other funding decisions. TA1-developed computational models will be assessed for performance against cognitivedevelopment milestone capabilities (for the core domains of objects, agents, and places) atincreasing levels of performance: prediction/expectation, experience learning, and problem solving. TA3-developed services will be assessed for performance on the AI2 Common Sense Benchmarkdatasets (Commonsense NLI, Commonsense NLI with Vision, Abductive NLI, Physical InteractionQA, and Social Interaction QA). Assessments of TA1-developed computational models and TA3-developed QA services will beconducted every six (6) mon

Oct 18, 2018 · NELL, KnowItAll Situation calculus, Region connection calculus, Qualitative process theory Scripts, Frames, Case-based reasoning ConceptNet, OpenMind CYC Ernest Davis and Gary Marcus. 2015. Commonsense reasoning and commonsense knowledge in artificial intelligence. . Dr. Doug Lenat, Cycorp, cyc.com Cyc Commonsense Knowledge Base

Related Documents:

This report provides an overview of DARPA, including the agency's organizational structure, characteristics (i.e., the "DARPA model"), and strategic priorities. The report also describes funding trends at DARPA and discusses select issues for possible congressional consideration,

Technology and their Impacts on GPS users John Burke Program Manager DARPA/MTO GPS Advisory Board 6 June 2019 1. DoD Distribution Statement A: Approved for public release; distribution is unlimited. 2 DARPA PNT programs overview Precision Inertial 1930 1940 1950 1960 1970 1980 1990 2000 2010 2020 NDB/ADF TACAN (VOR/DME)

decoration machine mortar machine paster machine plater machine wall machinery putzmeister plastering machine mortar spraying machine india ez renda automatic rendering machine price wall painting machine price machine manufacturers in china mail concrete mixer machines cement mixture machine wall finishing machine .

ladder-climbing problem as posed by the DARPA Robotics Challenge (DRC), which aims to develop robots with human-like navigation, manipulation, and perception capabilities in a hazardous disaster site. The type and location of the ladder are unspecified, so the robot must sense, plan, and execute a climbing motion largely autonomously. Beginning

FDA does not fund the development of MPS for regulatory use in the support of new products. History of FDA’s Involvement with MPS 2010: FDA and NIH Common Fund awarded grant money to Wyss to develop a heart-lung micromachine 2011: DARPA approached FDA’s Office of the Chief Scientist requesting to work together to develop a human body on a chip for me dical countermeasures. DARPA .

can achieve "digital intuition" about their own data, making assumptions and conclusions based on the connec-tions between that speci"c data and the general common sense that peo-ple have. Acquiring and Representing Common Sense Open Mind Common Sense takes a distributed approach to the problem of commonsense knowledge acquisi-tion.

Sense-of-urgency is the first core concept in this study. Several definitions exist on the concept of sense-of-urgency. For instance, sense-of-urgency is defined as employee’s determination to change [5]. Other scholars peg sense-of-urgency as a crucial element in successful change process [6]. On the other hand, some scholars define sense-of .

a) Plain milling machine b) Universal milling machine c) Omniversal milling machine d) Vertical milling machine 2. Table type milling machine 3. Planer type milling machine 4. Special type milling machine 5.2.1 Column and knee type milling machine The column of a column and knee