Agent Function Vacuum Cleaner World

AgentssensorsperceptsIntelligent Agents?environmentagentactionsRRussellll anddNNorvig:i 2actuators Agent – perceives the environment through sensors andacts on it through actuators Percept – agent’s perceptual input (the basis for itsactions) Percept Sequence – complete history of what has beenperceived.CISC4/681 Introduction to Artificial Intelligence1Agent FunctionCISC4/681 Introduction to Artificial Intelligence2Vacuum Cleaner ators Agent Function – maps a give percept sequenceinto an action; describes what the agent does. Externally – Table of actions Internally – Agent ProgramCISC4/681 Introduction to Artificial Intelligence3Agent CharacterizationCISC4/681 Introduction to Artificial Intelligence4Vacuum Cleaner World Meant to be a tool for analyzing systems –not characterizing them as agent versusnon-agent Lots of things can be characterized asagents (artifacts) that act on the world AI operates where– Artifacts have significant computationalresources– Task Environments require nontrivial decisionmakingCISC4/681 Introduction to Artificial Intelligence Percepts: which square (A or B); dirt? Actions: move right, move left, suck, do nothing Agent function: maps percept sequence intoactions Agent program: function’s implementation How should the program act?5AB Percepts: which square (A or B); dirt? Actions: move right, move left, suck, do nothing Agent function: maps percept sequence intoactions Agent program: function’s implementation How should the program act?CISC4/681 Introduction to Artificial Intelligence61

Rational Agent – does the right thingRationality Depends on:What does that mean? One that behavesas well as possible given the Environmentin which it acts. How should success bemeasured? On consequences. Performance measure1. Performance measure that defines criterion forsuccess.2. Agent’s prior knowledge of the environment.3. Actions the agent can perform.4 Agent4.Agent’ss percept sequence to datedate.– Embodies criterion for success Amount of dirt cleaned? Cleaned floors?– Generally defined in terms of desired effect onenvironment (not on actions of agent)– Defining measure not always easy!CISC4/681 Introduction to Artificial Intelligence7RationalityFor each possible percept sequence, a rationalagent should select an action that is expected tomaximize its performance measure, given theevidence provided by the percept sequence andwhatever built-in knowledge the agent has.CISC4/681 Introduction to Artificial IntelligenceRationalityFor each possible percept sequence, arational agent should select an action thatis expected to maximize its performancemeasure, given the evidence provided byperceptp sequenceqand whatever builtthe pin knowledge the agent has. Notice the rationality is dependent onEXPECTED maximization. Agent might need to learn how theenvironment changes, what actionsequences to put together, etc Notice that an agent may be rationalbecause the designer thought ofeverything, or it may have learned it itself(more autonomous)CISC4/681 Introduction to Artificial IntelligenceCISC4/681 Introduction to Artificial Intelligence9For each possible percept sequence, arational agent should select an action thatis expected to maximize its performancemeasure, given the evidence provided byperceptp sequenceqand whatever builtthe pin knowledge the agent has.10Properties of Task Environments(affect appropriate agent design)Task Environment The “problems” for which rational agentsare the “solutions”PEAS Description of Task Environment PerformanceP fMeasureM Environment Actuators (actions) Sensors (what can be perceived)CISC4/681 Introduction to Artificial Intelligence8 Fully observable vs partially observable– Fully observable gives access to completestate of the environment– Complete state means aspects relevant toaction choice– global vs local dirt sensor11CISC4/681 Introduction to Artificial Intelligence122

Properties of Task Environments(affect appropriate agent design) Deterministic vs Stochastic Single Agent vs Multi-agent– Deterministic – next state completelydetermined by current state and action– Uncertainty may arise because of defectiveactions or partially observable state (i.e.,agent might not see everything that affectsthe outcome of an action).– Single Agent – crossword puzzle– Multi-agent – chess, taxi driving? (are otherdrivers best described as maximizing aperformance element?)– Multi-agent means other agents may becompetitive or cooperative and may requirecommunication– Multi-agent may need communicationCISC4/681 Introduction to Artificial Intelligence13Properties of Task Environments(affect appropriate agent design)14 Static vs Dynamic– Episodic the agent’s experience divided intoatomic episodes– NextN t episodei d nott ddependentd t on actionstittakenkin previous episode. E.g., assembly line– Sequential – current action may affect futureactions. E.g., playing chess, taxi– short-term actions have long-term effects– must think ahead in choosing an action15Properties of Task Environments(affect appropriate agent design)– does environment change while agent isdeliberating?– StaticSt ti – crosswordd puzzlel– Dynamic – taxi driverCISC4/681 Introduction to Artificial Intelligence16Properties of Task Environments(affect appropriate agent design) Known vs Unknown Discrete vs Continuous.Can refer to– the state of the environment ((chess has finitenumber of discrete states)– the way time is handled (taxi drivingcontinuous – speed and location of taxi sweepthrough range of continuous values)– percepts and actions (taxi driving continuous– steering angles)CISC4/681 Introduction to Artificial IntelligenceCISC4/681 Introduction to Artificial IntelligenceProperties of Task Environments(affect appropriate agent design) Episodic vs SequentialCISC4/681 Introduction to Artificial IntelligenceProperties of Task Environments(affect appropriate agent design)17- This does not refer to the environment itself,but rather the agent’s knowledge of it and howit changeschanges.- If unknown, the agent may need to learnCISC4/681 Introduction to Artificial Intelligence183

Properties of Task Environments(affect appropriate agent design)Fully e agent Easy: Fully observable, Deterministic,Episodic, Static, Discrete, Single agent. Hard: Partially observable, Sochastic,Sequential, Dynamic, Continuous, MultiAgentCISC4/681 Introduction to Artificial IntelligenceEnvironment typesChess witha clockYesStrategicNoSemiYesNoChess withouta clockYesStrategicNoYesYesNoTaxi drivingNoNoNoNoNoNo The environment type largely determines the agentdesign19Agent Programs The real world is (of course) partially observable,stochastic, sequential, dynamic, continuous, multi-agentPossible Agent Program Need to develop agents – programs thattake the current percept as input from thesensors and return an action to theactuatorsactuators.CISC4/681 Introduction to Artificial Intelligence21Agent Programs22Simple Reflective Agent23AgentSensorsWhat the worldis like nowCondition action rulesWhat action Ishould do nowEnvivironment Need to develop agents – programs thattake the current percept as input from thesensors and return an action to theactuatorsactuators. The key challenge for AI is to find out howto write programs that, to the extentpossible, produce rational behavior from asmall amount of code.CISC4/681 Introduction to Artificial IntelligenceCISC4/681 Introduction to Artificial IntelligenceActuatorsCISC4/681 Introduction to Artificial Intelligence244

Simple Reflexive AgentSimply Reflexive Vacuum Agent Handles simplest kind of worldAgent embodies a set of condition-action rulesIf percept then actionAgent simply takes in a perceptpercept, determineswhich action could be applied, and does thataction. NOTE: Action dependent on current percept only Only works in fully observable environmentCISC4/681 Introduction to Artificial Intelligence Implements the agent function (describedin earlier table)25Model-Based Reflex Agent Upon getting a perceptSensorsWhat my actions doCondition action rulesAgentWhat action Ishould do now– Update the state (given the current state, theaction you just did, and the observations)– Choose a rule to apply (whose conditionsmatch the state)– Schedule the action associated with thechosen ruleEnvivironmentWhat the worldis like now26Model-Based Reflex AgentStateHow the world evolvesCISC4/681 Introduction to Artificial IntelligenceActuatorsCISC4/681 Introduction to Artificial Intelligence27CISC4/681 Introduction to Artificial IntelligenceGoal Based AgentUtility-Based AgentSensorsSensorsStateStateWhat it will be likeif I do action AAgentCISC4/681 Introduction to Artificial IntelligenceWhat action Ishould do nowHow the world evolvesWhat the worldis like nowWhat my actions doWhat it will be likeif I do action AUtilityHow happy I will bein such a stateWhat action Ishould do nowAgentActuators29CISC4/681 Introduction to Artificial IntelligenceEnvivironmentWhat my actions doEnvivironmentHow the world evolvesWhat the worldis like nowGoals28Actuators305

Learning Agent ComponentsLearning agents1. Learning Element – responsible for makingimprovements (on what ever aspect is beinglearned )2. Performance Element – responsible forselecting external actions. In previous parts,this was the entire agent!3. Critic – gives feedback on how agent is goingand determines how performance elementshould be modified to do better in the future4. Problem Generator – suggests actions for newand informative experiences.CISC4/681 Introduction to Artificial IntelligenceSummary Chapter 2Summary (cont) Agents interact with environments throughactuators and sensors The agent function describes what theagent does in all circumstances. TheTh performancefmeasure evaluateslthehenvironment sequence. A perfectly rational agent maximizesexpected performance. Agent programs implement (some) agentfunctions.CISC4/681 Introduction to Artificial Intelligence3233 PEAS descriptions define taskenvironments. Environments are categorized alongseveral dimensions:– Observable? Deterministic? Episodic? Static?Discrete? Single-agent? Several basic agent architectures exist:– Reflex, reflex with state, goal-based, utilitybasedCISC4/681 Introduction to Artificial Intelligence346

CISC4/681 Introduction to Artificial Intelligence 1 Russell and Norvig: 2 Agents? agent percepts sensors actions environment CISC4/681 Introduction to Artificial Intelligence 2 Agent – perceives the environment through sensors and acts on it through actuators Percept – agent’s perceptual input (the basis for its actions) Percept Sequence – complete history of what has been .