Game Theory And Cyber War: Paradigms For Understanding Human Decisions .

Game Theory and Cyber War:Paradigms for Understanding HumanDecisions in Cyber SecurityCoty Gonzalez (Carnegie Mellon University)In collaboration with: Noam Ben-Asher, Ph.D.Post-Doctoral Fellow – CMU; Now: Post-Doctoral Researcher – ARL1

Research Objectives– To establish a theoretical model of decision making in cybersecurity situations that answers questions such as: How do humans recognize and process possible threats? How do humans recognize, process and accumulate information to makedecisions regarding to cyber-defense? How do human risk perception and tendencies to perceive rewards andlosses influence their decisions in cyber-defense?– To provide a computational cognitive model of human decisionmaking in cyber-security situations that: Addresses challenges of cyber-security while accounting for human cognitivelimitations Provide concrete measures of a human’s decision making and behavior Suggest approaches to investigate courses of action and the effectiveness ofdefense strategies according to the dynamics of cyber-security situations.2

Research Approach Laboratory Experiments: Cognitive Modeling:Involves comparison of datafrom: computational cognitive– E.g., The “IDS security game”: Studythe dynamic process of decisions from models and from humans, bothperforming the same taskexperience– Computational representations ofhuman experiential judgment anddecision making process– Based on Instance-Based LearningTheory (IBLT, Gonzalez et al., 2003)– E.g., IBL models of stoppingdecisions: dynamic accumulation ofevidence before an attack is declared3

From individual to network behaviorModeling detection with InstanceBased Learning Theory (Dutt, Ahn,Gonzalez, 2011, 2012)From Individual Decisionsfrom Experience toBehavioral Game Theory:Lessons for Cyber Security(Gonzalez, 2013)Individual (Defender).Cognitive theories, Memory andindividual behaviorDefenderPair (Defender and Attacker).Interdependencies, Information,Behavioral Game TheoryDefenderAttackerPerspectives from CognitiveEngineering on CyberSecurity. (Cooke et al.,2012).The Cyber Warfare SimulationEnvironment and Multi-AgentModels (Ben-Asher, Rajivan,Cooke & Gonzalez, 2014;Cyber War: multiple attackersBen-Asher & Gonzalez, inDefendersPrep).Network (Multiple Defendersand Attackers).Behavioral Network Theory;Network science (& topology)Organizational Learning;Group Dynamics; Politicaland Social Science4

Experimental paradigms.Individual LevelDefenderIDS ToolRepeated Decisions fromExperienceMain behavioral results in: Ben-Asher & Gonzalez, 20145

Experimental paradigms.Pair LevelDefenderAttackerGame Theory 2x2 GamesPrisoner’s DilemmaChicken DilemmaPlayer 2 ActionDDC-1, -110, -10Player 2 ActionDCD-10, -1010, -1C-1, 101, 1Player 1 ActionPlayer 1 ActionC-10, 101, 1Repeated Decisionsfrom Experiencesimultaneous and sequential gamesMain behavioral results in:Gonzalez, Ben-Asher,Martin & Dutt, 20146

Experimental paradigms.Network LevelCyber War: multiple attackers/Defenders N players – Each player makes decisionswhether to: Attack, Defend, do Nothingagainst each of the other playersEach player is characterized by two essentialattributes:– Power– AssetsDecisions are led by the goal of maximizingown assets.Multi-round game.Decisions result in an Outcome (Gain orLoss) which changes the Assets available inthe following round.Actions have a cost: Cost of attack, cost ofdefend, cost of doing nothing is zeroRepeated Decisions fromExperience7

The Role of Power and Assets Power represents capabilities and abilities:– Investment in cyber infrastructure (e.g., computational power); Knowledge andsophistication (e.g., zero-day exploit); Vulnerabilities– The ability to execute an action successfully. successfully defend against an attack or successfully execute an attacksagainst other players– 𝑝 𝑠𝑢𝑐𝑐𝑒𝑠𝑠 𝑖 𝑃𝑜𝑤𝑒𝑟𝑖𝑃𝑜𝑤𝑒𝑟𝑖 𝑃𝑜𝑤𝑒𝑟𝑗Assets are the currency for maximization– A players’ goal is to maximize his/her own assets– An action results in obtaining (losing) a percentage g of Assets– The outcome in round t changes the value of Assets available in the nextround t 1– Assets are needed to be part of a war: there are costs (C) to attack and todefend (D)– A player with no assets is suspended for a fixed number of rounds (r)

Actions and Outcomes (Player i, Player j, change in Assets)Player j Action𝑂𝐴𝑖𝑗 𝑝(𝑠𝑢𝑐𝑐𝑒𝑠𝑠)𝑖 𝑔 𝐴𝑠𝑠𝑒𝑡𝑠𝑗 𝐶ADNOAijOAijOAijPlayer ONDijONNijOAjiODjiONNji𝑂𝐷𝑖𝑗 𝑝(𝑠𝑢𝑐𝑐𝑒𝑠𝑠)𝑗 𝑔 𝐴𝑠𝑠𝑒𝑡𝑠𝑖 𝐷A𝑂𝑁𝐴𝑖𝑗 𝑝(𝑠𝑢𝑐𝑐𝑒𝑠𝑠)𝑗 𝑔 𝐴𝑠𝑠𝑒𝑡𝑠𝑖D𝑂𝑁𝐷𝑖𝑗 0N𝑂𝑁𝑁𝑖𝑗 0

Dynamic Decision TheoryInstance-Based Learning Theory (IBLT)(Gonzalez, Lerch, & Lebiere, 2003) Proposes a generic DDMcognitive process:Recognition, Judgment, Choice,Execution, Feedback Formalizesrepresentations: Instance: tripled: Situation,Decision, Utility (SDU) Relies on mathematicalmechanisms proposed by ACT-R Represents processescomputationally: to provideconcrete predictions of humanbehavior in various task types

IBL model of choice: Individual1. Each experience combination iscreated as an instance in memory(e.g. A-10; N-8; A-1; N-5; A-5) whenthe outcome is experienced2. Each instance has a memory“activation” value based onfrequency, recency, similarity, etc.3. The probability of retrieving aninstance from memory depends onactivation4. For each option, memory instancesare “blended” to determine nextchoice by combining value andprobability5. Choose the option with themaximum blended valueNA10108155 .11

A formalization of an IBL model(Gonzalez & Dutt, 2011; Lejarraga et al., 2012)Defender1. Each Instance has an Activation: simplification of ACT-R’s mechanism (Anderson &Lebiere, 1998):FrequencyFree parameters:Recencyd : high d- More recencyNoise: s : high s - high variability2. Each Instance has a probability of retrieval is a function of memory Activation (A) of thatoutcome relative to the activation of all the observed outcomes for that option given by:3. Each Option has a Blended Value that combines the probability of retrieval and outcomeof the instances:4. Choose the option with the highest experienced expected value (“blended” value)12

Instance-Based Learning ModelPair LevelGonzalez, Ben-Asher, Martin & Dutt, 2014DefenderAttackerIBL-PDGame Theory 2x2GamesPrisoner’s Dilemma – An instance includes both players’ actions and outcomes[C, D, -10, 10], [C, C, 1, 1], [D, C, 10, -10], and [D, D, -1, 1]Player 2 ActionPlayer 1ActionDCDC-1, -110, -10-10, 101, 1Experiential & Descriptive Adding the “other” outcome to the blendingequation: And how do humans weigh the “other”information into their own decisions? (w f(t))?– Dynamic adaptation of expectations– Surprise is a function of the gap between the expectedoutcome and the outcomes actually received:

Predictions against human dataMain behavioral results in: Gonzalez, Ben-Asher, Martin & Dutt, 201414

Fitting the model’s parameters to data15

Instance-Based LearningNetwork LevelCyber War: multiple attackers/Defenders Each active agent evaluates the otheractive agents, one at a time Each active agent is evaluated bycalculating the possible outcome fromattacking it Then the agent evaluates how likely itis to actually obtain that outcome Each agent selects to attack the agentthat would yield the highest utility ofattacking Makes a decision whether to attack ornot, according to the highest blendedvalue of the two types of actions“attack” or “no attack”

Simulations and Results A network with 9 different types agents– Power (High, Medium, Low)– Asset Value (High, Medium, Low) Each network was simulated for 2500 trials. 60 simulations with the same network setting. Successful attack yields 20% of the opponent'sassets Downtime - An agent without assets issuspended for 10 trials IBL Agents with d 5 and σ 0.2517

Active Agents in the Network Within 500 trials the number of active agents becomes stable(mean 6.42, SD 0.16)Power influenced the overall proportion time agents were suspended:– High power agents 2% of the trials– Medium power agent 19% of the trials– Low power agents 50% of the trials High power allowed agents to maintain an active state, however evenhigh power did not guaranty that an agent will be active 100% of thetime

Role of Power over dynamics of AssetsPower influenced the dynamics of agents’ state and the network heterogeneity

Power and Assets Accumulation High power allowed accumulation of assets starting from earlystages of the interactionThe difference between Medium and Low power agents was evidentonly after 500 trialsThe relationship between accumulated assets and power is notlinear

Conclusions– Significant progress in the development of theoretical models of decisionmaking in cyber-security situations. Theoretical models evolved from Individual (Instance-Based Learning Theory) Pair-level (Behavioral Game Theory and IBL-Game Theory) Network Level (Network Theory and IBL-Network)– Development of experimental paradigms that served to collect humandata and conclude with behavioral phenomena: IDS tool, Binary choice repeated decisions, Game theory games, CyberWargame– Development of computational cognitive models based on theoreticaldevelopments including IBL model IBL-PD Cyber War simulations

Pair-level (Behavioral Game Theory and IBL-Game Theory) Network Level (Network Theory and IBL-Network) - Development of experimental paradigms that served to collect human data and conclude with behavioral phenomena: IDS tool, Binary choice repeated decisions, Game theory games, CyberWar game