Extensive Form Games - MIT OpenCourseWare

Extensive Form GamesMihai ManeaMIT

Extensive-Form GamesIIIIIIIN: finite set of players; nature is player 0 Ntree: order of movespayoffs for every player at the terminal nodesinformation partitionactions available at every information setdescription of how actions lead to progress in the treerandom moves by natureCourtesy of The MIT Press. Used with permission.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 20162 / 33

Game TreeI(X , ): treeIX : set of nodesIx y: node x precedes node yIφ X : initial node, φ x , x X \ {φ}I transitive (x y , y z x z) and asymmetric (x y y x)Ievery node x X \ {φ} has one immediate predecessor: x 0 x s.t.x 00 x & x 00 , x 0 x 00 x 0IZ {z @x , z x }: set of terminal nodesIz Z determines a unique path of moves through the tree, payoffui (z ) for player iMihai Manea (MIT)Extensive-Form GamesMarch 2, 20163 / 33

Information PartitionIinformation partition: a partition of X \ ZInode x belongs to information set h (x )Iplayer i (h ) N moves at every node x in information set hIi (h ) knows that he is at some node of h but does not know which oneIsame player moves at all x h, otherwise players might disagree onwhose turn it isIi (x ) : i (h (x ))Mihai Manea (MIT)Extensive-Form GamesMarch 2, 20164 / 33

ActionsIA (x ): set of available actions at x X \ Z for player i (x )IA (x ) A (x 0 ) : A (h ), x 0 h (x ) (otherwise i (h ) might play aninfeasible action)Ieach node x , φ associated with the last action taken to reach itIevery immediate successor of x labeled with a different a A (x ) andvice versaImove by nature at node x: probability distribution over A (x )Mihai Manea (MIT)Extensive-Form GamesMarch 2, 20165 / 33

StrategiesIHi {h i (h ) i }ISi Isi (h ): action taken by player i at information set h Hi under si SiIS IA strategy is a complete contingent plan specifying the action to betaken at each information set.IMixed strategies: σi (Si )Imixed strategy profile σ O (σ) (Z )Iui (σ) EO (σ) (ui (z ))Qh HiQi NA (h ): set of pure strategies for player iSi : strategy profilesMihai Manea (MIT)Qi N (Si ) probability distributionExtensive-Form GamesMarch 2, 20166 / 33

Strategic FormIThe strategic form representation of the extensive form game is thenormal form game defined by (N , S , u)IA mixed strategy profile is a Nash equilibrium of the extensive formgame if it constitutes a Nash equilibrium of its strategic form.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 20167 / 33

Grenade Threat GamePlayer 2 threatens to explode a grenade if player 1 doesn’t give him 1000.IPlayer 1 chooses between g and g.IPlayer 2 observes player 1’s choice, then decides whether to explodea grenade that would kill both.2g,1A g2Mihai Manea (MIT)A,Extensive-Form Games( , )( 1000, 1000)( , )(0, 0)March 2, 20168 / 33

Strategic Form Representation2g2AA( 1000, 1000)A g, , , ( , ),1g gAA,, , 0, 0 ( , )(0, 0),A,, 1000, 1000 , ,, , 1000, 10000, 0 Three pure strategy Nash equilibria. Only ( g , ,, ,) is subgame perfect.A is not a credible threat.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 20169 / 33

Behavior StrategiesIbi (h ) (A (h )): behavior strategy for player i (h ) at information set hIbi (a h ): probability of action a at information set hIbehavior strategy bi Iindependent mixing at each information setIbi outcome equivalent to the mixed strategyQh Hi (A (h ))σi (si ) Ybi (si (h ) h )(1)h HiIIs every mixed strategy equivalent to a behavior strategy?IYes, under perfect recall.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201610 / 33

Perfect RecallNo player forgets any information he once had or actions he previouslychose.IIf x 00 h (x 0 ), x x 0 , and the same player i moves at both x and x 0(and thus at x 00 ), then there exists x̂ h (x ) (possibly x̂ x) s.t.x̂ x 00 and the action taken at x along the path to x 0 is the same asthe action taken at x̂ along the path to x 00 .Ix 0 and x 00 distinguished by information i does not have, so he cannothave had it at h (x )Ix 0 and x 00 consistent with the same action at h (x ) since i mustremember his action thereIEquivalently, every node in h Hi must be reached via the samesequence of i’s actions.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201611 / 33

Equivalent Behavior StrategiesIRi (h ) {si h is on the path of (si , s i ) for some s i }: set of i’s purestrategies that do not preclude reaching information set h HiIUnder perfect recall, a mixed strategy σi is equivalent to a behaviorstrategy bi defined byσi (si )Pbi (a h ) {si Ri (h ) si (h ) a }P(2)σi (si )si Ri (h )when the denominator is positive.Theorem 1 (Kuhn 1953)In extensive form games with perfect recall, mixed and behavior strategiesare outcome equivalent under the formulae (1) & (2).Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201612 / 33

ProofIh1 , . . . , hk̄ : player i’s information sets preceding h in the treeIUnder perfect recall, reaching any node in h requires i to take thesame action ak at each hk ,Ri (h ) {si si (hk ) ak , k 1, k̄ }.IConditional on getting to h, the distribution of continuation play at h isgiven by the relative probabilities of the actions available at h underthe restriction of σi to Ri (h ),σi (si )Pbi (a h ) {si si (hk ) ak , k 1,k̄ & si (h ) a }Pσi (si ).{si si (hk ) ak , k 1,k̄ }Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201613 / 33

ExampleCourtesy of The MIT Press. Used with permission.Figure: Different mixed strategies can generate the same behavior strategy.IS2 {(A , C ), (A , D ), (B , C ), (B , D )}IBoth σ2 1/4(A , C ) 1/4(A , D ) 1/4(B , C ) 1/4(B , D ) andσ2 1/2(A , C ) 1/2(B , D ) generate—and are equivalent to—thebehavior strategy b2 with b2 (A h ) b2 (B h ) 1/2 andb2 (C h 0 ) b2 (D h 0 ) 1/2.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201614 / 33

Example with Imperfect RecallCourtesy of The MIT Press. Used with permission.Figure: Player 1 forgets what he did at the initial node.IS1 {(A , C ), (A , D ), (B , C ), (B , D )}Iσ1 1/2(A , C ) 1/2(B , D ) b1 (1/2A 1/2B , 1/2C 1/2D )b1 not equivalent to σ1(σ1 , L ): prob. 1/2 for paths (A , L , C ) and (B , L , D )(b1 , L ): prob. 1/4 to paths (A , L , C ), (A , L , D ), (B , L , C ), (B , L , D )IIIMihai Manea (MIT)Extensive-Form GamesMarch 2, 201615 / 33

Imperfect Recall and Correlations&RXUWHV\ RI 7KH 0,7 3UHVV 8VHG ZLWK SHUPLVVLRQ ISince both A vs. B and C vs. D are choices made by player 1, thestrategy σ1 under which player 1 makes all his decisions at onceallows choices at different information sets to be correlatedIBehavior strategies cannot produce this correlation, because when itcomes time to choose between C and D, player 1 has forgottenwhether he chose A or B.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201616 / 33

Absent Minded DriverPiccione and Rubinstein (1997)IA drunk driver has to take the third out of five exits on the highway(exit 3 has payoff 1, other exits payoff 0).IThe driver cannot read the signs and forgets how many exits he hasalready passed.IAt each of the first four exits, he can choose C (continue) or E(exit). . . imperfect recall: choose same action.IC leads to exit 5, while E leads to exit 1.IOptimal solution involves randomizing: probability p of choosing Cmaximizes p 2 (1 p ), so p 2/3.I“Beliefs” given p 2/3: (27/65, 18/65, 12/65, 8/65)IE has conditional “expected” payoff of 12/65, C has 0. Optimalstrategy: E with probability 1, inconsistent.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201617 / 33

ConventionsIRestrict attention to games with perfect recall, so we can use mixedand behavior strategies interchangeably.IBehavior strategies are more convenient.IDrop notation b for behavior strategies and denote by σi (a h ) theprobability with which player i chooses action a at information set h.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201618 / 33

SurvivorTHAI 21ITwo players face off in front of 21 flags.IPlayers alternate in picking 1, 2, or 3 flags at a time.IThe player who successfully grabs the last flag wins.Game of luck?Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201619 / 33

Backward InductionIAn extensive form game has perfect information if all information setsare singletons.ICan solve games with perfect information using backward induction.IFinite game penultimate nodes (successors are terminal nodes).IThe player moving at each penultimate node chooses an action thatmaximizes his payoff.IPlayers at nodes whose successors are penultimate/terminal choosean optimal action given play at penultimate nodes.IWork backwards to initial node. . .Theorem 2 (Zermelo 1913; Kuhn 1953)In a finite extensive form game of perfect information, the outcome(s) ofbackward induction constitutes a pure-strategy Nash equilibrium.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201620 / 33

Market EntranceIIncumbent firm 1 chooses a level of capital K1 (which is then fixed).IA potential entrant, firm 2, observes K1 and chooses its capital K2 .IThe profit for firm i 1, 2 is Ki (1 K1 K2 ) (firm i produces output Ki ,we use earlier demand function).IEach firm dislikes capital accumulation by the other.IA firm’s marginal value of capital decreases with the other’s.ICapital levels are strategic substitutes.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201621 / 33

Stackelberg CompetitionIProfit maximization by firm 2 requiresK2 I1 K1.2Firm 1 anticipates that firm 2 will act optimally, and therefore solves(max K1K11 K11 K1 2!).ISolution involves K1 1/2, K2 1/4, π1 1/8, and π2 1/16.IFirm 1 has first mover advantage.IIn contrast, in the simultaneous move game, K1 1/3, K2 1/3,π1 1/9, and π2 1/9.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201622 / 33

Centipede GameIIIIIPlayer 1 has two piles in front of her: one contains 3 coins, the other1.Player 1 can either take the larger pile and give the smaller one toplayer 2 (T ) or push both piles across the table to player 2 (C).Every time the piles pass across the table, one coin is added to each.Players alternate in choosing whether to take the larger pile (T ) ortrust opponent with bigger piles (C).The game lasts 100 rounds.What’s the backward induction solution?C12C1TTT(3, 1)(2, 4)(5, 3)Mihai Manea (MIT)C21TC2C(103, 101)T(101, 99) (100, 102)Extensive-Form GamesMarch 2, 201623 / 33

Chess Players and Backward InductionPalacios-Huerta and Volij (2009)Ichess players and college students behave differently in thecentipede game.IHigher-ranked chess players end the game earlier.IAll Grandmasters in the experiment stopped at the first opportunity.IChess players are familiar with backward induction reasoning andneed less learning to reach the equilibrium.IPlaying against non-chess-players, even chess players continue inthe game longer.IIn long games, common knowledge of the ability to do complicatedinductive reasoning becomes important for the prediction.Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201624 / 33

Subgame PerfectionIBackward induction solution is more than a Nash equilibrium.IActions are optimal given others’ play—and form anequilibrium—starting at any intermediate node: subgameperfection. . . rules out non-credible threats.ISubgame perfection extends backward induction to imperfectinformation games.IReplace “smallest” subgames with a Nash equilibrium and iterate onthe reduced tree (if there are multiple Nash equilibria in a subgame,all players expect same play).Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201625 / 33

SubgamesSubgame: part of a game that can be analyzed separately; strategicallyand informationally independent. . . information sets not “chopped up.”Definition 1A subgame G of an extensive form game T consists of a single node xand all its successors in T , with the property that if x 0 G and x 00 h (x 0 )then x 00 G. The information sets, actions and payoffs in the subgame areinherited from T .Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201626 / 33

False Subgames&RXUWHV\ RI 7KH 0,7 3UHVV 8VHG ZLWK SHUPLVVLRQ Mihai Manea (MIT)Extensive-Form GamesMarch 2, 201627 / 33