Introduction To Mean Field Games And Applications
IntroductionOur results and applicationsIntroduction toMean field games and applicationsWei YangDepartment of Mathematics and StatisticsUniversity of Strathclyde GlasgowWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsOutline1Introduction2Our results and applicationsWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsmean field game (MFG) theorya new branch of game theorydeveloped independently byLions, Lasry, Guéant (2006, 2007.)Caines, Huang, Malhamé (2005, 2006.)community meeting: June 2015 ParisWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsmean field game (MFG) theorya new branch of game theorydeveloped independently byLions, Lasry, Guéant (2006, 2007.)Caines, Huang, Malhamé (2005, 2006.)community meeting: June 2015 Parismaintaining a day to day interaction betweenmathematical research and real world applicationsThe co-founders include P.-L. Lions and J.-M. LasryCustomers include banks, energy councils, twitter (bigdata).Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applications”Large numbers are much simpler than small ones?” -Maybe!mean field game (MFG) theoryto study large (stochastic) dynamic gamesinspired by ideas from statistical particle physics (particlesare replace by agents with strategic interactions). to use the concept of mean fieldWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game Methodologyconsider an N-player stochastic dynamic gamestudy a mean field game (a limit for N ) which can beexpressed by a system of coupled equations: Fokker-Planck equation Hamilton-Jacobi-Bellman equationany solution to the mean field game is an -equilibrium tothe N-player game. an efficient decision-making process without paying toomuch attention to fine details of the system.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsAn N-player stochastic dynamic gameTo sense dynamics and costs.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsAn N-player stochastic dynamic gameT 0: a finite time horizon.U R: a compact control set.For i {1, . . . , N}, state dynamics {Zi (t), t 0} is described byN1XdZi (t) F (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1and the cost function is given asZJi (ui ) : E0TNh1 XNiL(t, Zi (t), ui (t), Zk (t)) dtk 1where F , L : [0, T ] R U R R. ui {ui (t) U, t 0}.Players are coupled through dynamics and costs.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsExample — a typical structure of interaction:dZi (t) N 1 X f t, Zi (t), ui (t) g t, Zk (t) dt σdWi (t)Nk 1andZJi (ui ) : E0TN h1 XN i l t, Zi (t), ui (t) h(t, Zk (t)) dtk 1where f , l : [0, T ] R U R and g, h : [0, T ] R R.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsExample — a typical structure of interaction:N 1Xg(t, Zi (t)) dt σdWi (t)dZi (t) f t, Zi (t), ui (t) Nj 1andZJi (ui ) : E0ThNi 1Xh(t, Zj (t)) dtl t, Zi (t), ui (t) Nj 1where f , l : [0, T ] R U R, g, h : [0, T ] R R.Dynamic and cost are closely related to its own states andcontrol, while receiving an impact of the population.To analyse Nash strategies {û1 , . . . , ûN }, full information isneeded!Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsThe smaller the number of variables is, the more explainary themodel is.To reduce the complexity by the concept of empirical measure.Empirical measure on RFor z(N) (z1 , z2 , . . . , zN ) RN define the empirical measureηzN(N) (B) N1Xδzi (B),N B B(R)i 1where δa is the Dirac measure at a R.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsDenote Z(N) (t) (Z1 (t), . . . , ZN (t)), for any t 0.N1XF (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1 ZNF (t, Zi (t), ui (t), y ) ηZ(N) (t) (dy ) dt σdWi (t) dZi (t) RWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsDenote Z(N) (t) (Z1 (t), . . . , ZN (t)), for any t 0.N1XF (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1 ZNF (t, Zi (t), ui (t), y ) ηZ(N) (t) (dy ) dt σdWi (t) dZi (t) RLet N and Z(t) (Z1 (t), . . . , ZN (t), . . . ).NAssume ηZ(t) : Z limN ηZ(N) (t) exits in weak sense, thendZi (t) F (t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt σdWi (t).RZT ZJi (ui ) E0 L(t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt.RWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsDenote Z(N) (t) (Z1 (t), . . . , ZN (t)), for any t 0.N1XF (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1 ZNF (t, Zi (t), ui (t), y ) ηZ(N) (t) (dy ) dt σdWi (t) dZi (t) RLet N and Z(t) (Z1 (t), . . . , ZN (t), . . . ).NAssume ηZ(t) : Z limN ηZ(N) (t) exits in weak sense, thendZi (t) F (t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt σdWi (t).RZT ZJi (ui ) E0 L(t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt.RUnderlying intuition: as N increases.the overall population’s behaviour (i.e. mean field)becomes relevant to a given agent’s dynamics.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsDenote Z(N) (t) (Z1 (t), . . . , ZN (t)), for any t 0.N1XF (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1 ZNF (t, Zi (t), ui (t), y ) ηZ(N) (t) (dy ) dt σdWi (t) dZi (t) RLet N and Z(t) (Z1 (t), . . . , ZN (t), . . . ).NAssume ηZ(t) : Z limN ηZ(N) (t) exits in weak sense, thendZi (t) F (t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt σdWi (t).RZT ZJi (ui ) E0 L(t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt.RηZ(t) (mean field) contains only statistical property of the massZ(t).Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsTo sense a mean field parameter.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsηZ(t) contains statistical property of Z(t) (Z1 (t), . . . , ZN (t), . . . )Let PZi (t) be the probability distribution of Zi (t). In thecontinuum limit N , Z dZi (t) F (t, Zi (t), ui (t), y ) PZi (t) (dy ) dt σdWi (t)RZT ZJi (ui ) E0R L(t, Zi (t), ui (t), y ) PZi (t) (dy ) dt.Underlying intuition: as N increases.at any time t 0, an individual’s distribution PZi (t) caneffectively represent the empirical distribution ηZ(t) .Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game (continuum limit for N )Model assumptionsplayers become infinitesimal and indistinguishable.the dynamics of the mass is the result of what a singleplayer doesplayers response to a mean-field.restating game theory as an interaction betweenindividual and the massWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game (limit for N )For a representative agent,The controlled dynamics (X (t), t 0)For a control policy u. (u(t) U, t 0),dX (t) f (X (t), u(t), µ(t))dt σdW (t)(1)where f : R U P(R) R.The solution is {(X (t), µ(t)), t 0} such that{X (t), t 0} is a solution to Eq. (1)µ(t) is the probability distribution of X (t) for any t 0.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game (limit for N )For a representative agent,The controlled dynamics (X (t), t 0)For a control policy u. (u(t) U, t 0),dX (t) f (X (t), u(t), µ(t))dt σdW (t) McKean-Vlasov(1)where f : R U P(R) R.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game (limit for N )For a representative agent,The controlled dynamics (X (t), t 0)For a control policy u. (u(t) U, t 0),dX (t) f (X (t), u(t), µ(t))dt σdW (t) McKean-Vlasov(1)where f : R U P(R) R.µ. {µ(t) P(R), t 0} represents the mean field.The cost functionFor a mean field µ. {µ(t) P(R), t 0},Z TJ(t, x, u., µ.) ExL(X (s), u(s), µ(s))ds(2)twhere L : R U P(R) R.To find a û. which is an optimal response to µ. and produces µ.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game (limit for N )The controlled dynamics (X (t), t 0)For a control policy u. (u(t) U, t 0),dX (t) f (X (t), u(t), µ(t))dt σdW (t) Fokker-Planck (FP) equation (forward Kolmogorov equation)The cost functionFor a mean field µ. {µ(t) P(R), t 0},Z TJ(t, x, u., µ.) ExL(x(s), u(s), µ(s))dst Hamilton-Jacobi-Bellman (HJB) equationWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game (limit for N )FP equationFor a control policy u. (u(t) U, t 0), σ2 d 2 dd m(t, x)m(t, x) f x, u(t), m(t, x) m(t, x) dtdx2 dx 2m(0, x) m0 (x).describe the aggregation of the action of all players.HJB equationFor a mean field µ. {µ(t) P(R), t 0}, V Vσ2 2V inff x, u(t), µ(t) L x, u(t), µ(t) t x2 x 2u(t) UV (T ,x) 0.the reaction of players to the massWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game (limit for N )FP equation - forward equationFor a control policy u. (u(t) U, t 0), σ2 d 2 dd m(t, x)m(t, x) f x, u(t), m(t, x) m(t, x) dtdx2 dx 2m(0, x) m0 (x).describe the aggregation of the action of all players.where the population actually end up, based on the initial dist.HJB equationFor a mean field µ. {µ(t) P(R), t 0}, V Vσ2 2V inff x, u(t), µ(t) L x, u(t), µ(t) t x2 x 2u(t) UV (T ,x) 0.the reaction of players to the massWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game (limit for N )FP equation - forward equationFor a control policy u. (u(t) U, t 0), σ2 d 2 dd m(t, x)m(t, x) f x, u(t), m(t, x) m(t, x) dtdx2 dx 2m(0, x) m0 (x).describe the aggregation of the action of all players.where the population actually end up, based on the initial dist.HJB equation - backward equationFor a mean field µ. {µ(t) P(R), t 0}, V Vσ2 2V inff x, u(t), µ(t) L x, u(t), µ(t) t x2 x 2u(t) UV (T ,x) 0.the reaction of players to the mass.decisions based on where you want to be in the futureWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMean field game (limit for N )Mean field equations - coupled system of two equationsFP equation - forward equationFor a control policy u. (u(t) U, t 0), σ2 d 2 dd m(t, x) f x, u(t), m(t, x) m(t, x) m(t, x)dtdx2 dx 2m(0, x) m0 (x)û(t) µ(t)(dx) m(t, x)dxHJB equation - backward equationFor a mean field µ. {µ(t) P(R), t 0}, V Vσ2 2V inff x, u(t), µ(t) L x, u(t), µ(t) t x2 x 2u(t) UV (T , x) 0.Let û(t) û(t, x, µ.) be the best response to the mean field.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsOutline1Introduction2Our results and applicationsWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsJoint work with Vassili Kolokoltsov at Warwick UniversityWe developed a unified framework where a larger class ofMarkov processes is considered The dynamic of the N players Z N (t) RN : t [0, T ] isassociated to a family of linear and bounded operatorsnoA[t, µ, u] L(C2 , C) : t [0, T ], µ P(Rd ), u U .Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsFor each (t, µ, u) [0, T ] P(Rd ) U, A[t, µ, u] : C2 7 C isassumed to generate a Feller process with values in Rd and tobe of the formA[t, µ, u]f (z) (h(t, z, µ, u), f (z)) R[t, µ]f (z)h : [0, T ] Rd P(Rd ) U RdR[t, µ] L(C2 , C) is of the form:1R[t, µ]f (z) (G(t, z, µ) , )f (z) (b(t, z, µ), f (z))2Z (f (z y ) f (z) ( f (z), y)1B1 (y ))ν(t, z, µ, dy ).RdWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsFor each (t, µ, u) [0, T ] P(Rd ) U, A[t, µ, u] : C2 7 C isassumed to generate a Feller process with values in Rd and tobe of the formA[t, µ, u]f (z) (h(t, z, µ, u), f (z)) R[t, µ]f (z)ExampleIf R[t, µ] 12 σ 2 with a constant σ, i.e,A[t, µ, u]f (z) (h(t, z, µ, u), f (z)) 12 σ 2 f (z), Z N (t) : t [0, T ] can also be described by the SDEdZ N (t) h(t, Z N (t), µt , ut ) dt σdWt .Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsOptimal control problem for each player.Given {η N (t) : t [0, T ]}, the optimal payoff of i-th player,i [1, N], starting at x and t is"Z#TVi,N (t, x) inf Exu.tNNL(s, Zi,s, ηsN , ui,s ) ds V T (Zi,T)L : [0, T ] Rd P(Rd ) U RV T : Rd RWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsFormally, if ηtN µt and Vi,N V as N such thatCoupled forward-backward equationsforward kinetic equation in weak form:(ddt (g, µt ) (A[t, µt , Γ(t, ., {µs : t s T })]g, µt ) (FE)µ0 µbackward HJB equation: V (t,x) Ht (x, V (t, x), µt ) R[t, µt ]V (t, x) 0 (BE) t Ht (x, p, µ) infu U (h(t, x, µ, u)p L(t, x, µ, u)) V VTTV (t, x) Γ(t, x, {νt : t [0, T ]}) (Xt : t [0, T ]) {µt : t [0, T ]}{νt : t [0, T ]}Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsFormally, if ηtN µt and Vi,N V as N such thatCoupled forward-backward equationsforward kinetic equation in weak form:(ddt (g, µt ) (A[t, µt , Γ(t, ., {µs : t s T })]g, µt ) (FE)µ0 µbackward HJB equation: V (t,x) Ht (x, V (t, x), µt ) R[t, µt ]V (t, x) 0 (BE) t Ht (x, p, µ) infu U (h(t, x, µ, u)p L(t, x, µ, u)) V VTTV (t, x) Γ(t, x, {νt : t [0, T ]}) (Xt : t [0, T ]) {µt : t [0, T ]}{νt : t [0, T ]}fixed point, consistencyWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsTheorem 1 (KY2013): mean field limitFor arbitrary T 0, there exists a solution to the Cauchyproblem for (FE) d (g, µt ) (A[t, µt , Γ(t, ., {µs : t s T })]g, µt )dt µ µ.0For T small enough, the solution is unique.Requirement: Γ has feedback regularity property.V. Kolokoltsov, W.Yang (2013). Existence of Solutions toPath-Dependent Kinetic Equations and Related Forward - BackwardSystems, Open Journal of Optimization, 2, 39-44.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsTheorem 2 (KY2013): optimal control(a) For a given flow {νt : t [0, T ]}, the Cauchy problem V (t,x) Ht (x, V (x), νt ) R[t, νt ]V (t, x) 0 tHt (x, p, µ) infu U (h(t, x, µ, u)p L(t, x, µ, u)) V T V Tis wellposed;Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsTheorem 2 (KY2013): optimal control - sensitivity(b) For any {νt1 : t [0, T ]}, {νt2 : t [0, T ]}, there exists k 0such thatsup kV (t, ·; {ν.1 }) V (t, ·; {ν.2 })kC1 k sup kνt1 νt2 k(C2 ) t [0,T ]t [0,T ]andsup k V (t, ·; {ν.1 }) V (t, ·; {ν.2 })kC k sup kνt1 νt2 k(C2 ) ;t [0,T ]t [0,T ]Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsTheorem 2 (KY2013): optimal control - feedback regularity(c) The unique optimal control function Γ(t, x; {νt : t [0, T ]})denote has the feedback regularity required by Theorem 1,i.e. for {νt1 : t [0, T ]}, {νt2 : t [0, T ]},sup Γ(t, x; {ν 1 .}) Γ(t, x; {ν 2 .}) k1 sup νs1 νs2 (C2 ) t,xs [0,T ]with some constant k1 0.prove, rather than assume, the feedback regularity propertyV. Kolokoltsov, W.Yang (2013). Sensitivity analysis for HJB equationswith an application to a coupled backward-forward system, submitted.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsTheorem 3 (KY2012): convergenceThe N-player games converge to the limit system, i.e.NµNt (µ0 ) µt (µ0 ) as N , t [0, T ]with a convergence rate O(1/N), where µt is a solution to (FE).Technical issue: smoothness of the solutions to equation (FE)with respect to initial data µ0 .Improvement to the existing models: convergence rate 1/N,instead of 1/ N. Kolokoltsov, Li, Yang (2012). Mean Field Games and NonlinearMarkov Processes, 63 pages, arXiv:1112.3744v. Kolokoltsov, Troeva, Yang (2014). On the rate of convergence forthe mean field approximation of controlled diffusions with largenumber of payers. Dynamic Games and Applications, 4, 208-230.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsDefinition: Nash equilibriumFor 0, a strategy profile Γ is a -Nash equilibrium ifJi (Γ) Ji (Γ i , ui ) , i 1, . . . , Nwhere (Γ i , ui ) denotes the profile obtained from Γ bysubstituting the strategy of player i with any eligible strategy ui .Theorem 4 (KY2012): Nash equilibriumAny solution of the limit modelΓ(t, x, {µs : t s T })represents an -equilibrium for an N players dynamic game,with O(1/N).In a general setting, solutions are not in a closed-form.V.Kolokoltsov, J. Li, W. Yang (2012). Mean Field Games andNonlinear Markov Processes, 63 pages, arXiv:1112.3744v.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsA setting with a major player:Inspection games in mean field settingconsider one inspector with N inspecteesintroduction of a deterministic major playerMarkov Chain on a finite state space {l1 , . . . , ld } (crimelevels)obtain a convergence result without a convergence rateKolokoltsov, Yang (2014). Inspection games in mean field setting, inpreparationWei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsMFG theory attracts much attention from Mathematical society.Other developments of MFGcooperative populationstability of the limit system over infinite horizonwith a stochastic major playermean field type controls···Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsReferences:J.-M. Lasry and P.-L. Lions (2007). Mean field games. JapaneseJournal of Mathematics, 2:1, 229-260.Guéant O., Lasry J.-M., Lions P.-L. (2010). Mean Field Gamesand Applications. Paris-Princeton Lectures on MathematicalFinance, Springer.Huang M., Caines P. E., Malhamé R. P. (2006). Large populationstochastic dynamic games: closed-loop Mckean-Vlasov systemsand the Nash certainty equivalence principle. Communicationsin information and systems, 6, 221-252.Gomes G., Saúde J. (2014). Mean field games models - a briefsurvey, Dynamic Games and Applications, 4:2, 110-154.Bensoussan A, Frehse J., Yam P. (2013). Mean field games andmean field type control. Springer, Berlin.Nguyen S. and Huang M. (2012). Mean Field LQG Games withMass Behavior Responsive to A Major Player. 51st IEEEConference on Decision and Control.Wei YangIntroduction to Mean field games and applications
IntroductionOur results and applicationsKolokoltsov V. , Li J, Yang W. (2011). Mean Field Games andNonlinear Markov Processes. arXiv:1112.3744vKolokoltsov V. Yang W. (2013). Existence of solutions topath-dependent kinetic equations and related forward-backwardsystems, Open J Optimisation, 2, 39-44.Kolokoltsov V. Yang W. (2013). Sensitivity analysis for HJBequation with an application to a couple backward-forwardsystem, arXiv:1303.6234v1.Kolokoltsov V. Troeva M. Yang W. (2014). On the
Wei Yang Introduction to Mean field games and applications. Introduction Our results and applications Mean field game Methodology consider an N-playerstochastic dynamic game study amean field game(a limit for N !1) which can be expressed bya system of coupled equations: Fokker-Planck equation Hamilton-Jacobi-Bellman equation any solution to the mean field game is an -equilibriumto the N .
Outline: Demystifying Mean Field Approximation 1 Construction of the Mean Field Approximation: 3 models Density Dependent Population Processes A Second Point of View: Zoom on One Object Discrete-Time Models 2 On the Accuracy of Mean Field : Positive and Negative Results Transient Analysis Steady-state Regime 3 The Re ned Mean Field Main Results
The Games organised at Olympia led to the development of the Panhellenic Games. These included: - The Games at Olympia (Olympic Games): every four years - The Games at Delphi (Pythian Games), 582 B.C.: every four years (third year of each Olympiad) - The Games at the Isthmus of Corinth (Isthmian Games), from 580 B.C.:
Section 3: Playground Markings Games 16 Section 4: Skipping, Hula Hoop & Elastics 25 Section 5: Catching games 32 Section 6: Relay games 41 Section 7: Ball games 48 Section 8: Fun games 59 Section 9: Frisbee games 66 Section 10: Parachute games 70 Section 11: Clapping and rhyming games 74 Useful websites 79
Olympic Winter Games medals Olympic Winter Games posters Olympic Summer Games posters Olympic Summer Games mascots Olympic Winter Games mascots The sports pictograms of the Olympic Summer Games The sports pictograms of the Olympic Winter Games The IOC, the Olympic Movement and the Olympic Games The Olympic programme evolution Torches and torch .
Regional Games and Multi-Sport Games (such as Pan American Games, African Games, European Games, Commonwealth Games, Mediterranean Games, Francophone Games, Youth Olympic Games) International Tournaments organised by the IJF (Grand Prix, Grand Slam, Masters) or under its auspices (continental open and cups),
There are five averages. Among them mean, median and mode are called simple averages and the other two averages geometric mean and harmonic mean are called special averages. Arithmetic mean or mean Arithmetic mean or simply the mean of a variable is defined as the sum of the observations divided by the number of observations.
Olympic Summer Games posters Olympic Summer Games mascots Olympic Winter Games mascots The IOC, the Olympic Movement and the Olympic Games The Olympic programme evolution The Olympic stadiums of the Summer Games The sports pictograms of the Olympic Summer Games The sports pictograms of the Olympic Winter Games .
KOREAN LANGUAGE PROGRAM (9 credits, Fall & Winter 2 semester course) FIRST LEVEL KOREAN Courses offered. This course is a continuation of First Level Korean.The goal is to give students necessary tools to speak, read and write Korean fluently. Continuing learning further sentence structures, we will also focus on contextual aspects. Special attention is given on using and recognizing minimal .
