3y ago

112 Views

2 Downloads

901.83 KB

44 Pages

Transcription

IntroductionOur results and applicationsIntroduction toMean field games and applicationsWei YangDepartment of Mathematics and StatisticsUniversity of Strathclyde GlasgowWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsOutline1Introduction2Our results and applicationsWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsmean field game (MFG) theorya new branch of game theorydeveloped independently byLions, Lasry, Guéant (2006, 2007.)Caines, Huang, Malhamé (2005, 2006.)community meeting: June 2015 ParisWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsmean field game (MFG) theorya new branch of game theorydeveloped independently byLions, Lasry, Guéant (2006, 2007.)Caines, Huang, Malhamé (2005, 2006.)community meeting: June 2015 Parismaintaining a day to day interaction betweenmathematical research and real world applicationsThe co-founders include P.-L. Lions and J.-M. LasryCustomers include banks, energy councils, twitter (bigdata).Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applications”Large numbers are much simpler than small ones?” -Maybe!mean field game (MFG) theoryto study large (stochastic) dynamic gamesinspired by ideas from statistical particle physics (particlesare replace by agents with strategic interactions). to use the concept of mean fieldWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game Methodologyconsider an N-player stochastic dynamic gamestudy a mean field game (a limit for N ) which can beexpressed by a system of coupled equations: Fokker-Planck equation Hamilton-Jacobi-Bellman equationany solution to the mean field game is an -equilibrium tothe N-player game. an efficient decision-making process without paying toomuch attention to fine details of the system.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsAn N-player stochastic dynamic gameTo sense dynamics and costs.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsAn N-player stochastic dynamic gameT 0: a finite time horizon.U R: a compact control set.For i {1, . . . , N}, state dynamics {Zi (t), t 0} is described byN1XdZi (t) F (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1and the cost function is given asZJi (ui ) : E0TNh1 XNiL(t, Zi (t), ui (t), Zk (t)) dtk 1where F , L : [0, T ] R U R R. ui {ui (t) U, t 0}.Players are coupled through dynamics and costs.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsExample — a typical structure of interaction:dZi (t) N 1 X f t, Zi (t), ui (t) g t, Zk (t) dt σdWi (t)Nk 1andZJi (ui ) : E0TN h1 XN i l t, Zi (t), ui (t) h(t, Zk (t)) dtk 1where f , l : [0, T ] R U R and g, h : [0, T ] R R.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsExample — a typical structure of interaction:N 1Xg(t, Zi (t)) dt σdWi (t)dZi (t) f t, Zi (t), ui (t) Nj 1andZJi (ui ) : E0ThNi 1Xh(t, Zj (t)) dtl t, Zi (t), ui (t) Nj 1where f , l : [0, T ] R U R, g, h : [0, T ] R R.Dynamic and cost are closely related to its own states andcontrol, while receiving an impact of the population.To analyse Nash strategies {û1 , . . . , ûN }, full information isneeded!Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsThe smaller the number of variables is, the more explainary themodel is.To reduce the complexity by the concept of empirical measure.Empirical measure on RFor z(N) (z1 , z2 , . . . , zN ) RN define the empirical measureηzN(N) (B) N1Xδzi (B),N B B(R)i 1where δa is the Dirac measure at a R.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsDenote Z(N) (t) (Z1 (t), . . . , ZN (t)), for any t 0.N1XF (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1 ZNF (t, Zi (t), ui (t), y ) ηZ(N) (t) (dy ) dt σdWi (t) dZi (t) RWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsDenote Z(N) (t) (Z1 (t), . . . , ZN (t)), for any t 0.N1XF (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1 ZNF (t, Zi (t), ui (t), y ) ηZ(N) (t) (dy ) dt σdWi (t) dZi (t) RLet N and Z(t) (Z1 (t), . . . , ZN (t), . . . ).NAssume ηZ(t) : Z limN ηZ(N) (t) exits in weak sense, thendZi (t) F (t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt σdWi (t).RZT ZJi (ui ) E0 L(t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt.RWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsDenote Z(N) (t) (Z1 (t), . . . , ZN (t)), for any t 0.N1XF (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1 ZNF (t, Zi (t), ui (t), y ) ηZ(N) (t) (dy ) dt σdWi (t) dZi (t) RLet N and Z(t) (Z1 (t), . . . , ZN (t), . . . ).NAssume ηZ(t) : Z limN ηZ(N) (t) exits in weak sense, thendZi (t) F (t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt σdWi (t).RZT ZJi (ui ) E0 L(t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt.RUnderlying intuition: as N increases.the overall population’s behaviour (i.e. mean field)becomes relevant to a given agent’s dynamics.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsDenote Z(N) (t) (Z1 (t), . . . , ZN (t)), for any t 0.N1XF (t, Zi (t), ui (t), Zk (t))dt σdWi (t)Nk 1 ZNF (t, Zi (t), ui (t), y ) ηZ(N) (t) (dy ) dt σdWi (t) dZi (t) RLet N and Z(t) (Z1 (t), . . . , ZN (t), . . . ).NAssume ηZ(t) : Z limN ηZ(N) (t) exits in weak sense, thendZi (t) F (t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt σdWi (t).RZT ZJi (ui ) E0 L(t, Zi (t), ui (t), y ) ηZ(t) (dy ) dt.RηZ(t) (mean field) contains only statistical property of the massZ(t).Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsTo sense a mean field parameter.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsηZ(t) contains statistical property of Z(t) (Z1 (t), . . . , ZN (t), . . . )Let PZi (t) be the probability distribution of Zi (t). In thecontinuum limit N , Z dZi (t) F (t, Zi (t), ui (t), y ) PZi (t) (dy ) dt σdWi (t)RZT ZJi (ui ) E0R L(t, Zi (t), ui (t), y ) PZi (t) (dy ) dt.Underlying intuition: as N increases.at any time t 0, an individual’s distribution PZi (t) caneffectively represent the empirical distribution ηZ(t) .Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game (continuum limit for N )Model assumptionsplayers become infinitesimal and indistinguishable.the dynamics of the mass is the result of what a singleplayer doesplayers response to a mean-field.restating game theory as an interaction betweenindividual and the massWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game (limit for N )For a representative agent,The controlled dynamics (X (t), t 0)For a control policy u. (u(t) U, t 0),dX (t) f (X (t), u(t), µ(t))dt σdW (t)(1)where f : R U P(R) R.The solution is {(X (t), µ(t)), t 0} such that{X (t), t 0} is a solution to Eq. (1)µ(t) is the probability distribution of X (t) for any t 0.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game (limit for N )For a representative agent,The controlled dynamics (X (t), t 0)For a control policy u. (u(t) U, t 0),dX (t) f (X (t), u(t), µ(t))dt σdW (t) McKean-Vlasov(1)where f : R U P(R) R.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game (limit for N )For a representative agent,The controlled dynamics (X (t), t 0)For a control policy u. (u(t) U, t 0),dX (t) f (X (t), u(t), µ(t))dt σdW (t) McKean-Vlasov(1)where f : R U P(R) R.µ. {µ(t) P(R), t 0} represents the mean field.The cost functionFor a mean field µ. {µ(t) P(R), t 0},Z TJ(t, x, u., µ.) ExL(X (s), u(s), µ(s))ds(2)twhere L : R U P(R) R.To find a û. which is an optimal response to µ. and produces µ.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game (limit for N )The controlled dynamics (X (t), t 0)For a control policy u. (u(t) U, t 0),dX (t) f (X (t), u(t), µ(t))dt σdW (t) Fokker-Planck (FP) equation (forward Kolmogorov equation)The cost functionFor a mean field µ. {µ(t) P(R), t 0},Z TJ(t, x, u., µ.) ExL(x(s), u(s), µ(s))dst Hamilton-Jacobi-Bellman (HJB) equationWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game (limit for N )FP equationFor a control policy u. (u(t) U, t 0), σ2 d 2 dd m(t, x)m(t, x) f x, u(t), m(t, x) m(t, x) dtdx2 dx 2m(0, x) m0 (x).describe the aggregation of the action of all players.HJB equationFor a mean field µ. {µ(t) P(R), t 0}, V Vσ2 2V inff x, u(t), µ(t) L x, u(t), µ(t) t x2 x 2u(t) UV (T ,x) 0.the reaction of players to the massWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game (limit for N )FP equation - forward equationFor a control policy u. (u(t) U, t 0), σ2 d 2 dd m(t, x)m(t, x) f x, u(t), m(t, x) m(t, x) dtdx2 dx 2m(0, x) m0 (x).describe the aggregation of the action of all players.where the population actually end up, based on the initial dist.HJB equationFor a mean field µ. {µ(t) P(R), t 0}, V Vσ2 2V inff x, u(t), µ(t) L x, u(t), µ(t) t x2 x 2u(t) UV (T ,x) 0.the reaction of players to the massWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game (limit for N )FP equation - forward equationFor a control policy u. (u(t) U, t 0), σ2 d 2 dd m(t, x)m(t, x) f x, u(t), m(t, x) m(t, x) dtdx2 dx 2m(0, x) m0 (x).describe the aggregation of the action of all players.where the population actually end up, based on the initial dist.HJB equation - backward equationFor a mean field µ. {µ(t) P(R), t 0}, V Vσ2 2V inff x, u(t), µ(t) L x, u(t), µ(t) t x2 x 2u(t) UV (T ,x) 0.the reaction of players to the mass.decisions based on where you want to be in the futureWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMean field game (limit for N )Mean field equations - coupled system of two equationsFP equation - forward equationFor a control policy u. (u(t) U, t 0), σ2 d 2 dd m(t, x) f x, u(t), m(t, x) m(t, x) m(t, x)dtdx2 dx 2m(0, x) m0 (x)û(t) µ(t)(dx) m(t, x)dxHJB equation - backward equationFor a mean field µ. {µ(t) P(R), t 0}, V Vσ2 2V inff x, u(t), µ(t) L x, u(t), µ(t) t x2 x 2u(t) UV (T , x) 0.Let û(t) û(t, x, µ.) be the best response to the mean field.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsOutline1Introduction2Our results and applicationsWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsJoint work with Vassili Kolokoltsov at Warwick UniversityWe developed a unified framework where a larger class ofMarkov processes is considered The dynamic of the N players Z N (t) RN : t [0, T ] isassociated to a family of linear and bounded operatorsnoA[t, µ, u] L(C2 , C) : t [0, T ], µ P(Rd ), u U .Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsFor each (t, µ, u) [0, T ] P(Rd ) U, A[t, µ, u] : C2 7 C isassumed to generate a Feller process with values in Rd and tobe of the formA[t, µ, u]f (z) (h(t, z, µ, u), f (z)) R[t, µ]f (z)h : [0, T ] Rd P(Rd ) U RdR[t, µ] L(C2 , C) is of the form:1R[t, µ]f (z) (G(t, z, µ) , )f (z) (b(t, z, µ), f (z))2Z (f (z y ) f (z) ( f (z), y)1B1 (y ))ν(t, z, µ, dy ).RdWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsFor each (t, µ, u) [0, T ] P(Rd ) U, A[t, µ, u] : C2 7 C isassumed to generate a Feller process with values in Rd and tobe of the formA[t, µ, u]f (z) (h(t, z, µ, u), f (z)) R[t, µ]f (z)ExampleIf R[t, µ] 12 σ 2 with a constant σ, i.e,A[t, µ, u]f (z) (h(t, z, µ, u), f (z)) 12 σ 2 f (z), Z N (t) : t [0, T ] can also be described by the SDEdZ N (t) h(t, Z N (t), µt , ut ) dt σdWt .Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsOptimal control problem for each player.Given {η N (t) : t [0, T ]}, the optimal payoff of i-th player,i [1, N], starting at x and t is"Z#TVi,N (t, x) inf Exu.tNNL(s, Zi,s, ηsN , ui,s ) ds V T (Zi,T)L : [0, T ] Rd P(Rd ) U RV T : Rd RWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsFormally, if ηtN µt and Vi,N V as N such thatCoupled forward-backward equationsforward kinetic equation in weak form:(ddt (g, µt ) (A[t, µt , Γ(t, ., {µs : t s T })]g, µt ) (FE)µ0 µbackward HJB equation: V (t,x) Ht (x, V (t, x), µt ) R[t, µt ]V (t, x) 0 (BE) t Ht (x, p, µ) infu U (h(t, x, µ, u)p L(t, x, µ, u)) V VTTV (t, x) Γ(t, x, {νt : t [0, T ]}) (Xt : t [0, T ]) {µt : t [0, T ]}{νt : t [0, T ]}Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsFormally, if ηtN µt and Vi,N V as N such thatCoupled forward-backward equationsforward kinetic equation in weak form:(ddt (g, µt ) (A[t, µt , Γ(t, ., {µs : t s T })]g, µt ) (FE)µ0 µbackward HJB equation: V (t,x) Ht (x, V (t, x), µt ) R[t, µt ]V (t, x) 0 (BE) t Ht (x, p, µ) infu U (h(t, x, µ, u)p L(t, x, µ, u)) V VTTV (t, x) Γ(t, x, {νt : t [0, T ]}) (Xt : t [0, T ]) {µt : t [0, T ]}{νt : t [0, T ]}fixed point, consistencyWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsTheorem 1 (KY2013): mean field limitFor arbitrary T 0, there exists a solution to the Cauchyproblem for (FE) d (g, µt ) (A[t, µt , Γ(t, ., {µs : t s T })]g, µt )dt µ µ.0For T small enough, the solution is unique.Requirement: Γ has feedback regularity property.V. Kolokoltsov, W.Yang (2013). Existence of Solutions toPath-Dependent Kinetic Equations and Related Forward - BackwardSystems, Open Journal of Optimization, 2, 39-44.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsTheorem 2 (KY2013): optimal control(a) For a given flow {νt : t [0, T ]}, the Cauchy problem V (t,x) Ht (x, V (x), νt ) R[t, νt ]V (t, x) 0 tHt (x, p, µ) infu U (h(t, x, µ, u)p L(t, x, µ, u)) V T V Tis wellposed;Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsTheorem 2 (KY2013): optimal control - sensitivity(b) For any {νt1 : t [0, T ]}, {νt2 : t [0, T ]}, there exists k 0such thatsup kV (t, ·; {ν.1 }) V (t, ·; {ν.2 })kC1 k sup kνt1 νt2 k(C2 ) t [0,T ]t [0,T ]andsup k V (t, ·; {ν.1 }) V (t, ·; {ν.2 })kC k sup kνt1 νt2 k(C2 ) ;t [0,T ]t [0,T ]Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsTheorem 2 (KY2013): optimal control - feedback regularity(c) The unique optimal control function Γ(t, x; {νt : t [0, T ]})denote has the feedback regularity required by Theorem 1,i.e. for {νt1 : t [0, T ]}, {νt2 : t [0, T ]},sup Γ(t, x; {ν 1 .}) Γ(t, x; {ν 2 .}) k1 sup νs1 νs2 (C2 ) t,xs [0,T ]with some constant k1 0.prove, rather than assume, the feedback regularity propertyV. Kolokoltsov, W.Yang (2013). Sensitivity analysis for HJB equationswith an application to a coupled backward-forward system, submitted.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsTheorem 3 (KY2012): convergenceThe N-player games converge to the limit system, i.e.NµNt (µ0 ) µt (µ0 ) as N , t [0, T ]with a convergence rate O(1/N), where µt is a solution to (FE).Technical issue: smoothness of the solutions to equation (FE)with respect to initial data µ0 .Improvement to the existing models: convergence rate 1/N,instead of 1/ N. Kolokoltsov, Li, Yang (2012). Mean Field Games and NonlinearMarkov Processes, 63 pages, arXiv:1112.3744v. Kolokoltsov, Troeva, Yang (2014). On the rate of convergence forthe mean field approximation of controlled diffusions with largenumber of payers. Dynamic Games and Applications, 4, 208-230.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsDefinition: Nash equilibriumFor 0, a strategy profile Γ is a -Nash equilibrium ifJi (Γ) Ji (Γ i , ui ) , i 1, . . . , Nwhere (Γ i , ui ) denotes the profile obtained from Γ bysubstituting the strategy of player i with any eligible strategy ui .Theorem 4 (KY2012): Nash equilibriumAny solution of the limit modelΓ(t, x, {µs : t s T })represents an -equilibrium for an N players dynamic game,with O(1/N).In a general setting, solutions are not in a closed-form.V.Kolokoltsov, J. Li, W. Yang (2012). Mean Field Games andNonlinear Markov Processes, 63 pages, arXiv:1112.3744v.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsA setting with a major player:Inspection games in mean field settingconsider one inspector with N inspecteesintroduction of a deterministic major playerMarkov Chain on a finite state space {l1 , . . . , ld } (crimelevels)obtain a convergence result without a convergence rateKolokoltsov, Yang (2014). Inspection games in mean field setting, inpreparationWei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsMFG theory attracts much attention from Mathematical society.Other developments of MFGcooperative populationstability of the limit system over infinite horizonwith a stochastic major playermean field type controls···Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsReferences:J.-M. Lasry and P.-L. Lions (2007). Mean field games. JapaneseJournal of Mathematics, 2:1, 229-260.Guéant O., Lasry J.-M., Lions P.-L. (2010). Mean Field Gamesand Applications. Paris-Princeton Lectures on MathematicalFinance, Springer.Huang M., Caines P. E., Malhamé R. P. (2006). Large populationstochastic dynamic games: closed-loop Mckean-Vlasov systemsand the Nash certainty equivalence principle. Communicationsin information and systems, 6, 221-252.Gomes G., Saúde J. (2014). Mean field games models - a briefsurvey, Dynamic Games and Applications, 4:2, 110-154.Bensoussan A, Frehse J., Yam P. (2013). Mean field games andmean field type control. Springer, Berlin.Nguyen S. and Huang M. (2012). Mean Field LQG Games withMass Behavior Responsive to A Major Player. 51st IEEEConference on Decision and Control.Wei YangIntroduction to Mean field games and applications

IntroductionOur results and applicationsKolokoltsov V. , Li J, Yang W. (2011). Mean Field Games andNonlinear Markov Processes. arXiv:1112.3744vKolokoltsov V. Yang W. (2013). Existence of solutions topath-dependent kinetic equations and related forward-backwardsystems, Open J Optimisation, 2, 39-44.Kolokoltsov V. Yang W. (2013). Sensitivity analysis for HJBequation with an application to a couple backward-forwardsystem, arXiv:1303.6234v1.Kolokoltsov V. Troeva M. Yang W. (2014). On the

Wei Yang Introduction to Mean ﬁeld games and applications. Introduction Our results and applications Mean ﬁeld game Methodology consider an N-playerstochastic dynamic game study amean ﬁeld game(a limit for N !1) which can be expressed bya system of coupled equations: Fokker-Planck equation Hamilton-Jacobi-Bellman equation any solution to the mean ﬁeld game is an -equilibriumto the N .

Related Documents: