Why Machine Learning And Games? Machine . - Microsoft
Why Machine Learning and Games?Machine Learning in Video GamesDrivatars Reinforcement LearningMachine Learning in Online GamesTrueSkill Halo 3The Path of GoConclusions
Test Beds for Machine Learning Perfect instrumentation andmeasurements Perfect control and manipulation Reduced cost Reduced risk Great way to showcase algorithmsImprove User Experience Create adaptive, believable game AI Compose great multiplayer matchesbased on skill and social criteria Mitigate Network latency usingprediction Create realistic character movement
Partially observable stochastic gamesStates only partially observedMultiple agents choose actionsStochastic pay-offs and state transitions depend on state and all theother agents’ actionsGoal: Optimise long term pay-off (reward)Just like life: complex, adversarial, uncertain, and we are in it forthe long run!
From single player’sperspectiveApproximateSolutionsWhat is the best AI? Partially Observable Markov Decision Process (POMDP) Reinforcement Learning Unsupervised Learning Supervised Learning Always takes optimal actions Delivers best entertainment value
Why Machine Learning and Games?Machine Learning in Video GamesDrivatars Reinforcement LearningMachine Learning in Online GamesTrueSkill Halo 3The Path of GoConclusions
Adaptive avatar fordrivingSeparate game modeBasis of all in-game AIBasis of “dynamic”racing line
XBOX Game Dynamic Racing Line Learning a Drivatar Using a Drivatar
“Built-In” AI BehaviourDevelopment ToolLearning SystemDrivatarRacing LineRacing LineBehaviour ModelVehicle Interaction and RacingStrategyControllerCar BehaviourAI DrivingDrivatarAI Driving
Segmentsa1a2a3a4
Why Machine Learning and Games?Machine Learning in Video GamesDrivatars Reinforcement LearningMachine Learning in Online GamesTrueSkill Halo 3The Path of GoConclusions
gamestateAgentparameter updatereward /punishmentactionLearning Algorithmgame stateGameaction
4.01ft / GROUND2ft / GROUNDgame states3ft / GROUND4ft / GROUND5ft / GROUND6ft / GROUND1ft / KNOCKED2ft / KNOCKED3ft / KNOCKED4ft / KNOCKED5ft / KNOCKED6ft / KNOCKED3 ft
Game state featuresReinforcement LearnerSeparation (5 binned ranges)Last action (6 categories)Mode (ground, air, knocked)Proximity to obstacleAvailable Actions19 aggressive (kick, punch)10 defensive (block, lunge)8 neutral (run)Q-Function RepresentationOne layer neural net (tanh)In-Game AI Code
Reward for decrease in Wulong Goth’s healthEarly in the learning process after 15 minutes of learning
Punishment for decrease in either player’s healthEarly in the learning process after 15 minutes of learning
Why Machine Learning and Games?Machine Learning in Video GamesDrivatars Reinforcement LearningMachine Learning in Online GamesTrueSkill Halo 3The Path of GoConclusions
Competition is central to our livesInnate biological traitDriving principle of many sportsChess Rating for fair competitionELO: Developed in 1960 by Árpád Imre ÉlőMatchmaking system for tournamentsChallenges of online gamingLearn from few match outcomes efficientlySupport multiple teams and multiple players per team
Given:Match outcomes: Orderings among teams consisting ofplayers, respectivelyQuestions:Skillfor each player such thatGlobal ranking among all playersFair matches between teams of players,, .,
Latent Gaussian performance model for fixed skillsPossible outcomes: Player 1 wins over 2 (and vice versa)s1s2p1p2y12
Gaussian Prior Factorss1s2s3s4Fast and efficient approximate message passing od Factors1223
LeaderboardGlobal ranking of all playersMatchmakingFor gamers: Most uncertain outcomeFor inference: Most informativeBoth are equivalent!
Xbox 360 LiveLaunched in September 2005Every game uses TrueSkill to match players 35 million players 4 million matches per day 2 billion hours of gameplay / monthHalo 3Launched on 25th September 2007Largest entertainment launch in history 200,000 player concurrently (peak: 1,000,000)
Halo 3 Game Matchmaking Skill Stats Tight Matches
Why Machine Learning and Games?Machine Learning in Video GamesDrivatars Reinforcement LearningMachine Learning in Online GamesTrueSkill Halo 3The Path of GoConclusions
Started about 4000 years ago in ancient China.About 60 million players worldwide.2 Players: Black and White.Board: 19 19 grid.Rules:Turn: stone placed on vertex.Capture.Aim: Gather territory
5th November 1997:Gary Kasparov beaten by Deep Blue.Best Go programs cannot beat amateurs.
Minimax search defeated.MinimaxlookaheadHigh Branching Factor.Go: 200Chess: 35EvaluationComplex Position Evaluation.Stone’s value derived from configuration of surrounding stones.
MCTerritory Hypothesis
LW W
LW W W
LW W LW
This nodeSeen: 3 timesWin: 2/3 times L LW W LW
L LW W LW
The Path of Go MSRC Go AI (written in F#) TrueSkill Match Making XNA Game Studio
Why Machine Learning and Games?Machine Learning in Video GamesDrivatars Reinforcement LearningMachine Learning in Online GamesTrueSkill Halo 3The Path of GoConclusions
Computer games can be used as test beds for research.Machine learning can be used to improve the user experience incomputer games.Both research and applications are in their infancy and there aremany open questions.XNA framework exists to plug in machine learning algorithms.For more question, please drop us a line
joaquinc @ microsoft.com
Create adaptive, believable game AI Compose great multiplayer matches . 3.2 4.06.0 10.0 . Game state features Separation (5 binned ranges) . XNA Game Studio . Why Machine Learning and Games? Machine Learning in Video Games Drivatars
Olympic Winter Games medals Olympic Winter Games posters Olympic Summer Games posters Olympic Summer Games mascots Olympic Winter Games mascots The sports pictograms of the Olympic Summer Games The sports pictograms of the Olympic Winter Games The IOC, the Olympic Movement and the Olympic Games The Olympic programme evolution Torches and torch .
Regional Games and Multi-Sport Games (such as Pan American Games, African Games, European Games, Commonwealth Games, Mediterranean Games, Francophone Games, Youth Olympic Games) International Tournaments organised by the IJF (Grand Prix, Grand Slam, Masters) or under its auspices (continental open and cups),
Section 3: Playground Markings Games 16 Section 4: Skipping, Hula Hoop & Elastics 25 Section 5: Catching games 32 Section 6: Relay games 41 Section 7: Ball games 48 Section 8: Fun games 59 Section 9: Frisbee games 66 Section 10: Parachute games 70 Section 11: Clapping and rhyming games 74 Useful websites 79
The Games organised at Olympia led to the development of the Panhellenic Games. These included: - The Games at Olympia (Olympic Games): every four years - The Games at Delphi (Pythian Games), 582 B.C.: every four years (third year of each Olympiad) - The Games at the Isthmus of Corinth (Isthmian Games), from 580 B.C.:
decoration machine mortar machine paster machine plater machine wall machinery putzmeister plastering machine mortar spraying machine india ez renda automatic rendering machine price wall painting machine price machine manufacturers in china mail concrete mixer machines cement mixture machine wall finishing machine .
Olympic Summer Games posters Olympic Summer Games mascots Olympic Winter Games mascots The IOC, the Olympic Movement and the Olympic Games The Olympic programme evolution The Olympic stadiums of the Summer Games The sports pictograms of the Olympic Summer Games The sports pictograms of the Olympic Winter Games .
Machine learning has many different faces. We are interested in these aspects of machine learning which are related to representation theory. However, machine learning has been combined with other areas of mathematics. Statistical machine learning. Topological machine learning. Computer science. Wojciech Czaja Mathematical Methods in Machine .
ASTM F 891 Cellular Core PVC DWV Pipe ASTM D 2665 PVC DWV Pipe & Fittings NSF Standard 14 Dimensional Standard Schedule 40 Iron Pipe Size (IPS) Cell Class 12454 PVC Solid Wall Pipe & Fittings 11432 PVC DWV Cellular Core Pipe Maximum Working Temperature 140 F Maximum Working Pressure 0 (zero) PSI PVC DWV is NOT a pressure-rated piping system .
