NflWAR: A Reproducible Method For Offensive Player .
nflWAR:A Reproducible Method forOffensive Player Evaluation in FootballRon YurkoSam VenturaMax HorowitzDepartment of StatisticsCarnegie Mellon UniversityNESSIS, 2017Ron Yurko (@Stat Ron)nflWARNESSIS, 20171 / 36
Reproducible Research with nflscrapRRecent work in football analytics is not easily reproducible:Reliance on proprietary and costly data sourcesData quality relies on potentially biased human judgementRon Yurko (@Stat Ron)nflWARNESSIS, 20172 / 36
Reproducible Research with nflscrapRRecent work in football analytics is not easily reproducible:Reliance on proprietary and costly data sourcesData quality relies on potentially biased human judgementnflscrapR:R package created by Maksim Horowitz to enable easy dataaccess and promote reproducible NFL researchCollects play-by-play data from NFL.com and formats into Rdata framesData is available for all games starting in 2009Available on Github, install with:devtools::install github(repo maksimhorowitz/nflscrapR)Ron Yurko (@Stat Ron)nflWARNESSIS, 20172 / 36
Pittsburgh Fans ReactPittsburgh Post-Gazette article by Liz Bloom covered recentnflscrapR research and status of statistics in footballRon Yurko (@Stat Ron)nflWARNESSIS, 20173 / 36
Pittsburgh Fans ReactPittsburgh Post-Gazette article by Liz Bloom covered recentnflscrapR research and status of statistics in footballAnd the comments.Ron Yurko (@Stat Ron)nflWARNESSIS, 20173 / 36
Another Comment.Ron Yurko (@Stat Ron)nflWARNESSIS, 20174 / 36
Tremendous Insight!Recognizes the key flaws of rawfootball statistics:Moving parts in every playNeed to assign credit to eachplayer involved in a playUltimately evaluate players interms of winsUsing nflscrapR we introduce nflWAR for offensive players:Reproducible framework for wins above replacementRon Yurko (@Stat Ron)nflWARNESSIS, 20175 / 36
Goals of nflWARProperly evaluate every playAssign individual player contribution on each playEvaluate relative to replacement levelConvert to a wins scaleEstimate the uncertainty in WARApply this framework to each available season, 2009-2016Ron Yurko (@Stat Ron)nflWARNESSIS, 20176 / 36
How to Value Plays?Expected Points (EP): Value of play is in terms ofE (points of next scoring play )How many points have teams scored when in similar situations?Several ways to model thisWin Probability (WP): Value of play is in terms of P(Win)Have teams in similar situations won the game?Common approach is logistic regressionCan apply nflWAR framework to both, but will focus on EP todayRon Yurko (@Stat Ron)nflWARNESSIS, 20177 / 36
How to Calculate EP?Response: Y {Touchdown (7), Field Goal (3), Safety (2),-Touchdown (-7), -Field Goal (-3), -Safety (-2),No Score (0)}Covariates: X {down, yards to go, yard line, .}“Nearest Neighbors”:Identify similar plays in historical data based on down, yards togo, yard line, etc. and take the averageRon Yurko (@Stat Ron)nflWARNESSIS, 20178 / 36
How to Calculate EP?Response: Y {Touchdown (7), Field Goal (3), Safety (2),-Touchdown (-7), -Field Goal (-3), -Safety (-2),No Score (0)}Covariates: X {down, yards to go, yard line, .}“Nearest Neighbors”:Identify similar plays in historical data based on down, yards togo, yard line, etc. and take the averageBut what defines a similar play?Ron Yurko (@Stat Ron)nflWARNESSIS, 20178 / 36
How to Calculate EP?Response: Y {Touchdown (7), Field Goal (3), Safety (2),-Touchdown (-7), -Field Goal (-3), -Safety (-2),No Score (0)}Covariates: X {down, yards to go, yard line, .}“Nearest Neighbors”:Identify similar plays in historical data based on down, yards togo, yard line, etc. and take the averageBut what defines a similar play?Linear Regression:E (Y X ) β0 β1 Xdown β2 XyardsRon Yurko (@Stat Ron)nflWARto go .NESSIS, 20178 / 36
How to Calculate EP?Response: Y {Touchdown (7), Field Goal (3), Safety (2),-Touchdown (-7), -Field Goal (-3), -Safety (-2),No Score (0)}Covariates: X {down, yards to go, yard line, .}“Nearest Neighbors”:Identify similar plays in historical data based on down, yards togo, yard line, etc. and take the averageBut what defines a similar play?Linear Regression:E (Y X ) β0 β1 Xdown β2 Xyardsto go .But is treating the next score as continuous appropriate?Ron Yurko (@Stat Ron)nflWARNESSIS, 20178 / 36
Distribution of Next ScoreRon Yurko (@Stat Ron)nflWARNESSIS, 20179 / 36
Linear Regression Approach.What are the assumptions of linear regression? i N(0, σ 2 ) (iid)Ron Yurko (@Stat Ron)nflWARNESSIS, 201710 / 36
Linear Regression Approach. IS A DISASTER!What are the assumptions of linear regression? i N(0, σ 2 ) (iid)Ron Yurko (@Stat Ron)nflWARNESSIS, 201711 / 36
Multinomial Logistic RegressionLogistic regression to model the probabilities ofY {Touchdown (7), Field Goal (3), Safety (2),-Touchdown (-7), -Field Goal (-3), -Safety (-2),No Score (0)}Specified with 6 logit transformations relative to No Score:P(Y Touchdown X x)) β10 β1T xP(Y No Score X x)P(Y Field Goal X x)) β20 β2T xlog (P(Y No Score X x).P(Y Touchdown X x)log () β60 β6T xP(Y No Score X x)log (Ron Yurko (@Stat Ron)nflWARNESSIS, 201712 / 36
Multinomial Logistic RegressionModel is generating probabilities, agnostic of value associatedwith each next score typeNext Score: Y {Touchdown (7), Field Goal (3), Safety (2), NoScore (0), -Safety (-2), -Field Goal (-3), -Touchdown (-7)}Situation: X {down, yards to go, yard line, .}Outcome probabilities: P(Y y X )Expected Points (EP) E (Y X ) Ron Yurko (@Stat Ron)nflWARPyP(Y y X ) yNESSIS, 201713 / 36
Expected Points AddedExpected Points Added (EPA) estimates a play’s value based onthe change in situation, providing a point valueEPAplayi EPplayi 1 EPplayiRon Yurko (@Stat Ron)nflWARNESSIS, 201714 / 36
Expected Points AddedExpected Points Added (EPA) estimates a play’s value based onthe change in situation, providing a point valueEPAplayi EPplayi 1 EPplayiFor passing plays can use air yards to calculate airEPA and yacEPA(yards after catch EPA):airEPAplayi EPinair playi EPstartyacEPAplayi EPplayi 1 EPinRon Yurko (@Stat Ron)playiair playinflWARNESSIS, 201714 / 36
Expected Points AddedExpected Points Added (EPA) estimates a play’s value based onthe change in situation, providing a point valueEPAplayi EPplayi 1 EPplayiFor passing plays can use air yards to calculate airEPA and yacEPA(yards after catch EPA):airEPAplayi EPinair playi EPstartyacEPAplayi EPplayi 1 EPinplayiair playiBut how much credit does each player deserve?e.g. On a pass play, how much credit does a QB get vs the receiver?One player is not solely responsible for a play’s EPARon Yurko (@Stat Ron)nflWARNESSIS, 201714 / 36
How to Allocate EPA?Using proprietary, manually collected data, Total QBR (Oliver et al.,2011) divides credit between those involved in passing playsRon Yurko (@Stat Ron)nflWARNESSIS, 201715 / 36
How to Allocate EPA?Using proprietary, manually collected data, Total QBR (Oliver et al.,2011) divides credit between those involved in passing playsPublicly available data only includes those directly involved:Passing:Individuals: passer, target receiver, tackler(s), interceptorContext: air yards, yards after catch, location, and if thepasser was hit on the playRushing:Individuals: rusher and tackler(s)Context: run gap and locationRon Yurko (@Stat Ron)nflWARNESSIS, 201715 / 36
Multilevel ModelingGrowing in popularity (and rightfully so):“Multilevel Regression as Default” - Richard McElreathNatural approach for data with group structure, and differentlevels of variation within each groupe.g. QBs have more pass attempts than receivers have targetsEvery play is a repeated measure of performanceBaseball example: Deserved Run Average(Judge et al., 2015)Ron Yurko (@Stat Ron)nflWARNESSIS, 201716 / 36
Multilevel ModelingKey feature is the groups are given a model - treating the levels ofgroups as similar to one another with partial poolingSimple example of varying-intercept model:2EPAi N(QBj[i] RECk[i] βxi , σEPA), for i 1, . . . , # of plays,2QBj Normal(µQB , σQB), for j 1, . . . , # of QBs,2RECk Normal(µREC , σREC), for k 1, . . . , # of ReceiversRon Yurko (@Stat Ron)nflWARNESSIS, 201717 / 36
Multilevel ModelingKey feature is the groups are given a model - treating the levels ofgroups as similar to one another with partial poolingSimple example of varying-intercept model:2EPAi N(QBj[i] RECk[i] βxi , σEPA), for i 1, . . . , # of plays,2QBj Normal(µQB , σQB), for j 1, . . . , # of QBs,2RECk Normal(µREC , σREC), for k 1, . . . , # of ReceiversUnlike linear regression, no longer assuming independenceProvides estimates for average play effects while providing necessaryshrinkage towards the group averagesRon Yurko (@Stat Ron)nflWARNESSIS, 201717 / 36
nflWAR ModelingUse varying-intercepts for each of the grouped variablesWith location and gap, create Team-side-gap as O-line proxye.g. PIT-left-end, PIT-left-guard, PIT-middleSeparate passing and rushing with different grouped variablesPassing: Offensive team, QB, receiver, defensive teamRushing: Team-side-gap, rusher, defensive teamRon Yurko (@Stat Ron)nflWARNESSIS, 201718 / 36
nflWAR ModelingUse varying-intercepts for each of the grouped variablesWith location and gap, create Team-side-gap as O-line proxye.g. PIT-left-end, PIT-left-guard, PIT-middleSeparate passing and rushing with different grouped variablesPassing: Offensive team, QB, receiver, defensive teamRushing: Team-side-gap, rusher, defensive teamEach individual intercept for player groups is an estimate for a player’seffect, individual points added (iPA)Intercepts for team groups are team points added (tPA)Multiply iPA/tPA by attempts to getindividual/team points above average (iPAA/tPAA)Ron Yurko (@Stat Ron)nflWARNESSIS, 201718 / 36
Rushing BreakdownWith EPA as the response, two separate models:RB/FB/WR/TE - designed rushing playsAdjust for rusher position as non-grouped variableQB - designed runs, scrambles, and sacksReplace Team-side-gap with offensive teamProvides iPArush and tPArushRon Yurko (@Stat Ron)side gapnflWARestimatesNESSIS, 201719 / 36
Group Variation for RB/FB/WR/TE Rushing ModelRon Yurko (@Stat Ron)nflWARNESSIS, 201720 / 36
Group Variation for QB Rushing ModelRon Yurko (@Stat Ron)nflWARNESSIS, 201721 / 36
Which Teams Ran Efficiently in 2016?Ron Yurko (@Stat Ron)nflWARNESSIS, 201722 / 36
Passing BreakdownCould simply use EPA, or take advantage of air yardsTwo separate models for airEPA and yacEPA, where both modelsconsider all pass attempts but the response depends on the model:Receptions assigned airEPA and yacEPA for respective modelsIncomplete passes use observed EPAEmphasize importance of completionsBoth adjust for QBs hit, receiver positions, and pass locationyacEPA model adjusts for air yardsProvides iPAair and iPAyac estimatesRon Yurko (@Stat Ron)nflWARNESSIS, 201723 / 36
Variation of Passing Intercepts (airEPA)Ron Yurko (@Stat Ron)nflWARNESSIS, 201724 / 36
Variation of Passing Intercepts (yacEPA)Ron Yurko (@Stat Ron)nflWARNESSIS, 201725 / 36
Passing Efficiency in 2016Ron Yurko (@Stat Ron)nflWARNESSIS, 201726 / 36
Relative to Replacement LevelFollowing an approach similar to openWAR (Baumer et al., 2015),defining replacement level based on rosterFor each team and position sort by number of attempts (separateRB/FB replacement level for rushing and receiving)Ron Yurko (@Stat Ron)nflWARNESSIS, 201727 / 36
Relative to Replacement LevelFollowing an approach similar to openWAR (Baumer et al., 2015),defining replacement level based on rosterFor each team and position sort by number of attempts (separateRB/FB replacement level for rushing and receiving)Player i 0 s iPAAi,total iPAAi,rush iPAAi,air iPAAi,yacCreates a replacement-level iPAA that “shadows” a player’sperformance, denote as iPAAreplacementiPlayer i 0 s individual points above replacement (iPAR) as:iPARi iPAAi,total iPAAreplacementi,totalRon Yurko (@Stat Ron)nflWARNESSIS, 201727 / 36
Convert to Wins“Wins & Point Differential in the NFL” - (Zhou & Ventura, 2017)(CMU Statistics & Data Science freshman research project)Ron Yurko (@Stat Ron)nflWARNESSIS, 201728 / 36
WAR!Fit a linear regression between wins and total score differential:Points per Win e.g. In 2016 β̂ScoreRon Yurko (@Stat Ron)Diff1β̂ScoreDiff 0.0319, roughly 31 points per winnflWARNESSIS, 201729 / 36
WAR!Fit a linear regression between wins and total score differential:Points per Win e.g. In 2016 β̂ScoreDiff1β̂ScoreDiff 0.0319, roughly 31 points per winand finally arrive at wins above replacement (WAR):WAR Ron Yurko (@Stat Ron)iPARPoints per WinnflWARNESSIS, 201729 / 36
QB WAR in 2016Ron Yurko (@Stat Ron)nflWARNESSIS, 201730 / 36
RB WAR in 2016Ron Yurko (@Stat Ron)nflWARNESSIS, 201731 / 36
TE WAR in 2016Ron Yurko (@Stat Ron)nflWARNESSIS, 201732 / 36
WR WAR in 2016Ron Yurko (@Stat Ron)nflWARNESSIS, 201733 / 36
Recap and Future of nflWARProperly evaluating every play with EPA generated with multinomiallogistic regression modelMultilevel modeling provides an intuitive way for estimating playereffects and can be extended with data containing every player onthe field for every playRon Yurko (@Stat Ron)nflWARNESSIS, 201734 / 36
Recap and Future of nflWARProperly evaluating every play with EPA generated with multinomiallogistic regression modelMultilevel modeling provides an intuitive way for estimating playereffects and can be extended with data containing every player onthe field for every playNaive to assume player has same effect for every play!Need to estimate the uncertainty in the different types of iPA togenerate intervals of WAR valuesRefine the definition of replacement-level,e.g. what about down specific players?Ron Yurko (@Stat Ron)nflWARNESSIS, 201734 / 36
Carnegie Mellon Sports Analytics ConferenceClear your calendars for Oct 28th!And visit www.cmusportsanalytics.com/conferencefor more information! #CMSACRon Yurko (@Stat Ron)nflWARNESSIS, 201735 / 36
AcknowledgementsMax Horowitz for creating nflscrapRSam Ventura for advising every step in the processJonathan Judge for answering questions on multilevel modelingRebecca Nugent and CMU Statistics and Data Science for all oftheir instruction, motivation, and support!Ron Yurko (@Stat Ron)nflWARNESSIS, 201736 / 36
References I[Gelman and Hill, 2007]. [Baumer et al., 2015]. [Oliver, ].[Hastie et al., 2009]. [Carroll et al., 1998].[Pasteur and David, 2017]. [Goldner, 2017]. [Berri and Burke, 2017].[Carter and Machol, 1971]. [Burke, ]. [out, ]. [num, ].Football outsiders.numberfire.Baumer, B., Jensen, S., and Matthews, G. (2015).openwar: An open source system for evaluating overall playerperformance in major league baseball.Journal of Quantitative Analysis in Sports, 11(2).Ron Yurko (@Stat Ron)nflWARNESSIS, 20171/4
References IIBerri, D. and Burke, B. (2017).Measuring productivity of nfl players.In Quinn, K., editor, The Economics of the National FootballLeague, pages 137–158. Springer, New York, New York.Burke, B.Advanced football analytics.Carroll, B., Palmer, P., Thorn, J., and Pietrusza, D. (1998).The Hidden Game of Football: The Next Edition.Total Sports, Inc., New York, New York.Carter, V. and Machol, R. (1971).Operations research on football.Operations Research, 19(2):541–544.Ron Yurko (@Stat Ron)nflWARNESSIS, 20172/4
References IIIGelman, A. and Hill, J. (2007).Data Analysis Using Regression and Multilevel/HierarchicalModels.Cambridge University Press, Cambridge, United Kingdom.Goldner, K. (2017).Situational success: Evaluating decision-making in football.In Albert, J., Glickman, M. E., Swartz, T. B., and Koning, R. H.,editors, Handbook of Statistical Methods and Analyses in Sports,pages 183–198. CRC Press, Boca Raton, Florida.Hastie, T., Tibshirani, R., and Friedman, J. (2009).The Elements of Statistical Learning: Data Mining, Inference,and Prediction.Springer, New York, New York.Ron Yurko (@Stat Ron)nflWARNESSIS, 20173/4
References IVOliver, D.Guide to the total quarterback rating.Pasteur, R. D. and David, J. A. (2017).Evaluation of quarterbacks and kickers.In Albert, J., Glickman, M. E., Swartz, T. B., and Koning, R. H.,editors, Handbook of Statistical Methods and Analyses in Sports,pages 165–182. CRC Press, Boca Raton, Florida.Ron Yurko (@Stat Ron)nflWARNESSIS, 20174/4
For passing plays can use air yards to calculate airEPA and yacEPA (yards after catch EPA): airEPA play i EP in air play i EP start play i yacEPA play i EP play i 1 EP in air play i But how much credit does each player deserve? e.g. On a pass play, how much credit does a QB get vs the receiver? One player is not solely responsible for a .
Bruksanvisning för bilstereo . Bruksanvisning for bilstereo . Instrukcja obsługi samochodowego odtwarzacza stereo . Operating Instructions for Car Stereo . 610-104 . SV . Bruksanvisning i original
10 tips och tricks för att lyckas med ert sap-projekt 20 SAPSANYTT 2/2015 De flesta projektledare känner säkert till Cobb’s paradox. Martin Cobb verkade som CIO för sekretariatet för Treasury Board of Canada 1995 då han ställde frågan
service i Norge och Finland drivs inom ramen för ett enskilt företag (NRK. 1 och Yleisradio), fin ns det i Sverige tre: Ett för tv (Sveriges Television , SVT ), ett för radio (Sveriges Radio , SR ) och ett för utbildnings program (Sveriges Utbildningsradio, UR, vilket till följd av sin begränsade storlek inte återfinns bland de 25 största
Hotell För hotell anges de tre klasserna A/B, C och D. Det betyder att den "normala" standarden C är acceptabel men att motiven för en högre standard är starka. Ljudklass C motsvarar de tidigare normkraven för hotell, ljudklass A/B motsvarar kraven för moderna hotell med hög standard och ljudklass D kan användas vid
LÄS NOGGRANT FÖLJANDE VILLKOR FÖR APPLE DEVELOPER PROGRAM LICENCE . Apple Developer Program License Agreement Syfte Du vill använda Apple-mjukvara (enligt definitionen nedan) för att utveckla en eller flera Applikationer (enligt definitionen nedan) för Apple-märkta produkter. . Applikationer som utvecklas för iOS-produkter, Apple .
EPA Test Method 1: EPA Test Method 2 EPA Test Method 3A. EPA Test Method 4 . Method 3A Oxygen & Carbon Dioxide . EPA Test Method 3A. Method 6C SO. 2. EPA Test Method 6C . Method 7E NOx . EPA Test Method 7E. Method 10 CO . EPA Test Method 10 . Method 25A Hydrocarbons (THC) EPA Test Method 25A. Method 30B Mercury (sorbent trap) EPA Test Method .
och krav. Maskinerna skriver ut upp till fyra tum breda etiketter med direkt termoteknik och termotransferteknik och är lämpliga för en lång rad användningsområden på vertikala marknader. TD-seriens professionella etikettskrivare för . skrivbordet. Brothers nya avancerade 4-tums etikettskrivare för skrivbordet är effektiva och enkla att
or a small group of countries, we explore possible drivers behind the decline in income inequality in Latin America as a whole. To undertake this task, we utilize an array of methodologies—including correlation and econometric techniques. To start, we look at simple correlations between changes in policy variables and changes in income inequality
