A Simple Framework For Building Predictive Models

1y ago
2.39 MB
18 Pages
Last View : 12d ago
Last Download : 11m ago
Upload by : Camryn Boren

A Simple Framework for BuildingPredictive ModelsA Little Data Science Business GuideAuthored by:Hal Kalechofsky, Ph.D.September 2016

TableofContents1. Introduction.31.1 Working Definition of Predictive Models.41.2 Problem Statement.41.3 Goal of this Publication.42. Audience.53. Business Usage of Predictive Models.53.1 Use Cases.64. Modelling Framework.64.1 Recommended Predictive Modelling Business Requirements.64.2 The Modelling Process.74.2.1 Plan the model.74.2.2 Build the model.74.2.3 Implement the model.84.3 Modelling Longevity and Considerations.84.3.1 Types of Predictive Models.94.4 Supervised and Unsupervised Learning. 104.5 Practical Considerations When Working with the Data. 105. Cognitive Architecture. 135.1 Data Management Strategy. 135.2 Basic Model Architecture. 146. How Do You Know Your Model is Successful?. 146.1 Statistical Uplift. 146.2 Business Success Measures. 157. Appendix A: References. 168. About the Author. 17A Simple Framework for Building Predictive Models 2

1. Introduction“Statistics are like a bikini: What they reveal is interesting, but what they conceal is vital.”Most humans seem to want to predict the future. It is a natural human desire. To know in advanceabout the weather, the stock market, the next card coming up in blackjack - what a power tohave over the natural world!After decades of research anddevelopment, computer science andinformation technologies are nowreaching a point where predictive modelsare an important, if not indispensable,part of the business ecosystem. Manytechnologies, including fast computingpower, inexpensive storage, cloudcomputing, voice recognition, mobiledevices, artificial intelligence (cognitivecomputing), and advanced applicationsoftware are combining to make thispossible. The world is generating vastamounts of data. To cite just one example,the amount of data that will be generatedby the Internet of Things (IoT) in the yearsto come will dwarf what we today call“Big Data.” Recent trends are allowingpredictive analytics and models to be democratized and spread to smaller organizations andindividual users, avoiding the need for large software budgets or armies of data scientists tocreate and analyze the insights generated.Business leaders in all industries will want to not only be aware of the data science forces shapingour future economy, but also to be well versed in how best to use and to make the most of thesecoming opportunities.As human beings, unlike most other animals, we have the power to shape our environment.One main way of doing this is to assess situations based on data and evidence and then planand strategize for the future. One might go so far as to say this is an evolutionary necessity forsurvival, or at least for improved chances for survival. It is interesting how our tools (of whichinformation technology is one powerful example) reflect what we need (or think we need)to survive throughout history. Just think back through the wheel, fire, steel, weapons, travel,medicine and information technology.Viewed in this way, predictive computing models are an extension of our natural survivalinstinct. In this data-intensive world, predictive models are more important than ever in orderto make sense out of what is around us and to estimate, assess, or plan for what might happenin the future.A Simple Framework for Building Predictive Models 3

1.1 Working Definition of Predictive Models“It’s hard to make predictions, especially about thefuture.”A short definition of a predictive model is: Using data to make decisions.A longer definition might be: Using data to make decisions and to take actionsusing models that are empirically derived andstatistically valid.Predictive modelling is a commonly used statisticaltechnique to predict future behavior. Predictivemodelling solutions are a form of data-miningtechnology that works by analyzing historical andcurrent data and generating a model to help predictfuture outcomes.Predictive models are created whenever data is usedto train a predictive modelling technique. To put itanother way: data predictive modelling technique predictive model.A predictive model is the result of combining data andmathematics, where learning can be translated into thecreation of a mapping function between a set of inputdata fields and a response or target variable.1.2 Problem Statement“I know that half my marketing spend works . I justdon’t know which half.”Business metrics do a great job at summarizing thepast. However, if you want to predict how customerswill respond in the future, there is one place to turn —predictive analytics.There are many business reasons why it is important tomake use of predictive models. Any time you would liketo know something about the future, it is useful to havea predictive model. Although an educated guess is fine,imagine being right more than half the time! Imaginebeing right most of the time or all of the time.Here are just a few reasons to consider using predictiveanalytics and models in your business: Future Potential Revenue or Cost Drivers- How can I know my product is right forthis market?- What is the best way to continuously lowermy cost of goods sold? Risk Management- How can I identify fraud or increase trust infinancial transactions?- Which business scenario is the likeliest towin out?- How can I build and maintain advantageover my competitors? Operations- How can I save money, resources and timeby anticipating when and how physicalassets are likely to break down, andpreventing it? Customer Relationships- What will my customers want in the future?- How can I solve customer problems beforethey happen?It is sometimes said that decisions made on wrong dataare better than decisions made on no data at all. This isbecause even if the data are wrong, at least one can usehuman skepticism and questioning techniques to learnfrom it. Without data, this is not possible.Business leaders often use intuition, a “gut feeling” orrevenue streams to forecast future market conditions.Sometimes they are right, sometimes not. This couldbe called an example of an emotional or gut-levelpredictive model. Given all the data we have gatheredin our high-tech world, it can be useful to supplementthis instinct with predictable information so that onecan evaluate the decisions to make.1.3 Goal of this PublicationThe goal of this paper is to present a simple frameworkfor developing predictive or statistical models formodern business purposes.Modern means the first few decades of the 21stcentury, with all of our high-tech measurements andcomputational apparatus brought to bear. What wewere able to do with predictive modelling a generationor more ago and what we can do at the present timemay share mathematical techniques, yet are different intheir historical anchor points.This is not intended to be an academic or universityresearch type of paper; this paper is presenting aframework, not an exposition of deep data sciencetechniques. In the modern world, we have educatedmany competent analysts and data scientists and havebuilt many software systems to apply the necessaryA Simple Framework for Building Predictive Models 4

predictive modelling techniques, so it is not needed tocover that here.This paper is meant to be a primer, not a detailed essay.It does not instruct people on how to build models,but covers the steps involved and the practical issuesto consider. This paper is designed to be relatively shortand to deliver information efficiently in a short amountof the reader’s time.2. AudienceThis paper is written for a variety of audiences. Thereader should be familiar with modern computersystems and have some level of curiosity. Manybusiness people these days are well versed in technologyand technical reasoning. Trained statisticians and datascientists who are practitioners of predictive modelsmay know the theory and the mathematics, but mayget something out of the business discussions.A list of role/resource types that may benefit from thispaper are listed below: Project Leaders: who desire to have a moredetailed understanding of predictive modellingmethods and techniques to better manage andinteract with their practitioners Business Analysts: who must develop andinterpret the models, communicate the resultsand make actionable recommendations Big Data Analysts: who are under increasingpressure to transform their deluge of data froma liability to an asset Functional Analytic Users: Customer RelationshipManagers, Risk Analysts, Business Forecasters,Statistical Analysts, Social Media and Web DataAnalysts, Fraud Detection Analysts, AuditSelection Managers, Direct Marketing Analysts,Medical Diagnostic Analysts, Market Timers Data Scientists: who desire to extend thescientist aspect of the role with formal processand hands-on methodological practice IT Professionals: who wish to gain a betterunderstanding of the data preparation, analyticsand analytic sandbox development requirementsto more fully support the growing demand foranalytic IT support Anyone overwhelmed with data and starved foractionable insights3. Business Usage of Predictive Models“If you predict it, you own it.”This section describes some of the business usages ofpredictive analytics and predictive models.Predictive analytics encompasses a variety of statisticaltechniques from predictive modelling, machinelearning, and data mining that analyze current andhistorical facts to make predictions about future orotherwise unknown events. When most lay peoplediscuss predictive analytics, they are usually discussingit in terms of predictive modelling. Indeed, predictivemodelling is at the heart of predictive analytics.Predictive analytics is used in marketing, ecommerce,financial services, actuarial science and il,travel, medical and healthcare, child protection,pharmaceuticals, capacity planning, supply chain, andother fields.A Simple Framework for Building Predictive Models 5

3.1. Use Cases4. Modelling FrameworkThe table below lists predictive analytics businessapplications. The first column names the type ofbusiness benefit, and the second column identifies thetype of customer prediction required – that is, whichbehavior or action must be predicted to undertakeeach business application. As there are many suchapplications, this list includes only the most pervasivein commercial deployment to date.“The best way to predict the future is to create it.”BusinessApplicationWhat isPredictedReference orCompanyCase StudyCustomer retentionCustomerdefection/churn/attritionReed Elsevier,and Telenor,Published articleeCommerceHow to sellsuccessfully onlineAmazon.comDirect marketingCustomer responseCharles SchwabPublished tflix Prizeleader, HSBC &Amazon.comBehavior-basedadvertisingGoogle, Yahoo!Which ad customer and ClickwillForensics,click“ 1 million”case studyEmail targetingMessage that willgenerate customerresponseCredit scoringDebtor risk or fraud Wells FargoFundraising fornonprofitsDonation amountInsurance pricingand selectionInsuranceApplicant response,Journal, Pinnacolinsured riskAssuranceWorld WildlifeFundNRAThere are many more applications of predictive analytics,including collections, supply chain optimization, humanresource decision support for recruitment and humancapital retention, and market research survey analysis.The way predictive scores help your business dependson the customer behavior predicted – just aim predictiveanalytics towards the right customer predictiongoal and fire away. This is why it is sometimes said,“If you predict it, you own it!”4.1 Recommended Predictive ModellingBusiness RequirementsWhile they may share many characteristics andtechniques, building statistical or predictive modelsfor a scientific or university research context can bedifferent than a similar exercise in a business context.For business-oriented projects, here are some basiccore modelling requirements and considerations. Formulate a clear and transparent framework:- What are the objectives? What are wemodelling for?- Predictive models alone do not createbusiness value, but rather need to beeffectively deployed either into a decision making or business process. Make room for using business knowledge in theprocess:- Involve business users in the modellingprocess to ensure clear requirements,communication and purpose verification.- Business experience and heuristics maycontain valuable nuggets for how to use thedata.- Do solutions pass the “Does it makebusiness sense” test? Make sure that data are appropriate formodelling:- Quality, scope and quantity suitable to meetobjectives- Structured in line with underlying processesbeing modeled to maximize the signal Don’t just make best statistical fits to historicaldata:- Test predictive power and consistency ofpatterns over time and across validationsamples Think about managing the model:- How to integrate the model into existingbusiness systems- After implementing, how to track andmanage the interactions, and how tomaintain, tweak, or update the modelgoing forward in an efficient mannerA Simple Framework for Building Predictive Models 6

4.2 The Modelling ProcessScope“The devil is in the details.”A clear specific objective for the model is required.Each model is developed for its own specific purposeand cannot be used effectively in another situation.For example a model that predicts ecommerce onlinecustomer churn cannot be used to predict credit cardchurn. An example of a clearly defined model objectivecontains the event or action that the model is to predictand the period when it is likely to happen. For example,the objective could be to predict customers that arelikely to miss a credit card payment within the nextmonth.Let’s take a look at the process of predictive modelling.There is a wealth of information on the Web about this.However, to understand the strategic areas, it is usefulto break down the process of prediction into just a fewessential components.The below diagram is a good depiction of the predictivemodelling process across a wide range of businesses.Prepare the DatasetThe dataset created to use for the model might broadlycover diverse information, such as product, case,behavioral, demographic, geographic, competitordetails or weather. The variables that are not included inthe dataset will not form part of the prediction. Variablescover both static fields such as income and triggerssuch as change in spend. Both technical and businesspeople need to be involved in the decisions relatingto the contents of the dataset. Focus needs to be onthe behavioral information as this is more powerful forpredictions than demographic data. It is useful to giveconsideration to which customers to exclude from themodel build process.Broadly speaking, predictive analysis and modelling canbe divided into three parts: Plan, Build, and Implement.4.2.1 Plan the ModelThe next step includes scoping and planning. In thePlanning section, the sub-phase activities here arecalled Scope and Prepare. This part of the exercisemight take around 40% of the total time.To build a predictive model, you first need to assemblethe datasets that will be used for training. You mustformulate clear objectives, cleanse and organize thedata, perform data treatment including missing valuesand outlier fixing, make a descriptive analysis of thedata with statistical distributions, and create data setsused for the model-building.Customers need to be excluded if they are goingto impair the performance of the model. Potentialexclusions include things like bad debt, staff and newcustomers as they have insufficient history. The actualexclusions applied relates to the specific purpose of themodel. For example, if the model is to predict customersthat are likely to carry bad debts, bad debt customerswould be included in the model. In practice, exclusionsare applied as filters when creating the dataset andshould be noted down.4.2.2 Build the ModelHere you will write model code, build the model,calculate scores, and validate the data. In the Buildsection, the Sub-phases are known as Model andValidate. This might take around 20% of the overall time.This is the part that can be left with the data scientistsor technical analysts. The technical aspects of buildingmodels are not covered in this paper.A Simple Framework for Building Predictive Models 7

The model will be built using a sample from the data setcreated. The resulting model will contain a subset of theoriginal list of variables considered for the model. This isacceptable and happens because some of the variablesconsidered for the model will be correlated with eachother, for example, a product type and product family,or the floor number and height of building. Othervariables will have been discarded as they add little ornothing to the model’s predictive power.Deploy the ModelIt’s time to apply the model. The model will be builton a subset of data. Once the model is complete andhas been validated, it will be run over the customer,product, or case base.AssessThe model developed will be an equation that, whenapplied to the customer base, will allocate a scoreto each customer. The score represents a customer’slikelihood ‘to do’ whatever the model is predicting,such as predict a customer’s expected order value orlikelihood to churn within a month. If you are buildinga model for product failures, you may have a score thatcalculates the likelihood of a product to fail in a giventime frame.It is typical to generate rankings from your model– depending what you are modelling – customersor products. You will want to understand theperformance of your model. The ranking scores allowa customer or product base to be ranked in order ofthe predicted score, such as from most likely to leastlikely to churn or from most popular product to least. Inreality, some customers will churn and some customerswill not. Therefore, in absolute terms, these predictionswill not be accurate. The ranked list provides a superbbase upon which to vary the treatment, and thereforethe level of service or marketing spend, to groups ofcustomers.Validate the ModelMonitorCalculate a ScoreTypically models are validated against a hold out group.This group contains customers that have not beenincluded in the development of the model. As such,they represent a group of previously unseencustomers that are representative of the customerbase. To achieve an accurate prediction of lift, the holdout group must not be made up of the customersthat have been excluded from the modeldevelopment process. The same idea applies ifyou are building a product failure score model.4.2.3 Implement the ModelHere you will deploy and apply the model, rankcustomers or products, use the model outcomes forsome business purpose, estimate model performance,assess and monitor the model, and drive initiativesbased on the model. The Sub-phases here are knownas Deploy, Assess, and Monitor.You need to think about accessing, storing, and usingthe data. This might take around 40% of the overalltime, require IT department work, and can be anongoing operational exercise when the model is usedcontinuously for the business.You want to consider how often to run the model acrossthe base and generate scores. You will want to monitorthe performance of the model on an ongoing basis, andperhaps take advantage of new data or techniques asthey become available.4.3 Modelling Longevity and Considerations“The simplest solution is the best.”It is useful to think of the deployment of predictivemodels in a continuous improvement scenario. Youdon’t just build the model and leave it alone. The modelmay take nourishment, watering, monitoring, and careand can improve over time as conditions change andnew data come in.A model does not have to be extremely complicatedin order to make good predictions. Like in coding,simplicity is a good guideline. It is useful to keepOccam’s Razor in mind when considering how to build amodel: Some people say Occam’s Razor is “the simplestsolution is the best.” However, if you look at the originaltext, it says “Do not multiply entities beyond necessity.”Generally, the analyst may make assumptions whenbuilding a model. However, nature does not assumeanything before forcing an event to occur. The fewerassumptions there are in a predictive model, the greaterA Simple Framework for Building Predictive Models 8

will be the predictive power. Clearly, attributes that arenot in the model will have no effect on the model’spredictions.The speed of changing business conditions is a factor inhow well the model will perform over time. For example,information technology is a faster-moving industry thaninsurance. Once a model is in use and driving actions,there is a requirement for tracking and managing theinteractions. It is a good idea to refresh model scoresover time, sometimes frequently depending on theindustry.Control files will be required to test initiatives and totest the model performance. The results from this willhelp determine how frequently the model needs tobe refreshed. As a rule of thumb, a model needs to bereviewed and possibly rebuilt annually.4.3.1 Types of Predictive Models“There are three types of lies -- lies, damn lies, andstatistics.”Many types of predictive models have been developedover the years that are useful for different classes ofproblems. Applying these techniques is the domain ofstatisticians and data scientists, and is intentionally notcovered in this short paper.1. Business RulesA business rule is a rule that defines or constrains someaspect of business and always resolves to either trueor false. Business rules are intended to assert businessstructure or to control or influence the behavior of thebusiness.2. Classification and Decision TreesA decision tree is a decision support tool that uses atree-like graph or model of decisions and their possibleconsequences, including chance event outcomes,resource costs, and utility. It is one way to display analgorithm.3. Naive BayesIn machine learning, naive Bayes classifiers are a familyof simple probabilistic classifiers based on applyingBayes’ theorem with strong (naive) independenceassumptions between the features. The techniqueconstructs classifiers: models that assign class labels toproblem instances, represented as vectors of featurevalues, where the class labels are drawn from somefinite set.4. Linear RegressionIn statistics, regression analysis is a statistical processfor estimating the relationships among variables. Linearregression is an approach for modelling the relationshipbetween a scalar dependent variable Y and one ormore explanatory variables (or independent variables),denoted X. The case of one explanatory variable iscalled simple linear regression. More than one variableis called multivariate.5. Logistic RegressionIn statistics, logistic regression, or logit regression, orlogit model is a regression model where the dependentvariable (DV) is categorical or binary.6. Neural Networks (NNs)An Artificial Neural Network (ANN) is an informationprocessing paradigm that is inspired by the waybiological nervous systems, such as the brain, processinformation.7. Machine LearningMachine learning is a type of artificial intelligence (AI)that provides computers with the ability to learn withoutbeing explicitly programmed. Machine learning focuseson the development of computer programs that canteach themselves to grow and change when exposedto new data.8. Support Vector Machines (SVMs)In Machine Learning, an SVM is a supervised learningmodel with associated learning algorithms that analyzedata used for classification and regression analysis. AnSVM maps input data vectors into a higher dimensionalspace, where an “optimal hyperplane” that separatesthe data is constructed. An SVM is a discriminativeclassifier formally defined by a separating hyperplane.In other words, given labeled training data (supervisedlearning), the algorithm outputs an optimal hyperplanewhich categorizes new examples.9. Natural Language Processing (NLP)Since NLP (content analytics) feeds predictive andprescriptive analytics, it is included here. NLP factA Simple Framework for Building Predictive Models 9

extraction can be used with descriptive statisticalformulae to make use of the wealth of unstructureddata.4.5 Practical Considerations When Working withthe Data4.4 Supervised and Unsupervised Learning1) Asking the Right Questions“Statistics can be made to prove anything - even thetruth.”“Correlation does not imply causality.”What are the business objectives of the project? Does itmake business sense?SVMs, decision trees, NNs and regression models usesupervised learning to create the mapping functionbetween a set of input data fields and a target variable.The known outcome is then used as a teacher whosupervises the learning of her pupil. Whenever the pupilmakes a mistake, the teacher provides her with the rightanswer in the hopes that the pupil will eventually get itright. For instance, when presented with a specific set ofinputs, her output will match the target.Awash in reams of data, it is critical that companiesask the right questions. Being specific is important.Predictive analytics is most efficient when used todetermine the answer to a narrow inquiry, such as thelikelihood of customer A to buy product X at time Y forprice Z, rather than the likelihood of customers buyingproduct X.Unsupervised learning requires no teacher or target.Clustering techniques fall into this category. Data pointsare simply grouped together based on their similarity. Inan ecommerce website analysis, online shoppers mightbe grouped into window shoppers or power buyers. Incase of customer churn, a clustering technique couldpotentially assign different clusters to churners andnon-churners even though the outcome is not availableduring model training.It is one thing to have someone say, “Build me apredictive model to mint money.” Is that realistic? If thegoal looks realistic, one must find out if there are evendata available to do something reasonable. Sometimesthe data will not be accessible, or it will not be orderedor structured well enough.Black-box is a term used to identify certain predictivemodelling techniques that are not capable of explainingtheir reasoning because you can’t really know whatis happening in the Black-box. Although extremelypowerful, techniques such as NNs and SVMs fall intothis category. Very often, an explanation or reason forthe model decision is required; for example, when a riskscore is used to decline a loan application or a creditcard transaction.Whenever explaining is a must, you need to considerusing a predictive modelling technique that clearlydescribes the reasons for its decisions. Scorecards fitsuch criteria very well. Based on regression models,scorecards are a popular technique used by financialinstitutions to assess risk. With scorecards, all data fieldsin an input record are associated with specific reasoncodes. Thus, it is possible to explain to the customerwhy a given decision was rendered.2) Identify Available Data3) Data Accuracy and CleanupData scientists must be aware that not all data isaccurate, arrive at an estimate of bad data, and correctfor it in their studies. Data can be bad for any numberof reasons, including self-reporting errors, corruptedfiles, poorly phrased questions, incomplete dataaggregation, and poor standardization methods.It is critical that data scientists quickly recognize andfilter bad data f

predictive analytics and predictive models. Predictive analytics encompasses a variety of statistical techniques from predictive modelling, machine learning, and data mining that analyze current and historical facts to make predictions about future or otherwise unknown events. When most lay people discuss predictive analytics, they are usually .

Related Documents:

Bruksanvisning för bilstereo . Bruksanvisning for bilstereo . Instrukcja obsługi samochodowego odtwarzacza stereo . Operating Instructions for Car Stereo . 610-104 . SV . Bruksanvisning i original

10 tips och tricks för att lyckas med ert sap-projekt 20 SAPSANYTT 2/2015 De flesta projektledare känner säkert till Cobb’s paradox. Martin Cobb verkade som CIO för sekretariatet för Treasury Board of Canada 1995 då han ställde frågan

service i Norge och Finland drivs inom ramen för ett enskilt företag (NRK. 1 och Yleisradio), fin ns det i Sverige tre: Ett för tv (Sveriges Television , SVT ), ett för radio (Sveriges Radio , SR ) och ett för utbildnings program (Sveriges Utbildningsradio, UR, vilket till följd av sin begränsade storlek inte återfinns bland de 25 största

Hotell För hotell anges de tre klasserna A/B, C och D. Det betyder att den "normala" standarden C är acceptabel men att motiven för en högre standard är starka. Ljudklass C motsvarar de tidigare normkraven för hotell, ljudklass A/B motsvarar kraven för moderna hotell med hög standard och ljudklass D kan användas vid

LÄS NOGGRANT FÖLJANDE VILLKOR FÖR APPLE DEVELOPER PROGRAM LICENCE . Apple Developer Program License Agreement Syfte Du vill använda Apple-mjukvara (enligt definitionen nedan) för att utveckla en eller flera Applikationer (enligt definitionen nedan) för Apple-märkta produkter. . Applikationer som utvecklas för iOS-produkter, Apple .

och krav. Maskinerna skriver ut upp till fyra tum breda etiketter med direkt termoteknik och termotransferteknik och är lämpliga för en lång rad användningsområden på vertikala marknader. TD-seriens professionella etikettskrivare för . skrivbordet. Brothers nya avancerade 4-tums etikettskrivare för skrivbordet är effektiva och enkla att

Den kanadensiska språkvetaren Jim Cummins har visat i sin forskning från år 1979 att det kan ta 1 till 3 år för att lära sig ett vardagsspråk och mellan 5 till 7 år för att behärska ett akademiskt språk.4 Han införde två begrepp för att beskriva elevernas språkliga kompetens: BI

**Godkänd av MAN för upp till 120 000 km och Mercedes Benz, Volvo och Renault för upp till 100 000 km i enlighet med deras specifikationer. Faktiskt oljebyte beror på motortyp, körförhållanden, servicehistorik, OBD och bränslekvalitet. Se alltid tillverkarens instruktionsbok. Art.Nr. 159CAC Art.Nr. 159CAA Art.Nr. 159CAB Art.Nr. 217B1B