Predictive HCM Using Machine Learning Data Management Platforms - Oracle
Predictive HCM Using Machine Learning Data Management Platforms Move the Algorithms; Not the Data! Charlie Berger, MS Engineering, MBA Sr. Director Product Management, Machine Learning, AI and Cognitive Analytics charlie.berger@oracle.com www.twitter.com/CharlieDataMine Nancy Estell Zoder Director, Product Strategy, HCM Applications Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright 2017, Oracle and/or its affiliates. All rights reserved. 2
Agenda 1. Concepts / Architecture 2. Product Functionality Overview 3. Predicting Employee Attrition Demo 4. “Predictive” HCM Application 5. Getting Started Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Anticipating Voluntary Employee Attrition HCM Early Warning Systems Can Avoid Huge Costs ? X ? X ? X t-help-you-predictCopyright 2017, Oracle and/or its affiliates. All rights reserved. Employee-Turnover-1.jpg
Predicting Employee Top Performers Important to Correctly Identify the Top Performers ? ! ! http://www.managers.org.uk/ /media/Images/Insights/PoachingStaff.jpg Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Example Organizations for Predictive HCM Knowledge Workers: Extremely Valuable and Difficult & Costly to Replace Financial Services Pharm/Life Sciences – Financial Advisors, Customer Service, Account Managers – Researchers, Data Scientists Healthcare Insurance – Physicians, Nurses, Managers, Medical Researchers – Claims Adjusters, Accountants, Financial Managers Software, IT Hospitality, Travel, Services – Chefs, Property Managers, Pilots, Flight Attendances, Cruise Directors Marketing – Mgrs, Creative, Data Scientists – Developers, Architects, PMs Military – Leadership, Pilots, Special Forces Sports – Boston Red Sox, New England Patriots, Boston Celtics, Soccer Copyright 2017, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Internal/Restricted/Highly Restricted 6
Dilbert on Predictive HCM Copyright 2017, Oracle and/or its affiliates. All rights reserved.
What is Machine Learning and Predictive Analytics? Automatically sifting through large amounts of data to create models that find previously hidden patterns, discover valuable new insights and make predictions Identify most important factor (Attribute Importance) Predict customer behavior (Classification) Predict or estimate a value (Regression) Find profiles of targeted people or items (Decision Trees) Segment a population (Clustering) Find fraudulent or “rare events” (Anomaly Detection) Determine co-occurring items in a “baskets” (Associations) Copyright 2016, Oracle and/or its affiliates. All rights reserved. A1 A2 A3 A4 A5 A6 A7
Traditional vs. Oracle Machine Learning/Predictive Analtyics Traditional— “Move the data” —“Don’t move the data!” Copyright 2017, Oracle and/or its affiliates. All rights reserved. 9
Traditional vs. Oracle Machine Learning/Predictive Analtyics Traditional— “Move the data” — “Move the algorithms” Simpler, Smarter Data Management Analytics / Machine Learning Architecture Copyright 2017, Oracle and/or its affiliates. All rights reserved. 10
Rapidly Build, Evaluate & Deploy Analytical Methodologies Leveraging a Variety of Data Sources and Types Transactional POS data SQL Joins and arbitrary SQL transforms & queries – power of SQL Modeling Approaches Inline predictive model to augment input data Consider: Demographics Generates SQL scripts Past purchases and workflow API for Recent purchases deployment Comments & tweets Unstructured data also mined by algorithms Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Oracle’s Machine Learning/Advanced Analytics Platforms Machine Learning Algorithms Embedded in the Data Management Platforms “Analytics Producers” “Analytics Consumers” Data Scientists, R Users, Citizen Data Scientists BI Analysts, Managers Functional Users (HCM, CRM) Data Management Advanced Analytical Platform Big Data SQL Big Data Cloud Service “Oracle Machine Learning” Big Data Cloud ORAAH—Machine Learning Algorithms, Statistical Functions R Integration for Scalable, Parallel, Distributed Execution Database Cloud “Oracle Machine Learning” Database Edition Machine Learning Algorithms, Statistical Functions R Integration for Scalable, Parallel, Distributed, in-DB Execution Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Oracle’s Machine Learning/Advanced Analytics Fastest Way to Deliver Enterprise-wide Predictive Analytics Traditional Analytics Major Benefits Oracle Advanced Analytics Data Import Data Mining Model “Scoring” Data remains in Database & Hadoop Model building and scoring occur in-database Use R packages with data-parallel invocations Leverage investment in Oracle IT Data Prep. & Transformation avings Data Mining Model Building Eliminate data duplication Eliminate separate analytical servers Data Prep & Transformation Deliver enterprise-wide applications GUI for ML/Predictive Analytics & code gen R interface leverages database as HPC engine Data Extraction Model “Scoring” Embedded Data Prep Model Building Data Preparation Hours, Days or Weeks Copyright 2016, Oracle and/or its affiliates. All rights reserved. Secs, Mins or Hours
Predictive Appl: HCM Cloud—Workforce Predictions Complete, Integrated, Embedded, Automated and Interactive “Predictive HCM” Solution Integrated data management embedded predictive analytics Full 360 degree employee view Single source of HCM data data Interactive dashboards and “What if” analysis Customizable if desired to add input variables to predictive models Additional relevant data and “engineered features” Oracle Database 12c Historical data Mobile Oracle Cloud solutions Historical or Current Data to be “scored” for predictions Assembled historical data Predictions & Insights Sensor data, Text, unstructured data, transactional data, spatial data, etc. Copyright 2017, Oracle and/or its affiliates. All rights reserved. 1
“Why Oracle? Because that’s where the data is!” – Larry Ellison, Executive Chairman and CTO of Oracle Corporation Copyright 2017, Oracle and/or its affiliates. All rights reserved. 15
Product Functionality Overview Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Dilbert on Big Data Copyright 2017, Oracle and/or its affiliates. All rights reserved. 17
Oracle’s Machine Learning/Advanced Analytics Fastest Way to Deliver Scalable Enterprise-wide Predictive Analytics Key Features Parallel, scalable machine learning algorithms and R integration In-Database Hadoop—Don’t move the data Data analysts, data scientists & developers Drag and drop workflow, R and SQL APIs Extends data management into powerful advanced/predictive analytics platform Enables enterprise predictive analytics deployment applications Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Oracle’s Machine Learning & Adv. Analytics Algorithms CLASSIFICATION – Naïve Bayes – Logistic Regression (GLM) – Decision Tree – Random Forest – Neural Network – Support Vector Machine CLUSTERING – Hierarchical K-Means – Hierarchical O-Cluster – Expectation Maximization (EM) ANOMALY DETECTION – One-Class SVM TIME SERIES REGRESSION – Linear Model – Generalized Linear Model – Support Vector Machine (SVM) – Stepwise Linear regression – Neural Network – LASSO ATTRIBUTE IMPORTANCE FEATURE EXTRACTION – Principal Comp Analysis (PCA) – Non-negative Matrix Factorization – Singular Value Decomposition (SVD) – Explicit Semantic Analysis (ESA) TEXT MINING SUPPORT – Algorithms support text type – Tokenization and theme extraction – Explicit Semantic Analysis (ESA) for document similarity A1 A2 A3 A4 A5 A6 A7 – Minimum Description Length – Principal Comp Analysis (PCA) – Unsupervised Pair-wise KL Div – CUR decomposition for row & AI STATISTICAL FUNCTIONS – Basic statistics: min, max, median, stdev, t-test, F-test, Pearson’s, Chi-Sq, ANOVA, etc. ASSOCIATION RULES – A priori/ market basket – Holt-Winters, Regular & Irregular, with and w/o trends & seasonal – Single, Double Exp Smoothing PREDICTIVE QUERIES – Predict, cluster, detect, features R PACKAGES – CRAN R Algorithm Packages through Embedded R Execution – Spark MLlib algorithm integration SQL ANALYTICS – SQL Windows, SQL Patterns, SQL Aggregates EXPORTABLE ML MODELS OAA (Oracle Data Mining Oracle R Enterprise) and ORAAH combined OAA includes support for Partitioned Models, Transactional, Unstructured, Geo-spatial, Graph data. etc, – C and Java code for deployment Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Manage and Analyze All Your Data Data Scientists, R Users, Citizen Data Scientists Architecturally, Many Options and Flexibility SQL / R Boil down the Data Like Big Data SQL / R Object Store “Engineered Features” – Derived attributes that reflect domain knowledge—key to best models e.g: Counts Totals Changes over time Copyright 2017, Oracle and/or its affiliates. All rights reserved. 20
Oracle Advanced Analytics 12.2 Unofficial Model Build Time Performance OAA 12.2 Algorithms Rows (Ms) T7-4 (Sparc & Solaris) X5-4 (Intel and Linux) Model Build Time (Secs / Degree of Parallelism) Attributes Importance 640 28s / 512 K Means Clustering Expectation Maximization 640 159 161s / 256 455s / 512 Wow! That’s Fast! 44s / 72 268s / 144 588s / 144 Naive Bayes Classification GLM Classification GLM Regression 320 17s / 256 23s / 72 640 154s / 512 363s / 144 640 55s / 512 93s / 144 In 24 hours, could build new predictive models for entire Support Vector Machine (IPM solver) 640 404sattributes, / 512 1411s / 144 United States Population, for 400 4 times! Support Vector Machine (SGD solver) 640 84s / 256 188s / 72 The way to read their results is that they compare 2 chips: X5 (Intel and Linux) and T7 (Sparc and Solaris). They are measuring scalability (time in seconds) with increase degree of parallelism (dop). The data also has high cardinality categorical columns translates inreserved. 9K mining Copyright 2016, Oracle and/orwhich its affiliates. All rights attributes (when algorithms require explosion). There are no comparisons to 12.1 and it is fair to say that the 12.1 algorithms could not run on data of this size.
“1-Click”, 100% Automated, SQL Predictive Queries Oracle’s Machine Learning Accelerates New Possibilities Predictive Queries “1-Click” immediate machine learning model build AND model apply as a SQL query Classification & regression – Multi-target problems Clustering query Anomaly query Feature extraction query “1-Click” Predictions! Automatically creates multiple anomaly detection models “Grouped By” and “scores” by partition via powerful highly automated SQL query Copyright 2016, Oracle and/or its affiliates. All rights reserved.
Machine Learning 101 1. Define a Business Problem — “Find Top Performers” 2. Assemble “the Right” Input Data and “Engineered Features” Engineered Features Employee Variables – – – – Age Income Income Growth % over past n time periods – – – – – – # dependents Healthcare Marital Status Mobile spouse? Outside work interests Machine Learning/ “Model” Prediction Family Variables Y F(X1, X2, .Xn) ? Work Variables – Years same manager – % travel – Environmental Variables – Other local employers hiring? – Benefits vs. local employers – HCM Predictive Workforce already does this for you! Copyright 2016, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved. 3
Build Predictive Models on an Attribute Oracle’s Machine Learning Accelerates New Possibilities Machine Learning Model Function(X1, X2, .X) Y (LTV BIN); Probability Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Build Predictive Models on an Attribute Oracle’s Machine Learning Accelerates New Possibilities Machine Learning Model Function(X1, X2, .X) Y (LTV BIN); Probability Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Build Predictive Models on an Attribute Oracle’s Machine Learning Accelerates New Possibilities Machine Learning Models Function(X1, X2, .X) Y2 (BankFunds Copyright 2017, Oracle and/or its affiliates. All rights reserved. Y (LTV BIN); Probability
Build Predictive Models on an Attribute Oracle’s Machine Learning Accelerates New Possibilities Machine Learning Models Function(X1, X2, .X) Y2 (BankFunds Copyright 2017, Oracle and/or its affiliates. All rights reserved. Y (LTV BIN); Probability
More Data Variety—Better Predictive Models Engineered Features – Derived attributes/variable that reflect domain knowledge—key to best models 100% Increasing sources of relevant data can boost model accuracy Naïve Guess or Random Responders Model with “Big Data” and hundreds -- thousands of input variables including: Demographic data Purchase POS transactional data “Unstructured data”, text & comments Spatial location data Long term vs. recent historical behavior Web visits Sensor data etc. 100% Model with 20 variables Model with 75 variables Model with 250 variables 0% Population Size Copyright 2017, Oracle and/or its affiliates. All rights reserved.
A Quick Peak Under the HCM Workforce Predictions Hood How Did They Do That? Copyright 2016, Oracle and/or its affiliates. All rights reserved.
HCM Predictive Workforce Predictive Analytics Applications Employee Attrition Model: Y Function(X1, X2, .X) Human Capital Management Powered by OAA Oracle Advanced Analytics factoryinstalled predictive analytics Employees likely to leave and predicted performance Top reasons, expected behavior Real-time "What if?" analysis Link to Oracle HCM on O.com HCM Predictive Workforce demo Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Attributes Used in Workforce Predictions (can be extended) Age Amount of leave taken in the previous year Amount of sickness in the current year Amount of sickness in the previous year Average salary change Average time in each department Average time in each grade Average time in each job Average time in each position Average time with each manager Change in current performance Current appraising manager Current assignment status Current department Current enterprise Current grade Current job Current legal employer Current location Current manager performance rating Current or most recent manager Current performance rating Current performance self rating Current position Difference between manager rating and self rating Disabled Ethnicity FTE Gender Has a second passport Home city Home country Increase in sickness absences over previous year Increase in sickness over previous year Latest salary change Legislation Length of service Manager's performance Marital status Nationality Normal end time Normal start time Normal working hours Number of different departments Number of different grades Number of grade changes in the last 2 years Number of manager changes in the last 5 years Number of sickness absences in the previous year Number of stock options compared to others in the same grade Potential profit on stock Previous manager performance rating Ratio of vested to unvested options Reason for latest salary change Time in current department Time in current grade Time in current job Time in current position Time since last leave Time since last marital status change Time since last probation ended Time since last received options Time since last sickness Time since latest salary change Time until contract expiration Time until next salary review Time until the next performance review Time until work permit or visa expiration Time with current manager Tobacco user Total enterprise leave Willing to relocate domestically Willing to relocate internationally Worker category Worker is a rehire Worker is an employee Worker's performance compared to peers Worker's stock options compared to peers Copyright 2016, Oracle and/or its affiliates. All rights reserved. 44
Machine Learning & Advanced Analytical Methodologies Data Preparation & Adv. Analytical Processes Runs In-Data Mgmt Platform Additional relevant data and “engineered features” Historical or Current Data to be “scored” for predictions Oracle Database 12c Historical data Assembled historical data Predictions & Insights Sensor data, Text, unstructured data, transactional data, spatial data, etc. Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2016, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Internal/Restricted/Highly Restricted 46
Fraud Prediction Demo Automated In-DB Analytical Methodology drop table CLAIMS SET; exec dbms data mining.drop model('CLAIMSMODEL'); create table CLAIMS SET (setting name varchar2(30), setting value varchar2(4000)); insert into CLAIMS SET values ('ALGO NAME','ALGO SUPPORT VECTOR MACHINES'); insert into CLAIMS SET values ('PREP AUTO','ON'); commit; begin dbms data mining.create model('CLAIMSMODEL', 'CLASSIFICATION', 'CLAIMS', 'POLICYNUMBER', null, 'CLAIMS SET'); end; / -- Top 5 most suspicious fraud policy holder claims select * from (select POLICYNUMBER, round(prob fraud*100,2) percent fraud, rank() over (order by prob fraud desc) rnk from (select POLICYNUMBER, prediction probability(CLAIMSMODEL, '0' using *) prob fraud from CLAIMS where PASTNUMBEROFCLAIMS in ('2to4', 'morethan4'))) where rnk 5 order by percent fraud desc; Copyright 2017, Oracle and/or its affiliates. All rights reserved. Automated Monthly “Application”! Just add: Create View CLAIMS2 30 As Select * from CLAIMS2 Where mydate SYSDATE – 30 Time measure: set timing on;
Predicted Voluntary Termination for Team Copyright 2016, Oracle and/or its affiliates. All rights reserved. 48
Predicted Performance for Individual Worker Copyright 2016, Oracle and/or its affiliates. All rights reserved. 49
Copyright 2016, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Internal/Restricted/Highly Restricted 50
HCM Predictive Workforce Predictive Analytics Applications Fusion Human Capital Management Powered by OAA Oracle Advanced Analytics factoryinstalled predictive analytics Employees likely to leave and predicted performance Top reasons, expected behavior Real-time "What if?" analysis Link to Oracle HCM on O.com HCM Predictive Workforce demo Copyright 2017, Oracle and/or its affiliates. All rights reserved.
ML Model Deployment for Real-Time Scoring Real-Time Scoring, Predictions and Recommendations On-the-fly, single record apply with new data (e.g. from call center) Select prediction probability(CLAS DT 1 15, 'Yes' USING 7800 as bank funds, 125 as checking amount, 20 as credit balance, 55 as age, 'Married' as marital status, 250 as MONEY MONTLY OVERDRAWN, 1 as house ownership) from dual; Social Media Call Center Likelihood to respond: Get AdviceBranch Office R Mobile Web Email Copyright 2017, Oracle and/or its affiliates. All rights reserved. Oracle Cloud
HCM Workforce Predictions vs. Traditional Aproaches Single Analytical Platform vs. Sum of Parts Oracle HCM Workforce Predictions Traditional Approaches (Single, Integrated Solution) (SAS, SPSS, R, Data Scientist, Consultant) Simpler , More Powerful Integrated Architecture Data movement Immediate Information, Insights and Predictions Slow Built-in Predictive HCM “What if?” Analysis Data Duplication HCM Application Lacks security Sum of Parts at best Outer loop for Predictive No interactive “What if?” possible Copyright 2017, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle 5
Machine Learning & Advanced Analytics Getting Started Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Getting Started—Oracle ML/AA Resources & Links Oracle Advanced Analytics Overview Information Oracle's Machine Learning and Advanced Analytics 12.2c and Oracle Data Miner 4.2 New Features preso Oracle Advanced Analytics Public Customer References Oracle’s Machine Learning and Advanced Analytics Data Management Platforms white paper on OTN Oracle INTERNAL ONLY OAA Product Management Wiki and Beehive Workspace (contains latest presentations, demos, product, etc. information) YouTube recorded Oracle Advanced Analytics Presentations and Demos, White Papers Oracle's Machine Learning & Advanced Analytics 12.2 & Oracle Data Miner 4.2 New Features YouTube video Library of YouTube Movies on Oracle Advanced Analytics, Data Mining, Machine Learning (7 “live” Demos e.g. Oracle Data Miner 4.0 New Features, Retail, Fraud, Loyalty, Overview, etc.) Overview YouTube video of Oracle’s Advanced Analytics and Machine Learning Getting Started/Training/Tutorials Link to OAA/Oracle Data Miner Workflow GUI Online (free) Tutorial Series on OTN Link to OAA/Oracle R Enterprise (free) Tutorial Series on OTN Link to Try the Oracle Cloud Now! Link to Getting Started w/ ODM blog entry Link to New OAA/Oracle Data Mining 2-Day Instructor Led Oracle University course. Oracle Data Mining Sample Code Examples Send an email now to charlie.berger@oracle.com and my “away message” will send you many of these links” Additional Resources, Documentation & OTN Discussion Forums Oracle Advanced Analytics Option on OTN page OAA/Oracle Data Mining on OTN page, ODM Documentation & ODM Blog OAA/Oracle R Enterprise page on OTN page, ORE Documentation & ORE Blog Oracle SQL based Basic Statistical functions on OTN Oracle R Advanced Analytics for Hadoop (ORAAH) on OTN Analytics and Data Summit , All Analytics, All Data, No Nonsense. March 20-22, 2018, Redwood Shores, CA Copyright 2017, Oracle and/or its affiliates. All rights reserved.
www.biwasummit.org www.analyticsanddatasummit.org Copyright 2017, Oracle and/or its affiliates. All rights reserved.
Copyright 2017, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle 58
Predictive Appl: HCM Cloud—Workforce Predictions Integrated data management embedded predictive analytics Full 360 degree employee view Single source of HCM data data Interactive dashboards and "What if" analysis Customizable if desired to add input variables to predictive models Mobile Oracle Cloud solutions 1 4
SAP HCM Reporting/Validation Query Manager : Fiori App Document BuilderTM HCM Reporting as a Service Variance Monitor for HCM SAP HCM Transformation System Landscape Optimization (SLO) SAP HCM Migration SAP HCM Carve Out SAP HCM CC Carve Out SAP HCM Merger SAP
E-Business Suite and HCM Cloud E-Business Suite and ERP/SCM Cloud E-Business Suite and CX Cloud 10 Oracle E-Business Suite and Practical Coexistence Scenarios Extend with SaaS –Hybrid is the New Normal 1.EBS ERP to Oracle HCM Cloud 2.EBS Payroll with Oracle HCM Cloud 3.EBS HCM to Oracle Taleo Cloud 4.EBS HCM to Oracle Talent Management Cloud .
39 HCM HCM6 Chao Chih Hung 500,000 40 HCM HCM7 ĐỗThịCúc 100,000 41 HCM HCM7 Phạm ThịThơm 100,000 42 HCM HCM8 Phạm ThịCúc 100,000 43 HCM HCM3 Yang Kuo Lien 500,000 44 HN HN1 Dương Ngọc Cường 100,000 45 HN HN1 Lê ThịÁi Ninh 500,000 46 HN HN1
predictive analytics and predictive models. Predictive analytics encompasses a variety of statistical techniques from predictive modelling, machine learning, and data mining that analyze current and historical facts to make predictions about future or otherwise unknown events. When most lay people discuss predictive analytics, they are usually .
SAP Predictive Analytics Data Manager Automated Modeler Expert Modeler (Visual Composition Framework) Predictive Factory Hadoop / Spark Vora SAP Applications SAP Fraud Management SAP Analytics Cloud HANA Predictive & Machine Learning Spatial Graph Predictive (PAL/APL) Series Data Streaming Analytics Text Analytics
SAP S/4HANA customers may also using SAP ERP HCM on-premise and might not be willing/able to move to SuccessFactors in the near term and like to continue to use SAP ERP HCM. SAP HCM can be run in either a separate instance or single instance with SAP S/4HANA. The integration among both on-premise instances (SAP HCM with
gate the HCM performance to achieve feature extraction and classification and then compare the performance when replacing HCM with Fast Discrete Cosine Transform (FDCT). Unlike FDCT, HCM could extract the entire features without any loss. Also the experiments have shown that HCM is generally better than FDCT and provides a
n Julian Le Grand supports the introduction of stronger market incentives to prompt improved performance among secondary care providers. He: – notes the positive effect market incentives have had in primary schools – argues that new structures (such as new systems of regulation and performance measurement) will help minimise undesirable consequences – suggests that, in 1991, the NHS .
