Applying Predictive Analytics To Public Sector Applications

Applying Predictive Analytics to PublicSector ApplicationsBenjamin Chard, Senior Client TechnicalProfessional, IBM1 2012 IBM Corporation

Traditional Commercial Applications of Predictive Analytics2

Government Challenges with Unique Solutions§ Threat Detection: Combining predictive with traditionalinvestigative rules-based technology.§ Unusual Events: Modeling “normal” behavior to detectoutliers.§ Non-Traditional Data: Evaluating and RationalizingJobs Descriptions for Government Employer§ Force Deployment: Optimal deployment of force toprevent the next adverse event.§ Service Life / Time to Failure: Predictive maintenanceto maximize readiness.

Government Challenges with Unique Solutions§ Threat Detection: Combining predictive with traditionalinvestigative rules-based technology.§ Unusual Events: Modeling “normal” behavior to detectoutliers.§ Non-Traditional Data: Evaluating and RationalizingJobs Descriptions for Government Employer§ Force Deployment: Optimal deployment of force toprevent the next adverse event.§ Service Life / Time to Failure: Predictive maintenanceto maximize readiness.

Combining Targeting Rules and PredictiveHigh5th GenerationText Analytics(Free Form Data)Overall TargetingEffectiveness4th GenerationPredictive Models (Learnsfrom Data)3rd GenerationWeighted Rule SetsAnalytical Tools (Data Query)2nd GenerationTargeting Criteria (Binary)1st GenerationHuman ExperienceLowPattern ComplexityHigh

Threat Detection – Combining Predictive and RulesDevelop a Predictive Model for assisting in the detectionof violations that maximizes the effectiveness ofexisting investigative systems§ Selected Violations: Select examination records that were selected for investigation. Afterinvestigation not all cases resulted in a true violation: 93% False Positives. Each case wasidentified as having a true Violation or Not after investigation.§ Priority List. Seizures Cases were further filtered to focus on the available Watch Lists.§ Rules Fired Data: These cases were also joined to the Rules Fired data with the goal toexamine how predictive modeling can assist and complement existing selection systems§ Merge With Additional Data SourcesCan we use Predictive Models to reduce the 93% Wasted Examsrate whilst ensuring we still maximize the number of violationscaptured.

Predictive Results – Goodness of FitThe Predictive approach created models that are over 98%accurate when predicting a violation, and over 96% accuratewhen predicting no violationActual ViolationViolationViolationNo ViolationNo ViolationModel Predicted ViolationViolationNo Violation (False Negative)No ViolationViolation (False Positive)# 1%The Predictive approach is approximately 10 times more efficientActual (1st 3 Generations)Model Prediction# Exams # Violations Found % Effectiveness % Total Violations Found110747416.69%100.00%108372867.22%98.25%

Scoring New Data§ Score: Propensity of Violation§ Data: Connect to Data’ containing cases whereexistence of an Violation is unknown.§ Impact: Sort entries by score and examine cases withhighest propensity for violation.Connect toIncomingDataTraditionalRules BasedTargetingPredictiveModel: UsesRules FiredDeploy intoOperationalSetting

Government Challenges with Unique Solutions§ Threat Detection: Combining predictive with traditionalinvestigative rules-based technology.§ Unusual Events: Modeling “normal” behavior to detectoutliers.§ Non-Traditional Data: Evaluating and RationalizingJobs Descriptions for Government Employer§ Force Deployment: Optimal deployment of force toprevent the next adverse event.§ Service Life / Time to Failure: Predictive maintenanceto maximize readiness.

Unusual Events: Predictive Analytics for Insider ThreatInsider Threat operations are historically “Reactive” in nature, leads are based solely on policyviolations, and audit data volumes continue to grow with no correlation (by design) across silos.A Predictive Analytic approach provides a number of benefits ,100% inclusion of all data * Persistent risk scoring of constantly changing behaviors * “Analyticmonitoring” to automatically score actual activity vs. forecasted activity * Baselining toautomate the identification of behavioral changes* Early warning of disgruntlement (Sentiment)Resulted in the early (Proactive) warning/identification of offenders; Reduced data overloadand focused few investigators on highest priority incidents (10 high priority leads vs. 1,000daily alarms); Ad-hoc Director-level requests were responded to in minutes to hours vs.days/weeks.10

Predicting Unusual Events: Insider Threat§ Data Transformation§ Leverage data generated by compliance, security, and riskmanagement tools§ Leverage relevant data from HR, log files, E-mail servers, chatlogs, etc.§ Baseline Employee Activity§ Identify anomalous behavior§ Model and Predict§ Determine which individuals are at high risk for insider crimes,and the likely impact of those crimes§ Deploy§ Embed all processes into an existing reporting system, portal,or any other 3rd party process

26 day (longterm) exponentialmoving averageNumber of MBs(uploaded ordownloaded)12 day (short term) exponentialmoving averageA large differencebetween short term andlong term averagesindicates a strong trend

Government Challenges with Unique Solutions§ Threat Detection: Combining predictive with traditionalinvestigative rules-based technology.§ Unusual Events: Modeling “normal” behavior to detectoutliers.§ Non-Traditional Data: Evaluating and RationalizingJobs Descriptions for Government Employer§ Force Deployment: Optimal deployment of force toprevent the next adverse event.§ Service Life / Time to Failure: Predictive maintenanceto maximize readiness.

Non-Traditional Data: Classifying and Rationalizing JobDescriptions1616

Business AnalyticsSearch: Employers looking for Applicants§ Provide Minimumqualifications criteria§ Provide ‘Key Word’Search for specificqueries.§ PA- statisticalmatching thatevaluates applicantsdata in its entirety. 2010 IBM Corporation

Business AnalyticsSearch: Applicants Looking for Jobs§ Provide Minimumqualifications criteria§ Provide ‘Key Word’ Searchwith for specific queries suchas Job Title.§ PA: Provide prioritized jobsthat match on majority ofdata on my application.§ PA: Selects jobs that I amwell qualified for that I maynot have previouslyconsidered. 2010 IBM Corporation

Business AnalyticsSearch Results§ PA: Provide ranked listsbased on similarity /dissimilarity score derivedfrom application and jobcharacteristics§ PA: Provide ranking basedon “propensity to succeed”using historical resultsfrom previous applicants 2010 IBM Corporation

Business AnalyticsConvert Pick lists toopen ended questions§ Uses applicants ownlanguage to describe skills§ Automatically extract keyconcepts from description.§ Linguistic basedalgorithms automaticallyrationalize and categorizeskills based on type of taskor other criteria. 2010 IBM Corporation

Job Descriptions – Comparing Candidate jobs toexisting group2121

Job Descriptions – Comparing candidate jobs toexisting group2222

Government Challenges with Unique Solutions§ Threat Detection: Combining predictive with traditionalinvestigative rules-based technology.§ Unusual Events: Modeling “normal” behavior to detectoutliers.§ Non-Traditional Data: Evaluating and RationalizingJobs Descriptions for Government Employer§ Force Deployment: Optimal deployment of force toprevent the next adverse event.§ Service Life / Time to Failure: Predictive maintenanceto maximize readiness.

Force DeploymentThe agency uses predictiveanalytics to proactively deploypolice forces and prevent crimeUnderstandsgeographicallywhere events arelikely to happenOperationalAlertPlanningShift CommanderEnabling factors(Weather, Policepresence, )Risk AssessmentCrash in Zone 123AIfDay SaturdayAnd EntertainmentEventAnd DayAfterPaydayAnd DispatchZone 004Then V Crime Yes (65, 0.98)Risk Assessment: area 008ReallocationPatrol OfficerIfCondition RainAnd Temp 60And PatrolActivityPS HighThen V Crime No (0.92)Notices change inweather conditionsLikelihoodof CrimeHistoricalcrime incidents(RMS, CAD)Triggers(City events,Paydays, Time,Holiday )InteractiveDecisioning

Government Challenges with Unique Solutions§ Threat Detection: Combining predictive with traditionalinvestigative rules-based technology.§ Unusual Events: Modeling “normal” behavior to detectoutliers.§ Non-Traditional Data: Evaluating and RationalizingJobs Descriptions for Government Employer§ Force Deployment: Optimal deployment of force toprevent the next adverse event.§ Service Life / Time to Failure: Predictive maintenanceto maximize readiness.

Time to Failure: Predictive MaintenanceCorrosion Classification For Ship Maintenance§ Problem – Understanding when and why a ship is likely to go to a Corrosion Class 3 or 4 (i.e.the most corroded) is key to helping reduce costs and maximise readiness§ Data Description: Ship hull classes, maintenance dates and time between maintenance, andinformation on where and how large the corrosion spots were on each hull

Predictive Maintenance – Weibull AnalysisWeibull Analysis to develop risk & survival curves and identify probability of whenCorrosion Class 3 or 4 occurs for a particular class of ship hull.Time to Failure &Service LifeAcceptable period of usein service. Calculation ofMean/Median time toFailure (MTTF).

Predictive Maintenance – Going Beyond WeibullApplication of Predictive Analytics§ Corrosion Class– Classification techniques that use the shipservice time combined with the Ship Class to predict CorrosionClass 3 or 4§ Anomaly detection –Discover anomalies that use the site of thecorrosion on the ship hull and how large the corrosive patch was.§ Residual Analysis - Calculate difference between Mean Time ToFailure and actual time to failure. Build supervised models to explorethis difference and better understand actual Service Life.§ Data Quality – Extract concepts and sentiment from writtenmaintenance reports to build a better understanding of maintenancerequirements.

Government Challenges with Unique Solutions§ Threat Detection: Combining predictive with traditionalinvestigative rules-based technology.§ Unusual Events: Modeling “normal” behavior to detectoutliers.§ Non-Traditional Data: Evaluating and RationalizingJobs Descriptions for Government Employer§ Force Deployment: Optimal deployment of force toprevent the next adverse event.§ Service Life / Time to Failure: Predictive maintenanceto maximize readiness.

examine how predictive modeling can assist and complement existing selection systems ! Merge With Additional Data Sources Can we use Predictive Models to reduce the 93% Wasted Exams rate whilst ensuring we still maximize the number of violations captured. Threat Detection - Combining Predictive and Rules