Integrating MATLAB Analytics Into Business-Critical .

1

Integrating MATLAB Analyticsinto Business-Critical Applications이정수Applications EngineerMathWorks 2016 The MathWorks, Inc.2

Problem statement§Democratization: Is it possible to make the results and insights from theseanalytics available to all stakeholders in an organization?§Agility: Is it possible to accelerate the development of data analytics?§Production: Is it possible to build data analytics algorithms in a scalablemanner that is suitable for production usage and rigor?3

Nothing new In 1454 Gutenberg put his press to commercial use The printing press was an important step towardsthe democratization of knowledge [ ]Now that more people had access to knowledge both newand old, more people could discuss these works [ ]On the other hand, the printing press was criticized forallowing the dissemination of information which may havebeen incorrect.https://en.wikipedia.org/wiki/Printing press4

Taking MATLAB analytics into sVisualizationAccess to dataExpressivelanguageIntegration withbest-in-class tools5

Taking MATLAB analytics into sVisualizationAccess to dataExpressivelanguageIntegration withbest-in-class ityUsabilityExtendibilityPortabilityReadability6

Taking MATLAB analytics into productionDevelopmentCode preparationTestingProduction7

Taking MATLAB analytics into productionDevelopmentCode preparationTestingProduction8

Example: code preparationTask:Make analytics developed in MATLABavailable to Excel usersInput:Time series from race track- Time stamp- Latitude- Longitude- RPM- Forward acceleration- Lateral accelerationOutput: Lap statistics- Lap number, start and duration- Statistics per lap: max RPM, max forwardacceleration, max lateral acceleration9

Example: code preparationWhat needs to be deployed?§Access to initial data: NO§§Exploratory calculations: NOExploratory visualisation: NOInteractive parameter tuning: NO§Computation of lap stats: YES§10

Example: code preparationStill to address:§ Interleaved data access, processing andvisualisation§ What should be the input/output of thedeployed functions?§ What are the code dependencies?§ Performance optimisation11

Code preparation step 1: decouple code layersPresentation(Client)Logic(Algorithm)Data12

Code preparation step 2: fit into production contextPresentation(Client)Logic(Algorithm)Data13

Code preparation step 2: fit into production contextPresentation(Client)§Microsoft ExcelLayers– tools, scale, performanceWeb§Interfaces– types, scale, performanceLogic(Algorithm)§Dependencies– state, configurationData14

Taking MATLAB analytics into productionDevelopmentCode preparationTestingProduction15

Testing: is it just a stage?Development§§Check whetherresults “look”correctWrite a script tocheck if results arewithin predictedbandsCode preparation§Write code tocheck whetherresults stay thesame after codeoptimizationTesting§§§Write a formal suiteof tests coveringyour codeRun testsautomaticallyUser acceptanceProduction§Operations16

Example: testingSourceControlReport- Test results- Coverage- Comments- 17

Taking MATLAB analytics into productionPrototypingCode preparationTestingProduction18

MATLAB deployment TLABCompiler SDKExcelAdd-inHadoopC/C PythonMATLABProductionServer.NETJavaMATLAB Compiler enables sharing MATLAB programs without integration programmingMATLAB Compiler SDK provides implementation and platform flexibility for software developersMATLAB Production Server provides the most efficient development path for secure andscalable web and enterprise applications19

The range of application platforms20

Production Deployment of MATLAB Programs§Algorithm developers– MATLAB to design, test, and refine their algorithms§System administrators– manage the deployment of MATLAB analyticswithin a production environment using MPS§Application developers– integrate the lightweight client librariesincluded with MPS into their enterprise applications21

Integration with Enterprise Applications22

Develop and Test Framework – for MATLAB Production Server23

Reference ArchitectureMATLAB analytics with Data Visualization tools24

MATLAB analytics with Plotly25

Reference diagram for PlotlyUserWeb BrowserWeb ServerWeb AppPlotly ServerMATLAB Production ServerMATLAB ComponentRuntimeMPS ClientLibraryMATLABAnalyticsMATLAB Compiler SDK26

MATLAB analytics with TIBCO Spotfire27

Reference diagram for TIBCO SpotfireMobileWebDesktopMathWorks. MPSExtensionTIBCOSpotfireWeb PlayerMathWorks. MPSExtensionMATLAB Production ServerHTTP(s)MATLABAnalyticsTIBCOSpotfireServer28

Reference ArchitectureMATLAB analytics with SQL / NoSQL29

MATLAB analytics with Microsoft SQL ServerMicrosoftSQL ServerOn-DatabaseOff-DatabaseODBC/JDBC DriverODBCJDBC“Processes”“Transactions”C# Common Language Runtime(CLR)MATLABProduction ServerMATLABDatabase Toolbox.NET AssemblyMATLAB Component RuntimeMPS B Compiler SDK30

MATLAB analytics with MongoDBMongoDBWeb ServerWeb AppRouter (Mongos)Client LibraryMATLAB Production ServerApplication ServerBatch JobMATLABAnalyticsDataShardShardShardClient LibraryMongoDB ApplicationDriverUsersBusiness ApplicationMetadataConfig ServerConfig ServerConfig ServerClient Library31

MATLAB analytics with Microsoft SQL Server123MATLABMATLABProduction ServerDevelopMATLABAnalyticsRefineTest32

Integration with production systems: benefitsRapid development and deployment of MATLABanalyticsMATLABAnalytics expressed in MATLAB with nearly allavailable toolbox functionalityMATLAB Compiler SDKMATLAB Production ServerRobust, scalable analytics available to entireorganizationMicrosoft Excel, Plotly,Spotfire, SQL server,MongDB, etc.MATLABMATLABProduction ServerDevelopMATLABAnalyticsRefineTest33

Conclusions: MATLAB analytics in production§Democratization: Yes.– Integrate analytics with enterprise systems§Agility: Yes.– Access and explore data from within MATLAB during prototype development– Establish process allowing rapid iterations between ideas and production system§Production Quality: Yes.– MATLAB products and services provide a single-stack solution when used withsupporting technologies to address production data analytics demands34

Taking MATLAB analytics into tprocessesVisualizationAccess to dataExpressive highlevel languageIntegration withbest-in-class toolsCode preparation§§§§§ArchitectureCode optimizationData scalabilityClass systemError handlingTesting§§Verification andvalidation– Correctness– ityPortabilityReadability35

19 MATLAB Excel Add-in Hadoop MATLAB Compiler Standalone Application MATLAB deployment targets MATLAB Compiler enables sharing MATLAB programs without integration programming MATLAB Compiler SDK provides implementation and platform flexibility for software developers MATLAB Production Server provides the most efficient development path for secure and scalable web and enterprise applications