Azure IoT Workshop

AzureAzure IoT WorkshopReal-time asset trackingBenjamin Cabé // @kartbenPrincipal Program ManagerAzure IoTNov. 3, 2019

AgendaIntroducing the problem – Contoso Art ShippingTypical IoT architectureHands-on labsGoing furtherFollow along at: http://aka.ms/iot-workshop/asset-tracking

Introducing the problem

ContosoMonitoring of high-value parcelsfor international company Contoso Art Shipping

Problem statement (1/3)Contoso’s goods are getting tampered, ruined in transitIncreased customerdissatisfactionIncreasedinsurance claimsLosing customers,revenueHigh personnel coststo follow-up on claims

Problem statement (2/3)Contoso lacksability to monitorGoods that are gettingtampered with during transportationIn-transit productconditionLocation of theparcels60%Nearlyof insuranceclaims for damaged art arerelated to its transport

Problem statement (3/3)Goodsgettingtamperedpackage is outfor deliveryContoso already has an ERPand mobile app in place forbasic tracking informationThey would like to enablereal-time tracking ofshipments

Contoso Art Shipping’s requirementsTrack in real-time the physical location and condition of their parcelsEfficiently store this data so that it can be accessed and queriedGet alerts when abnormal conditions are detected and visualize them on a mapIntegrate with their existing business applicationScale from a very small deployment to something much bigger bonus points if they can be supported by great developer tools

Architecture of an IoT solutionThingsInsightsActions

A More Realistic View Cloud-to-devicecommandsEnterprise integrationDevice recoveryData storageDeploymentBusiness process integrationCold path analyticsOperations monitoringSolution scaleInternationalizationCost managementHigh availabilityUpdating devicesTransport protocolsDevice lifecycleThingsDriversDisaster recoveryDevice updatesHW certificationWarm path analyticsInsightsHot path analytics ---- End-to-End Security ---- On device analyticsManufacturing scaleDevice commercializationSecuring dataProvisioningdevicesActionsData ownershipIndustry and government complianceData visualization and why IoT needs simplifying

Provisionand senddata fromdevices toCloudDevice ManagementThingsInsightsActions

Provisionand senddata fromdevices toCloudStream processing andrules evaluations over dataStore dataDevice ManagementThingsInsightsActions

Provisionand senddata fromdevices toCloudStream processing andrules evaluations over dataStore dataDevice ManagementIntegrate with Business processThingsInsightsActions

Configure and controlVisualize data and learningsProvisionand senddata fromdevices toCloudStream processing andrules evaluations over dataStore dataDevice ManagementIntegrate with Business processThingsInsightsActions

aka.ms/iot-workshop/asset-trackingHands-on lab

Step 1 – Connecting a device to Azure IoTTrack in real-time the physical location and condition of their parcelsLearning goals: How to setup the messaging infrastructure for connecting your IoT devicesUnderstand basic security and device management conceptsExplore IoT Plug-and-Play conceptsDiscover VS Code IoT extensions and Azure IoT Explorer

Provisionand senddata fromdevices toCloudDevice ManagementThingsInsightsActions

IoT Hub and Device Provisioning ServiceDeviceProvisioning ServiceAzure IoT HubBi-directionalcommunicationEnterprise scale& integrationEnd-to-endsecurityIoT-scale automatedprovisioningMillions of DevicesBillions of messagesPer Device CertificatesZero-touch provisioningMulti-language, open sourceSDKsScale up and downPer Device Enable/DisableDeclarative Message RoutesTLS SecurityCentralize your provisioningworkflowFile UploadX.509 SupportLoad balance across multipleIoT HubsWebSockets & MultiplexingIP Whitelisting/BlacklistingRe-provisioning supportAzure MonitorShared Access PolicesSupports TPM X.509Azure Resource HealthFirmware/Software UpdatesConfiguration ManagementAzure Security Center SupportHTTPS/AMQPS/MQTTSSend TelemetryReceive CommandsDevice ManagementDevice TwinsQueries & Jobs

IoT Device LifecycleReplace or decommissiondevices after failure, upgradecycle or service lifetimeMonitor device inventory,health & security whileproviding proactiveremediation of issuesRETIREPLANMONITORGroup devices and control accessaccording to your organization'sneedsREGISTERCONFIGUREProvide updates, configuration & applicationsto assign the purpose of each deviceSecurely authenticate devices,on-board for managementand provision for service

IoT Device LifecycleReplace or decommissiondevices after failure, upgradecycle or service lifetimeMonitor device inventory,health & security whileproviding proactiveremediation of issuesRETIREPLANMONITORGroup devices and control accessaccording to your organization'sneedsREGISTERCONFIGUREProvide updates, configuration & applicationsto assign the purpose of each deviceSecurely authenticate devices,on-board for managementand provision for service

Quick orientationProvisioninghappensherePower BIWeb/Mobile AppDevicesStorage blobsCosmosDBBack endsystems andprocessesIoT HubStreamAnalyticsEvent Hub* Azure MLWeb JobsLogic Apps

What is provisioning?RegistrationConfigurationProvisioning

Why provisioning is hard today

Azure IoT Hub Device Provisioning ServiceSimplify with zero touch provisioningMinimize manual connection requirementsCross-geo provisioning supportDPSIoT Hub USIoT Hub JapanIoT Hub IndiaMultitenancy supportEnhanced securityFor any device compatible with IoT HubDPS knows exactly which IoT Hub to connect and provision

An IoT device’s relationship to DPS

High level provisioning1. Asks for hubDevice4. Returns hub2. Creates IDDPS5. Connects3. Returns IDIoT hub

IoT Plug and Play In Platform ContextCapability ModelRepository(Optional)IoT DeviceInterfaceD2CMessage RoutesTelemetryC2D/MethodsPropertiesDevice TwinCommandsCustomSolutionInterfaceD2CIoT EdgeIoT DeviceIoT EdgeIoT DeviceMessage RoutesTelemetryC2D/MethodsPropertiesDevice TwinCommandsInterfaceD2CMessage RoutesTelemetryC2D/MethodsPropertiesDevice TwinCommandsAzureIoT Hub & DPSAzure IoTCentral

Where are capability models stored?Device sends capability model ID and version expected for the solution to knowIf unknown, the following are the model retrieval options for the solution:

IoT Plug and Play ns.comCapability model and interface workspace and publishingrepository experiencePublishing integrated in VS Code and Azure CLI for bothinterfaces and capability modelsAutomated validation, collision checks and versioning supportSearch, filter, sort, view models & their graphs in modelrepository UXWorks out-of-the-box with any Azure IoT solutionWill be made available as an open-source projectMicrosoft will also host a fully managed, multi-tenant instancefor always up-to-date for Azure Certified devices; integratedinto certification flow

CapabilityModelDeveloperTooling

Easy to modeldevicecapabilities, easyto generatedevice softwareskeletonDevices that justwork out of thebox with no coderequiredDeviceCapabilityModelJSON-LDSchemaEasy to certifyplug and playdevicesEasy forcustomers andpartners to findplug and playdevices that justworkGenerated Device AgentAzure IoT Device SDKEasy to developdevice softwareand ensure it justworks with IoTsolutions

IoT Hub, DPS, PnPStep 1 – etup IoT Hub and DPSConfigure asset tracking deviceExplore capabilities thanks to PnP

Step 2 – Setting up an IoT data pipelineEfficiently store this data so that it can be accessed and queriedLearning goals: How to implement short-term and long-term retention using Azure Time Series InsightsHow to use TSI built-in data explorer to perform data analytics

Provisionand senddata fromdevices toCloudDevice ManagementThingsInsightsActions

Provisionand senddata fromdevices toCloudStream processing andrules evaluations over dataStore dataDevice ManagementThingsInsightsActions

What is Azure Time Series InsightsA Fully Managed Platform as a Service (PaaS) Solution Built for IoTFully managed, end-to-end PaaSsolution to ingest, process, store,and query highly contextualized,time-series-optimized, IoT-scaledataConnect to a variety of datasolutions using TSI’s flexible dataplatformUse rich analytics APIs and UX forad-hoc exploration and operationalintelligenceUse JavaScript control library forbuilding custom analytics apps onthe TSI platform

IoT Data extualizationOften used withother dataInfiniteretention

Canonical IoT Data PipelineNotifyOperatoror SystemExplore andmonitorPerformoperationalanalysisModels forlearning &predictionsCOLLECTPROCESSSTOREANALYZECollect time series datacoming from differentsources, formats & ratesParse, Normalize,Enrich, TransformStore in multi-layered(hot, warm, cold) timeseries data storeoptimized for analysisRun ad-hoc queries for monitoringOperationalize using asset-based analysisUse models for machine learninginference and make predictions

Azure Time Series Insights

Time Series InsightsStep 2 – etup TSI environmentSetup event sourcesExplore data

Step 3 – Anomaly detectionGetting alerts when abnormal conditions are detectedLearning goals: How to extract insights from real-time IoT Data using Azure Stream AnalyticsHow to turn alerts into actionsHow to store alerts into Time Series Insights, alongside telemetry

Azure Stream AnalyticsIn a Nutshell

Unlocking Real-time Insights Time to Insight is Critical Reducing decision latency can unlock business value Insights are Perishable Window of opportunity for insights to be actionable Ask Questions to Data in Motion Can’t wait for data to get to rest before running computation

Real-time Stream Processing Simple Event Processing FilterTransformEnrichSplitRoute Event Stream Processing [Simple event processing] AggregateRules Complex Event Processing [Event Stream Processing] Pattern detectionTime windowsJoins & correlations

Scenario ExamplesReal-time Fraud DetectionStreaming ETLPredictive MaintenanceCall Center AnalyticsIT Infrastructure and NetworkMonitoringCustomer Behavior PredictionLog AnalyticsReal-time Cross Sell OffersFleet monitoring and Connected CarsReal-time Patient MonitoringSmart GridReal-time Marketingand many more

Streaming PipelineApplicationsArchiving for longterm storage/batch analyticsEvent HubsDevices &StreamAnalyticsGatewaysPowerBIIoT HubsReal-time dashboardBlobsAutomation tokick-off workflowsMachine LearningReference DataEventproductionEvent Queuing& StreamIngestionStreamAnalyticsStorage &Batch AnalysisPresentation &Action

Stream Analytics Query Language (SAQL)Declarative SQL like language todescribe transformations Filters (“Where”) Projections (“Select”) Time-window and property-based aggregates(“Group By”) Time-shifted joins (specifying time bounds withinwhich the joining events must occur)and all combinations thereofData ManipulationDate and TimeSELECTFROMWHEREHAVINGGROUP BYCASE WHEN THEN ELSEINNER/LEFT OUTER JOINUNIONCROSS/OUTER APPLYCAST INTOORDER BY ASC, DSCDateNameLenDatePart Day, Month, Year ateAddLower, dowing Scaling ExtensionsWITHPARTITION NSQUARESQRTGeospatial T DISTANCEST WITHINST OVERLAPSST INTERSECTS

Stream Analytics JobUsers construct and deploy jobs to ASAJob definition includes inputs, a query, and outputInputs are from where the job reads the data streamQuery runs for perpetuity unless explicitly stopped and transforms the input streamOutput is where the job sends the job results to

Azure Stream AnalyticsStep 3 – etup ASA jobCreate query to detect anomaliesVisualize anomalies in TSI

Step 4 – Map visualizationLearning goals: How to use Azure Maps web controlHow to combine Time Series Insights and Azure Maps web SDKsAdvanced Azure Maps features, e.g heatmap

Azure MapsOverview