What's New In MATLAB And Simulink For ADAS And Automated Driving

What’s New in MATLAB and Simulinkfor ADAS and Automated DrivingMark CorlessAutomated Driving Segment Manager 2020 The MathWorks, Inc.1

Some common questions from automated driving engineersPerceptionPlanningControlHow can Ianalyze & synthesizescenarios?How can Idesign & deployalgorithms?How can Iintegrate & testsystems?2

Some common questions from automated driving engineersPerceptionPlanningControlHow can Ianalyze & synthesizescenarios?How can Idesign & deployalgorithms?How can Iintegrate & testsystems?3

Analyze and synthesize scenariosReal-world data workflowsAccessVisualizeLabelSynthetic scenario workflowsCreate scenesModel actorsModel sensorsEnablesopen loopworkflowsEnablesopen loop andclosed loopworkflows4

Access recorded and live dataCANROSROS 2.0HERE HD Live MapForward Collision Warningwith CAN FD and TCP/IPWork with Specialized ROSMessagesUse HERE HD Live Map Datato Verify Lane ConfigurationsAutomated Driving ToolboxTMVehicle Network ToolboxTMInstrument Control ToolboxTMROS ToolboxTMAutomated Driving ToolboxTM5

Visualize vehicle dataDetectionsImagesMapsVisualize Sensor Coverage,Detections, and TracksAnnotate Video Using Detections inVehicle CoordinatesDisplay Data onOpenStreetMap BasemapAutomated Driving ToolboxTMAutomated Driving ToolboxTMAutomated Driving ToolboxTM6

Label camera and lidar dataVisualizemultiple signalsInteractivelylabelAutomatelabeling ExportlabelsLoad multiple time-overlapped signalsrepresenting the same sceneSynchronously explore dataGet Started with the Ground Truth LabelerAutomated Driving ToolboxTMUpdated7

Label camera and lidar dataVisualizemultiple signalsInteractivelylabelAutomatelabeling ExportlabelsInteractively label sensor data–––––Rectangular region of interest (ROI)Polyline ROIPixel ROI (semantic segmentation)Cuboid (lidar)ScenesGet Started with the Ground Truth LabelerAutomated Driving ToolboxTMUpdated8

Label camera and lidar dataVisualizemultiple signalsInteractivelylabelAutomatelabeling ExportlabelsGet started with built-in detectionand tracking algorithmsExtend workflow by registeringcustom automation algorithmsGet Started with the Ground Truth LabelerAutomated Driving ToolboxTMUpdated9

Label camera and lidar dataVisualizemultiple signalsInteractivelylabelAutomatelabeling ExportlabelsExport to workspace or fileEnables workflows to customizeformat of labels for integration withother toolsGet Started with the Ground Truth LabelerAutomated Driving ToolboxTMUpdated10

Analyze and synthesize scenariosReal-world data workflowsAccessVisualizeLabelSynthetic scenario workflowsCreate scenesModel actorsModel sensorsEnablesopen loopworkflowsEnablesopen loop andclosed loopworkflows11

Synthesize scenarios to test algorithms and systemsScenesCuboidTestingControls, sensor fusion, planningSensingProbabilistic vision (detection list)Probabilistic lane (detection list)Probabilistic radar (detection list)Lidar (point cloud)12

Synthesize scenarios to test algorithms and systemsScenesCuboidUnreal EngineTestingControls, sensor fusion, planningControls, sensor fusion, planning, perceptionSensingProbabilistic vision (detection list)Probabilistic lane (detection list)Probabilistic radar (detection list)Lidar (point cloud)Monocular camera (image, labels, depth)Fisheye camera (image)Probabilistic radar (detection list)Lidar (point cloud)13

Graphically author scenarios with Driving Scenario Designer Design scenes– Roads, Lane markings– Pre-built scenes (Euro NCAP) Import roads– OpenDRIVE, HERE HD Live Map Add actors– Size, Radar cross-section (RCS)– Trajectories Export scenarios– MATLAB code, Simulink modelDriving Scenario DesignerAutomated Driving ToolboxTMUpdated14

Synthesize driving scenarios from recorded dataVisualizevideoImportroadsCreate egotrajectoryCreate targettrajectories SimulatescenarioImport roads from OpenDRIVECreate ego trajectory from GPSCreate target trajectories object listsScenario Generation from RecordedVehicle DataAutomated Driving ToolboxTM15

Model sensors in cuboid driving scenarios Vision object detectionsVision lane detectionsRadar detectionsLidar point cloudVisionRadarLidarCuboid Driving Scenario SimulationAutomated Driving ToolboxTMUpdated16

Model sensors in Unreal Engine driving scenariosMonocular image LidarMonocular camera– Image– Depth– Labels Fisheye camera imageLidar point cloudRadar detectionsDepthRadarFisheyeLabels3D Simulation for Automated DrivingAutomated Driving ToolboxTMUpdated17

Model monocular camera sensor in Unreal Engine driving scenarioDefinetrajectoryModel lsVisualize Depth and SemanticSegmentation Data in 3DEnvironmentAutomated Driving ToolboxTM18

Design with cuboid and Unreal Engine driving scenariosScenesTrajectoriesCustomize scenesCuboid Versions of 3D SimulationScenes in Driving Scenario DesignerSpecify Vehicle Trajectoriesfor 3D SimulationCustomize 3D Scenes forAutomated DrivingAutomated Driving ToolboxTMAutomated Driving ToolboxTMAutomated Driving ToolboxTM19

Design 3D scenes for automated driving simulationUpdate 1New base productDoes not require MATLABExternal SimulatorsMATLAB & Simulink20

Design scenes with road, marking, and prop assets Roads and markingsTraffic signalsGuard railsTreesSignsElevation dataAssetsRoadRunnerTMUpdate 121

Design scenes and export to driving simulatorDesignscenesExportmeshesImport tosimulator SimulateEdit roadsEdit road materialsAdd road markingsExporting to CARLARoadRunnerTMUpdate 122

Design scenes and export to driving simulatorDesignscenesExportmeshesImport tosimulator SimulateInstall pluginExport from RoadRunnerImport into CARLA/UnrealExporting to CARLARoadRunnerTMUpdate 123

Design scenes and export to driving simulatorDesignscenesExportmeshesImport tosimulator SimulateMove vehicle in automated drivingsimulationVisualize pixels IDs for semanticsegmentationExporting to CARLARoadRunnerTMUpdate 124

Export scenes to file formats and driving simulators Export to common file formats foruse in third-party applicationsFBX(meshes)– Filmbox (.fbx), OpenDRIVE (.xodr)– Unreal Engine , CARLA– Unity , LGSVL, GeoJSON– VIRES Virtual Test Drive, Metamoto– IPG Carmaker, Cognata, Baidu Apollo– Tesis Dynaware, TaSS PreScan– Universal Scene Description te 125

Integrate RoadRunner with MATLAB and Simulink workflowsRoadRunnerExport scenedescription(.FBX, .XML)Unreal EngineImport togameMATLAB & SimulinkConnect .XODR)Import todrivingscenario26

Import, visualize, and edit OpenDRIVE filesImportOpenDRIVEVisualizeEdit ExportValidate OpenDRIVE fileImport and visualizeEdit roads and sceneExport to common driving simulatorformats (including OpenDRIVE)Importing OpenDRIVE FilesRoadRunnerTMUpdate 127

Analyze and synthesize scenariosReal-world data workflowsAccessVisualizeLabelSynthetic scenario workflowsCreate scenesModel actorsModel sensorsEnablesopen loopworkflowsEnablesopen loop andclosed loopworkflows28

Some common questions from automated driving engineersPerceptionPlanningControlHow can Ianalyze & synthesizescenarios?How can Idesign & deployalgorithms?How can Iintegrate & testsystems?29

Design and deploy algorithmsPlanning & control ntrolsLateralcontrolsPerception workflowsDetectionTracking &sensor fusionLocalization30

Design controls and decision logic for ADASAdaptive Cruise ControlLane Keep AssistLane Following(longitudinal control)(Lateral control)(longitudinal lateral control)Adaptive Cruise Control withSensor FusionLane Keeping Assist with LaneDetectionLane Following Control withSensor FusionAutomated Driving ToolboxTMModel Predictive Control ToolboxTMEmbedded Coder Automated Driving ToolboxTMModel Predictive Control ToolboxTMEmbedded Coder Model Predictive Control ToolboxTMAutomated Driving ToolboxTMEmbedded Coder 31

Design planning and controls for highway lane sModeldynamics VisualizeresultsPlot candidate trajectoriesPlot selected optimal trajectoryPlot trajectory historyLane Change for Highway DrivingNavigation ToolboxTMModel Predictive Control ToolboxTMAutomated Driving ToolboxTMUpdated32

Design planning and controls for automated parkingDesignplanner & controlsDeploy toROS 2 nodePlanner & Controller Nonlinear MPCAutomated Parking Valet withSimulinkAutomated Parking Valet withROS 2 in SimulinkParking Valet using NonlinearModel Predictive ControlAutomated Driving ToolboxTMAutomated Driving ToolboxTMROS ToolboxTMEmbedded Coder Automated Driving ToolboxTMModel Predictive Control ToolboxTMNavigation ToolboxTM33

Design controls with reinforcement learningTrainnew networkTrain toimitate existing controllerTrain frompretrained networkTrain DQN Agent for LaneKeeping AssistImitate MPC Controller for LaneKeep AssistTrain DDPG Agent withPretrained Actor NetworkReinforcement Learning ToolboxTMReinforcement Learning ToolboxTMModel Predictive Control ToolboxTMReinforcement Learning ToolboxTM34

Design and deploy algorithmsPlanning & control ntrolsLateralcontrolsPerception workflowsDetectionTracking &sensor fusionLocalization35

Deploy deep learning networksNVIDIA GPUIntel MKL-DNNARMCode Generation for ObjectDetection by Using SingleShot Multibox DetectorGenerate C Code forObject Detection Using YOLOv2 and Intel MKL-DNNCode Generation for SemanticSegmentation Application onARM NeonDeep Learning ToolboxTMGPU CoderTMDeep Learning ToolboxTMMATLAB Coder Deep Learning ToolboxTMMATLAB Coder 36

Track-level Fusion of Radar and Lidar sDetectboundingboxes3D cuboidof clustereddetectionsTrackradarTracklidar3D cuboidtracksFusetracksTracks3D cuboidtracks2D rectangulartracksTrack-Level Fusion of Radar andLidar DataAutomated Driving ToolboxTMComputer Vision ToolboxTMSensor Fusion and Tracking ToolboxTM37

Fuse lidar point cloud with radar rackradarFusetracks AssessmetricsCreate sceneAdd actorsAdd lidar point cloud sensorAdd radar detection sensorTrack-Level Fusion of Radar andLidar DataAutomated Driving ToolboxTMComputer Vision ToolboxTMSensor Fusion and Tracking ToolboxTM38

Fuse lidar point cloud with radar rackradarFusetracks AssessmetricsRemove ground planeSegment and cluster detectionsFit bounding box to clustersTrack-Level Fusion of Radar andLidar DataAutomated Driving ToolboxTMComputer Vision ToolboxTMSensor Fusion and Tracking ToolboxTM39

Fuse lidar point cloud with radar rackradarFusetracksAssessmetrics Design conventional joint probabilisticdata association (JPDA) multi-objecttracker Track vehicles during lane change withinteracting multiple model unscentedKalman filter (IMM-UKF)Track-Level Fusion of Radar and Lidar DataAutomated Driving ToolboxTMComputer Vision ToolboxTMSensor Fusion and Tracking ToolboxTM40

Fuse lidar point cloud with radar rackradarFusetracks AssessmetricsDesign extended object trackerwith Gaussian Mixtureprobability hypothesis densityfilter (GM-PHD)Track-Level Fusion of Radar andLidar DataAutomated Driving ToolboxTMComputer Vision ToolboxTMSensor Fusion and Tracking ToolboxTM41

Fuse lidar point cloud with radar rackradarFusetracks AssessmetricsDesign track level fusionVisualizeTrack-Level Fusion of Radar andLidar DataAutomated Driving ToolboxTMComputer Vision ToolboxTMSensor Fusion and Tracking ToolboxTM42

Fuse lidar point cloud with radar detectionsSynthesizescenarioMissed cks False TracksAssessmetricsAssess missed tracksAssess false tracksAssess generalized optimal subpattern assignment metric(GOSPA)Track-Level Fusion of Radar andLidar DataAutomated Driving ToolboxTMComputer Vision ToolboxTMSensor Fusion and Tracking ToolboxTM43

Design object tracking and sensor fusionMeasureTuneGenerate codeIntroduction to Tracking MetricsTuning a Multi-Object TrackerGenerate C Code for a TrackerSensor Fusion and TrackingToolboxTMSensor Fusion and TrackingToolboxTMSensor Fusion and TrackingToolboxTMMATLAB Coder 44

Design localization algorithmsInertial fusion(IMU & GPS)Estimate Position and Orientationof a Ground VehicleSensor Fusion and TrackingToolboxTMSLAM(Monocular camera)Monocular Visual SimultaneousLocalization and Mapping(SLAM)Computer Vision ToolboxTMSLAM(Lidar)Design Lidar SLAM Algorithm using3D Simulation EnvironmentAutomated Driving ToolboxTMComputer Vision ToolboxTMNavigation ToolboxTM45