CONCEPTS, ALGORITHMS & PRACTICAL APPLICATIONS IN

1y ago
14 Views
2 Downloads
2.87 MB
61 Pages
Last View : 19d ago
Last Download : 2m ago
Upload by : Nadine Tse
Transcription

CONCEPTS, ALGORITHMS & PRACTICALAPPLICATIONS IN 2D AND 3D COMPUTERVISIONCsaba BeleznaiSenior ScientistCenter for Vision, Automation & ControlAutonomous SystemsAIT Austrian Institute of Technology GmbHVienna, AustriaMichael Rauter, Christian Zinner, Andreas Zweng,Andreas Zoufal, Julia Simon, Daniel Steininger,Markus Hofstätter und Andreas Kriechbaum

MOTIVATION Research is evolution so is your learning processRECOGNITIONSEGMENTATIONRECONSTRUCTIONTIME GRANDCHALLENGESBalance: becoming a domain expert vs. being a „globalist“Researchers tend to favour certain paradigms - Learn to outline trends, look upstreamRevisit old problems to see them under new lightSpecialize the general & Generalize the specific2Factorize your know-how (code, topics, ) into components sustainable, scalable

VISUAL OBJECT RECOGNITION TRENDSHuman-level for 20122019timeAmount of imagedata (for training)IMAGE DATABARRIER20122019time

AIT AUSTRIAN INSTITUTE OF TECHNOLOGYFederal Ministry for Transport,Innovation and Technology50,46%Federation of AustrianIndustries49,54%AITAustrianInstituteof ofTechnologyAITAustrian InstituteTechnologyEnergyHealth &BioresourcesDigital Safety &SecurityVision, Automation &ControlMobility novation Systems &PolicySeibersdorfLabor GmbHNuclearEngineeringSeibersdorfGmbH1300 employeesBudget: 140 Mio Business Model: 40:30:304

VISION, AUTOMATION & CONTROLHigh-PerformanceVision3D Vision andModelingComplex DynamicalSystemsWorldwide fastest visionsensor technologyRobust and flexible3D vision technologyAdvanced handling andsmart productionF r o mS e n s o rT oD e c i s i o n5

AIT AUTONOMOUS & ASSISTIVE SYSTEMSDriver AssistanceSystem for TramsAssistance Systems forConstruction MachinesDriverless Missions in Crisis& Disaster ManagementAutonomousLocal RailwayAutonomousBus

ENABLING METHODOLOGIES FORASSISTED OR SELF-DRIVINGMobile platforms: sensory signals local context (situation) decisionsvehicle controlmotion analysislocalizationrecognitionobjects (type, location, pose)environmentpositioningmappingsensor fusionvehicle modelobject trackingdynamic modelcomputationstate predictionvehicle controlprobabilitybased behaviorelementssafetycomplianceego-motion computationsensor/data fusionDeep learning basedRELATEDdetection & segmentationKNOW-HOWVision algorithms testingLocalization,map buildingSparse motionestimation, tracking7

INTELLIGENT PERCEPTION FOR MOBILE MACHINES

AUTONOMOUS OFFROAD VEHICLE

IntroductionA frequently asked question14.07.201910

Example for robust visionExample: Crop detection Radial symmetry Near regular structure11

IntroductionMotivation Challenges when developing Vision Systems: Complexity Algorithmic, Systemic, Data Non-linear search for a solutionRESEARCHDEVELOPMENTAlg. AMATLABC Alg. Bbranch & boundresearch methodologyAlg. CIDEAAPPLICATIONPRODUCT

2DReal-time optical flow based particle advectionfor object detection and tracking13

MOTIVATION – I.OBJECT DETECTION PIPELINESSpatial distribution ofposterior probabilityScore map (DPM, R-CNN, )Delineated objectsBounding boxesVote mapOccupancy mapback-projected similarity map More complexparametricrepresentationsInstance segmentationDPM: Deformable Part ModelsR-CNN: Region-based Convolutional Neural Networks14.07.201914

RELATED STATE-OF-THE-ART Clustering detectionsweakly constrained structural priorCenter-surround filterNon-maximum suppressionRothe et al., 2014Neubeck & Van Gool, 2006Mean Shift, CAMShiftComaniciu & Meer, 2002Bradski 1998MeanShift and CamShift iterations Detection by voting/segmentation/learningimplicit or explicit structural priorImplicit Shape ModelLeibe et al. 2005Markov Point Processes for object configurationsLeibe et al. 2005Verdie, 2014Structured random forestsDollar & Zitnick 2013Kontschieder et al. 2011Kontschieder et al. 2011CNN‘s for Non-Max. SuppressionHosang et al. 2016, Wan et al. 2015Hosang et al. 2016

Optical flow driven advectionAdvection: transport mechanism induced by a force fieldtiti 1Dense optical flow fieldVy,iVx,iA particle trajectoryinduced by the OF field16

Particle advection with FW-BW consistency A simple but powerful testForward:SuccessfulBackward:Consistency check: x Failure x: mean offset

Pedestrian Flow AnalysisPublic dataset: Grand Central Station, NYC: 720x480 pixels, 2000 particles, runs at 35 fps

SHAPE-GUIDED TRACKLET GROUPINGFOR COMPACT OBJECTSOptical flow driven particle trackletsThe ith tracklet: 𝑇𝑖𝒗 – velocity vector𝑤 – weight (scalar)STEP 1: sampling 𝑥𝑡 , 𝑦𝑡𝑡 1.𝑁 , 𝒗, 𝑤STEP 2: weight generationfrom orientation similarity(w.r.t. center tracklet)Clustering directly performed in thediscrete tracklet-domainSTEP 3: local shape, scaleand center estimationSTEP 4: find nearesttracklet to mode estimateWSingle parameter:W – initial scaleEstimated cluster parameters mode locationrepeat from STEP 1 untilconvergence

3DSTEREO DEPTH INFORMATIONCHARACTERISTICS AND USAGE20

PASSIVE STEREO BASED DEPTH MEASUREMENT 3D stereo-camera system developed by AIT Area-based, local-optimizing, correlationbased stereo matching algorithm Specialized variant of the Census Transform Resolution: typically 1 Mpixel Run-time: 14 fps (Core-i7, multithreaded, SSE-optimized) Excellent “depth-quality-vs.-computational-costs” ratio USB 2 interfaceAdvantage: Depth ordering of people Robustness against illumination,shadows, Enables scene analysis

STEREO CAMERA CHARACTERISTICSTrinocular setup: 3 baselines possible 3 stereo computations withresults fused into one disparityimagefar-rangenear-rangesmallmediumlarge baseline22

Data characteristicsIntensity imageDisparity imageyPlanar surface in 3D spacey(x,y) image coordinates, d disparityd(x,y)23d

2.5D vs. 3D algorithmic approachesComputed top view of the 3Dpoint cloud3D approachHeight (world)Stereo setup2.5D approachnoisy measurementcorrect measurementGround plane (world)

LEFT ITEM DETECTIONAdditional knowledge (compared to existing video analytics solutions): Stationary object (Geometry introduced to a scene) Object geometric properties (Volume, Size) Spatial location (on the ground)

METHODOLOGYINTENSITYChange detectionOrtho-transformBackgroundmodelGround planeestimationStereo disparityDEPTHInput imagesProcessing intensity and depth dataOrtho-mapgenerationObjectdetection andvalidation inthe orthomapCombinationof proposals ValidationFinalcandidates

Left Item Detection – Demos14.07.201927

Clustering in discrete two-dimensional distributions28

Object detection as clustering(a)14.07.201929

A Frequently Occurring TaskAnalysis of discrete two-dimensional distributions LEARNEDCODEBOOK

EXAMPLES Description of the Binary-Shape-driven 2D clustering Shape learning Shape clustering, delineation Results Occupancy map clustering Text line delineation Object delineation by shape-guidedtracklet grouping14.07.201931

TASK DEFINITIONIntermediate probabilistic representationsLocal groupingprior, structure-specific knowledge2D distributionsgenerate consistentobject hypothesesChallenge: arbitrarily shaped distributions multiple nearby modes noise, clutterDefinitions:mode location of maximum densitycomputed using a kernel Kdensity estimation of variable x𝑓 𝑥 𝐾 𝑎 𝑥 𝑤(𝑎)𝑎32

Shape learningShape learning – Case: Compact clusters1. Binary mask from manual annotation orfrom synthetic dataSpatial resolution of local structure2. Sampling using an analysis windowdiscretized into a ni ni grid3. Building a codebook of binary shapeswith a coarse-to-fine spatial Codebook:

Example Codebook – Case: Compact clusters14.07.201934

Shape learningShape learning – Case: Line structuresBinary mask from manually annotatedtext linesSpatial resolution of local structureCodebook:lowmid high

Shape delineationShape delineation – I.Step 1: Fast Mode SeekingThree integral images:andMode location:COMPACT CLUSTERSLINE STRUCTURESStep 2: Local density analysisDensity measurefor each resolution levelfor the binary structureEnumerating all binary shapes at each resolution level Finding best matching entry:

Shape delineationShape delineation – II.Recursive search for end points, starting frommode locations: Line-centered structuresRelative line end locations define:-Search directionLine end positions Off-the-linestructuresEND POINTCANDIDATESVOTE MAP

Experimental results - Case: Compact clustersHuman detection by occupancy map clustering:Passive stereo depth sensing depth data projected orthogonal to theground plane13 mOccupancy map (1246 728 pix.) clustering: 56 fps, overall system (incl. stereo computation): 6 fps

Experimental results - Case: Compact clusters

Experimental results - Case: Line structures (Text line segmentation)Input imageSimple binarizationBinarization is very sensitive to employed thresholdProbability distribution for textProposed schemeOur scheme has no threshold, only local structuralpriors

Experimental results - Case: Text line segmentation

2D 3DQueue length detection using depth andintensity information42

Queue Length Waiting Time estimationWhat is waiting time in a queue?Time measurement relating to lastpassenger in the queueCheckpointWaiting timeWhy interesting?Example: Announcement of waiting times (App) customer satisfactionExample: Infrastructure operator load balancing

Queue analysis Challenging problemWaiting time LengthVelocity1. What is the shape and extent of the queue?2. What is the velocity of the propagation?Shape No predefined shape (context/situation-dependent and time-varying)simple complexMotion not a pure translational pattern Propagating stop-and-go behaviour with a noisy „background“ Signal-to-noise ratio depends on the observation distanceDEFINITION: Collective goal-oriented motion pattern of multiple humansexhibiting spatial and temporal coherence

Visual queue analysis - Overview How can we detect (weak) correlation?Correlation in space and timetyxSource: Parameswaran et al. Design and Validation of aSystem for People Queue Statistics Estimation, VideoAnalytics for Business Intelligence, 2012 Much data is necessary Simulating crowding phenomena in Matlab Social force model (Helbing 1998)goal-driven kinematics – force fieldrepulsion by wallsrepulsion by „preserving privacy “

Queue analysisSimulation tool Creating infinite number of possible queueing zonesTwo simulated examples (time-accelerated view):46

Queue analysis (length, dynamics)Straight lineStaged scenarios, 1280x1024 pixels, computational speed: 6 fpsMeander style

Pairwise spatio-temporalcorrelation analysis Correlation data weights Mode seeking and tracking Queue delineation posed as anopenTravelingSalesmanProblem (oTSP) with a fixed start Queue forward velocityestimation using a deformableelastically connected chain

Adaptive estimation of the spatial extent of the queueing zoneEstimated configuration(top-view)stereo sensorDetection resultsLeft part of the image is intentionally blurredfor protecting the privacy of by-standers,who were not part of the experimental setup.

Scene-aware heatmap

2DEnd-to-end video text recognition51

Overview The End-to-End Video Recognition ProcessINPUTOUTPUTCharacterizing dynamicelements: running ocation(single frames)(x, y, w, h)SegmentationLocation(frame span)(x, y, w, h)Recognition,PropagationBinary imageregionsEvaluation: High accuracy at each stage is necessaryVery high recall throughout the chainIncreasing Precision toward the end of the chainTextText(e.g. in ASCII)

Algorithmic chain - MotivationMain strategies for text detection:What is text (when appearing in images)?:An oriented sequence of characters in close proximity, obeying a certain regularity(spatial offset, character type, color).Sample text region complex background

Improved text detection – synthetic text generation(Classification using Aggregated Channel Features)(a)(b)

Convolutional Neural Network based OCR - TrainingGenerated single characters (0-9, A-Z, a-z): include spatial jitter, font variations role of jitter: characters can be recognized despite an offset at detection time6000 „A“6000 „0“6000 „Z“

Convolutional Neural Network based OCR - ResultsAnalysis window is scanned along the textline, and likelihood ration (score1/score2) isplotted in the row (below textline) belonging to the maximum classification score.Optimum partitioning56

14.07.201957

Implementation detailsOur development conceptMethod, atlab engineshared libraryPorting MATLAB / PYTHON: Broad spectrum of algorithmic libraries, Well-suited for image analysis, Visualisation, debugging, Rapid development Method, Prototype, Demonstrator C/C Real-time capabilityC/C

LEARNED REPRESENTATIONS OF HIGHDISCRIMINATIVE POWER

MODEL-BASED VISION - USE CASERelevance:1. DeepLearning: coarse pose estimation (coarse initialization/re-initialization)2. Edge based model tracking (continuous fine 6DoF pose estimation)Pose regressortrained fromsynthetic data:14.07.201960

THANK YOU!CSABA BELEZNAISenior ScientistCenter for Vision, Automation & ControlAutonomous SystemsAIT Austrian Institute of Technology GmbHGiefinggasse 4 1210 Vienna AustriaT 43(0) 664 825 1257 F 43(0) 50550-4170csaba.beleznai@ait.ac.at http://www.ait.ac.at

CONCEPTS, ALGORITHMS & PRACTICAL APPLICATIONS IN 2D AND 3D COMPUTER VISION Michael Rauter, Christian Zinner, Andreas Zweng, Andreas Zoufal, Julia Simon, Daniel Steininger, Senior Scientist Markus Hofstätter und Andreas Kriechbaum Center for Vision, Automation & Control Auton

Related Documents:

THIRD EDITION Naveed A. Sherwani Intel Corporation. KLUWER ACADEMIC PUBLISHERS NEW YORK, BOSTON, DORDRECHT, LONDON, MOSCOW. eBook ISBN: 0-306-47509-X . Graph Search Algorithms Spanning Tree Algorithms Shortest Path Algorithms Matching Algorithms Min-Cut and Max-Cut Algorithms

Graph algorithms Geometric algorithms . Textbook Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms, Third Edition, McGraw-Hill, 2009. 4 Suggested Reading Polya, How to Solve it, Princeton University Press, 1957. Preparata and Shamos, Computational Geometry, an . limitations of algorithms? Computability .

Swarm Intelligence and bio-inspired computation have become increasingly popular in the last two decades. Bio-inspired algorithms such as ant colony algorithms, bat algorithms, bee algorithms, firefly algorithms, cuckoo search and particle swarm optimization have been

Metaheuristic Algorithms Genetic Algorithms: A Tutorial “Genetic Algorithms are good at taking large, potentially huge search spaces and navigating them, looking for optimal combinations of things, solutions you might not otherwise find in a lifetime.” - Salvatore Mangano Computer Design, May 1995 Genetic Algorithms: A Tutorial

algorithms, which are very different in principle than the above algorithms, are covered in Chapter 6. Genetic algorithms—search and optimization algorithms that mimic natural evolution and genetics—are potential optimization algorithms and have been applied to many engineering design problems in the recent past.

In this course we study algorithms for combinatorial optimization problems. Those are . and so it is unlikely that we can design exact e cient algorithms for them. For such problems, we will study algorithms that are worst-case e cient, but that output . make us give a second look at the theory of linear programming duality. Online Algorithms.File Size: 832KB

use of these algorithms. The algorithms assist the staff concerned in making analyses and taking decisions. This does not detract from the fact that – viewed from the aspect in the year 2021 – there is room for improvement, as algorithms are set to be used more and more often in the years ahead. If algorithms

teaching 1, and Royal Colleges noting a reduction in the anatomy knowledge base of applicants, this is clearly an area of concern. Indeed, there was a 7‐fold increase in the number of medical claims made due to deficiencies in anatomy knowledge between 1995 and 2007.