Shape From Silhouettes I CV Book Szelisky 11.6

1y ago
3 Views
1 Downloads
1.94 MB
41 Pages
Last View : 23d ago
Last Download : 2m ago
Upload by : Jenson Heredia
Transcription

Shape from Silhouettes ICV book Szelisky 11.6.2Guido GerigCS 6320, Spring 2012(slides modified from Marc PollefeysUNC Chapel Hill, some of the figures and slidesare adapted from M. Pollefeys, J.S. Franco, J.Matusik’s presentations, and referenced papers)

Shape from silhouettesSlides fromLazebnik,MatusikYerexand othersAutomatic 3D Model Construction for Turn-Table Sequences,A.W. Fitzgibbon, G. Cross, and A. Zisserman, SMILE 1998

Big Picture Multi-cameraenvironments Dynamic sceneOutdoor data capturing with 9 video cameras behind the Ackland Museum,UNC-Chapel Hill, 2006/8/24. Pictured by Jae Hak Kim. N cameras observe the scene and produce Nvideo streams What can we do with this data?

Motivation: MoviesSinha Sudipta, UNC PhD 2008

Motivation: MoviesSinha Sudipta, UNC PhD 2008

Motivation: MoviesSinha Sudipta, UNC PhD 2008

What can we do with this data? Reconstruct scene objects:–shape from silhouettes–photo-consistencyCalibrate cameras– recover epipolar geometryFit specific models (articulated models)

Outline Silhouettes––––––basic conceptsextract silhouettesfundamentals about using silhouettesreconstruct shapes from silhouettesuse uncertain silhouettescalibrate from silhouettes Perspectives and interesting ideas

Silhouettes of objects of interest Silhouettes are the regions where objects ofinterest project in images Silhouettes can generally be obtained usinglow level information (fast) They give information about the global shapeof scene objects

How to extract silhouettes? Sometimes done manually (for offlineapplications, ground truth and verifications) Region based-extraction (automatic)– silhouette extraction is a 2-region imagesegmentation problem, w/ specificsolutions: chroma keying (blue, green background) background subtraction (pre-observed static ordynamic background)(refer to segmentation course)

How to extract silhouettes? Contour-based extraction focus on silhouette outline instead of regionitself–snakes, active contours: fitting of a curve to highgradients in image, local optimizationYilmaz&Shah ACCV04

How to extract silhouettes? (cont.) Background subtractionSimple thresholdingTrain an appearance model foreach pixel, from a set ofbackground images RGB 3D-Gaussian modelHSV modelGMM modelNon-parametric model(histogram/kernel densityfunction) Apply the pixel color to the model,then classify it to beforeground/background We will talk about this in moredetail later

Why use a Visual Hull? Good shape representationCan be computed efficientlyNo photo-consistency requiredAs bootstrap of many fancyrefinement background foregroundbackground-foreground

Outline Silhouettes––––––basic conceptsextract silhouettesfundamentals about using silhouettesreconstruct shapes from silhouettesuse uncertain silhouettescalibrate from silhouettes Perspectives and cool ideas

What is shape fromsilhouette? The silhouette, or occluding contourof an object in an image containssome information about the 3Dshape of the object. Given a single silhouette image ofan object, we know that the 3Dobject lies inside the volumegenerated by back-projecting thesilhouette area using the cameraparameters.

What is shape fromsilhouette? With multiple views of thesame object, we canintersect the generalizedcones generated by eachimage, to build a volumewhich is guaranteed tocontain the object. The limiting smallestvolume obtainable in thisway is known as the visualhull of the object.

One-view silhouette geometry

Multi-view silhouette geometry:the Visual HullVisual hull Maximal volume consistentwith silhouettes[Laurentini94] [Baumgart74] Viewing cone Can be seen as theintersection of viewing conesProperties: Containment property: contains real scene objectsConverges towards the shape of scene objects minusconcavities as N increasesProjective structure: simple management of visibilityproblems

Visual Hull: A 3D Example

Outline Silhouettes––––––basic conceptsextract silhouettesfundamentals about using silhouettesreconstruct shapes from silhouettesuse uncertain silhouettescalibrate from silhouettes Perspectives and cool ideas

Algorithms Standard voxel basedmethodMarching Intersections Exact polyhedral methods Image-based visual hulls

Voxel based– First the object space is split up into a 3Dgrid of voxels.– Each voxel is intersected with eachsilhouette volume.– Only voxels that lie inside all silhouettevolumes remain part of the final shape.

Visual hull as voxel grid Identify 3D region using voxel carving– does a given voxel project inside all silhouettes? pros: simplicity cons: bad precision/computation timetradeoff

Classical voxel grid improvement:octrees Same principle, but refinement throughspace subdivision[Szeliski TR 90’]

Marching intersections– The object space is again split up into a 3D grid.– The grid used is made of 3 sets of rays, rather thanvoxels.– Rays are aligned with the 3 axes, and store points ofentry/exit into the volume– Each silhouette cone can be converted to themarching intersections data structure.– Then merging them is reduced to 1D intersectionsalong each ray.M. Tarini et al, Marching intersections, Anefficient Approach to Shape from Silhouette

Marching intersections example

Marching intersections -ConceptM. Tarini et al,Marchingintersections, Anefficient Approachto Shape fromSilhouette Given a curve Select reference grid Intersections between curve and horizontal andvertical lines: MI Create look-up-table for each non-empty box

Marching intersections Silhouettes Convert conoid structures to MI datastructure Intersection tested in 2D image: purely 2Doperation Intersection of conoids: AND operations on MIdatastructures

Marching intersections SilhouettesFinal step: Convert MI datastructure representingall intersections to triangular mesh

Marching intersections Silhouettes

Example Compute visual hull with silhouette images from multiplecalibrated camerasCompute Silhouette ImageVolumetric visual hull computationDisplay the result

Algorithms Standard voxel basedmethod Exact polyhedral methods Image-based visual hulls

Exact Polyhedral Methods– First, silhouette images are converted topolygons. (convex or non-convex, withholes allowed)– Each edge is back projected to form a 3dpolygon.– Then each polygon is projected onto eachimage, and intersected with each silhouettein 2D.– The resulting polygons are assembled toform the polyhedral visual hullWojciech Matusik, An Efficient Visual Hull Computation Algorithm

Exact Polyhedral MethodsWojciech Matusik, An Efficient Visual Hull Computation Algorithm

Exact Polyhedral - example

Metric Cameras and Visual-HullReconstruction from 4 viewsFinal calibration quality comparable to explicit calibration procedure

Image-based Visual HullsIBVH Results Approximately constant computation perpixel per camera Parallelizes Consistent withinput silhouettes

IBVH Results

Image-based Visual Hullshttp://www.youtube.com/watch?v Lw9aFaHobao

What if my silhouettes arenoisy? Idea: we wish to find the content of the scene from images, as aprobability grid Modeling the forward problem - explaining image observationsgiven the grid state - is easy. It can be accounted for in a sensormodel. Bayesian inference enables the formulation of our initial inverseproblem from the sensor model Simplification for tractability: independent analysis andprocessing of voxels

Idea Unreliable silhouettes: do not make decision about theirlocation Do sensor fusion: use all image information simultaneously Fusion of “probability information”: Related to Radon Transform(used for Computer Tomography Reconstruction)

polygons. (convex or non-convex, with holes allowed) -Each edge is back projected to form a 3d polygon. -Then each polygon is projected onto each image, and intersected with each silhouette in 2D. -The resulting polygons are assembled to form the polyhedral visual hull Wojciech Matusik, An Efficient Visual Hull Computation Algorithm

Related Documents:

Non-parametric model (histogram/kernel density function) Apply the pixel color to the model, . - use uncertain silhouettes - calibrate from silhouettes Perspectives and cool ideas. What is shape from . Advanced modeling - Sullivan & Ponce '98, Cross & Zisserman '00, Matusik et al. '02

Texts of Wow Rosh Hashana II 5780 - Congregation Shearith Israel, Atlanta Georgia Wow ׳ג ׳א:׳א תישארב (א) ׃ץרֶָֽאָּהָּ תאֵֵ֥וְּ םִימִַׁ֖שַָּה תאֵֵ֥ םיקִִ֑לֹאֱ ארָָּ֣ Îָּ תישִִׁ֖ארֵ Îְּ(ב) חַורְָּ֣ו ם

National Silhouettes of Kappa Alpha Psi Fraternity, Inc Policies and Procedures Manual 1. Membership A. Conditions/Requirements for membership Membership is restricted to the wife or widow of a financial member of Kappa Alpha Psi Fraternity, Inc. Sweethearts and lady friends of Kappa

: Show 3D Text.: Show 2D Text with a back shape.: Show 3D Text with a back shape.: Show a text hole in shape.: Show 3D Text in the border of shape. One Click add a object: Click the item to add a object quickly. Change Shape or Text Style: Double click the style icon can change node's shape or

Step 7 - Adding Shape # 2 Duplicate the shape from previous step by first selecting the arrow (A). While holding down the alt key click on the shape and drag it below the yellow rectangle. This will duplicate the shape. Now hold down the command key (apple) and drag the four points to transform your shape properly. Step 8 - Duplicating Shape #2

Shape Analysis & Measurement The extraction of quantitative feature information from images is the objective of image analysis. The objective may be: – shape quantification – count the number of structures – characterize the shape of structures. 3 Shape Measures

Sevtsuk, A. “”How we shape our cities, and then they shape us”, MAJA: the Estonian Architectural Review, 2-2012 (72), pp. 10-15, 2012. !! 1! How we shape our cities, and then they shape us. The quality of the built envi

For 6 BP the Molecular Shape is: Octahedral. Try These! Determine the geometric and molecular shape for the following: BrCl 4-CH 2F 2 HCN AsCL 5 SO 2 NH 4 SO 3 BF 4-SO 4 2-Shape and Polarity Shape and Polarity are directly related . Dipole u In diatomic molecules bond polarity applies to the overall molecule