Shape From Silhouettes I

1m ago
1 Views
0 Downloads
1.56 MB
49 Pages
Last View : 1m ago
Last Download : n/a
Upload by : Francisco Tran
Transcription

Shape from Silhouettes IGuido GerigCS 6320, Spring 2015Credits: Marc Pollefeys, UNC Chapel Hill, some of the figuresand slides are also adapted from J.S. Franco, J. Matusik’spresentations, and referenced papers)

Shape from silhouettesSlides fromLazebnik,MatusikYerexand othersAutomatic 3D Model Construction for Turn-Table Sequences,A.W. Fitzgibbon, G. Cross, and A. Zisserman, SMILE 1998

Big Picture Multi-cameraenvironments Dynamic sceneOutdoor data capturing with 9 video cameras behind the Ackland Museum,UNC-Chapel Hill, 2006/8/24. Pictured by Jae Hak Kim. N cameras observe the scene and produce Nvideo streams What can we do with this data?

Motivation: MoviesSinha Sudipta, UNC PhD 2008

Motivation: 3D from MoviesSinha Sudipta, UNC PhD 2008

Motivation: 3D from Movies:Replay from arbitrary viewpointsSinha Sudipta, UNC PhD 2008

What can we do with this data? Reconstruct scene objects:–shape from silhouettes–photo-consistencyCalibrate cameras– recover epipolar geometryFit specific models (articulated models)

Outline Silhouettes––––––basic conceptsextract silhouettesfundamentals about using silhouettesreconstruct shapes from silhouettesuse uncertain silhouettescalibrate from silhouettes Perspectives and interesting ideas

Silhouettes of objects of interest Silhouettes are the regions where objects ofinterest project in images Silhouettes can generally be obtained usinglow level information (fast) They give information about the global shapeof scene objects

How to extract silhouettes? Sometimes done manually (for offlineapplications, ground truth and verifications) Region based-extraction (automatic)– silhouette extraction is a 2-region imagesegmentation problem, w/ specificsolutions: chroma keying (blue, green background) background subtraction (pre-observed static ordynamic background)(refer to segmentation course)

How to extract silhouettes? Contour-based extraction focus on silhouette outline instead of regionitself–snakes, active contours: fitting of a curve to highgradients in image, local optimizationYilmaz&Shah ACCV04

How to extract silhouettes? (cont.) Background subtractionSimple thresholdingTrain an appearance model foreach pixel, from a set ofbackground images RGB 3D-Gaussian modelHSV modelGMM modelNon-parametric model(histogram/kernel densityfunction) Apply the pixel color to the model,then classify it to beforeground/background We will talk about this in moredetail later

Why use a Visual Hull? Good shape representationCan be computed efficientlyNo photo-consistency requiredAs bootstrap of many fancyrefinement background foregroundbackground-foreground

Outline Silhouettes––––––basic conceptsextract silhouettesfundamentals about using silhouettesreconstruct shapes from silhouettesuse uncertain silhouettescalibrate from silhouettes Perspectives and cool ideas

What is shape fromsilhouette? The silhouette, or occluding contourof an object in an image containssome information about the 3Dshape of the object. Given a single silhouette image ofan object, we know that the 3Dobject lies inside the volumegenerated by back-projecting thesilhouette area using the cameraparameters.

What is shape fromsilhouette? With multiple views of thesame object, we canintersect the generalizedcones generated by eachimage, to build a volumewhich is guaranteed tocontain the object. The limiting smallestvolume obtainable in thisway is known as the visualhull of the object.

Literature Theory– Laurentini ’94, Petitjean ’98, Laurentini ’99 Solid cone intersection:– Baumgart ’74 (polyhedra), Szeliski ’93 (octrees) Image-based visual hulls– Matusik et al. ’00, Matusik et al. ’01 Advanced modeling– Sullivan & Ponce ’98, Cross & Zisserman ’00,Matusik et al. ’02 Applications– Leibe et al. ’00, Lok ’01, Shlyakhter et al. ’01

One-view silhouette geometry

Multi-view silhouette geometry:the Visual HullVisual hull Maximal volume consistentwith silhouettes[Laurentini94] [Baumgart74] Viewing cone Can be seen as theintersection of viewing conesProperties: Containment property: contains real scene objectsConverges towards the shape of scene objects minusconcavities as N increasesProjective structure: simple management of visibilityproblems

Visual Hull: A 3D Example

Convex Hull: ComputationalGeometry ProblemIn mathematics, the convex hull orconvex envelope for a set of points Xin a real vector space V is theminimal convex set containing X.Convex hull: Elasticband analogy:Concave parts ofobject not part ofhull.In computational geometry, it iscommon to use the term "convexhull" for the boundary of the minimalconvex set containing a given nonempty finite set of points in the plane.Unless the points are collinear, theconvex hull in this sense is a simpleclosed polygonal chain.

Convex Hull: ComputationalGeometry ProblemHint: Calculate the convex hull based on the Delauneytriangulation and its dual, the Voronoi diagram.

Outline Silhouettes––––––basic conceptsextract silhouettesfundamentals about using silhouettesreconstruct shapes from silhouettesuse uncertain silhouettescalibrate from silhouettes Perspectives and cool ideas

What representation for scene objects?Voxel gridSurfaceVolumetric approachesSurface approachesImage-basedapproachesPolyhedron meshA priori knowledgeex: articulated model

General idea and assumptions 2 main families of approaches for VH:– focus on visual hull as volume: locate portions ofspace that don't project in silhouettes (carving) use 2D silhouette regions in images– focus on visual hull as surface: locate theboundary surface of the visual hull use 2D silhouette contours in images General assumptions:– very good silhouettes are extracted– views are calibrated parameters and positions are known

Computational complexity Intersection of many volumes can be slow Simple polyhedron-polyhedron intersectionalgorithms are inefficient To improve performance, most methods:– Quantize volumesand/or– Perform Intersection computations in 2D not 3D

Algorithms Standard voxel basedmethodMarching Intersections Exact polyhedral methods Image-based visual hulls

Voxel based– First the object space is split up into a 3Dgrid of voxels.– Each voxel is intersected with eachsilhouette volume.– Only voxels that lie inside all silhouettevolumes remain part of the final shape.

Visual hull as voxel grid Identify 3D region using voxel carving– does a given voxel project inside all silhouettes? pros: simplicity cons: bad precision/computation timetradeoff

Classical voxel grid improvement:octrees Same principle, but refinement throughspace subdivision[Szeliski TR 90’]

Marching intersectionsTarini et al., 2002– The object space is again split up into a 3D grid.– The grid used is made of 3 sets of rays, rather thanvoxels.– Rays are aligned with the 3 axes, and store points ofentry/exit into the volume– Each silhouette cone can be converted to themarching intersections data structure.– Then merging them is reduced to 1D intersectionsalong each ray.M. Tarini et al, Marching intersections, Anefficient Approach to Shape from Silhouette

Marching intersections example

Marching intersections example

Marching intersections -ConceptM. Tarini et al,Marchingintersections, Anefficient Approachto Shape fromSilhouette Given a curve Select reference grid Intersections between curve and horizontal andvertical lines: MI Create look-up-table for each non-empty box

Marching intersections Silhouettes Convert conoid structures to MI datastructure Intersection tested in 2D image: purely 2Doperation Intersection of conoids: AND operations on MIdatastructures

Marching intersections SilhouettesFinal step: Convert MI datastructure representingall intersections to triangular mesh

Marching intersections Silhouettes

Example: Student Project Compute visual hull with silhouette images from multiplecalibrated camerasCompute Silhouette ImageVolumetric visual hull computationDisplay the result

Algorithms Standard voxel basedmethod Exact polyhedral methods Image-based visual hulls

Exact Polyhedral MethodsWojciech Matusik et al.– First, silhouette images are converted topolygons. (convex or non-convex, withholes allowed)– Each edge is back projected to form a 3dpolygon.– Then each polygon is projected onto eachimage, and intersected with each silhouettein 2D.– The resulting polygons are assembled toform the polyhedral visual hullWojciech Matusik, An Efficient Visual Hull Computation Algorithm

Exact Polyhedral MethodsWojciech Matusik, An Efficient Visual Hull Computation Algorithm

Exact Polyhedral - example

Wojciech Matusik et al.

Wojciech Matusik et al.

Metric Cameras and Visual-HullReconstruction from 4 viewsFinal calibration quality comparable to explicit calibration procedure

IBVH Results Approximately constant computation perpixel per camera Parallelizes Consistent with input silhouetteshttp://www.youtube.com/watch?v Lw9aFaHobao

IBVH Results

Image Based Visual Hullshttp://www.youtube.com/watch?v Lw9aFaHobaoSee also: http://www.youtube.com/watch?v UdmBW4kDcok

Video ShadingMoviehttp://www.google.com/url?q uRpSG9iYW8%26image-based-visualhulls%3D&ei Zc8ES8wggaKyA73OrcIK&sa X&oi video result&resnum 5&ct thumbnail&ved 0CCQQuAIwBA&usg AFQjCNETWA Eqgy8 10mJUmc540zpx8T6A

Non-parametric model (histogram/kernel density function) Apply the pixel color to the model, . - use uncertain silhouettes - calibrate from silhouettes Perspectives and cool ideas. What is shape from . Advanced modeling - Sullivan & Ponce '98, Cross & Zisserman '00, Matusik et al. '02

Related Documents:

Texts of Wow Rosh Hashana II 5780 - Congregation Shearith Israel, Atlanta Georgia Wow ׳ג ׳א:׳א תישארב (א) ׃ץרֶָֽאָּהָּ תאֵֵ֥וְּ םִימִַׁ֖שַָּה תאֵֵ֥ םיקִִ֑לֹאֱ ארָָּ֣ Îָּ תישִִׁ֖ארֵ Îְּ(ב) חַורְָּ֣ו ם

National Silhouettes of Kappa Alpha Psi Fraternity, Inc Policies and Procedures Manual 1. Membership A. Conditions/Requirements for membership Membership is restricted to the wife or widow of a financial member of Kappa Alpha Psi Fraternity, Inc. Sweethearts and lady friends of Kappa

: Show 3D Text.: Show 2D Text with a back shape.: Show 3D Text with a back shape.: Show a text hole in shape.: Show 3D Text in the border of shape. One Click add a object: Click the item to add a object quickly. Change Shape or Text Style: Double click the style icon can change node's shape or

Step 7 - Adding Shape # 2 Duplicate the shape from previous step by first selecting the arrow (A). While holding down the alt key click on the shape and drag it below the yellow rectangle. This will duplicate the shape. Now hold down the command key (apple) and drag the four points to transform your shape properly. Step 8 - Duplicating Shape #2

Shape Analysis & Measurement The extraction of quantitative feature information from images is the objective of image analysis. The objective may be: – shape quantification – count the number of structures – characterize the shape of structures. 3 Shape Measures

Sevtsuk, A. “”How we shape our cities, and then they shape us”, MAJA: the Estonian Architectural Review, 2-2012 (72), pp. 10-15, 2012. !! 1! How we shape our cities, and then they shape us. The quality of the built envi

For 6 BP the Molecular Shape is: Octahedral. Try These! Determine the geometric and molecular shape for the following: BrCl 4-CH 2F 2 HCN AsCL 5 SO 2 NH 4 SO 3 BF 4-SO 4 2-Shape and Polarity Shape and Polarity are directly related . Dipole u In diatomic molecules bond polarity applies to the overall molecule

mean shape from multiple shape samples which are represented by unlabeled point-sets. An iterative bootstrap process is used wherein multiple shape sample point-sets are non-rigidly deformed to the emerging mean shape, with subsequent estimation of the mean shape based on these non-rigid alignments. The process is entirely symmet-

shapes is called Mensuration. These shapes exist in 2 dimension or 3 dimensions. Let’s learn the difference between the two. Difference Between 2D and 3D shapes 2D Shape 3D Shape If a shape is surrounded by three or more straight lines in a plane, then it is a 2D shape. If a shape is surrounded by a no. o

Thermal Model Simulation 15 Temperature-time histories for peak heating profile: 0 200 400 600 800 1000 1200 1400 1600 1800 0 20 40 60 80 Temp (K) Time (s) U-shape 0.165 mm TC1 U-shape 0.305 mm TC1 U-shape 0.396 mm TC1 L-shape 0.305 mm coated TC1 U-shape no TC1 Peak heating 0 200 400 600 800 1000 1200 1400 1600 1800 0 50 100 150 200 250 300 .

bridging of features across domains, as we do. Shape representations. While shape modeling and re-construction methods exist [44, 31], shape has been used for recognition to a very limited extent. Shape-based 2D im-age recognition includes [3, 46, 26] but these do not utilize hierarchical convolutional representations. More recently,

work is thus to develop an efficient computer method that can predict 1) shape-specific structural performance from given discrete shape data, and 2) structural performance variation over the shape population. Our approach builds on statistical analysis of shape vari-ations, a.k.a. statistcal shape modeling (SSM). SSM has

by looking at the shape of bodies in clothing. Costume designers can “read” a piece of clothing in the same way that an architect reads a building, by looking at details of design and construction to figure out age, history, and somet

polygons. (convex or non-convex, with holes allowed) -Each edge is back projected to form a 3d polygon. -Then each polygon is projected onto each image, and intersected with each silhouette in 2D. -The resulting polygons are assembled to form the polyhedral visual hull Wojciech Matusik, An Efficient Visual Hull Computation Algorithm

Method of Finite Elements I: Shape Functions. Why shape functions? Discretization leads to solution in the nodes, but no information concerning the space in between Shape functions required to approximate quantities between nodes . i.e. continuous across element discrete nodal quantities

facial landmark localization, it remains an unsolved prob-lem when applied to facial shape tracking in the real world video due to the challenging factors such as expression, illu-mination, occlusion, pose, image quality and so on. A suc-cessful facial shape tracking includes at least two character-istics.

Important new developments have appeared since the most recent direct survey on shape correspondence published almost a . shape reconstruction, r,defor- . terize a shape em

into shape optimization and topology optimization. For shape optimization, the theory of shape design sensitivity analysis was established by Zolésio and Haug.1,2 Bendsøe and Kikuchi3 proposed the homogenization method for structural topology optimization by introducing microstructu

Tell students to sort the real-world three-dimensional objects by these following attributes: 1. A shape that has one vertex. 2. A shape that every face is a rectangle. 3. A shape that every face is a square. 4. A shape that has two faces. Have students act out these three attributes of solid shapes

language that is spoken in the environment which they live, since the ability to distinguish the phonemes of one’s language environment is crucial to language acquisition. It is this ability which allows French children adopted by Japanese parents to speak the language of their environment (Jackendoff). Language Development 3 Exposure to language thus influences infants’ acquisition of .