LAYERED RELIEF TEXTURES

2y ago
25 Views
2 Downloads
749.09 KB
8 Pages
Last View : 20d ago
Last Download : 2m ago
Upload by : Genevieve Webb
Transcription

LAYERED RELIEF TEXTURESSergey Parilov, Wolfgang StuerzlingerDepartment of Computer ScienceYork University, 4700 Keele StreetM3J 1P3, Toronto, OntarioCanada{parilov wolfgang}@cs.yorku.cawww.cs.yorku.ca/ wolfgangABSTRACTIn this paper we present an Image-Based Rendering method for post-warping LDI’s in real-time onexisting systems. The algorithm performs accurate splatting at low computational costs, reduces memoryaccess bottlenecks, enables us to trade-off the quality for the speed, and is simple to implement.Keywords: image-based rendering, image warping, real time rendering, relief textures, layered depth images,splatting.1.INTRODUCTIONOver the last few years, the growing needs of thegraphics community and the recent advances intechnology have resulted in the considerable increasein the rendering speed of polygon-based graphicshardware. Although it is possible to render up to 300thousand triangles per frame in practice, thehardware still cannot create photo-realistic images inreal-time. Among the reasons are the complexity ofcreating realistic polygonal models in general, andthe inability of rendering the complex ones at highframe rates. In special cases the quality of the imagescan be improved with bump-maps, light maps andsimilar advanced rendering techniques. This stilldoes not solve the problem for acquired imagery andpoint sampled models.Image-Based Rendering by Warping(IBRW) was proposed as a method to overcomethese difficulties by using images as the model forrepresenting the source data. Current IBRWtechniques can create photo-realistic renderings atthe cost of the low speed that prohibits the use ofIBRW in real-time interactive applications. Despiteseveral research projects aimed at creating IBRWcapable hardware, even the simplest IBRW methodshave not been demonstrated until now.We present a low computational cost IBRWmethod targeted to support insertion of realisticobjects into polygonal scenes. We employ thetexture mapping capabilities of existing graphicshardware, and perform all the other operations insoftware. In this paper, we show how the LRT (Layered Relief Textures)method can achieve a high-quality reconstructionimage with small computational cost; and demonstrate IBRW at real-time rates formedium-complexity models.The paper is organized as follows. In thefollowing section we present a brief overview of theexisting methods and describe the basic warpingalgorithms on which we build our method. Section 3describes a way to mitigate the inability of one of theprevious IBRW methods (which this work buildsonto) to render objects from arbitrary views. Section4 discusses the problem of efficient imagereconstruction from point-samples. We compare thequality of our method with that of the existingtechniques. In section 5, we propose the method touse the cache memory in a more efficient way.Section 6 discusses our implementation and theresults.2.PREVIOUS WORKSeveral different image-based rendering methodsbased on different data structures have beenpresented over the last few years. In this work, welimit the discussion only to images augmented withper-pixel disparity1, as defined in [Mcmil97].1the quantity inversely proportional to depth

For Image-Based Rendering in general, thequality of the final result depends primarily on thequality of the source images. Using photographs orimages synthesized with global illuminationtechniques IBR methods can generate highly realisticresults [Mcmil97, Debev96, Debev98]. Acquiringmodels for IBR implies computation of cameraparameters for source images and obtaining depthvalues for each source image pixel. For real images,both these tasks can be addressed with computervision methods. For synthesized images thisinformation is easy to store during image generation.IBRW algorithms essentially solve twoproblems - geometrical warp problem defined as themap from the source to the destination imageaccording to the camera configuration, and the signalreconstruction problem to reconstruct the intensitiesdefined by the samples of the source images[Mcmil97, Shade98, Olive00, Popes98]. The(planar) forward warp derived in [Mcmil97] 1u u s k1d,1 k 3 dv v s k 2d1 k3dand determine the position (u,v) of a source imagepoint (us,vs,d) in the intermediate image, where d isthe depth (or displacement), and k1,k2,k3 are thecoefficients that depend only on the cameraconfiguration and are independent of the sourcepoint co-ordinates [Olive00]. 1x2 & ä( x1 )P2 (C1 C2 ) P2 P1 x1computes the location of the point x2 in thedestination image corresponding to a point x1 in thesource image, where ä() is the disparity of a point,and P1, C1, and P2, C2 define camera configuration ofthe source and the destination images respectively.The occlusion compatible order guarantees correctvisibility between samples with a simple painter'salgorithm. In [Mcmil97], it is defined as a sourceimage plane traversal such that along every line inthe source image passing through the epipole2 thepoints farther away from the epipole are visited first.The cost of the computation is a matrixmultiplication per source image pixel.As the occlusion compatible warping orderis only valid for a pair of source and destinationimage, the regions in the destination image whichshould be depicting parts of the scene not visiblefrom the source image camera appear empty.Layered Depth Images (LDI), defined as a "view ofthe scene from a single input camera view, but withmultiple pixels along each line of sight" [Shade98],solve the problem. Furthermore, LDI’s supportocclusion compatible traversals. While the LDI’srely on the McMillan's warping equation, theincremental warping scheme is presented in[Shade98].Relief textures [Olive00] present analternative that requires slightly less arithmeticoperations per source sample and exploits standardgraphics hardware to perform part of the2computations. The key innovation of this approach isto decompose the warping process into two separatepasses - the pre-warp and the texture mapping. Thepre-warping stage handles only the parallax effectsby pre-warping the source image into an intermediateimage. On the second stage, the intermediate imageis used to texture map the polygon in 3D spacecorresponding to the source image's viewplane. Thisaccounts for the perspective distortion, zoom, androtation due to the current viewpoint and producesthe same result as a conventional warp. The prewarping equations are defined asthe projection of the center of projection of onecamera on the viewplane of the otherThe main shortcoming of relief texturing isthat it does not support multiple pixels along one lineof sight, thus being unable to handle objects thatfeature occlusions inside the volume represented bythe relief texture. In other words, only a subset ofobjects can be handled correctly and artefacts appearif the fundamental assumptions are violated (e.g. theareas behind the chimneys in [Olive00]).On modern graphics hardware, rendering anrectangular array of pixels after a conventional warphas run-time cost comparable to that of rendering atexture mapped triangle. Consequently, an additionaltexture mapping operation is practically free.Because sampling rates of the sourceimages rarely match the desired output resolutioninterpolation between samples is a major issue inIBRW methods. Many IBRW techniques usedifferent kinds of splatting of varying quality ofreconstruction and speed. The reconstruction errorsintroduced by splatting are analyzed in [Mark97].In the original warping approach the splatfor each sample was computed on the fly. As thecomputation of exact splat footprints is extremelycomputationally intensive different approximationsare used in practice, thus degrading the quality.Fixed sized splats have been used previously toachieve high warping speed [Aliag99]. In [Shade98],the authors use a small pre-computed fixed set ofGaussian splats to achieve somewhat better quality.In [Pfist00], a splatting method that allows highquality reconstructions with high computation costsis presented. In [Rusin00], the authors present a

hierarchical technique to visualize extremely largemodels.Relief textures [Olive00] present analternative that requires slightly less arithmeticoperations per source sample and exploits standardgraphics hardware to perform part of thecomputations. The key innovation of this approach isto decompose the warping process into two separatepasses – pre-warp and texture mapping. The prewarping pass warps the source image to take parallaxinto account. The second pass maps the output of thefirst pass as a texture onto a 3D rectangle in space.This accounts for perspective distortion, zoom androtation due to the current viewpoint and producesthe same result as a standard warp. Note that theresult of a traditional warp still needs to be renderedto the screen after the warp. Rendering the resultingrectangular array of pixels has run-time cost that iscomparable to rendering one texture-mappedtriangle. Therefore, Oliveira’s approach is slightlyless costly than the McMillan’s approach or the LDIsplatting approach. The main shortcoming of relieftexturing is that it is unable to handle objects thatfeature occlusions inside the volume represented bythe relief texture.That IBRW can achieve better speeds thanpolygonal computer graphics still needs to bedemonstrated in the general case. IBR methods suchas QuickTime VR can achieve the speed at theexpense of limiting the viewer motion to a locationand allowing only rotations. Hierarchical imagecaches or impostor hierarchies can be rendered inreal-time, but rely on the existence of a geometricdescription [Schaufler96, Shade96]. Therefore, theycannot be applied to acquired imagery. More generalIBR methods based on McMillan's warping methodusually achieve a maximum speed of up to 10 framesper second [Shade98, Popes98, Chang99].Because image warping implies many memoryaccesses, both reads and writes, memory bandwidthbecomes a significant issue. In [Mark97], the authorhas addressed the issue with algorithms that providemore efficient use of cache memory. Anothersolution is to limit the amount of data beingconsidered by clipping, although little work has beendone in this area [Shade98, Popescu98].The approach presented in this paperaddresses the issues mentioned by a combination oftechniques. We combine layered depth images andrelief textures and call the resulting data structureLayered Relief Texture (LRT). Similar to LDIs, weuse multiple samples along the line of sight, and,similar to relief textures, we use simplified equationsto pre-warp the samples and render them on thescreen with the help of texture mapping hardware.This extends the relief textures approach to deal witharbitrary objects. Furthermore, the nature of prewarping equation enables us to use a differentsolution to the splat size computation problem toachieve good image quality.We use tiling of both the source and thepre-warped images such that every pre-warpedimage tile fits into L2 cache to address memorybottlenecks. The tiles of the pre-warped image canbe rendered separately and selectively. This allowsus to easily clip the image and to re-use the results ofthe warp of the previous frame if the estimated erroris below some threshold or if the time allowed forthe frame generation is over (similar to [Lengy97,Schau96]).3.LAYERED RELIEF TEXTURESLayered Relief Textures (LRTs) combine theadvantage of relief textures with layered depthimages (LDIs). We base our approach on theoptimised warping method as described in [Olive00].This relief textures approach offloads part of thecomputational effort from the CPU to the graphicshardware. First the image is pre-warped and thentexture mapping is used to complete the operation.For correct image generation, the object renderedshould be complete, i.e. with no visible gaps andholes due to absence of some parts in source data setbeing used. In [Olive00], the authors propose torender several relief textures of the object per frame.Consequently, the same point of the object can bewarped several times from different relief textures.Obviously, this degrades performance.3.1 OVERFLOW HANDLINGIf a 3D-point is far from the source viewplane of arelief texture, its projection in the destination imagecan fall outside the projection of the polygon beingtextured and such point cannot be drawn. In[Olive00], authors call this effect an overflow. Todeal with this difficulty, the overflows can beaccumulatedontwoauxiliarypolygons,perpendicular to the source view plane [Olive00].Everything that projects outside of the polygoncorresponding to the source viewplane is drawn onthese auxiliary polygons. While this solves theproblem, it requires rendering several polygons foreach relief texture and results in complicatedocclusion-compatible orders. We have chosen toaddress this problem by dynamically resizing thetexture.We associate a bounding polyhedron withthe entire relief texture, with its vertices stored ascoordinates on the source image plane with

displacements. A simplified convex hull (with alimited number of vertices) or a bounding box can beused. Before pre-warping the texture itself, we prewarp the vertices of the bounding polyhedron andcompute its bounding box. We are guaranteed thatall source image pixels will project to within theprojection of this box.splat shape approximation must be fast to compute.The method we propose does not make use ofnormal vectors and other auxiliary information, butrelies only on the displacement values of thesamples. We approximate splats by perspectivelydistorted axis-aligned rectangles.By computing the extent of the projectedbounding box we can infer how much the sourceviewplane needs to be rescaled (shrunk or stretched)and shifted to fit all pre-warped points. We then rescale the texture correspondingly and render it.Figure 1 illustrates the problem, how the relieftextures approach handles the problem and how dowe address it.4.1 SPLAT SHAPE AND SIZE COMPUTATIONAssume that neighbouring source image pixels are ata distance du from each other horizontally, and dvvertically. Then, a pixel at (u,v) can be modelled as arectangle with its corners having coordinates(u du, v dv).LDI warping processdvduSource ImagePixelOriginal view planewith overflow (toppart of cube)[Olive00]: Auxiliaryview planes used toaccumulate overflowNew approach: Scaledand shifted view planeaccommodatesoverflowHandling of Relief Texture OverflowFigure 1Changes in the size of the polygon beingtextured are equivalent to inverse changes in the sizeof the resolution in comparison with the originaltexture. When the angle between the observationdirection and the source viewplane becomes toosharp the viewplane stretches by a large factor, thuslowering the effective resolution and decreasing thequality. We avoid this by storing multiple LRTs withdifferent viewing directions and switching to anotherLRT with a less severe viewing angle. If all the usedLRTs are consistent with each other, this switch ispractically invisible. The only potential source ofinaccuracies is a slight misalignment due to thediscrete nature of textures and the polygon renderinghardware, but we find in practice that these effectsare usually less than 1 pixel. An additional advantageis that we can dynamically adjust the resolution ofthe desired image if the quality can be sacrificed forthe speed of rendering.4.SPLATTINGA drawback of the presented combination of LDIsand relief textures is that we cannot use theinterpolation method presented in [Olive00] due tointra-object visibility. Instead, we have to resort tosplatting. This however turns out to be an advantage,as we can create better approximations to thefootprint of the splats in the image plane thanprevious approaches. For efficiency however, theCorrect Footprint forWarped PixelApproximation to theCorrect Footprint (Square)LRT warping processdv ’dvduSourceImage Pixeldu’Footprint in thePre-Warped ImageCorrect Footprint inDestination ImageApproximation of Splat ShapeFigure 2Having only points with depth and withoutany further information, such as the presence ofnormals to the surface or connectivity betweensamples, a point is simply an evidence of thepresence of a surface. Note that this assumption istrue for most LDI data sets, due to the existence ofmultiple layers. Consequently, we can only assumethat a sample is a flat, axis-aligned region of somesurface facing directly towards the camera.According to the pre-warping equations a pre-warpof such an axis-aligned rectangular sample will resultin an axis-aligned rectangular sample in the prewarped image. Later, the texture-mapping pass willrender the splat correctly on the screen. To pre-warpwarp a source image pixel, we warp its center to findthe center of the splat in the pre-warped image. Theamount of the horizontal and vertical extent of therectangular splat with respect to the original samplesize depends only on the depth, and does not dependon the co-ordinates in the image plane. That is, thesize of the splat can be found asdv ' dv k1d,1 k3ddu' du k 2 d,1 k 3d

where du, dv are horizontal and vertical size of thesource image pixel, and d is the depth. Figure 2illustrates the approach and compares it with thesplatting directly to the screen.4.2 COMPARISON WITH OTHER METHODSWe found this splatting method to be simple yetgiving good visual quality. One of the reasons is thatthe final result is a perspectively distorted splat,which is not orthogonal with respect to the axes ofthe desired view in general. This will generate abetter image compared to other approaches that splatdirectly into screen space. Furthermore, as it hasbeen shown in [Rusin00], simple rectangular splatsare much faster to render than Gaussians, and giventhe time allotted for the frame generation they tendto produce better results.In the original LDI paper, the authors use afixed set of 1x1, 3x3, 5x5, and 7x7 Gaussian splats,that are output directly to the destination image[Shade98]. In this case, the introducedreconstruction errors depend primarily on twofactors - the error due to the approximation of aperspectively distorted splat by a square and amaximum of half a pixel error due to the final imagesampling. In this work, we do not splat directly intothe destination image, but into an intermediate imagethat will be used as a texture map. Since the prewarp stage does not handle rotations or perspectivedistortion and both source and intermediate imageare orthogonal, any axis-aligned rectangle withconstant disparity corresponds to an axis-alignedrectangle in the intermediate image. Under theassumption of rectangular source samples, we get asplat that is optimal with this approach.During the texture mapping stage the splatundergoes perspective distortion. When the texturemapping is done, each splat is rendered as a nonaxis-aligned perspectively distorted quadrilateral,which matches the ideal splat shape much moreclosely than splats produced by many otherapproaches. Since we use arbitrary rectangles assplats, the error in the first pass is mainly due to theround-off errors in the positions of splats in theintermediate image, which are no more than half apixel. During the second pass this error is reduced ormagnified depending on the perspective distortion ofthe view. Furthermore, polygon rasterization andtexture mapping artefacts introduce an additionalerror of less than half a pixel.The error of our method is definitely lessthan the error of the LDI approach when the view istilted or when the object is viewed at sharp angles as(a)(b)Renderings of a fern model, (a) LDI; (b) LRT.Figure 3(a)(b)Parts of the fern renderings zoomed inFigure 4the perspective distortion gets larger in these cases.Furthermore, our approach generates better resultswhen the resulting image is smaller than the originalimage, i.e. when minification occurs, as we canbenefit from mip-mapping. The LDI splats willintroduce severe aliasing artifa

Keywords: image-based rendering, image warping, real time rendering, relief textures, layered depth images, splatting. 1. INTRODUCTION Over the last few years, the growing needs of the graphics community and the recent advances in technology have resulted in the considerable increase in the rendering s

Related Documents:

To Wrap Textures Around Quadric Objects There are 2 ways to do this: you can use quadric textures or automatic texture generation. Quadric textures are easy but don't give much control. Automatic textures give some control but don't ten

a bas-relief from a new photograph, the input image is relit to match the two standard illumination conditions, and the mapping function is used to produce two bas-relief images. A bas-relief surface is constructed from each bas-relief im-age using SFS, and the resulting two bas-relief surfaces are averaged to give the final bas-relief.

Dynamic textures are sequences of images of moving scenes that exhibit certain stationarity properties in time; these include sea-waves, smoke, foliage, whirlwind etc. We present a characterization of dynamic textures that poses the problems of modeling, learning, recognizing and synthesizing dynamic textures on a firm analytical footing.

Relief Valves Relief Valves Set pressure: 4 atm (gauge) Helium relief valve Anderson Greenwood pilot operated relief valve, Model No. 25905K34/S with a discharge area of 1186 mm 2 Nitrogen relief valve Flowsafe F84-8 ¾ X 1F SS 0.261 in 2 Relief Valve Magnet Failu

Relief valves 4.1 Cartridge Valves Technical Information Relief Valves 520L0588 Rev DB August 2010 Quick Reference Thermal Relief Model No. Cavity Description Flow* Pressure Page 1 2T P P104 899 CP208-4 SDC08-2 Relief Valve, Thermal Relief, Poppet Type 1.1 l/min [0.3 US gal/ min] 415 bar [6000 psi] 04.6

RELIEF VENTS CANADA: 1.800.387.7217 U.S.: 1.800.576.1200 WEB: www.thalermetal.com EMAIL: info@thalermetal.com Please be advised Thaler products may undergo improvements from time to time and are subject to change without notice. Thaler Or Equal RV-1A/-RV-1C Relief Vent RV-2A/-RV-2C Relief Vent RV-3A/-RV-3C One Way Relief Vent Relief Vent .

Various structural texture analysis approaches have been proposed, ranging from using different shapes of structuring elements [12] to conceiving real textures as distorted ver-sions of ideal textures [13]. However, these methods appear to be limited in practicality since they can only describe very regular textures [1].

Storage Systems for Automotive Applications Wasim Sarwar1*, Timothy Engstrom1, Monica Marinescu1, Nick Green2, Nigel . As with PHEVs, in a large ESS the use of active thermal management (system consisting of heating and cooling loop) provides good value, therefore a large thermal operating window is not required. Further, a comparatively shorter cycle life is sufficient in order to meet the .