Introduction To Nonlinear Image Processing Introduction To .
1Introduction to nonlinear image processingIntroduction to Nonlinear Image ProcessingIPAM Summer School on Computer VisionJuly 22, 2013Iasonas KokkinosCenter for Visual ComputingEcole Centrale Paris / INRIA Saclay
2Introduction to nonlinear image processingMean and medianoutlierObservations in a 3x3 window:[1, 2, 100, 1, 3, 2, 1, 5, 3]Mean:(1 2 100 1 3 2 1 5 3)/9 13.1Median:1) Sort:[1, 1, 1, 2, 2, 3, 3, 5, 100]2) Pick mid-pointrobust to outliersnon-linear"
Introduction to nonlinear image processingMean3
Introduction to nonlinear image processingMedian4
Introduction to nonlinear image processingGaussian blur5
6Introduction to nonlinear image processingImage Processingf!imagePrevious lecture:!filter(f )image( f h) (f ) This lecture: remove this constraintfreedom! at the expense of control(h)
7Introduction to nonlinear image processingMulti-scale Gaussian smoothingg (x, y) 12 52exp 2x y2 22 10 20
8Introduction to nonlinear image processingGaussian scale space1expRewrite Gaussian: gt (x, y) 4 tScale space: 2x y4t2 u(x, y, t) gt (x, y) u0 (x, y),2t 2‘time’t 0u(x, y, 0) u0 (x, y)A. Witkin, Scale-space filtering, IJCAI, 1983.J. Koenderink, The structure of images, Biological Cybernetics, 1984J. Babaud, A. P. Witkin, M. Baudin, and R. O. Duda, ‘Uniqueness of the Gaussian kernel forscale-space filtering’, PAMI, 1986.A. Yuille, T.A. Poggio: Scaling theorems for zero crossings. PAMI, 1986.T. Lindeberg, Scale-Space Theory in Computer Vision, Kluwer, 1994L. Florack, Image Structure, Kluwer, 1997B. Romeny, Front-End Vision and Multi-Scale Image Analysis, Kluwer, 2003.J. Weickert, Linear scale space has first been proposed in Japan. JMIV, 1999.
Introduction to nonlinear image processingHeat diffusion and image processingGaussian satisfies:@g@2g@2g 22@t@x@yScale-space satisfies:u(x, y, t) gt (x, y) u0 (x, y)Associative property:f [g h] [f g] hScale-space satisfies:@u@2u @2u 22@t@x@yu(x, y, 0) u0 (x, y)Heat diffusion PDE (Partial Differential Equation)9
Introduction to nonlinear image processingHeat diffusion and image processing10
11Introduction to nonlinear image processingHeat diffusion in 1Dut uxxd dx dudx
12Introduction to nonlinear image processingHeat diffusion in 1D – inhomogeneous materialdut dx conductivitydc udx 12
Introduction to nonlinear image processingHeat diffusion in 2DHomogeneous materialux ! ru (ux , uy )dux ! div(ru) @ ux @ uydx@x@yut uxx uyyInhomogeneous materiald(cux ) ! div(cru)dxut div(cru)13
14Introduction to nonlinear image processingPerona-Malik DiffusionImage-dependent conductivityut div (g ( ru ) ru) u(x, y, 0) u0 (x, y)g(s) exp 2sa2 Diffusion stops at strong image gradients (structure-preserving)CLMC formulation: ru ! rG u P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, PAMI 1990F. Catte, P.L. Lions, J.M. Morel, T. Coll, Image selective smoothing and edge detection bynonlinear diffusion, SIAM J. Numer. Analysis, 1992
15Introduction to nonlinear image processingNonlinear vs. linear diffusion(a) Linear diffusion at t 2(b) Nonlinear diffusion at t 4.4
Introduction to nonlinear image processingExtension to vectorial images Extension of nonlinear diffusion to vectorial images:generalizationwhere:16
17Introduction to nonlinear image processingNonlinear diffusion for color image denoising(a) Color Image with Noise(b) Perona-Malik diffusion
Introduction to nonlinear image processingVariational interpretation of heat diffusion Cost functional: Euler-Lagrange: Heat diffusion: modifies temperature to decrease E quickly18
19Introduction to nonlinear image processingVariational techniques Denoising as functional minimization– Functional: encodes undesirable properties Total Variation:u1(x)u2(x)TV[u1] TV[u2]Minimization flowTV[f] TV[g] TV[h]w1(x)Rudin, L. I.; Osher, S.; Fatemi, E."Nonlinear total variation based noiseremoval algorithms". Physica D 60, 1992w2(x)TV[w1] TV[w2]
20Introduction to nonlinear image processingTotal Variation diffusion(a) Noisy image(b) Total Variation diffusion
21Introduction to nonlinear image processingPerona-Malik versus Total Variation(a) Perona-Malik diffusiong(s) exp 2sK2 (b) Total-Variation diffusion1g(s) s
22Introduction to nonlinear image processingWhat is the ‘right’ cost functional?Can we learn the cost function?S.C. Zhu, Y. N. Wu and D. Mumford, ‘FRAME’, IJCV 1997 Filters: Use Gabors, Difference-of-Gaussians, Gaussian filters,Random Fields: Construct distribution that reproduces theirhistogramsAnd Maximum Entropy: while being as random as possibleNatural images: natural image statisticsTraining:Sample:
Introduction to nonlinear image processing23From FRAME to GRADEØ GRADE: Gibbs Reaction & Diffusion EquationØ GRADE: maximize image probability using Euler-Largange PDEsS.C. Zhu, Y. N. Wu, D. Mumford, ‘Filters, Random Fields and Max. Ent.’, IJCV 1997.S.C. Zhu and D. Mumford, ‘Gibbs Reaction and Diffusion Equation’, PAMI 1998.S. Roth and M. Black, ‘Fields of Experts’, IJCV 2009
24Introduction to nonlinear image processingSecond Moment MatrixDistribution of gradients: P2I0 ,y 0 xxJ Px0 ,y 0 Ix IyP0 ,y 0 Ix IyxP2Ix0 ,y 0 y
25Introduction to nonlinear image processingSecond Moment MatrixDistribution of gradients:J Xx0 ,y 0T(Ix , Iy ) (Ix , Iy )
26Introduction to nonlinear image processingSecond Moment MatrixDistribution of gradients:J Xx0 ,y 0T(rG u) (rG u)
27Introduction to nonlinear image processingSecond Moment MatrixDistribution of gradients:hTJ G (rG u) (rG u)i
28Introduction to nonlinear image processingSecond Moment MatrixhTJ G (rG u) (rG u) Eigenvectors: directions ofmaximal and minimal variation of u Eigenvalues: amounts of minimal andmaximal variation ui
29Introduction to nonlinear image processingAnisotropic diffusionstructure tensorNonlinear diffusionut div (g ( ru ) ru)Nonlinear Anisotropic diffusiondiffusion tensorut div (T (J (ru ) ru)Slide credit: A. RoussosJ. Weickert, Coherence-Enhancing Diffusion Filtering, Image and Vision Computing, 1999.R. Kimmel, R. Malladi, and N. Sochen. Images as Embedded Maps and Minimal Surfaces, IJCV, 2001.D. Tschumperlé, and R. Deriche (2005), Vector-valued image regularization with PDE’s. PAMI, 2005.A. Roussos and P. Maragos, Reversible Interpolation of vectorial Images, IJCV 2009M. Lindenbaum, M. Fischer and A. Bruckstein, "On Gabor's contribution to image enhancement”, 1994.D. Gabor, ‘Information theory in electron microscopy’, 1965
Introduction to nonlinear image processingAnisotropic Diffusion exampleSlide credits: A. Roussos30
Introduction to nonlinear image processingAnisotropic Diffusion exampleSlide credits: A. Roussos31
Introduction to nonlinear image processingNonlinear anisotropic diffusionExtension to vectorial imagesStructure tensor for vectorial image:Slide credits: A. Roussos32
33Introduction to nonlinear image processingNonlinear anisotropic diffusionNoisy inputSlide credits: A. RoussosNonlinear Anisotropic Diffusion
34Introduction to nonlinear image processingNonlinear vs. Nonlinear and anisotropic diffusionNonlinear DiffusionSlide credits: A. RoussosNonlinear Anisotropic Diffusion
35Introduction to nonlinear image processingNonlinear vs. Nonlinear and anisotropic diffusionNonlinear DiffusionSlide credits: A. RoussosNonlinear Anisotropic Diffusion
36Introduction to nonlinear image processingInpainting problemInpainting region D(a) Image with missing region(c1) 300 iterations(b) Initial prediction (constant)(c2) 600 iterations(c3) 1500 iterations(converged)M. Bertalmío, G. Sapiro, V. Caselles and C. Ballester., "Image Inpainting", SIGGRAPH 200T. F. Chan and J. Shen, "Mathematical Models for Local Nontexture Inpainting", SIAM, 200
37Introduction to nonlinear image processingApplication: text removal(a) InputSlide credits: A. Roussos(b) Total Variation Inainting
Introduction to nonlinear image processingApplication: fake bravadoSlide credits: A. Roussos38
39Introduction to nonlinear image processingPDE-based interpolationInterpolation problem(a) Low resolution input image(b) Zero-Padding initialization(c) 4x4 PDE-based magnificationA. Roussos and P. Maragos, Reversible Interpolation of vectorial Images, IJCV 2009
40Introduction to nonlinear image processingPDE-based interpolation(a) InputSignal(b) Bicubic interpolation (4 x 4)(d) PDE-based interpolation(4x 4)A. Roussos and P. Maragos, Reversible Interpolation of vectorial Images, IJCV 2009
Introduction to nonlinear image processing41PDE-based interpolation(a) Low-resolution Input(b) Initialization: Zero-Padding(c) 4x4 PDE-based interpolationA. Roussos and P. Maragos, Reversible Interpolation of vectorial Images, IJCV 2009
42Introduction to nonlinear image processingPDE-based interpolation(a) Input(c) TV based Interpolation(b) Bilinear Interpolation(d) Structure-tensor based interpolationA. Roussos and P. Maragos, Reversible Interpolation of vectorial Images, IJCV 2009
Introduction to nonlinear image processing43Further studyFast numerical solutionsG. Papandreou and P. Maragos, Multigrid Geometric Active Contour Models,TIP, 2007.J. Weickert and B. H. Romeny, 'Efficient Schemes for Nonlinear Diffusion Filtering', TIP '98A. Chambolle, 'An Algorithm for Total Variation Minimization and Applications', JMIV 2004T. Goldstein, S. Osher The Split Bregman method for L1-regularized problems, SIAM 2009Not covered in this talkBilteral Filter, Non-Local MeansBuades, B. Coll, and J. Morel, “A non-local algorithm for image denoising,” CVPR, 2005C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” ICCV 1998P Milanfar, " A Tour of Modern Image Filtering ", IEEE Signal Processing Magazine, 2013Online softwarehttp://www.ipol.im/
Introduction to nonlinear image processing 14 Perona-Malik Diffusion P. Perona and J. Malik, Scale-space and edge detection using anisotropic diffusion, PAMI 1990 F. Catte, P.L. Lions, J.M. Morel, T. Coll, Image selective smoothing and edge detection by nonlinear diffusion, SIAM J. Numer. Analysis, 1992 u(x,y, 0) u 0(x,y)
Introduction QThis has led to a growing interest in the development of nonlinear image processing methods in recent years QDue to rapidly decreasing cost of computing, image storage, image acquisition, and display, complex nonlinear image processing algorithms have also become more practical for implementation
Keywords: Image filtering, vertically weighted regression, nonlinear filters. 1. Introduction Image filtering and reconstruction algorithms have played the most fundamental role in image processing and ana-lysis. The problem of image filtering and reconstruction is to obtain a better quality image θˆfrom a noisy image y {y
The input for image processing is an image, such as a photograph or frame of video. The output can be an image or a set of characteristics or parameters related to the image. Most of the image processing techniques treat the image as a two-dimensional signal and applies the standard signal processing techniques to it. Image processing usually .
widespread use of nonlinear digital processing in a variety of applications. Most of the currently available image processing software packages include nonlinear techniques (e.g. median filters and morphological filters). A multi- plicity of nonlinear digital image processing techniques have appeared in the literature.
ysis, Mathematical Morphology, Nonlinear Image Representation Table of Contents Part I: Introduction and Theoretical Aspects Abbreviations 2 1 Introduction 3 1.1 Intensity-based Image Processing Frameworks 3 1.2 Spatially-Adaptive Image Processing 3 1.3 Extrinsic vs Intrinsic Approaches 4 1.4 General Adaptive-Neighborhood Image Processing 4
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 7, NO. 7, JULY 1998 979 Nonlinear Image Estimation Using Piecewise and Local Image Models Scott T. Acton, Member, IEEE, and Alan C. Bovik, Fellow, IEEE Abstract— We introduce a new approach to image estimation based on a flexible constraint framework that encapsulates mean-ingful structural image .
L2: x 0, image of L3: y 2, image of L4: y 3, image of L5: y x, image of L6: y x 1 b. image of L1: x 0, image of L2: x 0, image of L3: (0, 2), image of L4: (0, 3), image of L5: x 0, image of L6: x 0 c. image of L1– 6: y x 4. a. Q1 3, 1R b. ( 10, 0) c. (8, 6) 5. a x y b] a 21 50 ba x b a 2 1 b 4 2 O 46 2 4 2 2 4 y x A 1X2 A 1X1 A 1X 3 X1 X2 X3
ARCHITECTURAL STANDARDS The following Architectural Standards have been developed to aid homeowners, lot owners, architects, builders, and other design professionals in the understanding of what are the appropriate details to preserve a timeless Daufuskie Architecture. The existing residents of the island can rely on these guidelines to encourage quality, attention to detail, and by creating a .
