Color Consistency Correction Based On Remapping .

2y ago
37 Views
2 Downloads
8.27 MB
8 Pages
Last View : Today
Last Download : 2m ago
Upload by : Konnor Frawley
Transcription

Color Consistency Correction Based on Remapping Optimization for ImageStitchingMenghan XiaJian Yao*Renping XieMi ZhangSchool of Remote Sensing & Information Engineering, Wuhan University, Wuhan, Hubei, P.R. China*Email: jian.yao@whu.edu.cn Web: http://cvrs.whu.edu.cnJinsheng XiaoElectronic Information School, Wuhan University, Wuhan, Hubei, P.R. ChinaAbstractColor consistency correction is a challenging problem inimage stitching, because it matters several factors, including tone, contrast and fidelity, to present a natural appearance. In this paper, we propose an effective color correctionmethod which is feasible to optimize the color consistencyacross images and guarantee the imaging quality of individual image meanwhile. Our method first apply well-directedalteration detection algorithms to find coherent-content regions in inter-image overlaps where reliable color correspondences are extracted. Then, we parameterize the colorremapping curve as transform model, and express the constraints of color consistency, contrast and gradient in anuniform energy function. It can be formulated as a convexquadratic programming problem which provides the globaloptimal solution efficiently. Our method has a good performance in color consistency and suffers no pixel saturation or tonal dimming. Experimental results of representative datasets demonstrate the superiority of our method overstate-of-the-art algorithms.1. IntroductionAs the growing popularity of image capture equipmentsand photo sharing, we now are in an image world where theimage data can be obtained easily by using SLR cameras,smart phones, or downloading from the social networksand public data platforms1 . This exciting data availability enables many visual applications, such as image recognition [7], panoramic imaging [1, 11, 27], image rendering [31, 32] and virtual navigation [19, 9]. In many cases,images of the same scene might show noticeable tonal inconsistence because of different atmosphere illumination,exposure time and camera response function. Such photo1 USGSFigure 1: Panorama composited with source images takenby cameras with different imaging settings. Color inconsistence is still noticeable in the scene even processed byseamline selection and multi-band blending in Enblend2 .metric difference could particularly influence the visual effects of multiple images based rendering mission [29, 18].This paper focus on studying the color correction problemfor multiview image stitching, as exemplified in Figure 1.Within the technique pipeline of image stitching, colorcorrection is a critical step in presenting the composited image with an natural and consistent tone. Apart from removing the overall tonal disparity, it also facilitates the following seamline selection [10] and blending [22]. However, obtaining a satisfactory color correction result is a non-trivialtask: For one thing, a rigorous correction model that cangenuinely express the inter-image color transforming relations remains an open issue; For another, the original distribution of pixel value indicates semantic information (objectstructure, contrast, saturation, etc), which might might bedegraded or destroyed when adjusting pixels for the leastcolor disparity. This is a solved problem for two image involved color transfer techniques that allows complex operations to enhance process quality, however it is hard to beextended to multiple images scheme uniformly. Cascadingmanipulation strategy [14, 26] subjects to linearly amplifiedaccumulation error, making the global consistency inacces2 Available(satellite image): http://glovis.usgs.gov/12977at: http://enblend.sourceforge.net/

sible. As for multiview color correction, existing methodsmainly employed simple correction models or neglected detail preservation and contrast consideration in the global optimization for consistency [29, 18]. Thus, a robust and effective multiview color correction algorithm remains to beinvestigated further.As to the problems stated above, this paper presents aneffective method integrating detail preservation, global contrast and color consistency into an uniform optimizationframework. It can guarantee the optimality of the processing result. Like regular approaches, we leverage the colorstatistics of overlap regions to obtain the color correspondences. Specially, to improve the accuracy and robustnessof this, the classic Iteratively Reweighted Multivariate Alteration Detection (IR-MAD) [13] is used to detect and remove the disturbance of altered content in overlaps. Inspired by the flexible model used in [5], we parameterize the quadratic spline curve as transform function, whichcan express different constraints in the optimization function effectively. Besides, the optimization problem can besolved efficiently by virtue of convex quadratic programming. Through comparing to the latest methods, our approach illustrated better performance in color consistencyand dynamic range optimization.The rest of the paper is structured as follows. In Section 2 an overview of related work is given. In Section 3the scope of the problem is formally defined and in Section 4 the proposed color correction approach is detailed. InSection 5 experimental results are evaluated. Finally, conclusions and future work are presented in Section 6.2. Related WorksRegarding color processing, there are two relevant techniques: two image based color transfer [4, 8], and multiviewcolor correction [1, 6, 15]. Existing methods are reviewedbelow respectively.2.1. Color TransferThe concept of color transfer was first proposed byReinhard et al. [4], which aims at propagating one image’s color characteristic to another. From this baseline approach, many other following works were proposed. Before 2010, they mainly focus on solving the decorrelationbetween color channels [16] and content-based elaborating color transferring [23], which were well summarizedin [30]. Since then, more emphasis was laid on grain-free,detail preservation and artifacts suppression. To heightenthe detail performance, Xiao et al. [28] proposed a gradientpreserving model to convert the transfer processing to anoptimization balancing the color distribution and the detail performance. Different from a stepwise strategy, Suet al. [20] proposed to perform color mapping and detailboosting separately through gradient-aware decomposition,to obtain a grain-free and detail preserved result. Based onthis idea, they developed more complete framework to suppress corruptive artifacts [21].To avoid color distortion, Nguyen et al. [12] appliedXiao et al.’s algorithm [28] in luminance channel, followedby the color gamut alignment via rotating around luminanceaxis. It was effective in preventing color distortion but lacking in color fidelity because of its simple color transformmodel. To make a more accurate color mapping, Hwanget al. [8] proposed to correct each pixel’s color with anindependent affine model, which is the solution of probabilistic moving least square based on feature color correspondences. Besides, some works presented high-levelcolor transfer application based on semantic segmentationor content recognition [25, 3]. Different with these traditional methods, learning-based color transfer methods wereattempted to train out the proper color mapping relationship [24, 2].2.2. Multiview Color CorrectionHere, multiview color correction refers to correct thecolor of three or more images for consistency in a globalway, which excludes applying color transfer across imagesequence or network to realize color consistency. Brownet al. [1] first proposed to performed the gain compensation for multiple images in the way of global optimization, which has been applied in panorama software Autostitch. Under the linear optimization framework, Xiong etal. [29] extended this method by employing gamma modelin luminance channel and linear model in chromatic channels. Qian et al. [17] presented a manifold method to remove color inconsistence, which made full use of both thecorrespondences in overlap regions and the intrinsic colorstructures. However, its requirement on accurate geometricalignment limited its application range.In 3D modeling, Shen et al. [18] exploited linear function as color correction model over color histogram to generate consistent textures. It is efficient but unable to process great color discrepancy. In photo editing applications,Hacohen et al. [6] proposed to model the remapping curvedirectly, which were optimized for consistent appearanceof photos using color correspondences obtained from nonrigid dense matching [5]. This color model is flexibleenough to correct even large tonal disparity but its dependence on dense matching makes it computationally expensive. To address it, Park et al. [15] presented a robust lowrank matrix factorization method to estimate its correctionparameters, which just need sparse feature matching. But,its color correction ability is not as high.3. Problem FormulationFirst of all, our color correction method aims to be applied on the sequential images whose aligning models have22978

34 !-#!5 67#8 94 /'"2!"#" %&'" () *"( ',%&-)! %Figure 2: Piecewise quadratic spline: the color mappingfunction used in our method. Leveraging the fixed horizontal distribution of red anchor knots, the curve just covers thedomain of the original intensity values [vmin , vmax ].been estimated in advance. Thus, the adjacent relationshipsand overlap regions are used as given information in our approach. Second, our color correction algorithm runs on theassumption that all the color inconsistence among imagesis caused by different imaging conditions which affect eachimage as a whole.Seeking for the optimal consistency, our approach expresses all the quality requirements in the form of constraint on model parameters, which are then solved in aglobal optimization. To realize this, we define the transformation model as three monotonically increasing mappingcurves (one per channel), each of which is formulated as apiecewise quadratic spline with m control knots (m 6as default). As illustrated in Figure 2 , these red knots{(νk , ν̃k )}mk 1 are half free on the coordinate plane, where{νk }marefixed evenly on the horizontal axis to controlk 1the mapping curve effectively, while {ν̃k }mk 1 are free to determine the shape of the mapping curve as the actual modelparameters. Thus, the color mapping function for image Iican be parameterized as:iFi arg{(ν̃1i , ν̃2i , ., ν̃m)c }3c 1 ,(1)where c is the index number of each channel. Particularity, having the curve cover the intensity domain of eachoriginal image can save the troublesome extrapolation fornon-overlap regions. Besides, the detailed definition of ouroptimization function are described in Section 4.2.4. Color Correction%)./01 /'"234 !-#!5 67#8 /'"2)!*! !,-! %&'(!!"# %&'(!!).%//)0'" () *"( ',%&-)!"#!"Figure 3: Visual comparison between the corrected resultwith alteration filtering mask applied and that with nonemask applied when alteration exists.consistency optimization.4.1. Reliable Color CorrespondenceBased on the adjacent relations, we extract color correspondences in the overlaps of each image pair. Forefficiency problem, we adopt the statistical measures ofcolor histogram, instead of matching image color by pixels.That’s to say, we take the same quantiles (color value pairsof the same frequency) in the cumulative color histogramsof the shared contents as correspondences. In general case,the overlap regions can be regarded as the shared contents,even if a bit of misalignment exists. However, if obviouslyaltered objects exist there, their pixels as outliers should beexcluded from the overlaps in advance of histogram counting. To do so, we utilize a famous alteration detection algorithm IR-MAD [13] in overlap regions to generate filteringmasks. However, it is computationally expensive to run IRMAD on the images of their original size, especially forhigh-resolution remote sensing images. Thus, we downsample each overlap region into the size of 250K pixels atfirst and then apply the IR-MAD on them to get a coarsemask. Here is an example that applying alteration filteringmask improves the accuracy of color correction result whenalteration exists in Figure 3.4.2. Remapping Model OptimizationWith color correspondences of image pairs, energy function can be designed by using the parameters that remain tobe optimized. Given a group of images {Ii }ni 1 , we attemptto seek a group of color transformations {Fi }ni 1 whichmaintains a good balance between three quality goals: color: pixels depicting the same content have the samecolor across images; gradient: original structural details remain on thetransformed images; contrast: all the transformed images have a reasonablewide dynamic range;Our color correction method consists of two steps: colorcorrespondence extraction and model parameters optimization. Particularly, the color correction is performed inY CbCr space, because its luminance channel and chromaticchannels are separated and each channel has specific upperand lower bounds, which facilitate the quality-aware color32979

As stated in Section 3, all these quality requirements shouldbe expressed as the constraints on parametric model. In ourmethod, the global optimization is conducted independentlyin each channel. Without losing generality, the transformation function Fi of Ii in certain channel is denoted asifi arg(ν̃1i , ν̃2i , ., ν̃m) for simplicity. So, in any channel,the energy function can be formulated as:E wij Edata (fi , fj ) λn Eregulation (fi )i 1Ii Ij ̸ subject to: Crigid (fi ), i [1, n], (2)where wij is the weight proportionalto the area of the over lap region between Ii and Ij ( wij 1), and λ serves tobalance the data term and the regulation term. In our experiments, λ ξ Mm is used, where M denotes the amount ofcolor correspondences extracted in each overlap (M 16as default) and ξ [0.5, 5] is recommended. Actually, ouralgorithm is insensitive to λ since this two terms are noncontrary. The color consistency across images is expressedby data term that penalizes the deviation between remappedcorresponding color values:Edata (fi , fj ) Mk 1fi (vki ) fj (vkj )2,(3)where {vki , vkj }Mk 1 are color correspondences between Iiand Ij . As an unary term, Eregulation (fi ) enforces certainconstraints on the color transformations softly, includingregularizing the parameters and stretching dynamic range.Eregulation (fi ) m 1k 1fi (v̇ki ) v̇ki )2ii η fi (v0.05) fi (v0.95) 2 , (4)m 1where {v̇ki }k 1denote the horizontal coordinates of jointpoints joining different local curve segments (marked indifferent colors in Figure 2). Exactly, we have v̇ki iνki νk 1,k2 {1, 2, ., m 1}. The importance of this termlies in two aspects: (i) keeping transformed images close totheir original appearance slightly as default solution; (ii) allthe model parameters are optimized as a whole even if nocolor correspondences falls into the scope of some anchors(parameters). In addition, the latter negative term preventsthe dynamic range to narrow down where vαi depicts the αpercentile of the CDF of Ii . This term is very necessary toavoid an easy-appearing optimizing result that corrected images present a consistent but dimming tone, because remapping intensity into a lower range conforms to the minimalenergy principle of Ecolor . Particularly, η 5 and η 0 areused in luminance channel and chromatic channels respectively. Namely, we only impose dynamic range constraintin luminance.!"!# %&'()*' )",-*' )", # *' )", .Figure 4: Channel information display of a color image inY CbCr space. The major gradient information is containedin luminance channel Y .As a qualified color mapping curve, the quadratic splineshould meet two basic requirements: increasing monotonicity and mapping domain lying within gamut. They are guaranteed through the following constraint term:Crigid (fi ) :{τb fi′ (vki ) τu , vki [vmin , vmax ],iivstart fi (v0.01), fi (v0.99) vend ,(5)where τb and τu defines the bottom and upper boundary ofthe mapping curve’s slope domain, and [vstart , vend ] is thegamut range of related channel. Actually, the definition ofslop domain, especially its bottom boundary, is a trade-offproblem between detail preservation and color consistency,since a higher value of bottom boundary (such as τb 1.0)can prevent gradient detail loss from intensity levels merging, while it also inevitably restricts the freedom of parameters to achieve a better color consistency. In our method, weset the slop domain [0.3, 5] for chromatic channels Cb andCr , and [0.5, 5] for luminance channel Y where most of thegradient information is contained, as the example shown inFigure 4.4.3. Implementation DetailsOur correction function, as a piecewise interpolationmodel, can not be expressed explicitly. We now describehow to express the function mapping fi and partial derivaitive fi′ with the actual model parameters {ν̃1i , ν̃2i , ., ν̃m}.iGiven a color value vk , supposing it falls in the controlscope of knots {(νpi , ν̃pi )}p 2, then its remapped value ṽkipcan be obtained via the following quadratic spline interpolation equation: 1ii t2 νp 2], vki [(1 2t t2 )νpi (1 2t 2t2 )νp 12(6) ii ṽki 1 [(1 2t t2 )ν̃pi (1 2t 2t2 )ν̃p 1 t2 ν̃p 2],2where the only unknowns are interpolation coefficient t [0, 1] and ṽki , so ṽki can be expressed by model parameters{ν̃pi }p 2, which is the essence of fi . In fact, we tend to solvepthe interpolation coefficients of all color correspondencesbefore the global optimization, then the function mappingequals to a linear interpolation.According to Eq. (6), we can express the slope value of42980

remapping curve at vki as:iii(ν̃pi 2ν̃p 1 ν̃p 2)t ν̃p 1 ν̃pi ṽ i / t ,iii v i / t(νpi 2νp 1 νp 2)t νp 1 νpi(7)iiwhere νpi 2νp 1 νp 2 0 since {νpi }mareconi 1stants distributing on the horizontal axis evenly (as shownin Figure 2). So, fi′ (vki ) equals to a linear function witht [0, 1] as the free variable. Then, the domain constraintfi′ [τb , τu ] can be expressed as linear inequalities: ν̃ i ν̃pi τb p 1 τu , t 0, i νp 1 νpi(8)ii ν̃p 2 ν̃p 1 τb i τu , t 1,νp 1 νpifi′ (vki ) !"#" "So far, we can substitute Eq. (6) and Eq. (8) into Eq. (3),Eq. (4) and Eq. (5). Then, the energy function Eq. (2) turns aquadratic polynomial, which can be transformed to the standard form of constrained quadratic programming, then isminimized by convex quadratic programming3 efficiently.%"5. Experimental ResultsWe compare the proposed approach against two latest methods in correcting color consistency of multipleimages. The one is linear model based Shen et al.’smethod [18], the other is albedo reflect model based Parket al.’s method [15]. As two typical applications of imagestitching, ground panorama and remote sensing image mosaicking are used to evaluate the performance of color correction approaches. We use the default values of parametersdescribed in Section 4.2 for all the experiments.Generally, making a visually pleasing consistent resultis the primary goal of color correction for image stitching.Thus, visual effects serve as the major evaluation criterionof algorithms. As supplement, quantitative evaluations oncolor Discrepancy (CD) and gradient loss (GL) are also conducted, as the metric equations show: H(Îij , Îji ) CD ,w̄ij Nbin Ii Ij ̸ (9)n 1 Go(Ii , Îi ) GL , n i 1Npixwhere Îi depicts the corrected image of source image Ii ,and Îij denotes the overlap region of Îi shared with Îj .w̄ij is a normalized weight proportional to the area of theoverlap between Ii and Ij ( w̄ij 1). H( ) calculatesthe difference between histograms of images by bins, while Go( ) computes the difference between gradient orientation maps of images by pixels. Nbin is the number of thebins of a histogram, and Npix is the amount of pixels of Ii .3 QuadProg :htt

2.2. Multiview Color Correction Here, multiview color correction refers to correct the color of three or more images for consistency in a global way, which excludes applying color transfer across image sequence or network

Related Documents:

3D Color Correction–1st Order Polynomial Regression 3D Color Correction–2nd Order Polynomial Regression 3D Color Correction–Look-up Table 24.7 15.7 15.5 15.0 0.5 18 ΔE* 94is calculated according to calibration ball. 3D Color Correction – LuminanceDistribution Curve pixel 1 pixel n

the best color correction for elements in your document. You can specify the color correction in a supported driver: Windows 95 / Windows 98 PostScript driver: On the TekColor tab, select the desired color correction. Some of the color corrections are grouped under Office Color and Press Match Color. Windows 2000 / Windows NT PostScript driver:

FPS-1032 FPS-1031 1U 1P HP Business InkJet 1000 OK HP Color Laserjet 1500L OK HP Color Laserjet 1600 OK HP Color Laserjet 2500 OK OK HP Color LaserJet 2550 OK OK HP Color LaserJet 2550L/LN OK HP Color LaserJet 2600 OK HP Color LaserJet 2605 OK OK HP Color LaserJet 2700n OK HP Color LaserJet 2840 OK HP Color LaserJet 3700 OK OK HP Color LaserJet 4000 OK HP Color LaserJet 4100 OK

Types of Eventual Consistency 57 Casual Consistency 57 Read-Your-Writes Consistency 57 Session Consistency 58 Monotonic Read Consistency 58 Monotonic Write Consistency 58 Four Types of NoSQL Databases 59 Key-Value Pair Databases 60 Keys 60 Values 64 Differences Between Key-Value and Relational Databases 65 Document Databases 66

o next to each other on the color wheel o opposite of each other on the color wheel o one color apart on the color wheel o two colors apart on the color wheel Question 25 This is: o Complimentary color scheme o Monochromatic color scheme o Analogous color scheme o Triadic color scheme Question 26 This is: o Triadic color scheme (split 1)

Photomask Japan 2017 10 Curvilinear ILT MPC on VSB: Both Dose and Shape Correction Needed Small feature's resist profile will be bad, shape correction won't help Resist image correction is better to be done by dose correction Shot Size Exposed Resist 100 nm 60 nm Edge Slope Etching and nonlinearity better to be corrected by shape correction .

TL-PS110U TL-WPS510U TL-PS110P 1 USB WiFi 1 Parallel HP Business InkJet 1000 OK OK HP Color Laserjet 1500L OK OK HP Color Laserjet 1600 OK OK HP Color Laserjet 2500 OK OK OK HP Color LaserJet 2550 OK OK OK HP Color LaserJet 2550L/LN OK OK HP Color LaserJet 2600 OK OK HP Color LaserJet 2605 OK OK OK HP Color LaserJet 2700n OK OK HP Color LaserJet 2840 OK OK HP Color LaserJet 3700 OK OK OK

agile software development methodologies (SDMs) utilizing iterative development, prototyping, templates, and minimal documentation requirements. This research project investigated agile SDM implementation using an online survey sent to software development practitioners worldwide. This survey data was used to identify factors related to agile SDM implementation. The factors that significantly .