Style Transfer For Headshot Portraits - People MIT CSAIL

3y ago
21 Views
2 Downloads
8.03 MB
51 Pages
Last View : 2m ago
Last Download : 2m ago
Upload by : Mika Lloyd
Transcription

Style Transfer for Headshot PortraitsYiChang ShihSylvain ParisConnelly BarnesMIT CSAILAdobeUniversity of VirginiaWilliam T. FreemanFrédo DurandMIT CSAILMIT CSAIL

Professional portraits look betterOrdinary photoProfessional photo

The goal: make good portraits easy Makelook likeOrdinary photoProfessional photo Transfer the style from the example photo Automatic

We work on headshots What we match: retouching, texture, lighting What we do not match: pose, expression,clothing, focal length, aperture

Preview our resultInputExampleOutput

Hard problem: color transfer is not sufficient Humans are intolerant to artifacts on facesInputExampleOur method[HaCohen et al. 2010](lighting and detailsare missing)

Related work: global transfer[Bae et al. 2006, Sunkavalli et al. 2010 ] Work well on landscapesInputModel Do not work as well on portraitsOutput by Bae et al. [2006]

Related work: global transfer[Bae et al. 2006, Sunkavalli et al. 2010 ] Work well on landscapesInputModel Do not work as well on portraitsOutput by Bae et al. [2006]

Related work: local style transfer Time hallucination [Shih et al. 2013, Laffont et al. 2014]Input: afternoonExample images Requires two images: before and afterOutput: night

Related work: face enhancement[Joshi et al. 2010, Shih et al. 2013 ] Image restoration: deblurring, denoising Blurred input faceExamplesOutput: deblurred face We focus on photographic stylization.

Problem statement Input: a casual frontal portrait and an example Output:‐ The input portrait rendered in the example style‐ Automatic‐ The style includes texture, tone, and color

Key idea #1: local transfer Local: eyes, nose, skin, etc. are treated differentlyInputExample

Key idea #1: local transfer Local: eyes, nose, skin, etc. are treated differentlyInputExample

Key idea #2: multi‐scale transfer Textures at different scales are treated differentlyPortrait #1Portrait #2

Key idea #2: multi‐scale transfer Textures at different scales are treated differentlyPortrait #1Portrait #2

Overview of the algorithm1. Dense matching between the input and example2. Multiscale transfer of local statistics3. Post processing on eyes and backgroundInputExampleStep 1: matchingStep 2: transfer Step 3: post processing

Step 1: dense matching Rigid warp SIFT flow to align semantic features[Liu et al. 2008]InputExampleWarped example

Step 2: multi‐scale local transferInputExample

Step 2: multi‐scale local transfer1. Construct Laplacian stacks for the input and the exampleInputExample

Step 2: multi‐scale local transfer1. Construct Laplacian stacks for the input and the exampleInputExample2. Local matchat each scale

Step 2: multiscale transfer of local statistics1. Construct Laplacian stacks for the input and the exampleInput2. Local matchat each scaleExample3. Collapse the matched stacks to create the output of this stepOutput

Step 2: multi‐scale local transfer1. Construct Laplacian stacks for the input and the exampleInput2. Local matchat each scaleExample3. Collapse the matched stacks to create the output of this stepOutput

Local energyExample LaplacianLocal energyℓGaussian kernel at this scale

At each scale: match local energyInput energyExample energy

At each scale: match local energyComputethe gain mapExample LaplacianLocal energy S[E]Gain map Input LaplacianLocal energy S[I]

At each scale: match local energyComputethe gain mapLocal energy S[E]Example LaplacianGain map Input LaplacianModulatethe input LaplacianLocal energy S[I] Input LaplacianGain mapOutput Laplacian

Robust transfer Clamp the gain map to avoid artifactscaused by moles or glasses on the exampleInputExampleWithout robust transferOur robust transfer

Laplacian using a face mask Preserve the hair boundary using normalizedconvolution and a face maskInputExampleWithout using the mask(the edges disappear)Our method(the edges are preserved)

Step 3: post‐processing Adding eye highlights Replacing the backgroundInputExampleWithout eye highlights Adding eye highlights(Our final result)

Algorithm recapInputExampleStep 1.Dense alignment

Algorithm recapInputExampleStep 2.Step 1.Dense alignment Local transfer

Algorithm recapInputExampleStep 2.Step 1.Dense alignment Local transferStep 3.Eyes andbackground

Automatic example selection Retrieve the best examples based on the facesimilarity between the inputInputThe top three retrieved results

Automatic example selection The results are robust to the example choicesInputStyle transferred results using the top three examples

ResultsInputExamples are shown in the insetsStyle 1Style 2Style 3

Close‐upInputExampleOutput

ExampleOutput

More resultsInputStyle 1Style 2Style 3

Outdoor inputInputStyle 1Style 2Style 3

Extra resultsInputStyle 1Style 2Style 3

ComparisonsInputExampleGlobal transfer[Bae et al. 2006]Our result

InputExampleHistogram transfer [Reinhard et al. 2001]Our method[Pitié et al. 2007][Sunkavalli et al. 2010]Photoshop Match Color

Different success levels: good results The inputs are well litInputOutput

Hard case Matting (face mask) failureInputOutput

Limitations Require the input and the example to have similar facialattributes, e.g., skin color Cannot handle hard shadows on the inputInputExampleFailure output

Evaluation 94 headshot inputsfrom Flickr Available on ourwebsite

Extension to videos

Conclusion We introduce a style transfer algorithm tailored forheadshot portraits. Based on multiscale transfer of local image statisticsInputExampleOutput

Code and data are available Matlab code Flickr evaluation datasetpeople.csail.mit.edu/yichangshih/portrait web/

Acknowledgments We thank Kelly Castro for discussing with ushow he works and for his feedback, MichaelGharbi and Krzysztof Templin for being ourportrait models. We acknowledge the funding from QuantaComputer and Adobe.

Conclusion We introduce a style transfer algorithm tailored forheadshot portraits. Based on multiscale transfer of local image statisticsInputExampleOutput

Ordinary photo Professional photo. We work on headshots What we match: retouching, texture, lighting What we do not match: pose, expression, clothing, focal length, aperture . Input: a casual frontal portrait and an example

Related Documents:

Bruksanvisning för bilstereo . Bruksanvisning for bilstereo . Instrukcja obsługi samochodowego odtwarzacza stereo . Operating Instructions for Car Stereo . 610-104 . SV . Bruksanvisning i original

resume. If you do not have a quality headshot, you will also need to secure a photographer and schedule a headshot session. Please bring in your current headshot and resume for the first mock audition. On Jan 29 please email your appointment time and photographer’s info for your headshot

Is a good headshot and resume reall that important? How do I get the best headshot and how much should it cost? What are today’s headshot trends? What is the secret to getting a headshot that really works for me an

Headshot Morph 1000 is the world's most comprehensive facial morph system. Specially designed for photo-modelling and controlled modification of Headshot photo-generated models, they can be used to produce virtually scan-level results. Headshot morphs can also be used for general modelling, and in tandem with existing Character Creator morphs.

HEADSHOT. 3,599* Sponsor recognition at the headshot café Exclusive access to member information for those who come to the headshot café Branded USB's for headshots Ability to preload ad on to USB's that will be given to each attendee who gets a headshot * Limited availability † Availability based on one month cycles of .

10 tips och tricks för att lyckas med ert sap-projekt 20 SAPSANYTT 2/2015 De flesta projektledare känner säkert till Cobb’s paradox. Martin Cobb verkade som CIO för sekretariatet för Treasury Board of Canada 1995 då han ställde frågan

service i Norge och Finland drivs inom ramen för ett enskilt företag (NRK. 1 och Yleisradio), fin ns det i Sverige tre: Ett för tv (Sveriges Television , SVT ), ett för radio (Sveriges Radio , SR ) och ett för utbildnings program (Sveriges Utbildningsradio, UR, vilket till följd av sin begränsade storlek inte återfinns bland de 25 största

Hotell För hotell anges de tre klasserna A/B, C och D. Det betyder att den "normala" standarden C är acceptabel men att motiven för en högre standard är starka. Ljudklass C motsvarar de tidigare normkraven för hotell, ljudklass A/B motsvarar kraven för moderna hotell med hög standard och ljudklass D kan användas vid