CSC420: Intro To Image . - University Of Toronto
CSC420: Intro to Image UnderstandingIntroductionSanja FidlerJanuary 11, 2021Sanja FidlerIntro to Image Understanding1 / 60
The TeamInstructor:Sanja Fidler (fidler@cs.toronto.edu)Office: all office hours will be hosted onlineOffice hours: Mon 11am-12pm. Please send me an email toschedule.TAs:Sayyed Nezhadi (snezhadi@cs.toronto.edu)Frank Shen (shenti11@cs.toronto.edu)Sanja FidlerIntro to Image Understanding2 / 60
Course InformationClass time: Monday at 9-11amLocation: OnlineTutorials: TUT0101 on Monday 11am-12pm (online), TUT0102on Monday 12-1pm (online), demos and Q&A, we’ll do it ondemandClass Website:http://www.cs.toronto.edu/ fidler/teaching/2021/CSC420.htmlThe class will use Quercus for announcements and discussionsSanja FidlerIntro to Image Understanding3 / 60
Course InformationClass time: Monday at 9-11amLocation: OnlineTutorials: TUT0101 on Monday 11am-12pm (online), TUT0102on Monday 12-1pm (online), demos and Q&A, we’ll do it ondemandClass Website:http://www.cs.toronto.edu/ fidler/teaching/2021/CSC420.htmlThe class will use Quercus for announcements and discussionsSanja FidlerIntro to Image Understanding3 / 60
Course InformationTextbook: We won’t directly follow any book, but extra readingin this textbook will be useful:Rick SzeliskiComputer Vision: Algorithms and Applicationsavailable free online:http://szeliski.org/Book/Links to other material (papers, code, etc) will be posted on theclass webpageSanja FidlerIntro to Image Understanding4 / 60
Course PrerequisitesCourse Prerequisites:Data structuresLinear AlgebraVector calculusNumerical AnalysisWithout this you’ll need some serious catching up to do!Knowing some basics in this is a plus:Python, Matlab, C Machine LearningNeural NetworksSolving assignments sooner rather than laterSanja FidlerIntro to Image Understanding5 / 60
RequirementsEach student expected to complete 4 assignments and a projectAssignments:Short theoretical questions and programming exercisesWill be given roughly every two weeks (starting second week of class)You will have a week to hand in the solution to each assignmentYou need to solve the assignment aloneSanja FidlerIntro to Image Understanding6 / 60
RequirementsEach student expected to complete 4 assignments and a projectAssignments:Short theoretical questions and programming exercisesWill be given roughly every two weeks (starting second week of class)You will have a week to hand in the solution to each assignmentYou need to solve the assignment aloneProject:You will be able to choose from a list of projects or come up with yourown project (discussed prior with your instructor)Need to hand in a report and do an oral presentationCan work individually or in pairsSanja FidlerIntro to Image Understanding6 / 60
RequirementsEach student expected to complete 4 assignments and a projectAssignments:Short theoretical questions and programming exercisesWill be given roughly every two weeks (starting second week of class)You will have a week to hand in the solution to each assignmentYou need to solve the assignment aloneProject:You will be able to choose from a list of projects or come up with yourown project (discussed prior with your instructor)Need to hand in a report and do an oral presentationCan work individually or in pairsSanja FidlerIntro to Image Understanding6 / 60
GradingGrade breakdownAssignments: 60% (15% each)Project oral exam: 30%(project) 10%(oral exam)For the project you will need to doShort project proposalProject reportProject presentation (oral)Oral exam: During the project presentation, you will be askedquestions about the class materialSanja FidlerIntro to Image Understanding7 / 60
Term Work DatesTerm WorkPost DateDue DateAssignment 1Jan 22Jan 29Assignment 2Feb 5Feb 12Assignment 3March 5March 12Assignment 4March 19March 26Project ReportApril 15Project PresentationTBDAll dates are for 2021Dates are approximate (depend on what material we cover in class)Sanja FidlerIntro to Image Understanding8 / 60
Programming Language?Your assignments / project can be implemented either in Python,Matlab, or C . Python is preferred, but not a requirement.Most code and examples we will provide during the class will bein Python and Matlab.Choose wiselySanja FidlerIntro to Image Understanding9 / 60
LatenessDeadline The solutions to assignments / project should besubmitted by 11.59pm on the date they are due.Anything from 1 minute late to 24 hours will count as onelate day.Lateness Each student will be given a total of 3 free late days.This means that you can hand in three of the assignmentsone day late, or one assignment three days late. It is up tothe you to make a good planning of your work. After youhave used the 3 day budget, the late assignments willnot be accepted.Sanja FidlerIntro to Image Understanding10 / 60
SyllabusTentative syllabusIntroLinear filters, edgesImage featuresKeypoint detectionMatchingStereo, multi-viewStereo, multi-viewObject recognitionObject detectionNeural NetworksSegmentationSanja FidlerIntro to Image Understanding11 / 60
IntroductionSanja FidlerIntro to Image Understanding12 / 60
Let’s begin!Introduction to Intro to Image UnderstandingWhat is Computer Vision?Why study Computer Vision?Which cool applications can we do with it?Is vision a hard problem?Sanja FidlerIntro to Image Understanding13 / 60
What is Computer Vision?Sanja FidlerIntro to Image Understanding14 / 60
What is Computer Vision?A field trying to develop automatic algorithms that would “see”Sanja FidlerIntro to Image Understanding15 / 60
Embodied AgentsUnderstand the scene in order to take actions: perception, planning,reasoningFigure: How do I make dinner in this household?Many simulators: Carla, Thor, House3D, VirtualHome, etcSanja FidlerIntro to Image Understanding16 / 60
What is Computer Vision?What does it mean to see?[text adopted from A. Torralba]To know what is where by looking – Marr, 1982Sanja FidlerIntro to Image Understanding17 / 60
What is Computer Vision?What does it mean to see?[text adopted from A. Torralba]To know what is where by looking – Marr, 1982Understand where things are in the worldSanja FidlerIntro to Image Understanding17 / 60
What is Computer Vision?What does it mean to see?[text adopted from A. Torralba]To know what is where by looking – Marr, 1982Understand where things are in the worldWhat are their 3D/material properties?imageSanja FidlerIntro to Image Understanding17 / 60
What is Computer Vision?What does it mean to see?[text adopted from A. Torralba]To know what is where by looking – Marr, 1982Understand where things are in the worldWhat are their 3D/material properties?What actions are taking place?Depth pic from http://vladlen.infoSanja FidlerIntro to Image Understanding17 / 60
What is Computer Vision?What does it mean to see?[text adopted from A. Torralba]To know what is where by looking – Marr, 1982Understand where things are in the worldWhat are their 3D/material properties?What actions are taking place?Pic from www.cobblehillpuzzles.comSanja FidlerIntro to Image Understanding17 / 60
“Full” Image Understanding?Full understanding of an image?Sanja FidlerIntro to Image Understanding18 / 60
“Full” Image Understanding?Full understanding of an image? You can answer any question about it[M. Malinowski, M. Fritz, A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input,NIPS, 2014]Sanja FidlerIntro to Image Understanding18 / 60
“Full” Image Understanding?Full understanding of an image? You can answer any question about itSanja FidlerIntro to Image Understanding18 / 60
“Full” Image Understanding?Full understanding of an image? You can answer any question about itSanja FidlerIntro to Image Understanding18 / 60
“Full” Image Understanding?Full understanding of an image? You can answer any question about itSanja FidlerIntro to Image Understanding18 / 60
“Full” Image Understanding?Full understanding of an image? You can answer any question about itSanja FidlerIntro to Image Understanding18 / 60
“Full” Image Understanding?Full understanding of an image? You can answer any question about itSanja FidlerIntro to Image Understanding18 / 60
“Full” Image Understanding?Full understanding of an image? You can answer any question about itSanja FidlerIntro to Image Understanding18 / 60
“Full” Image Understanding?Full understanding of an image? You can answer any question about itSanja FidlerIntro to Image Understanding18 / 60
Why study Computer Vision?Sanja FidlerIntro to Image Understanding19 / 60
Why study Computer Vision?You are curious how to one day make the robot walk your doghttp://www.cs.toronto.edu/ fidler/videos/robotsmovies.movSanja FidlerIntro to Image Understanding20 / 60
Why study Computer Vision?. and fold your laundryhttps://www.youtube.com/watch?v gy5g33S0GzoSanja Fidlerhttps://www.youtube.com/watch?v KKUaVzf3OqwIntro to Image Understanding21 / 60
Why study Computer Vision?. and drive you to workAmnon Shashua’s Mobileye autonomous driving systemhttps://www.youtube.com/watch?v 4fxFDypHZLsSanja FidlerIntro to Image Understanding22 / 60
Why study Computer Vision?Allows you to manipulate your imagesScene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007Sanja FidlerIntro to Image Understanding23 / 60
Why study Computer Vision?Allows you to manipulate your imagesScene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007Sanja FidlerIntro to Image Understanding23 / 60
Why study Computer Vision?Allows you to manipulate your imagesScene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007Sanja FidlerIntro to Image Understanding23 / 60
Why study Computer Vision?Allows you to manipulate your imagesScene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007Sanja FidlerIntro to Image Understanding23 / 60
Why study Computer Vision?Allows you to manipulate your imagesScene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007Sanja FidlerIntro to Image Understanding23 / 60
Why study Computer Vision?Allows you to manipulate your imagesScene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007Sanja FidlerIntro to Image Understanding23 / 60
Why study Computer Vision?Allows you to manipulate your imageshttps://www.youtube.com/watch?v p5U4NgVGAwgGauGan, Ming-Yu Liu et al., http://nvidia-research-mingyuliu.com/gaugan/]Sanja FidlerIntro to Image Understanding24 / 60
Why study Computer Vision?Change style of images[Gatys, Ecker, Bethge. A Neural Algorithm of Artistic Style. Arxiv’15.]Sanja FidlerIntro to Image Understanding25 / 60
Why study Computer Vision?Change style of videoshttps://www.youtube.com/watch?v Khuj4ASldmU[Ruder, Dosovitskiy, Brox. Artistic style transfer for videos, 2016]Sanja FidlerIntro to Image Understanding26 / 60
Why study Computer Vision?Change style of videoshttps://arxiv.org/pdf/1701.04928.pdfSanja FidlerIntro to Image Understanding27 / 60
Why study Computer Vision?. and make cool videos using a single imagehttp://www.cs.cmu.edu/ om3d/3D Object Manipulation in a Single Photograph using Stock 3D Models,Kholgade, Simon, Efros, Sheikh, SIGGRAPH 2014Sanja FidlerIntro to Image Understanding28 / 60
Why study Computer Vision?Fancy visualization and game analysis in sportsSanja FidlerIntro to Image Understanding29 / 60
Why study Computer Vision?Fancy visualization and special e ects in movies[Source: http://cvfxbook.com andSanja Fidlerhttp://vimeo.com/100095868]Intro to Image Understanding30 / 60
Why study Computer Vision?Reconstruct the world in 3D from online photos!https://www.youtube.com/watch?v IgBQCoEfiMsPhotosynth: https://photosynth.net/Nerf: https://www.youtube.com/watch?v yPKIxoN2Vf0Sanja FidlerIntro to Image Understanding31 / 60
Why study Computer Vision?Figure: Modiface: Toronto-based startupSanja FidlerIntro to Image Understanding32 / 60
Why study Computer Vision?Play with facesSanja FidlerIntro to Image Understandinghttps://www.faceapp.com/(try it!)33 / 60
Why study Computer Vision?Play with facesSanja FidlerIntro to Image Understanding33 / 60
Why study Computer Vision?Play with facesSanja FidlerIntro to Image Understanding33 / 60
Why study Computer Vision?Play with facesSanja FidlerIntro to Image Understanding33 / 60
Why study Computer Vision?Generate new faceshttps://www.youtube.com/watch?v kSLJriaOumAStyleGAN, Tero Karras et al., https://github.com/NVlabs/stylegan]Sanja FidlerIntro to Image Understanding34 / 60
Why study Computer Vision?Generate image descriptions automatically[Source: L. Zitnick, NIPS’14 Workshop on Learning Semantics]Sanja FidlerIntro to Image Understanding35 / 60
Why study Computer Vision?Generate images from descriptions automatically[DALL-E: https://openai.com/blog/dall-e/]Sanja FidlerIntro to Image Understanding36 / 60
Why study Computer Vision?Have a computer do math for youFigure: Photomath: https://photomath.net/,Sanja Fidlerhttp://www.youtube.com/watch?v XlbVB50mIh4Intro to Image Understanding37 / 60
Why study Computer Vision?You can do movie-like ForensicsFigure: Source: Nayar and Nishino, “Eyes for Relighting”[Source: N. Snavely]Sanja FidlerIntro to Image Understanding38 / 60
Why study Computer Vision?Source: Nayar and Nishino, “Eyes for Relighting”[Source: N. Snavely]Sanja FidlerIntro to Image Understanding39 / 60
Why study Computer Vision?Figure: Source: Nayar and Nishino, “Eyes for Relighting”[Source: N. Snavely]Sanja FidlerIntro to Image Understanding39 / 60
Why study Computer Vision?Some more CSICan you see something on the wall?Torralba & Freeman, CVPR’12Sanja FidlerIntro to Image Understanding40 / 60
Why study Computer Vision?Some more CSIFidlerTorralbaSanja& Freeman,CVPR’12 Intro to Image Understanding40 / 60
Why study Computer Vision?Recognizing movie posters (in mobile phones)!"# %&'())* ''''''''''''''''',---./ 010.2 34'Source: S. LazebnikFrom student last year: phone appSanja FidlerIntro to Image Understanding41 / 60
How It All Began.Sanja FidlerIntro to Image Understanding42 / 60
How It All Began.[Slide credit: A. Torralba]Sanja FidlerIntro to Image Understanding43 / 60
50 years and thousands of PhDs later.Popular benchmarks: KITTI, PASCAL, Cityscapes, MS-COCOReasoning demo: http://vqa.cloudcv.org/Sanja FidlerIntro to Image Understanding44 / 60
Why is vision hard?Sanja FidlerIntro to Image Understanding45 / 60
Why is vision hard?Half of the cerebral cortex in primates is devoted to processing visualinformation. This is a lot. Means that vision has to be pretty hard!Sanja FidlerIntro to Image Understanding46 / 60
Why is vision hard?All this is dog.Sanja Fidler[slide adopted from: R. Urtasun]Intro to Image Understanding47 / 60
Why is vision hard?Biederman, 1987[slide credit: R. Urtasun]Sanja FidlerIntro to Image Understanding48 / 60
Why is vision hard?Lots of data to process:Thousands to millions of pixelsin an image100 hours of video added toYouTube per minute [source:YouTube]Over 6 billion hours of videoare watched each month onYouTube – almost an hour forevery person on Earth [source:YouTube]Sanja FidlerIntro to Image Understanding49 / 60
Why is vision hard?Lots of data to process: 5000 new tagged photos added to Flickr per minute (7M per day) 60M photos uploaded to Instagram every day [source: Instagram]Sanja FidlerIntro to Image Understanding50 / 60
Exploit so Much Data!Figure: Vemodalen: The Fear That Everything Has Already Been Done,https://www.youtube.com/watch?v 8ftDjebw8aA[Source: L. Zitnick, NIPS’14 Workshop on Learning Semantics]Sanja FidlerIntro to Image Understanding51 / 60
Why is vision hard?Human vision seems to work quite well.How well does it really work?Let’s play some games!Sanja FidlerIntro to Image Understanding52 / 60
How good are humans?Which square is lighter, A or B?[Slide credit: A. Torralba]Sanja FidlerIntro to Image Understanding53 / 60
How good are humans?Which square is lighter, A or B?[Slide credit: A. Torralba]Sanja FidlerIntro to Image Understanding53 / 60
How good are humans?Figure: 2006 Walt AnthonyWhich red line is longer?[Slide credit: A. Torralba]Sanja FidlerIntro to Image Understanding54 / 60
How good are humans?Figure: 2006 Walt AnthonyWhich red line is longer?[Slide credit: A. Torralba]Sanja FidlerIntro to Image Understanding54 / 60
How good are humans?Figure: Ames roomAssumptions can be wrong[Slide credit: A. Torralba]Sanja FidlerIntro to Image Understanding55 / 60
How good are humans?Figure: Chabris & Simons, https://www.youtube.com/watch?v vJG698U2MvoCount the number of times the white team pass the ballConcentrate, it’s difficult!https://www.youtube.com/watch?v vJG698U2MvoSanja FidlerIntro to Image Understanding56 / 60
How good are humans?Figure: Simons et al., http://www.perceptionweb.com/perception/perc1000/a d ex1.mov (more videoshere: http://www.perceptionweb.com/misc.cgi?id p3104)Is something happening in the picture?Sanja FidlerIntro to Image Understanding57 / 60
How good are humans?Figure: Torralba et .870/slides/blur.aviCan you describe what’s going on in the video?Sanja FidlerIntro to Image Understanding58 / 60
How good are humans?Figure: Torralba et .870/slides/highres.aviCan you describe what’s going on in the video?Sanja FidlerIntro to Image Understanding59 / 60
What do I need.What do I need to become a good Computer Vision researcher?Technical capabilitiesGood programming skillsImaginationEven better intuitionLots of persistenceSome luck always helpsSanja FidlerIntro to Image Understanding60 / 60
Scene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007 . Scene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007 Sanja Fidler Intro to Image Understanding 23/60. Why study Computer Vision? Allows you to manipulate your images Scene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007
Scene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007 . Scene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007 Sanja Fidler Intro to Image Understanding 23/63. Why study Computer Vision? Allows you to manipulate your images Scene Completion using Millions of Photographs, Hays & Efros, SIGGRAPH 2007
L2: x 0, image of L3: y 2, image of L4: y 3, image of L5: y x, image of L6: y x 1 b. image of L1: x 0, image of L2: x 0, image of L3: (0, 2), image of L4: (0, 3), image of L5: x 0, image of L6: x 0 c. image of L1– 6: y x 4. a. Q1 3, 1R b. ( 10, 0) c. (8, 6) 5. a x y b] a 21 50 ba x b a 2 1 b 4 2 O 46 2 4 2 2 4 y x A 1X2 A 1X1 A 1X 3 X1 X2 X3
Publication 1398-5.2 – PDF 1997 Table of Contents IntroTable of Contents Table of Contents Intro-3 List of Figures Intro-9 List of Tables Intro-13 Preface Intro-17 Who Should Use this Manual.Intro-17 ULTRA 100 Series Product Receiving and Storage Responsibility. .
ART V02A Intro to Hist of Western Art I 3 ARHS 200 Art of Western World I 3 EHAP, TCNA ART V02B Intro to Hist of West Art II 3 ARHS 2XXX Intro to Hist of West Art II 3 EHAP, EHAP ART V02C Intro to Non-Western Art 3 ARHS 2XXX Intro to Non-Western Art 3 ART V02D Art of Ancient Mediterranean 3
Actual Image Actual Image Actual Image Actual Image Actual Image Actual Image Actual Image Actual Image Actual Image 1. The Imperial – Mumbai 2. World Trade Center – Mumbai 3. Palace of the Sultan of Oman – Oman 4. Fairmont Bab Al Bahr – Abu Dhabi 5. Barakhamba Underground Metro Station – New Delhi 6. Cybercity – Gurugram 7.
Image Deblurring with Blurred/Noisy Image Pairs Lu Yuan1 Jian Sun2 Long Quan2 Heung-Yeung Shum2 1The Hong Kong University of Science and Technology 2Microsoft Research Asia (a) blurred image (b) noisy image (c) enhanced noisy image (d) our deblurred result Figure 1: Photographs in a low light environment. (a) Blurred image (with shutter speed of 1 second, and ISO 100) due to camera shake.
Digital Image Fundamentals Titipong Keawlek Department of Radiological Technology Naresuan University Digital Image Structure and Characteristics Image Types Analog Images Digital Images Digital Image Structure Pixels Pixel Bit Depth Digital Image Detail Pixel Size Matrix size Image size (Field of view) The imaging modalities Image Compression .
analisis akuntansi persediaan barang dagang berdasarkan psak no 14 (studi kasus pada pt enseval putera megatrading tbk) kementerian riset teknologi dan pendidikan tinggi politeknik negeri manado – jurusan akuntansi program studi sarjana terapan akuntansi keuangan tahun 2015 oleh: novita sari ransun nim: 11042014
