Lecture 1: Machine Learning Problem - University Of Adelaide

1y ago
33 Views
3 Downloads
1.12 MB
52 Pages
Last View : 14d ago
Last Download : 2m ago
Upload by : Hayden Brunner
Transcription

Course infoMachine LearningReal life problemsLecture 1: Machine Learning ProblemQinfeng (Javen) Shi28 July 2014Intro. to Stats. Machine LearningCOMP SCI 4401/7401Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTable of Contents I1Course info2Machine LearningWhat’s Machine Learning?Types of LearningOverfittingOccam’s Razor3Real life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsEnrolmentEnrol yourself in Forum for messages, assignments and slidesLink: o to Course “ISML-S2-2014”Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsAssessmentThe course includes the following assessment components:Final written exam at 55% (open book).Three assignments at 15% each (report and code).Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsRequired SkillsAbility to program in Matlab, C/C is required.Knowing some basic statistics, probability, linear algebra andoptimisation would be helpful, but not essential. They will becovered when needed.Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsRecommended books1Pattern Recognition and Machine Learning by Bishop,Christopher M.2Kernel Methods for Pattern Analysis by John Shawe-Taylor,Nello Cristianini3Convex Optimization by Stephen Boyd and LievenVandenbergheBook 1 is for machine learning in general. Book 2 focuses onkernel methods with pseudo code and some theoretical analysis.Book 3 gives introduction to (Convex) Optimization.Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsExternal coursesLearning from the Data by Yaser Abu-Mostafa in Caltech.Machine Learning by Andrew Ng in Stanford.Machine Learning (or related courses) by Nando de Freitas inUBC (now Oxford).Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorMachine LearningUsing data to uncover an underlying process.Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorFormulationInput: x X(feature)Output: y Y(label)Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorFormulationInput: x X(feature)Output: y Y(label)Underlying process (unknown) f : X YQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorFormulationInput: x X(feature)Output: y Y(label)Underlying process (unknown) f : X YData: {(xi , yi )}Ni 1Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorFormulationInput: x X(feature)Output: y Y(label)Underlying process (unknown) f : X YData: {(xi , yi )}Ni 1 LearnDecision function g : X Y, such that g f .Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorFormulationInput: x X(feature)Output: y Y(label)Underlying process (unknown) f : X YData: {(xi , yi )}Ni 1 LearnDecision function g : X Y, such that g f .For a new x0 , predict y 0 g (x0 ).Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorExamplesXYQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorExamplesX(age, education, occupation, .)Qinfeng (Javen) Shi Yincome 50k p.a.?Lecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorExamplesX(age, education, occupation, .)Qinfeng (Javen) Shi Yincome 50k p.a.? {0, 1, ., 9}Lecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorExamplesX(age, education, occupation, .)Qinfeng (Javen) Shi Yincome 50k p.a.? {0, 1, ., 9} {John, Jenny , .}Lecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorExamplesX(age, education, occupation, .) Yincome 50k p.a.? {0, 1, ., 9} {John, Jenny , .}To learn decision function g : X Y. What’s g like?Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorDecision functionsInner productFor vectors x [x 1 , x 2 , · · · , x d ] , w [w 1 , w 2 , · · · , w d ] , theinner productdXhx, wi xiwi.i 1RdWe write x, w to say they are d-dimensional real numbervectors. We consider all vectors as column vectors by default.Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

What’s Machine Learning?Types of LearningOverfittingOccam’s RazorCourse infoMachine LearningReal life problemsDecision functionsInner productFor vectors x [x 1 , x 2 , · · · , x d ] , w [w 1 , w 2 , · · · , w d ] , theinner productdXhx, wi xiwi.i 1RdWe write x, w to say they are d-dimensional real numbervectors. We consider all vectors as column vectors by default.Sign functionFor any scalar a R, sign(a) 1 1Qinfeng (Javen) Shiif a 0otherwiseLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorDecision functionsTypical decision functions for classificationBinary-classMulti-class1:g (x; w) sign(hx, wi).g (x; w) argmax(hx, wy i).y Ywhere w, wy are the parameters, and x, w, wy Rd .1for b R, more general form hx, wi b can be rewritten as h[x; 1], [w; b]iQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorDecision functionsTypical decision functions for classificationBinary-classMulti-class1:g (x; w) sign(hx, wi).g (x; w) argmax(hx, wy i).y Ywhere w, wy are the parameters, and x, w, wy Rd .ParameterisationTo learn g is to learn w or wy .1for b R, more general form hx, wi b can be rewritten as h[x; 1], [w; b]iQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorTypes of Learning1Supervised Learning2Unsupervised Learning3Semi-supervised LearningQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorSupervised LearningDefinitionGiven input-output data pairs {(xi , yi )}ni 1 sampled from anunknown but fixed distribution p(x, y ), the goal is to learng : X Y, g G s.t. p(g (x) 6 y ) is small.p(g (x) 6 y ) (i.e. expected testing error) is generalisation error.Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorSupervised LearningCoin recognition (vending machines and parking meters).Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorSupervised LearningWe have (input, correct output) in the training data.MassMass50c 250c 2 110c 120c10c5c20c5cSizeQinfeng (Javen) ShiSizeLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorUnsupervised LearningInstead of (input, correct output), we have (input, ?).MassSizeQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorSemi-supervised LearningWe have some (input, correct output), and some (input, ?).Mass50c 2 110c20c5cSizeQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorOverfittingFitting the training data too well cause a problem.TrainingTestingQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorOverfittingTrain on training data (testing data are hidden from us).TrainingTestingQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorOverfittingTwo possible models. Which model fits the training data better?TrainingTestingQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorOverfittingTwo possible models. Which model fits the testing data better?TrainingTestingQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorOverfittingReveal the testing data.TrainingTestingQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorOccam’s Razor“The simplest model that fits the data is also the most plausible.”Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorOccam’s Razor“The simplest model that fits the data is also the most plausible.”Two questions:1What does it mean for a model to be simple?Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorOccam’s Razor“The simplest model that fits the data is also the most plausible.”Two questions:1What does it mean for a model to be simple?2Why simpler is better?Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorSimpler means less complexModel complexity – two types:1 complexity of the function g : order of a polynomial, MDLa straight line (order 0 or 1) is simpler than a quadraticfunction (order 2).computer program: 100 bits simpler than 1000 bits2complexity of the space G: G , VC dimension, noise-fitting, .Often used in proofs.Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsWhat’s Machine Learning?Types of LearningOverfittingOccam’s RazorSimpler is better1What do you mean by “better”?smaller generalisation error (e.g. smaller expected testingerror).2Why simpler is better?Practically implemented by regularisation techniques, whichwill be covered in Lecture 2.Theoretically answered by generalisation bounds, which will becovered in Learning Theory in Lecture 12.Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentTypical assumptions1Small-scale dataModel fits in the memoryData fit in the memory or at least the diskComputer is fast enough23{(xi , yi )}Ni 1 are independent and identically distributed (i.i.d.)samples from p(x, y )Underlying process (f (x) or p(x, y )) unknown but fixedQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentIn real life things are more complex1Small-scale dataLarge-scale Random ProjectionQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentIn real life things are more complex1Small-scale dataLarge-scale Random Projection2{(xi , yi )}Ni 1 are independent and identically distributed (i.i.d.)samples from p(x, y )Correlated Structured Learning and Graphical ModelsQinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentIn real life things are more complex1Small-scale dataLarge-scale Random Projection2{(xi , yi )}Ni 1 are independent and identically distributed (i.i.d.)samples from p(x, y )Correlated Structured Learning and Graphical Models3Underlying process unknown but fixedChanging environment Online Learning (with StructuredData)Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentLarge-scale dataAssumption 1: Small-scale data.Web topic classification: 4.4 million data, input vector 1.8million dimensions, and output 7k classes?argmaxy Y (hx, wy i)? No! “store all wy ” 100G memory.Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentLarge-scale dataAssumption 1: Small-scale data.Web topic classification: 4.4 million data, input vector 1.8million dimensions, and output 7k classes?argmaxy Y (hx, wy i)? No! “store all wy ” 100G memory.Our methods:Loading data, training and testing on 804, 414 news articles topredict the topics in 25.16s!Training 4.4 million data in 0.5 hours (normally 2000 days).Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentStructured dataAssumption 2: {(xi , yi )}Ni 1 are independent.Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentStructured dataAssumption 2: {(xi , yi )}Ni 1 are independent.Figure : Tennis action recognitionMost likely actions argmaxy1 ,y2 ,y3 ,y4 P(y1 , y2 , y3 , y4 Image).Qinfeng (Javen) ShiLecture 1: Machine Learning Problem

Course infoMachine LearningReal life problemsTypical assumptionsLarge-scale dataStructured dataChanging environmentStructured dataAssumption 2: {(xi , yi )}Ni 1 are independent.Figure : Tennis action recognitionMost likely actions argmaxy1 ,y2 ,y3 ,y4 P(y1 , y

Machine Learning Real life problems Lecture 1: Machine Learning Problem Qinfeng (Javen) Shi 28 July 2014 Intro. to Stats. Machine Learning . Learning from the Databy Yaser Abu-Mostafa in Caltech. Machine Learningby Andrew Ng in Stanford. Machine Learning(or related courses) by Nando de Freitas in UBC (now Oxford).

Related Documents:

Introduction of Chemical Reaction Engineering Introduction about Chemical Engineering 0:31:15 0:31:09. Lecture 14 Lecture 15 Lecture 16 Lecture 17 Lecture 18 Lecture 19 Lecture 20 Lecture 21 Lecture 22 Lecture 23 Lecture 24 Lecture 25 Lecture 26 Lecture 27 Lecture 28 Lecture

Lecture 1: A Beginner's Guide Lecture 2: Introduction to Programming Lecture 3: Introduction to C, structure of C programming Lecture 4: Elements of C Lecture 5: Variables, Statements, Expressions Lecture 6: Input-Output in C Lecture 7: Formatted Input-Output Lecture 8: Operators Lecture 9: Operators continued

Lecture 5-6: Artificial Neural Networks (THs) Lecture 7-8: Instance Based Learning (M. Pantic) . (Notes) Lecture 17-18: Inductive Logic Programming (Notes) Maja Pantic Machine Learning (course 395) Lecture 1-2: Concept Learning Lecture 3-4: Decision Trees & CBC Intro Lecture 5-6: Artificial Neural Networks .

decoration machine mortar machine paster machine plater machine wall machinery putzmeister plastering machine mortar spraying machine india ez renda automatic rendering machine price wall painting machine price machine manufacturers in china mail concrete mixer machines cement mixture machine wall finishing machine .

Lecture 1: Introduction and Orientation. Lecture 2: Overview of Electronic Materials . Lecture 3: Free electron Fermi gas . Lecture 4: Energy bands . Lecture 5: Carrier Concentration in Semiconductors . Lecture 6: Shallow dopants and Deep -level traps . Lecture 7: Silicon Materials . Lecture 8: Oxidation. Lecture

TOEFL Listening Lecture 35 184 TOEFL Listening Lecture 36 189 TOEFL Listening Lecture 37 194 TOEFL Listening Lecture 38 199 TOEFL Listening Lecture 39 204 TOEFL Listening Lecture 40 209 TOEFL Listening Lecture 41 214 TOEFL Listening Lecture 42 219 TOEFL Listening Lecture 43 225 COPYRIGHT 2016

Partial Di erential Equations MSO-203-B T. Muthukumar tmk@iitk.ac.in November 14, 2019 T. Muthukumar tmk@iitk.ac.in Partial Di erential EquationsMSO-203-B November 14, 2019 1/193 1 First Week Lecture One Lecture Two Lecture Three Lecture Four 2 Second Week Lecture Five Lecture Six 3 Third Week Lecture Seven Lecture Eight 4 Fourth Week Lecture .

ASTM D2310 - Machine-Made Reinforced Thermosetting Resin Pipe. 11. ASTM D2996 - Filament-Wound Reinforced Thermosetting Resin Pipe. 12. ASTM D4021 - Glass-Fiber-Reinforced Polyester Underground Petroleum Storage Tanks. 13. NFPA 30 - Flammable and Combustible Liquids Code. University of Houston Master Construction Specifications Insert Project Name AE Project Number: Fuel Oil Piping 22 50 00 .