Artificial Neural Networks - Computer Science
Introduction to Learning AlgorithmsHill ClimbingSimulated AnnealingSummaryArtificial Neural NetworksAn Introductory LookSayed Jahed Hussini & Hisham SalehWestern Michigan UniversityDepartment of Computer ScienceAdvanced Theory of ComputationDr. Elise de DonckerFebruary 4, 2016Hussini & SalehArtificial Neural Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsSource: ams-of-artificialneura-1712226908Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase utline1 Introduction23456ProblemSolutionConceptsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureApplicationsCase StudyBankruptcy PredictionBenefits/LimitationsQuestionsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase achines will follow a path that mirrors the evolutionof humans.“Ray Kurzweil”Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase roblemIn 2012, Google received over 2 million search queries perminuteSource: http://pennystocks.la/internet-inreal-time/Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase roblemIn 2012, Google received over 2 million search queries perminuteIn 2014 it received over 4 million search queries per minuteSource: http://pennystocks.la/internet-inreal-time/Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase roblemIn 2012, Google received over 2 million search queries perminuteIn 2014 it received over 4 million search queries per minuteEvery Second:Source: http://pennystocks.la/internet-inreal-time/Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase roblemWorld is full of dataSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase roblemWorld is full of dataIn today’s interconnected e-world, information can be storedand transmitted instantlySayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase roblemWorld is full of dataIn today’s interconnected e-world, information can be storedand transmitted instantlyChallange?Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase roblemWorld is full of dataIn today’s interconnected e-world, information can be storedand transmitted instantlyChallange?To generate useful knowledge from collecteddataSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase roblemQuestion How do we extract knowledge from noisy mass ofdata?Traditional computers are too dumb tounderstand patterns or do analysisSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase roblemQuestion How do we extract knowledge from noisy mass ofdata?Traditional computers are too dumb tounderstand patterns or do analysisSolution Empirical computer models that learnInterpretation requires data acquisition, cleaning(preparing the data for analysis),Key is to extract information about data fromrelationships buried within the data itself.Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase olutionHuman Brain is the most powerful computer every invented to doanalysisHowever it cannot handle huge amounts of dataSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase rtificial Neural Networks(ANN)Rather than programming computers to the most specificdetail of their tasks teach them how to do a job e.g: childrenSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase rtificial Neural Networks(ANN)Rather than programming computers to the most specificdetail of their tasks teach them how to do a job e.g: childrenWe must built empirical models that can find patterns rapidlyand accurately(to some extent) burried in dataSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase rtificial Neural Networks(ANN)Rather than programming computers to the most specificdetail of their tasks teach them how to do a job e.g: childrenWe must built empirical models that can find patterns rapidlyand accurately(to some extent) burried in dataArtificial Intelligence System - AI can do thisSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase rtificial Neural Networks(ANN)Rather than programming computers to the most specificdetail of their tasks teach them how to do a job e.g: childrenWe must built empirical models that can find patterns rapidlyand accurately(to some extent) burried in dataArtificial Intelligence System - AI can do thisANN is a case of AISayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureOutline1 Introduction23456ProblemSolutionConceptsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureApplicationsCase StudyBankruptcy PredictionBenefits/LimitationsQuestionsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureArtificial Neural Networks(ANN)ANN is an information processing paradigm that is inspired bythe way biological nervous systems, such as the brain, processinformationSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureArtificial Neural Networks(ANN)ANN is an information processing paradigm that is inspired bythe way biological nervous systems, such as the brain, processinformationThe key element in this paradigm is the novel structure ofinformation processingSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureArtificial Neural Networks(ANN)ANN is an information processing paradigm that is inspired bythe way biological nervous systems, such as the brain, processinformationThe key element in this paradigm is the novel structure ofinformation processingANNs, like people, learn by exampleCurrently, an ANN is configured for a specific application e.g:pattern recognition, data calssificationSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureHow human brain learns?Human brain is a dense network of approximately 1011neurons, each connected to, on average, 104 othersSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureHow human brain learns?Human brain is a dense network of approximately 1011neurons, each connected to, on average, 104 othersNeuron activity is excited or inhibited through connections toother neuronsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureHow human brain learns?Human brain is a dense network of approximately 1011neurons, each connected to, on average, 104 othersNeuron activity is excited or inhibited through connections toother neuronsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureFrom Natural to Artificial NeuronsTo build artificial neuron:Deduce essential features of neurons and their connectionsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureFrom Natural to Artificial NeuronsTo build artificial neuron:Deduce essential features of neurons and their connectionsProgram a system to simulate the featuresSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureFrom Natural to Artificial NeuronsTo build artificial neuron:Deduce essential features of neurons and their connectionsProgram a system to simulate the featuresDue to imprecise knowledge, our models are necessarily grossidealisations of real networks of neuronesSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureA simple neuronArtificial neuron is a device with many inputs and one outputSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureA simple neuronArtificial neuron is a device with many inputs and one outputTwo modes:TrainingUsingFiring Rule determines when a neuron should fire.Are very important in neural networks and accounts for theirhigh flexibilitySayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureA simple neuronArtificial neuron is a device with many inputs and one outputTwo modes:TrainingUsingFiring Rule determines when a neuron should fire.Are very important in neural networks and accounts for theirhigh flexibilityCalcualtions of when neuron should fire are based on inputpatternsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureA simple neuronArtificial neuron is a device with many inputs and one outputTwo modes:TrainingUsingFiring Rule determines when a neuron should fire.Are very important in neural networks and accounts for theirhigh flexibilityCalcualtions of when neuron should fire are based on inputpatternsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureA simple neuronSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitecturePerceptronsA type of atrificial neuron developed in 1905sTakes several binary inputs and produces a single ouputTo compute the output each input is given a weight, thatexpresses it’s importanceThe output is determined:(P0 if j wj xj threshholdoutput P1 if j wj xj ing.com/Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitecturePreceptronsExample: Decide whether to go to a festival or not:How is the weather?(x1 )How far is the festival grounds?(x2 )Does your boyfriend/girlfriend want to accompany you?(x3 ayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitecturePreceptronsExample: Decide whether to go to a festival or not:How is the weather?(x1 )How far is the festival grounds?(x2 )Does your boyfriend/girlfriend want to accompany you?(x3 )A complex ning.com/Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ddeeplearning.com/Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureSigmoidThe problem is that this isn’t what happens when our networkcontains perceptronsIn fact, a small change in the weights or bias of any singleperceptron in the network can sometimes cause the output ofthat perceptron to completely flip, say from 00 to ayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureSigmoidThe aforementioned problem is solved by another type ofartificial neuron called Sigmoid neuronSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureSigmoidThe aforementioned problem is solved by another type ofartificial neuron called Sigmoid neuronSimilar to perceptrons, but modified so that small changes intheir weights and bias cause only a small change in theiroutputSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureSigmoidThe aforementioned problem is solved by another type ofartificial neuron called Sigmoid neuronSimilar to perceptrons, but modified so that small changes intheir weights and bias cause only a small change in theiroutputIn Sigmoid neurons inputs instead of just being 0 and 1, cantake any value between 0 and 1Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ddeeplearning.com/Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureArchitectureFeed-Forward NetworksBackpropagation dive-into-recurrent-neural-networks/Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureANN Application DevelopmentNelson and Illingworth in ”A Practical Guide To Neural Networks”outline following steps on designing a neural network:1Variable selection2Data collection3Training, testing and validation setNetwork Architecture4Number of hidden layers and neuronsNumber of ouput neuronsTransfer function5Neural Network Training6implementationSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsOutline1 Introduction23456ProblemSolutionConceptsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureApplicationsCase StudyBankruptcy PredictionBenefits/LimitationsQuestionsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase rity(e.g: Baggagechecking in airports)Stock market predictionLoan approvalCredit ratingMedical diagnosisProcess/Quality controlPattern recognitionSayed Jahed Hussini and Hisham M. SalehRecognizing genesEcosystem evaluationKndowledge discoveryTime serie analysisSales forecastingTargetted marketingHR managementArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionOutline1 Introduction23456ProblemSolutionConceptsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureApplicationsCase StudyBankruptcy PredictionBenefits/LimitationsQuestionsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionProblem StatementThere have been a lot of work on developing neural networksto predict bankruptcy using financial ratios and discriminantanalysisSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionProblem StatementThere have been a lot of work on developing neural networksto predict bankruptcy using financial ratios and discriminantanalysisThe ANN paradigm selected in the design phase for thisproblem was a three-layer feedforward ANN usingbackpropagationSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionProblem StatementThere have been a lot of work on developing neural networksto predict bankruptcy using financial ratios and discriminantanalysisThe ANN paradigm selected in the design phase for thisproblem was a three-layer feedforward ANN usingbackpropagationThe data for training the network consisted of a small set ofnumbers for well-known financial ratios, and data wereavailable on the bankruptcy outcomes corresponding to knowndata setsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionApplication DesignThere are five input nodes, corresponding to five financial ratios:X1: Working capital/total assetsX2: Retained earnings/total assetsX3: Earnings before interest and taxes/total assetsX4: Market value of equity/total debtX5: Sales/total assetsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionApplication DesignThere are five input nodes, corresponding to five financial ratios:X1: Working capital/total assetsX2: Retained earnings/total assetsX3: Earnings before interest and taxes/total assetsX4: Market value of equity/total debtX5: Sales/total assetsA single ouput, based on the given input, will indicate a possiblebrankruptcy(0) or nonbankruptcy(1) for a given financial firmSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionApplication DesignThere are five input nodes, corresponding to five financial ratios:X1: Working capital/total assetsX2: Retained earnings/total assetsX3: Earnings before interest and taxes/total assetsX4: Market value of equity/total debtX5: Sales/total assetsA single ouput, based on the given input, will indicate a possiblebrankruptcy(0) or nonbankruptcy(1) for a given financial firmThe system must have data and financial ratios of the firms thatdid and did not go bankrupt in the pastSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionANN ArchitectureSource:www.cse.hcmut.edu.vn/ dtanh/download/ANSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionTraining and TestingTraining:The data set, consisting of 129 firms, was partitioned into atraining set and a test set. The training set of 74 firmsconsisted of 38 that went bankrupt and 36 that did notSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBankruptcy PredictionTraining and TestingTraining:The data set, consisting of 129 firms, was partitioned into atraining set and a test set. The training set of 74 firmsconsisted of 38 that went bankrupt and 36 that did notTesting:The test set consisted of 27 bankrupt and 28 non-bankruptfirms. The neural network was able to correctly predict 81.5%of the bankrupt cases and 82.1% of the nonbankrupt casesOverrall, the ANN did much better predicting 22 out of the27 actual cases (the discriminant analysis predicted only 16cases correctly)Source: R.L. Wilson and R. Sharda, “Bankruptcy Prediction UsingNeural Networks,” Decision Support Systems, Vol. 11, No. 5, June1994, pp. 545-557.Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsOutline1 Introduction23456ProblemSolutionConceptsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureApplicationsCase StudyBankruptcy PredictionBenefits/LimitationsQuestionsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsBenefitsUseful in pattern recognition, classification, abstraction andinterpretation of incomplete and noisy inputs e.g:handwrittingProviding some human characteristics to problem solving thatare difficult using standard system/softwareAbility to solve new kinds of problems. ANNs are particularlyeffective at solving problems whose solutions are difficult, ifnot impossible, to defineANNs tend to be more robust, and have the ability to copewith imcomplete or fuzzy data.Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsLimitationsANNs do not produce an explicit model even though newcases can be fed into it and new results obtainedANNs lack explanation capabilities. Justifications for results isdifficults to obtain because the connection weights usually donot have obvious interpretaionsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsGoogle’s DeepDreamSource: ams-of-artificialneura-1712226908Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsGoogle’s DeepDreamSource: ams-of-artificialneura-1712226908Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsOutline1 Introduction23456ProblemSolutionConceptsArtificial Neural NetworksHuman and Artificial NeuronsArtificial NeuronsANN ArtchitectureApplicationsCase StudyBankruptcy PredictionBenefits/LimitationsQuestionsSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase ns?Sayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase StudyBenefits/LimitationsQuestionsReferencesUhrig, R.E., ”Introduction to artificial neural networks,” inIndustrial Electronics, Control, and Instrumentation, 1995.,Proceedings of the 1995 IEEE IECON 21st InternationalConference on , vol.1, no., pp.33-37 vol.1, 6-10 Nov 1995Introduction to artificial neural networks,” in ElectronicTechnology Directions to the Year 2000, 1995. Proceedings. ,vol., no., pp.36-62, 23-25 May 1995Yuhong Li; Weihua Ma, ”Applications of Artificial NeuralNetworks in Financial Economics: A Survey,” inComputational Intelligence and Design (ISCID), 2010International Symposium on , vol.1, no., pp.211-214, 29-31Oct. 2010Huang, S.H.; Hong-Chao Zhang, ”Artificial neural networks inmanufacturing: concepts, applications, and perspectives,” inSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
IntroductionConceptsApplicationsCase //www.doc.ic.ac.uk/ nd/surprise9 plearning du.vn/ dtanh/download/ANN.pptSayed Jahed Hussini and Hisham M. SalehArtificial Nerual Networks
Introduction to Learning AlgorithmsHill ClimbingSimulated AnnealingSummaryArtificial Neural NetworksAn Introductory LookSayed Jahed Hussini & Hisham SalehWestern Michigan UniversityDepartment of Computer ScienceAdvanced Theory of ComputationDr. Elise de DonckerFebruary 4, 2016Hussini & SalehArtificial Neural Networks
Introduction to Learning AlgorithmsHill ClimbingSimulated AnnealingSummaryOutline1Introduction to Learning AlgorithmsWhat Are They?2Hill ClimbingThe algorithmDisadvantages3Simulated AnnealingDefinitionThe algorithmAdvantagesAcceptance Function and Parameter SettingHussini & SalehArtificial Neural Networks
Introduction to Learning AlgorithmsHill ClimbingSimulated AnnealingSummaryWhat Are They?Outline1Introduction to Learning AlgorithmsWhat Are They?2Hill ClimbingThe algorithmDisadvantages3Simulated AnnealingDefinitionThe algorithmAdvantagesAcceptance Function and Parameter SettingHussini & SalehArtificial Neural Networks
Introduction to Learning AlgorithmsHill ClimbingSimulated AnnealingSummaryWhat Are They?What Are Learning Algorithmsand How do they workAs Jahed discussed in his presentation, neural networksbecome accurate as they are trained more.Training a neural network and building its neuronconnections requires a set of algorithms that fall within therealm of machine learning. These algorithms are generalartificial intelligence algorithms that can applied to helptrain the neuron network.By the end of this presentation, it is my hope that you willhave an additional two algorithms to add to your list ofthings that confuse you.Hussini & SalehArtificial Neural Networks
Introduction to Learning AlgorithmsHill ClimbingSimulated AnnealingSummaryWhat Are They?The AlgorithmsHill Climbing:The first algorithm we will cover is the hill climbingalgorithm, a technique that allows you to conduct a "local"search. This is one of the simplest technique available inthe artificialSimulated Annealing:The algorithm is used to allow us to approximate theoptimal solution to a problem with too many possiblesolutions to reasonably consider all of them in the search.Simulated Annealing is an algorithm thats based on thesimilar annealing process in metallurgy. We will cover how itworks and its advantages in a few slides.Hussini & SalehArtificial Neural Networks
Introduction to Learning AlgorithmsHill ClimbingSimulated AnnealingSummaryThe algorithmDisadvantagesOutline1Introduction to Learning AlgorithmsWhat Are They?2Hill ClimbingThe algorithmDisadvantages3Simulated AnnealingDefinitionThe algorithmAdvantagesAcceptance Function and Parameter SettingHussini & SalehArtificial Neural Networks
Introduction to Learning AlgorithmsHill ClimbingSimulated AnnealingSummaryThe algorithmDisadvantagesThe AlgorithmThe basic idea in Hill Climbing is that the solution or goalstate is at the top of the highest hill and you must reach it.Algorithm:First generate an initial solution.Loop till the crest is reached.Check neighboring pointIf it is better, choose it as solutionOtherwise, you’ve reached the crestHussini & SalehArtificial Neural Networks
Introduction to Learning AlgorithmsHill ClimbingSimulated AnnealingSummaryThe algorithmDisadvantagesExampleFigure: Image from http://www35.homepage.villanov
Arti cial Neural Networks Human and Arti cial Neurons Arti cial Neurons ANN Artchitecture A simple neuron Arti cial neuron is a device with many inputs and one output Two modes: Training Using Firing Ruledetermines when a neuron should re. Are very important in neural networks and accounts for their high exibility
Artificial Neural Networks Develop abstractionof function of actual neurons Simulate large, massively parallel artificial neural networks on conventional computers Some have tried to build the hardware too Try to approximate human learning, robustness to noise, robustness to damage, etc. Early Uses of neural networks
A growing success of Artificial Neural Networks in the research field of Autonomous Driving, such as the ALVINN (Autonomous Land Vehicle in a Neural . From CMU, the ALVINN [6] (autonomous land vehicle in a neural . fluidity of neural networks permits 3.2.a portion of the neural network to be transplanted through Transfer Learning [12], and .
neural networks using genetic algorithms" has explained that multilayered feedforward neural networks posses a number of properties which make them particularly suited to complex pattern classification problem. Along with they also explained the concept of genetics and neural networks. (D. Arjona, 1996) in "Hybrid artificial neural
Neural networks use machine learning based on the concept of self-adjustment of internal control parameters. An artificial neural network is a non-parametric attempt to model the human brain. Artificial neural networks are pliable mathematical structures that are capable of identifying complex non-linear relationships among input and output data
9 Artificial Neural Networks Rise and fall of Neural NetworksRise and fall of Neural Networks In the 70’s and 80's, it was shown that multilevel perceptrons don’t have These shortcomings Paul J. Werbos invented 1974 the back-propagation having the ability to perform classification tasks beyond simple Perceptrons
Neural Networks Shumeet Baluja baluja@cs.cmu.edu School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Abstract Dean Pomerleau pomerleau @cs.cmu.edu School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 We have developed an artificial neural network based gaze tracking system which can be customized to .
inspiration for artificial neural networks. Neurons andThresholds Biological brains, made of connected neurons, are the inspiration for artificial neural networks. NOOUTPUT. Neurons andThresholds Biological brains, made of connected neurons, are the inspiration for artificial neural networks.
Artificial Neural Networks Lecture Notes - Part 1 Stephen Lucci, PhD Artificial Neural Networks Lecture Notes Stephen Lucci, PhD . They conduct signals t the cell body. Axon Hillock Ex tends from cell body - initial por ion o the axon. .
