MATLAB Optimization Toolbox
OptimizationToolboxFor Use with MATLAB ComputationVisualizationProgrammingUser’s GuideVersion 2
How to Contact The Newsgroupinfo@mathworks.comTechnical supportProduct enhancement suggestionsBug reportsDocumentation error reportsOrder status, license renewals, passcodesSales, pricing, and general information508-647-7000Phone508-647-7001FaxThe MathWorks, Inc.3 Apple Hill DriveNatick, MA athworks.comFor contact information about worldwide offices, see the MathWorks Web site.Optimization Toolbox User’s Guide COPYRIGHT 1990 - 2001 by The MathWorks, Inc.The software described in this document is furnished under a license agreement. The software may be usedor copied only under the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form without prior written consent from The MathWorks, Inc.FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation byor for the federal government of the United States. By accepting delivery of the Program, the governmenthereby agrees that this software qualifies as "commercial" computer software within the meaning of FARPart 12.212, DFARS Part 227.7202-1, DFARS Part 227.7202-3, DFARS Part 252.227-7013, and DFARS Part252.227-7014. The terms and conditions of The MathWorks, Inc. Software License Agreement shall pertainto the government’s use and disclosure of the Program and Documentation, and shall supersede anyconflicting contractual terms or conditions. If this license fails to meet the government’s minimum needs oris inconsistent in any respect with federal procurement law, the government agrees to return the Programand Documentation, unused, to MathWorks.MATLAB, Simulink, Stateflow, Handle Graphics, and Real-Time Workshop are registered trademarks, andTarget Language Compiler is a trademark of The MathWorks, Inc.Other product or brand names are trademarks or registered trademarks of their respective holders.Printing History: November 1990December 1996January 1999September 2000June 2001First printingSecond printing for MATLAB 5Third printing for Version 2 (Release 11)Fourth printing for Version 2.1 (Release 12)Revised for Version 2.1.1 (Release 12.1) (online only)
ContentsPrefaceWhat Is the Optimization Toolbox? . . . . . . . . . . . . . . . . . . . . . viiiNew Features in Version 2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixToolbox Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixFunction Handles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixLarge Structured Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixLarge Problems that Are Not Well-Scaled . . . . . . . . . . . . . . . . . . . xUsing This Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiAcknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiRelated Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiConfiguration Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvTechnical Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviMatrix, Vector, and Scalar Notation . . . . . . . . . . . . . . . . . . . . . xviTypographical Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . xviiTutorial1Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4Problems Covered by the Toolbox . . . . . . . . . . . . . . . . . . . . . . . . 1-4Using the Optimization Functions . . . . . . . . . . . . . . . . . . . . . . . 1-6i
Examples that Use Standard Algorithms . . . . . . . . . . . . . . . . 1-8Unconstrained Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9Nonlinear Inequality Constrained Example . . . . . . . . . . . . . . 1-10Constrained Example with Bounds . . . . . . . . . . . . . . . . . . . . . 1-12Constrained Example with Gradients . . . . . . . . . . . . . . . . . . . 1-13Gradient Check: Analytic Versus Numeric . . . . . . . . . . . . . . . 1-15Equality Constrained Example . . . . . . . . . . . . . . . . . . . . . . . . . 1-16Maximization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-17Greater-Than-Zero Constraints . . . . . . . . . . . . . . . . . . . . . . . . 1-17Additional Arguments: Avoiding Global Variables . . . . . . . . . 1-18Multiobjective Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-18Large-Scale Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-30Problems Covered by Large-Scale Methods . . . . . . . . . . . . . . . 1-31Nonlinear Equations with Jacobian . . . . . . . . . . . . . . . . . . . . . 1-34Nonlinear Equations with Jacobian Sparsity Pattern . . . . . . . 1-36Nonlinear Least Squares with Full Jacobian Sparsity Pattern 1-38Nonlinear Minimization with Gradient and Hessian . . . . . . . 1-40Nonlinear Minimization with Gradient and Hessian SparsityPattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-41Nonlinear Minimization with Bound Constraints and BandedPreconditioner . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-43Nonlinear Minimization with Equality Constraints . . . . . . . . 1-47Nonlinear Minimization with a Dense but Structured Hessianand Equality Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-48Quadratic Minimization with Bound Constraints . . . . . . . . . . 1-52Quadratic Minimization with a Dense but Structured Hessian 1-54Linear Least Squares with Bound Constraints . . . . . . . . . . . . 1-58Linear Programming with Equalities and Inequalities . . . . . . 1-59Linear Programming with Dense Columns in the Equalities . 1-60Default Parameter Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-63Changing the Default Settings . . . . . . . . . . . . . . . . . . . . . . . . . 1-63Displaying Iterative Output . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-65Output Headings: Medium-Scale Algorithms . . . . . . . . . . . . . 1-65Output Headings: Large-Scale Algorithms . . . . . . . . . . . . . . . 1-67Optimization of Inline Objects Instead of M-Files . . . . . . . 1-70iiContents
Typical Problems and How to Deal with Them . . . . . . . . . . 1-72Converting Your Code to Version 2 Syntax . . . . . . . . . . . . .Using optimset and optimget . . . . . . . . . . . . . . . . . . . . . . . . . .New Calling Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Example of Converting from constr to fmincon . . . . . . . . . . . .1-761-771-771-85Selected Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-88Standard Algorithms2Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2Optimization Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4Unconstrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5Quasi-Newton Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-7Line Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9Quasi-Newton Implementation . . . . . . . . . . . . . . . . . . . . . . . . 2-12Hessian Update . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12Line Search Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12Least Squares Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-19Gauss-Newton Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-20Levenberg-Marquardt Method . . . . . . . . . . . . . . . . . . . . . . . . . 2-22Nonlinear Least Squares Implementation . . . . . . . . . . . . . . 2-24Gauss-Newton Implementation . . . . . . . . . . . . . . . . . . . . . . . . 2-24Levenberg-Marquardt Implementation . . . . . . . . . . . . . . . . . . 2-24Constrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-26Sequential Quadratic Programming (SQP) . . . . . . . . . . . . . . . 2-27Quadratic Programming (QP) Subproblem . . . . . . . . . . . . . . . 2-28iii
SQP Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Updating the Hessian Matrix . . . . . . . . . . . . . . . . . . . . . . . . . .Quadratic Programming Solution . . . . . . . . . . . . . . . . . . . . . . .Line Search and Merit Function . . . . . . . . . . . . . . . . . . . . . . . .2-302-302-312-35Multiobjective Optimization . . . . . . . . . . . . . . . . . . . . . . . . . .Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Goal Attainment Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Algorithm Improvements for Goal Attainment Method . . . . .2-362-362-422-43Selected Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-46Large-Scale Algorithms3Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2Trust Region Methods for Nonlinear Minimization . . . . . . . 3-3Preconditioned Conjugate Gradients . . . . . . . . . . . . . . . . . . . 3-6Linearly Constrained Problems . . . . . . . . . . . . . . . . . . . . . . . . 3-8Linear Equality Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8Box Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8Nonlinear Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11Quadratic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12Linear Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13Large-Scale Linear Programming . . . . . . . . . . . . . . . . . . . . . 3-14Main Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-14Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17Selected Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18ivContents
Function Reference4Functions by Category . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Minimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Equation Solving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Least Squares (Curve Fitting) . . . . . . . . . . . . . . . . . . . . . . . . . . .Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Demos of Large-Scale Methods . . . . . . . . . . . . . . . . . . . . . . . . . .Demos of Medium-Scale Methods . . . . . . . . . . . . . . . . . . . . . . . .4-34-34-34-44-44-44-5Function Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6Optimization Options Parameters . . . . . . . . . . . . . . . . . . . . . 4-11fgoalattain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-16fminbnd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-28fmincon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-32fminimax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-45fminsearch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-54fminunc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-58fseminf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-68fsolve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-81fzero . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-92linprog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-96lsqcurvefit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-104lsqlin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-115lsqnonlin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-123lsqnonneg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-134optimget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-138optimset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-139quadprog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-143Indexv
viContents
PrefaceWhat Is the Optimization Toolbox? . . . . . . . . . . viiiNew Features in Version 2.1 . . . . .Toolbox Speed . . . . . . . . . . . .Function Handles . . . . . . . . . .Large Structured Problems . . . . . .Large Problems that Are Not Well-Scaled .ixixixixxUsing This Guide . . . . . . . . . . . . . . . . . . . xiAcknowledgments. . . . . . . . . . . . . . . . . .xiiRelated Products . . . . . . . . . . . . . . . . . . xiiiConfiguration Information. . . . . . . . . . . . . . xvTechnical Conventions . . . . . . . . . . . . . . . xviMatrix, Vector, and Scalar Notation . . . . . . . . . . . xviTypographical Conventions . . . . . . . . . . . . . xvii
PrefaceWhat Is the Optimization Toolbox?The Optimization Toolbox is a collection of functions that extend the capabilityof the MATLAB numeric computing environment. The toolbox includesroutines for many types of optimization including: Unconstrained nonlinear minimization Constrained nonlinear minimization, including goal attainment problems,minimax problems, and semi-infinite minimization problems Quadratic and linear programming Nonlinear least squares and curve-fitting Nonlinear system of equation solving Constrained linear least squares Sparse and structured large-scale problemsAll of the toolbox functions are MATLAB M-files, made up of MATLABstatements that implement specialized optimization algorithms. You can viewthe MATLAB code for these functions using the statementtype function nameYou can extend the capabilities of the Optimization Toolbox by writing yourown M-files, or by using the toolbox in combination with other toolboxes, orwith MATLAB, or Simulink .This Preface also provides information about: New features in Version 2.1 and 2.1.1 Using this guide, which includes a summary of the chapters Acknowledgements Related MathWorks products Checking if the Optimization Toolbox is installed on your system Technical and typographical conventions used in this manualviii
New Features in Version 2.1New Features in Version 2.1Version 2.1 of the Optimization Toolbox offers a number of advances overprevious versions: Overall toolbox speed is improved. Functions that take a function as an argument now support the use offunction handles. Large structured problems are supported. Support is improved in large-scale methods for problems that are notwell-scaled.Toolbox SpeedBy improving the speed of optimset and optimget, the overall speed of thetoolbox is improved. Applications that call these functions repeatedly shouldexhibit improved execution time.Function HandlesYou can now use the new @ syntax to specify a function handle for anyargument that expects a function. As in earlier versions of the toolbox, you canalso specify the function as a string or as an inline object.For more information about function handles, see: The function handle (@), func2str, and str2func reference pages The “Function Handles” section of “Programming and Data Types” in theMATLAB documentationLarge Structured ProblemsThe functions fmincon, fminunc, fsolve, lsqcurvefit, lsqlin, lsqnonlin,and quadprog now support solving large structured problems, i.e., problemsthat have large dense Hessian or Jacobian matrices that you do not want toform explicitly, but for which Hessian-matrix (or Jacobian-matrix) productsare efficient to compute.Two new options parameters, HessMult and JacobMult, provide access to thisnew feature. For more information, see “Optimization Options Parameters” onpage 4-11 and the individual reference pages.ix
PrefaceLarge Problems that Are Not Well-ScaledIn Version 2.1.1, large-scale finite differencing is improved numerically tohandle cases when an optimization problem is not well-scaled. These changespotentially improve the speed and accuracy of results when using thelarge-scale versions of lsqnonlin, lsqcurvefit, fsolve, fmincon and fminunc,particularly if the objective function is not well-scaled.x
Using This GuideUsing This GuideThis guide has the following chapters: The “Tutorial” chapter shows you how to solve a variety of differentoptimization problems. It includes a section that highlights large-scaleproblems. This chapter also provides information on how to use the toolboxfunctions in conjunction with Simulink using multiobjective optimization.Other sections include information about changing default parameters andusing inline objects. The “Standard Algorithms” and “Large-Scale Algorithms” chapters describethe algorithms used by the optimization functions. “Standard Algorithms”describes the problem formulations and algorithms for the medium-scalealgorithms. “Large-Scale Algorithms” focuses on algorithms used to solvelarge sparse or structured problems. The “Function Reference” chapter provides a detailed reference descriptionof each toolbox function. Reference descriptions include the function’ssyntax, a description of the different calling sequences available, anddetailed information about arguments to the function, including relevantoptimization options parameters. Reference descriptions may also includeexamples, a summary of the function’s algorithms, and references toadditional reading material.xi
PrefaceAcknowledgmentsThe MathWorks would like to acknowledge these contributors:Thomas F. Coleman researched and contributed the large scale algorithms forconstrained and unconstrained minimization, nonlinear least squares andcurve fitting, constrained linear least squares, quadratic programming, andnonlinear equations.Dr. Coleman is Professor of Computer Science and Applied Mathematics atCornell University. He is Director of the Cornell Theory Center and the CornellComputational Finance Institute. Dr. Coleman is Chair of the SIAM ActivityGroup on Optimization, and a member of the Editorial Boards of AppliedMathematics Letters, SIAM Journal of Scientific Computing, ComputationalOptimization and Applications, Communications on Applied NonlinearAnalysis, and Mathematical Modeling and Scientific Computing.Dr. Coleman has published 4 books and over 70 technical papers in the areasof continuous optimization and computational methods and tools forlarge-scale problems.Yin Zhang researched and contributed the large scale linear programmingalgorithm.Dr. Zhang is Associate Professor of Computational and Applied Mathematicsand is on the faculty of the Keck Center for Computational Biology at RiceUniversity. He is on the Editorial Board of SIAM Journal on Optimization, andAssociate Editor of Journal of Optimization: Theory and Applications.Dr. Zhang has published over 40 technical papers in the areas of interior-pointmethods for linear programming and computation mathematicalprogramming.xii
Related ProductsRelated ProductsThe MathWorks provides several products that are relevant to the kinds oftasks you can perform with the Optimization Toolbox. For more informationabout any of these products, see either: The online documentation for that product, if it is installed or if you arereading the documentation from the CD The MathWorks Web site, at http://www.mathworks.com; see the “products”sectionNote The toolboxes listed below all include functions that extend MATLAB’scapabilities. The blocksets all include blocks that extend Simulink’scapabilities.ProductDescriptionData Acquisition ToolboxMATLAB functions for direct access to live,measured data from MATLABDatabase ToolboxTool for connecting to, and interacting with,most ODBC/JDBC databases from withinMATLABFinancial Time SeriesToolboxTool for analyzing time series data in thefinancial marketsFinancial ToolboxMATLAB functions for quantitative financialmodeling and analytic prototypingFrequency DomainSystem IdentificationToolboxTool for frequency domain model identificationand validationGARCH ToolboxMATLAB functions for univariate GeneralizedAutoregressive Conditional Heteroskedasticity(GARCH) volatility modelingxiii
PrefacexivProductDescriptionLMI Control ToolboxConvex optimization algorithms for solvinglinear matrix inequalities (LMI), withapplication to robust control, multi-objectivecontrol, and gain schedulingNeural Network ToolboxComprehensive environment for neuralnetwork research, design, and simulationwithin MATLABNonlinear ControlDesign BlocksetSimulink block libraries that provide atime-domain-based optimization approach tosystem design; automatically tunesparameters based on user-definedtime-domain performance constraintsSignal ProcessingToolboxTool for algorithm development, signal andlinear system analysis, and time-series datamodelingSimulinkInteractive, graphical environment formodeling, simulating, and prototypingdynamic systemsSpline ToolboxTool for the construction and use of piecewisepolynomial functionsStatistics ToolboxTool for analyzing historical data, modelingsystems, developing statistical algorithms, andlearning and teaching statisticsSymbolic Math ToolboxTool for seamless integration of symbolicmathematics and variable precision arithmeticbased on Maple VSystem IdentificationToolboxTool for building accurate, simplified models ofcomplex systems from noisy time-series data
Configuration InformationConfiguration InformationTo determine if the Optimization Toolbox is installed on your system, type thiscommand at the MATLAB prompt.verWhen you enter this command, MATLAB displays information about theversion of MATLAB you are running, including a list of all toolboxes installedon your system and their version numbers.If the Optimization Toolbox is not installed, check the Installation Guide foryour platform for instructions on how to install it.Note For the most up-to-date information about system requirements, seethe system requirements page, available in the products area at theMathWorks Web site (http://www.mathworks.com).xv
PrefaceTechnical ConventionsMatrix, Vector, and Scalar NotationUpper-case letters such as A are used to denote matrices. Lower-case letterssuch as x are used to denote vectors, except where noted that it is a scalar.For functions, the notation differs slightly to follow the usual conventions inoptimization. For vector functions, we use an upper-case letter such as F inF ( x ) . A function that returns a scalar value is denoted with a lower-case lettersuch as f in f ( x ) .xvi
Typographical ConventionsTypographical ConventionsThis manual uses some or all of these conventions.ItemConvention UsedExampleExample codeMonospace fontTo assign the value 5 to A,enterA 5Function names/syntaxMonospace fontThe cos function finds thecosine of each array element.Syntax line example isMLGetVar ML var nameKeysBoldface with an initial capitalletterPress the Return key.Literal strings (in syntaxdescriptions in referencechapters)Monospace bold for literalsf freqspace(n,'whole')MathematicalexpressionsItalics for variablesThis vector represents thepolynomialMATLAB outputStandard text font for functions,operators, and constantsMonospace fontp x2 2x 3MATLAB responds withA 5Menu titles, menu items,dialog boxes, and controlsBoldface with an initial capitalletterChoose the File menu.New termsItalicsAn array is an orderedcollection of information.Omitted input arguments(.) ellipsis denotes all of theinput/output arguments frompreceding syntaxes.[c,ia,ib] union(.)String variables (from afinite list)Monospace italicssysc d2c(sysd,'method')xvii
Prefacexviii
1TutorialOverview. . . . . . . . . . . . . . . . . . . . . 1-2Introduction . . . . . . . . . . . . . . . . . . . . 1-4Examples that Use Standard Algorithms . . . . . . . 1-8Large-Scale Examples. . . . . . . . . . . . . . . 1-30Default Parameter Settings . . . . . . . . . . . . . 1-63Displaying Iterative Output . . . . . . . . . . . . . 1-65Optimization of Inline Objects Instead of M-Files . . . 1-70Typical Problems and How to Deal with Them. . . . 1-72Converting Your Code to Version 2 Syntax . . . . . . 1-76Selected Bibliography. . . . . . . . . . . . . . . 1-88
1TutorialOverviewThis “Tutorial” provides information on how to use the toolbox functions. It alsoprovides examples for solving different optimization problems. It is organizedinto these major topics.IntroductionSummarizes, in tabular form, the functions available for minimization,equation solving, and solving least squares or data fitting problems. It alsoprovides basic guidelines for using the optimization routines and introducesthe algorithms and line-search strategies that are available for solvingmedium- and large-scale problems.Examples that Use Standard AlgorithmsPresents medium-scale algorithms through a selection of minimizationexamples. These examples include unconstrained and constrained problems,as well as problems with and without user-supplied gradients. This section alsodiscusses maximization, greater-than-zero constraints, passing additionalarguments, and multiobjective examples.Large-Scale ExamplesPresents large-scale algorithms through a selection of large-scale examples.These examples include specifying sparsity structures, and preconditioners, aswell as unconstrained and constrained problems.Default Parameter SettingsDescribes the use of default parameter settings and tells you how to changethem. It also tells you how to determine which parameters are used by aspecified function, and provides examples of setting some commonly usedparameters.Displaying Iterative OutputDescribes the column headings used in the iterative output of bothmedium-scale and large-scale algorithms.Optimization of Inline Objects Instead of M-FilesTells you how to represent a mathematical function at the command line bycreating an inline object from a string expression.1-2
OverviewTypical Problems and How to Deal with ThemProvides tips to help you improve solutions found using the optimizationfunctions, improve efficiency of the algorithms, overcome common difficulties,and transform problems that are not typically in standard form.Converting Your Code to Version 2 SyntaxFor each function, compares a Version 1.5 call to the equivalent Version 2 call.This section also describes the Version 2 calling sequences and provides adetailed example of converting from constr to its replacement fmincon.Selected BibliographyLists published materials that support concepts implemented in the StatisticsToolbox.1-3
1TutorialIntroductionOptimization concerns the minimization or maximization of functions. TheOptimization Toolbox consists of functions that perform minimization (ormaximization) on general nonlinear functions. Functions for nonlinearequation solving and least squares (data-fitting) problems are also provided.This introduction includes the following sections: Problems Covered by the Toolbox Using the Optimization FunctionsProblems Covered by the ToolboxThe tables below show the functions available for minimization, equationsolving, and solving least squares or data fitting problems.Note The following tables list the types of problems in order of increasingcomplexity.Table 1-1: MinimizationTypeNotationFunctionScalar Minimizationmin f ( a ) such that a 1 a a 2afminbndUnconstrained Minimizationmin f ( x )xfminunc,fminsearchLinear Programmingmin f x such thatxTlinprogA x b, Aeq x beq, l x uQuadratic ProgrammingT1 Tmin --- x Hx f x such thatx 2A x b, Aeq x beq, l x u1-4quadprog
IntroductionTable 1-1: Minimization (Continued)TypeNotationFunctionConstrained Minimizationmin f ( x ) such thatxfminconc ( x ) 0, ceq ( x ) 0A x b, Aeq x beq, l x uGoal Attainmentmin γ such thatx, γfgoalattainF ( x ) – wγ goalc ( x ) 0, ceq ( x ) 0A x b, Aeq x beq, l x uMinimaxmin max { F i(x) }x {F }isuch thatfminimaxc ( x ) 0, ceq ( x ) 0A x b, Aeq x beq, l x uSemi-Infinite Minimizationmin f ( x ) such thatxfseminfK(x, w) 0 for all wc ( x ) 0, ceq ( x ) 0A x b, Aeq x beq, l x uTable 1-2: Equation SolvingTypeNotationFunctionLinear EquationsC x d , n equations, n variables\ (slash)Nonlinear Equation of OneVariablef(a) 0fzero1-5
1TutorialTable 1-2: Equation Solving (Continued)TypeNotationFunctionNonlinear EquationsF(x) 0 , n equations, n variablesfsolveTable 1-3: Least Squares (Curve Fitting)TypeNotationFunctionLinear Least Squaresmin C x – d 2 , m equations, n variablesxNonnegative Linear LeastSquaresmin C x – d 2xConstrained Linear LeastSquaressuch thatmin C x – d 2xA x b, Aeq x beq, l x ulsqlinNonlinear Least Squares2211min --- F(x) 2 --Fi ( x)2x 2isuch that l x ulsqnonlinNonlinear Curve Fitting21min --- F(x, xdata) – ydata 2x 2such that l x u22such that x 02 \ (slash)lsqnonneglsqcurvefitUsing the Optimization FunctionsMost of these optimization routines require the definition of an M-filecontaining the function to be minimized, i.e., the objective function.Alternatively, an inline object created from a MATLAB expression can be used.Maximization is achieved by supplying the routines with -f, where f is thefunction being optimized.Optimization options passed to the
comp.soft-sys.matlab Newsgroup support@mathworks.com Technical support . info@mathworks.com Sales, pricing, and general information 508-647-7000 Phone 508-647-7001 Fax The MathWorks, Inc. Mail 3 Apple Hill Drive Natick, MA 01760-2098 For contact information about worldwide offices, see the MathWorks Web sit
MatConvNet library: MATLAB toolbox implementing Convolutional Neural Networks (CNNs) for computer vision applications MATLAB MATLAB Parallel Computing Toolbox , Computer Vision System Toolbox and Statistics and Machine Learning Toolbox A little bit of MATLAB code
The toolbox is designed to work with both resting state scans and block designs. Installing the toolbox: 1. download conn.zip, unzip the file. 2. add ./conn/ directory to matlab path To start the toolbox: On the matlab prompt, type : conn (make sure your matlab path include the path to the connectivity toolbox)
Compiler MATLAB Production Server Standalone Application MATLAB Compiler SDK Apps Files Custom Toolbox Python With MATLAB Users With People Who Do Not Have MATLAB.lib/.dll .exe . Pricing Risk Analytics Portfolio Optimization MATLAB Production Server MATLAB CompilerSDK Web Application
Model-Based Calibration Toolbox 13, 21, 23, 24, 27 500 600 Control System Design and Analysis Control System Toolbox 200 200 System Identification Toolbox 200 200 Fuzzy Logic Toolbox 200 200 Robust Control Toolbox 4 200 200 Model Predictive Control Toolbox 4 200 23200 Aerospace Toolbox 200 200 Signal Processing and Communications
MATLAB tutorial . School of Engineering . Brown University . To prepare for HW1, do sections 1-11.6 – you can do the rest later as needed . 1. What is MATLAB 2. Starting MATLAB 3. Basic MATLAB windows 4. Using the MATLAB command window 5. MATLAB help 6. MATLAB ‘Live Scripts’ (for algebra, plotting, calculus, and solving differential .
19 MATLAB Excel Add-in Hadoop MATLAB Compiler Standalone Application MATLAB deployment targets MATLAB Compiler enables sharing MATLAB programs without integration programming MATLAB Compiler SDK provides implementation and platform flexibility for software developers MATLAB Production Server provides the most efficient development path for secure and scalable web and enterprise applications
MATLAB tutorial . School of Engineering . Brown University . To prepare for HW1, do sections 1-11.6 – you can do the rest later as needed . 1. What is MATLAB 2. Starting MATLAB 3. Basic MATLAB windows 4. Using the MATLAB command window 5. MATLAB help 6. MATLAB ‘Live Scripts’ (for
3. MATLAB script files 4. MATLAB arrays 5. MATLAB two‐dimensional and three‐dimensional plots 6. MATLAB used‐defined functions I 7. MATLAB relational operators, conditional statements, and selection structures I 8. MATLAB relational operators, conditional statements, and selection structures II 9. MATLAB loops 10. Summary
