NUMERICALSOLUTIONOF ORDINARYDIFFERENTIAL EQUATIONS
NUMERICAL SOLUTION OFORDINARY DIFFERENTIALEQUATIONSKendall Atkinson, Weimin Han, David StewartUniversity of IowaIowa City, IowaA JOHN WILEY & SONS, INC., PUBLICATION
Copyright c 2009 by John Wiley & Sons, Inc. All rights reserved.Published by John Wiley & Sons, Inc., Hoboken, New Jersey.Published simultaneously in Canada.No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any formor by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except aspermitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the priorwritten permission of the Publisher, or authorization through payment of the appropriate per-copy fee tothe Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400,fax (978) 646-8600, or on the web at www.copyright.com. Requests to the Publisher for permission shouldbe addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ07030, (201) 748-6011, fax (201) 748-6008.Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts inpreparing this book, they make no representations or warranties with respect to the accuracy orcompleteness of the contents of this book and specifically disclaim any implied warranties ofmerchantability or fitness for a particular purpose. No warranty may be created ore extended by salesrepresentatives or written sales materials. The advice and strategies contained herin may not besuitable for your situation. You should consult with a professional where appropriate. Neither thepublisher nor author shall be liable for any loss of profit or any other commercial damages, includingbut not limited to special, incidental, consequential, or other damages.For general information on our other products and services please contact our Customer CareDepartment with the U.S. at 877-762-2974, outside the U.S. at 317-572-3993 or fax 317-572-4002.Wiley also publishes its books in a variety of electronic formats. Some content that appears in print,however, may not be available in electronic format.Library of Congress Cataloging-in-Publication Data:Numerical Solution of Ordinary Differential Equations / Kendall E. Atkinson . . . [et al.].p. cm.—(Wiley series in ?)“Wiley-Interscience."Includes bibliographical references and index.ISBN ? (pbk.)1. Numerical analysis. 2. Ordinary differential equations.I. Atkinson, Kendall E. II. Series.MATLAB R is a trademark of The MathWorks, Inc. and is used with permission.The MathWorks does not warrant the accuracy of the text or exercises in this book.This book’s use or discussion of MATLAB R software or related products does notconstitute endorsement or sponsorship by The MathWorks of a particular pedagogicalapproach or particular use of the MATLAB R software.QA31.?.? 2008510.?-?Printed in the United States of America.10 9 8 7 6 5 4 3 2 1
To Alice, Huidi, and Sue
PrefaceThis book is an expanded version of supplementary notes that we used for a course onordinary differential equations for upper-division undergraduate students and beginning graduate students in mathematics, engineering, and sciences. The book introduces the numerical analysis of differential equations, describing the mathematicalbackground for understanding numerical methods and giving information on whatto expect when using them. As a reason for studying numerical methods as a partof a more general course on differential equations, many of the basic ideas of thenumerical analysis of differential equations are tied closely to theoretical behaviorassociated with the problem being solved. For example, the criteria for the stabilityof a numerical method is closely connected to the stability of the differential equationproblem being solved.This book can be used for a one-semester course on the numerical solution of differential equations, or it can be used as a supplementary text for a course on the theoryand application of differential equations. In the latter case, we present more aboutnumerical methods than would ordinarily be covered in a class on ordinary differentialequations. This allows the instructor some latitude in choosing what to include, andit allows the students to read further into topics that may interest them. For example,the book discusses methods for solving differential algebraic equations (Chapter 10)and Volterra integral equations (Chapter 12), topics not commonly included in anintroductory text on the numerical solution of differential equations.vii
viiiPREFACEWe also include MATLAB R programs to illustrate many of the ideas that areintroduced in the text. Much is to be learned by experimenting with the numericalsolution of differential equations. The programs in the book can be downloaded fromthe following sODE/This site also contains graphical user interfaces for use in experimenting with Euler’smethod and the backward Euler method. These are to be used from within theframework of MATLAB.Numerical methods vary in their behavior, and the many different types of differential equation problems affect the performance of numerical methods in a variety ofways. An excellent book for “real world” examples of solving differential equationsis that of Shampine, Gladwell, and Thompson [74].The authors would like to thank Olaf Hansen, California State University at SanMarcos, for his comments on reading an early version of the book. We also expressour appreciation to John Wiley Publishers.
CONTENTS12Introduction1Theory of differential equations: An introduction31.1 General solvability theory1.2 Stability of the initial value problem1.3 Direction fieldsProblems781113Euler’s method152.12.22.316212628293032Definition of Euler’s methodError analysis of Euler’s methodAsymptotic error analysis2.3.1 Richardson extrapolation2.4 Numerical stability2.4.1 Rounding error accumulationProblemsix
x3456CONTENTSSystems of differential equations373.1 Higher-order differential equations3.2 Numerical methods for systemsProblems394246The backward Euler method and the trapezoidal method494.1 The backward Euler method4.2 The trapezoidal methodProblems515662Taylor and Runge–Kutta methods675.15.2Taylor methodsRunge–Kutta methods5.2.1 A general framework for explicit Runge–Kutta methods5.3 Convergence, stability, and asymptotic error5.3.1 Error prediction and control5.4 Runge–Kutta–Fehlberg methods5.5 MATLAB codes5.6 Implicit Runge–Kutta methods5.6.1 Two-point collocation methodsProblems68707375788082868789Multistep methods956.16.26.396101104105106Adams–Bashforth methodsAdams–Moulton methodsComputer codes6.3.1 MATLAB ODE codesProblems7General error analysis for multistep methods7.17.27.3Truncation errorConvergenceA general error analysis7.3.1 Stability theory7.3.2 Convergence theory7.3.3 Relative stability and weak stabilityProblems111112115117118122122123
CONTENTS8Stiff differential equations8.1The method of lines for a parabolic equation8.1.1 MATLAB programs for the method of lines8.2 Backward differentiation formulas8.3 Stability regions for multistep methods8.4 Additional sources of difficulty8.4.1 A-stability and L-stability8.4.2 Time-varying problems and stability8.5 Solving the finite-difference method8.6 Computer codesProblems9Implicit RK methods for stiff differential equations9.1 Families of implicit Runge–Kutta methods9.2 Stability of Runge–Kutta methods9.3 Order reduction9.4 Runge–Kutta methods for stiff equations in practiceProblems10 Differential algebraic equations10.110.210.310.4Initial conditions and driftDAEs as stiff differential equationsNumerical issues: higher index problemsBackward differentiation methods for DAEs10.4.1 Index 1 problems10.4.2 Index 2 problems10.5 Runge–Kutta methods for DAEs10.5.1 Index 1 problems10.5.2 Index 2 problems10.6 Index three problems from mechanics10.6.1 Runge–Kutta methods for mechanical index 3 systems10.7 Higher index DAEsProblems11 Two-point boundary value problems11.1 A finite-difference method11.1.1 184185187188190
xiiCONTENTS11.1.2 A numerical example11.1.3 Boundary conditions involving the derivative11.2 Nonlinear two-point boundary value problems11.2.1 Finite difference methods11.2.2 Shooting methods11.2.3 Collocation methods11.2.4 Other methods and problemsProblems12 Volterra integral equations12.1 Solvability theory12.1.1 Special equations12.2 Numerical methods12.2.1 The trapezoidal method12.2.2 Error for the trapezoidal method12.2.3 General schema for numerical methods12.3 Numerical methods: Theory12.3.1 Numerical stability12.3.2 Practical numerical 215216217219223225227231Appendix A. Taylor’s Theorem235Appendix B. Polynomial interpolation241References245Index250
IntroductionDifferential equations are among the most important mathematical tools used in producing models in the physical sciences, biological sciences, and engineering. In thistext, we consider numerical methods for solving ordinary differential equations, thatis, those differential equations that have only one independent variable.The differential equations we consider in most of the book are of the formY ′ (t) f (t, Y (t)),where Y (t) is an unknown function that is being sought. The given function f (t, y)of two variables defines the differential equation, and examples are given in Chapter1. This equation is called a first-order differential equation because it contains afirst-order derivative of the unknown function, but no higher-order derivative. Thenumerical methods for a first-order equation can be extended in a straightforward wayto a system of first-order equations. Moreover, a higher-order differential equationcan be reformulated as a system of first-order equations.A brief discussion of the solvability theory of the initial value problem for ordinary differential equations is given in Chapter 1, where the concept of stability ofdifferential equations is also introduced. The simplest numerical method, Euler’smethod, is studied in Chapter 2. It is not an efficient numerical method, but it is anintuitive way to introduce many important ideas. Higher-order equations and systemsof first-order equations are considered in Chapter 3, and Euler’s method is extended1
2INTRODUCTIONto such equations. In Chapter 4, we discuss some numerical methods with betternumerical stability for practical computation. Chapters 5 and 6 cover more sophisticated and rapidly convergent methods, namely Runge–Kutta methods and the familiesof Adams–Bashforth and Adams–Moulton methods, respectively. In Chapter 7, wegive a general treatment of the theory of multistep numerical methods. The numericalanalysis of stiff differential equations is introduced in several early chapters, and itis explored at greater length in Chapters 8 and 9. In Chapter 10, we introduce thestudy and numerical solution of differential algebraic equations, applying some of theearlier material on stiff differential equations. In Chapter 11, we consider numericalmethods for solving boundary value problems of second-order ordinary differentialequations. The final chapter, Chapter 12, gives an introduction to the numerical solution of Volterra integral equations of the second kind, extending ideas introduced inearlier chapters for solving initial value problems. Appendices A and B contain briefintroductions to Taylor polynomial approximations and polynomial interpolation.
CHAPTER 1THEORY OF DIFFERENTIALEQUATIONS: AN INTRODUCTIONFor simple differential equations, it is possible to find closed form solutions. Forexample, given a function g, the general solution of the simplest equationY ′ (t) g(t)isY (t) Zg(s) ds cRwith c an arbitrary integration constant. Here, g(s) ds denotes any fixed antiderivative of g. The constant c, and thus a particular solution, can be obtained by specifyingthe value of Y (t) at some given point:Y (t0 ) Y0 .Example 1.1 The general solution of the equationY ′ (t) sin(t)isY (t) cos(t) c.3
4THEORY OF DIFFERENTIAL EQUATIONS: AN INTRODUCTIONIf we specify the conditionY π 2,3then it is easy to find c 2.5. Thus the desired solution isY (t) 2.5 cos(t).The more general equationY ′ (t) f (t, Y (t))(1.1)is approached in a similar spirit, in the sense that usually there is a general solutiondependent on a constant. To further illustrate this point, we consider some moreexamples that can be solved analytically. First, and foremost, is the first-order linearequationY ′ (t) a(t)Y (t) g(t).(1.2)The given functions a(t) and g(t) are assumed continuous. For this equation, weobtainf (t, z) a(t)z g(t),and the general solution of the equation can be found by the so-called method ofintegrating factors.We illustrate the method of integrating factors through a particularly useful case,Y ′ (t) λY (t) g(t)(1.3)with λ a given constant. Multiplying the linear equation (1.3) by the integrating factore λt , we can reformulate the equation as d λte Y (t) e λt g(t).dtIntegrating both sides from t0 to t, we obtainZ te λs g(s) ds,e λt Y (t) c t0wherec e λ t0 Y (t0 ).(1.4)So the general solution of (1.3) is Z tZ tλt λsY (t) eeλ(t s) g(s) ds.eg(s) ds ceλt c t0(1.5)t0This solution is valid on any interval on which g(t) is continuous.As we have seen from the discussions above, the general solution of the first-orderequation (1.1) normally depends on an arbitrary integration constant. To single out
5a particular solution, we need to specify an additional condition. Usually such acondition is taken to be of the formY (t0 ) Y0 .(1.6)In many applications of the ordinary differential equation (1.1), the independent variable t plays the role of time, and t0 can be interpreted as the initial time. So it iscustomary to call (1.6) an initial value condition. The differential equation (1.1) andthe initial value condition (1.6) together form an initial value problemY ′ (t) f (t, Y (t)),Y (t0 ) Y0 .(1.7)For the initial value problem of the linear equation (1.3), the solution is given bythe formulas (1.5) and (1.4). We observe that the solution exists on any open intervalwhere the data function g(t) is continuous. This is a property for linear equations.For the initial value problem of the general linear equation (1.2), its solution existson any open interval where the functions a(t) and g(t) are continuous. As we willsee next through examples, when the ordinary differential equation (1.1) is nonlinear,even if the right-side function f (t, z) has derivatives of any order, the solution of thecorresponding initial value problem may exist on only a smaller interval.Example 1.2 By a direct computation, it is easy to verify that the equationY ′ (t) [Y (t)]2 Y (t)has a so-called trivial solution Y (t) 0 and a general solutionY (t) 11 c e t(1.8)with c arbitrary. Alternatively, this equation is a so-called separable equation, and itssolution can be found by a standard method such as that described in Problem 4. Tofind the solution of the equation satisfying Y (0) 4, we use the solution formula att 0:4 1,1 cc 0.75.So the solution of the initial value problem isY (t) 1,1 0.75e tt 0.With a general initial value Y (0) Y0 6 0, the constant c in the solution formula(1.8) is given by c Y0 1 1. If Y0 0, then c 1, and the solution Y (t) existsfor 0 t . However, for Y0 0, the solution exists only on the finite interval
6THEORY OF DIFFERENTIAL EQUATIONS: AN INTRODUCTION[0, log(1 Y0 1 )); the value t log(1 Y0 1 ) is the zero of the denominator in theformula (1.8). Throughout this work, log denotes the natural logarithm.Example 1.3 Consider the equationY ′ (t) [Y (t)]2 .It has a trivial solution Y (t) 0 and a general solutionY (t) 1t c(1.9)with c arbitrary. This can be verified by a direct calculation or by the method describedin Problem 4. To find the solution of the equation satisfying the initial value conditionY (0) Y0 , we distinguish several cases according to the value of Y0 . If Y0 0,then the solution of the initial value problem is Y (t) 0 for any t 0. If Y0 6 0,then the solution of the initial value problem is1.t Y0 1Y (t) For Y0 0, the solution exists for any t 0. For Y0 0, the solution exists only onthe interval [0, Y0 1 ). As a side note, observe that for 0 Y0 1 with c Y0 1 1,the solution (1.8) increases for t 0, whereas for Y0 0, the solution (1.9) withc Y0 1 decreases for t 0.Example 1.4 The solution ofY ′ (t) λY (t) e t ,Y (0) 1is obtained from (1.5) and (1.4) asY (t) eλt Zteλ(t s) e s ds.0If λ 6 1, thenIf λ 1, then Y (t) eλt 1 1[1 e (λ 1)t ] .λ 1Y (t) e t (1 t) .We remark that for a general right-side function f (t, z), it is usually not possibleto solve the initial value problem (1.7) analytically. One such example is for theequation4Y ′ e t Y .In such a case, numerical methods are the only plausible way to compute solutions.Moreover, even when a differential equation can be solved analytically, the solution
GENERAL SOLVABILITY THEORY7formula, such as (1.5), usually involves integrations of general functions. The integrals mostly have to be evaluated numerically. As an example, it is easy to verify thatthe solution of the problem ′Y 2 t Y 1, t 0,Y (0) 1isY (t) et2Zt22e s ds et .0For such a situation, it is usually more efficient to use numerical methods from theoutset to solve the differential equation.1.1 GENERAL SOLVABILITY THEORYBefore we consider numerical methods, it is useful to have some discussions on properties of the initial value problem (1.7). The following well-known result concernsthe existence and uniqueness of a solution to this problem.Theorem 1.5 Let D be an open connected set in R2 , let f (t, y) be a continuousfunction of t and y for all (t, y) in D, and let (t0 , Y0 ) be an interior point of D.Assume that f (t, y) satisfies the Lipschitz condition f (t, y1 ) f (t, y2 ) K y1 y2 all (t, y1 ), (t, y2 ) in D(1.10)for some K 0. Then there is a unique function Y (t) defined on an interval[t0 α, t0 α] for some α 0, satisfyingY ′ (t) f (t, Y (t)),t0 α t t0 α,Y (t0 ) Y0 .The Lipschitz condition on f is assumed throughout the text. The condition (1.10)is easily obtained if f (t, y)/ y is a continuous function of (t, y) over D, the closureof D, with D also assumed to be convex. (A set D is called convex if for any twopoints in D the line segment joining them is entirely contained in D. Examples ofconvex sets include circles, ellipses, triangles, parallelograms.) Then we can useK max(t,y) D f (t, y), yprovided this is finite. If not, then simply use a smaller D, say, one that is boundedand contains (t0 , Y0 ) in its interior. The number α in the statement of the theoremdepends on the initial value problem (1.7). For some equations, such as the linearequation given in (1.3) with a continuous function g(t), solutions exist for any t, andwe can take α to be . For many nonlinear equations, solutions can exist only in
8THEORY OF DIFFERENTIAL EQUATIONS: AN INTRODUCTIONbounded intervals. We have seen such instances in Examples 1.2 and 1.3. Let us lookat one more such example.Example 1.6 Consider the initial value problemY ′ (t) 2t[Y (t)]2 ,Y (0) 1.Here f (t, y) 4ty, yf (t, y) 2ty 2 ,and both of these functions are continuous for all (t, y). Thus, by Theorem 1.5 thereis a unique solution to this initial value problem for t in a neighborhood of t0 0.This solution is1, 1 t 1.Y (t) 1 t2This example illustrates that the continuity of f (t, y) and f (t, y)/ y for all (t, y)does not imply the existence of a solution Y (t) for all t.1.2 STABILITY OF THE INITIAL VALUE PROBLEMWhen numerically solving the initial value problem (1.7)
Differential equations are among the most important mathematical tools used in pro-ducing models in the physical sciences, biological sciences, and engineering. In this text, we consider numerical methods for solving ordinary differential equations, that is, those differential equations that have only one independent variable.
3 Ordinary Differential Equations K. Webb MAE 4020/5020 Differential equations can be categorized as either ordinary or partialdifferential equations Ordinarydifferential equations (ODE's) - functions of a single independent variable Partial differential equations (PDE's) - functions of two or more independent variables
EQUATIONS AND INEQUALITIES Golden Rule of Equations: "What you do to one side, you do to the other side too" Linear Equations Quadratic Equations Simultaneous Linear Equations Word Problems Literal Equations Linear Inequalities 1 LINEAR EQUATIONS E.g. Solve the following equations: (a) (b) 2s 3 11 4 2 8 2 11 3 s
1.2 First Order Equations 5 1.3 Direction Fields for First Order Equations 14 Chapter 2 First Order Equations 2.1 Linear First Order Equations 27 2.2 Separable Equations 39 2.3 Existence and Uniqueness of Solutions of Nonlinear Equations 48 2.5 Exact Equations 55 2.6 Integrating Factors 63 Chapter 3 Numerical Methods 3.1 Euler’s Method 74
Chapter 1 Introduction 1 1.1 ApplicationsLeading to Differential Equations 1.2 First Order Equations 5 1.3 Direction Fields for First Order Equations 16 Chapter 2 First Order Equations 30 2.1 Linear First Order Equations 30 2.2 Separable Equations 45 2.3 Existence and Uniqueness of Solutionsof Nonlinear Equations 55
point can be determined by solving a system of equations. A system of equations is a set of two or more equations. To ÒsolveÓ a system of equations means to find values for the variables in the equations, which make all the equations true at the same time. One way to solve a system of equations is by graphing.
solving equations from previous grades and is a gateway into the entire unit on equations and inequalities. Conceptual Understanding: Mystery Letters 4-5 days I will review equations by Conceptual Understanding:Solving simple equations, multi-step equations, and equations
9.1 Properties of Radicals 9.2 Solving Quadratic Equations by Graphing 9.3 Solving Quadratic Equations Using Square Roots 9.4 Solving Quadratic Equations by Completing the Square 9.5 Solving Quadratic Equations Using the Quadratic Formula 9.6 Solving Nonlinear Systems of Equations 9 Solving Quadratic Equations
A. General guidance for academic writing The style of writing required for LSHTM assessments may call for different skills to those you have used in your previous education or employment. If you are not entirely confident in this, remember that the more academic writing you do, the better you will become at it. Aspects that may be new or unfamiliar, such as citing and referencing, should .
