Linear Algebra, Theory And Applications - Saylor Academy
Linear Algebra, Theory And ApplicationsKenneth KuttlerJanuary 29, 2012Saylor URL: http://www.saylor.org/courses/ma212/The Saylor Foundation
2Linear Algebra, Theory and Applications was written by Dr. Kenneth Kuttler of Brigham Young University forteaching Linear Algebra II. After The Saylor Foundation accepted his submission to Wave I of the OpenTextbook Challenge, this textbook was relicensed as CC-BY 3.0.Information on The Saylor Foundation’s Open Textbook Challenge can be found at www.saylor.org/otc/.Linear Algebra, Theory, and Applications January 29, 2012 by Kenneth Kuttler, is licensed under a CreativeCommons Attribution (CC BY) license made possible by funding from The Saylor Foundation's OpenTextbook Challenge in order to be incorporated into Saylor.org's collection of open courses available at: http://www.saylor.org" Full license terms may be viewed at: deSaylor URL: http://www.saylor.org/courses/ma212/The Saylor Foundation
Contents1 Preliminaries1.1 Sets And Set Notation . . . . . . . . . . . . . . . . . .1.2 Functions . . . . . . . . . . . . . . . . . . . . . . . . .1.3 The Number Line And Algebra Of The Real Numbers1.4 Ordered fields . . . . . . . . . . . . . . . . . . . . . . .1.5 The Complex Numbers . . . . . . . . . . . . . . . . . .1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . .1.7 Completeness of R . . . . . . . . . . . . . . . . . . . .1.8 Well Ordering And Archimedean Property . . . . . . .1.9 Division And Numbers . . . . . . . . . . . . . . . . . .1.10 Systems Of Equations . . . . . . . . . . . . . . . . . .1.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . .1.12 Fn . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.13 Algebra in Fn . . . . . . . . . . . . . . . . . . . . . . .1.14 Exercises . . . . . . . . . . . . . . . . . . . . . . . . .1.15 The Inner Product In Fn . . . . . . . . . . . . . . . .1.16 What Is Linear Algebra? . . . . . . . . . . . . . . . . .1.17 Exercises . . . . . . . . . . . . . . . . . . . . . . . . .1111121214151920212326313232333336362 Matrices And Linear Transformations2.1 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . .2.1.1 The ij th Entry Of A Product . . . . . . . . . . . .2.1.2 Digraphs . . . . . . . . . . . . . . . . . . . . . . .2.1.3 Properties Of Matrix Multiplication . . . . . . . .2.1.4 Finding The Inverse Of A Matrix . . . . . . . . . .2.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . .2.3 Linear Transformations . . . . . . . . . . . . . . . . . . .2.4 Subspaces And Spans . . . . . . . . . . . . . . . . . . . .2.5 An Application To Matrices . . . . . . . . . . . . . . . . .2.6 Matrices And Calculus . . . . . . . . . . . . . . . . . . . .2.6.1 The Coriolis Acceleration . . . . . . . . . . . . . .2.6.2 The Coriolis Acceleration On The Rotating Earth2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . .37374143454851535661626366713 Determinants3.1 Basic Techniques And Properties . . . . . .3.2 Exercises . . . . . . . . . . . . . . . . . . .3.3 The Mathematical Theory Of Determinants3.3.1 The Function sgn . . . . . . . . . . .7777818384.3Saylor URL: http://www.saylor.org/courses/ma212/The Saylor Foundation
4CONTENTS3.43.53.63.3.2 The Definition Of The Determinant .3.3.3 A Symmetric Definition . . . . . . . .3.3.4 Basic Properties Of The Determinant3.3.5 Expansion Using Cofactors . . . . . .3.3.6 A Formula For The Inverse . . . . . .3.3.7 Rank Of A Matrix . . . . . . . . . . .3.3.8 Summary Of Determinants . . . . . .The Cayley Hamilton Theorem . . . . . . . .Block Multiplication Of Matrices . . . . . . .Exercises . . . . . . . . . . . . . . . . . . . .4 Row Operations4.1 Elementary Matrices . . . . . . .4.2 The Rank Of A Matrix . . . . .4.3 The Row Reduced Echelon Form4.4 Rank And Existence Of Solutions4.5 Fredholm Alternative . . . . . . .4.6 Exercises . . . . . . . . . . . . . . . .To. . . . . . . . . . . .Linear. . . . . . .868788909294969798102. . . . . . . . . . . . .Systems. . . . . . . . .1051051101121161171185 Some Factorizations5.1 LU Factorization . . . . . . . . . . . . . . . . . . .5.2 Finding An LU Factorization . . . . . . . . . . . .5.3 Solving Linear Systems Using An LU Factorization5.4 The P LU Factorization . . . . . . . . . . . . . . .5.5 Justification For The Multiplier Method . . . . . .5.6 Existence For The P LU Factorization . . . . . . .5.7 The QR Factorization . . . . . . . . . . . . . . . .5.8 Exercises . . . . . . . . . . . . . . . . . . . . . . .1231231231251261271281301336 Linear Programming6.1 Simple Geometric Considerations .6.2 The Simplex Tableau . . . . . . . .6.3 The Simplex Algorithm . . . . . .6.3.1 Maximums . . . . . . . . .6.3.2 Minimums . . . . . . . . . .6.4 Finding A Basic Feasible Solution .6.5 Duality . . . . . . . . . . . . . . .6.6 Exercises . . . . . . . . . . . . . .1351351361401401431501521567 Spectral Theory7.1 Eigenvalues And Eigenvectors Of A Matrix . . . . .7.2 Some Applications Of Eigenvalues And Eigenvectors7.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . .7.4 Schur’s Theorem . . . . . . . . . . . . . . . . . . . .7.5 Trace And Determinant . . . . . . . . . . . . . . . .7.6 Quadratic Forms . . . . . . . . . . . . . . . . . . . .7.7 Second Derivative Test . . . . . . . . . . . . . . . . .7.8 The Estimation Of Eigenvalues . . . . . . . . . . . .7.9 Advanced Theorems . . . . . . . . . . . . . . . . . .7.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . .157157164167173180181182186187190Saylor URL: http://www.saylor.org/courses/ma212/.The Saylor Foundation
CONTENTS58 Vector Spaces And Fields8.1 Vector Space Axioms . . . . . . . . . . . . . .8.2 Subspaces And Bases . . . . . . . . . . . . . .8.2.1 Basic Definitions . . . . . . . . . . . .8.2.2 A Fundamental Theorem . . . . . . .8.2.3 The Basis Of A Subspace . . . . . . .8.3 Lots Of Fields . . . . . . . . . . . . . . . . . .8.3.1 Irreducible Polynomials . . . . . . . .8.3.2 Polynomials And Fields . . . . . . . .8.3.3 The Algebraic Numbers . . . . . . . .8.3.4 The Lindemannn Weierstrass Theorem8.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .And. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Vector. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Spaces . . . . .1991992002002012052052052102152192199 Linear Transformations9.1 Matrix Multiplication As A Linear Transformation . . . . .9.2 L (V, W ) As A Vector Space . . . . . . . . . . . . . . . . . .9.3 The Matrix Of A Linear Transformation . . . . . . . . . . .9.3.1 Some Geometrically Defined Linear Transformations9.3.2 Rotations About A Given Vector . . . . . . . . . . .9.3.3 The Euler Angles . . . . . . . . . . . . . . . . . . . .9.4 Eigenvalues And Eigenvectors Of Linear Transformations .9.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . .22522522522723423723824024210 Linear Transformations Canonical Forms10.1 A Theorem Of Sylvester, Direct Sums . .10.2 Direct Sums, Block Diagonal Matrices . .10.3 Cyclic Sets . . . . . . . . . . . . . . . . .10.4 Nilpotent Transformations . . . . . . . . .10.5 The Jordan Canonical Form . . . . . . . .10.6 Exercises . . . . . . . . . . . . . . . . . .10.7 The Rational Canonical Form . . . . . . .10.8 Uniqueness . . . . . . . . . . . . . . . . .10.9 Exercises . . . . . . . . . . . . . . . . . .24524524825125525726226626927311 Markov Chains And Migration Processes11.1 Regular Markov Matrices . . . . . . . . .11.2 Migration Matrices . . . . . . . . . . . . .11.3 Markov Chains . . . . . . . . . . . . . . .11.4 Exercises . . . . . . . . . . . . . . . . . .27527527927928412 Inner Product Spaces12.1 General Theory . . . . . . . . . . . .12.2 The Gram Schmidt Process . . . . .12.3 Riesz Representation Theorem . . .12.4 The Tensor Product Of Two Vectors12.5 Least Squares . . . . . . . . . . . . .12.6 Fredholm Alternative Again . . . . .12.7 Exercises . . . . . . . . . . . . . . .12.8 The Determinant And Volume . . .12.9 Exercises . . . . . . . . . . . . . . .287287289292295296298298303306Saylor URL: http://www.saylor.org/courses/ma212/.The Saylor Foundation
6CONTENTS13 Self Adjoint Operators13.1 Simultaneous Diagonalization . . . . . . . . . . .13.2 Schur’s Theorem . . . . . . . . . . . . . . . . . .13.3 Spectral Theory Of Self Adjoint Operators . . . .13.4 Positive And Negative Linear Transformations .13.5 Fractional Powers . . . . . . . . . . . . . . . . . .13.6 Polar Decompositions . . . . . . . . . . . . . . .13.7 An Application To Statistics . . . . . . . . . . .13.8 The Singular Value Decomposition . . . . . . . .13.9 Approximation In The Frobenius Norm . . . . .13.10Least Squares And Singular Value Decomposition13.11The Moore Penrose Inverse . . . . . . . . . . . .13.12Exercises . . . . . . . . . . . . . . . . . . . . . .30730731031231731932232532732933133133414 Norms For Finite Dimensional Vector Spaces14.1 The p Norms . . . . . . . . . . . . . . . . . . .14.2 The Condition Number . . . . . . . . . . . . .14.3 The Spectral Radius . . . . . . . . . . . . . . .14.4 Series And Sequences Of Linear Operators . . .14.5 Iterative Methods For Linear Systems . . . . .14.6 Theory Of Convergence . . . . . . . . . . . . .14.7 Exercises . . . . . . . . . . . . . . . . . . . . .33734334534835035436036315 Numerical Methods For Finding Eigenvalues15.1 The Power Method For Eigenvalues . . . . . . . . . . . . .15.1.1 The Shifted Inverse Power Method . . . . . . . . .15.1.2 The Explicit Description Of The Method . . . . .15.1.3 Complex Eigenvalues . . . . . . . . . . . . . . . . .15.1.4 Rayleigh Quotients And Estimates for Eigenvalues15.2 The QR Algorithm . . . . . . . . . . . . . . . . . . . . . .15.2.1 Basic Properties And Definition . . . . . . . . . .15.2.2 The Case Of Real Eigenvalues . . . . . . . . . . .15.2.3 The QR Algorithm In The General Case . . . . . .15.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . .371371375376381383386386390394401.A Positive Matrices403B Functions Of Matrices411C Applications To Differential EquationsC.1 Theory Of Ordinary Differential EquationsC.2 Linear Systems . . . . . . . . . . . . . . . .C.3 Local Solutions . . . . . . . . . . . . . . . .C.4 First Order Linear Systems . . . . . . . . .C.5 Geometric Theory Of Autonomous SystemsC.6 General Geometric Theory . . . . . . . . . .C.7 The Stable Manifold . . . . . . . . . . . . .417417418419421428432434D Compactness And Completeness439D.0.1 The Nested Interval Lemma . . . . . . . . . . . . . . . . . . . . . . . . 439D.0.2 Convergent Sequences, Sequential Compactness . . . . . . . . . . . . . 440Saylor URL: http://www.saylor.org/courses/ma212/The Saylor Foundation
CONTENTS7E The Fundamental Theorem Of Algebra443F Fields And Field ExtensionsF.1 The Symmetric Polynomial Theorem . . . .F.2 The Fundamental Theorem Of Algebra . . .F.3 Transcendental Numbers . . . . . . . . . . .F.4 More On Algebraic Field Extensions . . . .F.5 The Galois Group . . . . . . . . . . . . . .F.6 Normal Subgroups . . . . . . . . . . . . . .F.7 Normal Extensions And Normal SubgroupsF.8 Conditions For Separability . . . . . . . . .F.9 Permutations . . . . . . . . . . . . . . . . .F.10 Solvable Groups . . . . . . . . . . . . . . .F.11 Solvability By Radicals . . . . . . . . . . . .G Answers To Selected ExercisesG.1 Exercises . . . . . . . . . . .G.2 Exercises . . . . . . . . . . .G.3 Exercises . . . . . . . . . . .G.4 Exercises . . . . . . . . . . .G.5 Exercises . . . . . . . . . . .G.6 Exercises . . . . . . . . . . .G.7 Exercises . . . . . . . . . . .G.8 Exercises . . . . . . . . . . .G.9 Exercises . . . . . . . . . . .G.10 Exercises . . . . . . . . . . .G.11 Exercises . . . . . . . . . . .G.12 Exercises . . . . . . . . . . .G.13 Exercises . . . . . . . . . . .G.14 Exercises . . . . . . . . . . .G.15 Exercises . . . . . . . . . . .G.16 Exercises . . . . . . . . . . .G.17 Exercises . . . . . . . . . . .G.18 Exercises . . . . . . . . . . .G.19 Exercises . . . . . . . . . . .G.20 Exercises . . . . . . . . . . .G.21 Exercises . . . . . . . . . . .G.22 Exercises . . . . . . . . . . .G.23 Exercises . . . . . . . . . . .c 2012,Copyright ⃝.Saylor URL: 9490491492492493494494494495495495496496496496The Saylor Foundation
8Saylor URL: http://www.saylor.org/courses/ma212/CONTENTSThe Saylor Foundation
PrefaceThis is a book on linear algebra and matrix theory. While it is self contained, it will workbest for those who have already had some exposure to linear algebra. It is also assumed thatthe reader has had calculus. Some optional topics require more analysis than this, however.I think that the subject of linear algebra is likely the most significant topic discussed inundergraduate mathematics courses. Part of the reason for this is its usefulness in unifyingso many different topics. Linear algebra is essential in analysis, applied math, and even intheoretical mathematics. This is the point of view of this book, more than a presentationof linear algebra for its own sake. This is why there are numerous applications, some fairlyunusual.This book features an ugly, elementary, and complete treatment of determinants earlyin the book. Thus it might be considered as Linear algebra done wrong. I have done thisbecause of the usefulness of determinants. However, all major topics are also presented inan alternative manner which is independent of determinants.The book has an introduction to various numerical methods used in linear algebra.This is done because of the interesting nature of these methods. The presentation hereemphasizes the reasons why they work. It does not discuss many important numericalconsiderations necessary to use the methods effectively. These considerations are found innumerical analysis texts.In the exercises, you may occasionally see at the beginning. This means you ought tohave a look at the exercise above it. Some exercises develop a topic sequentially. There arealso a few exercises which appear more than once in the book. I have done this deliberatelybecause I think that these illustrate exceptionally important topics and because some peopledon’t read the whole book from start to finish but instead jump in to the middle somewhere.There is one on a theorem of Sylvester which appears no fewer than 3 times. Then it is alsoproved in the text. There are multiple proofs of the Cayley Hamilton theorem, some in theexercises. Some exercises also are included for the sake of emphasizing something which hasbeen done in the preceding chapter.9Saylor URL: http://www.saylor.org/courses/ma212/The Saylor Foundation
10Saylor URL: http://www.saylor.org/courses/ma212/CONTENTSThe Saylor Foundation
Preliminaries1.1Sets And Set NotationA set is just a collection of things called elements. For example {1, 2, 3, 8} would be a setconsisting of the elements 1,2,3, and 8. To indicate that 3 is an element of {1, 2, 3, 8} , it iscustomary to write 3 {1, 2, 3, 8} . 9 / {1, 2, 3, 8} means 9 is not an element of {1, 2, 3, 8} .Sometimes a rule specifies a set. For example you could specify a set as all integers largerthan 2. This would be written as S {x Z : x 2} . This notation says: the set of allintegers, x, such that x 2.If A and B are sets with the property that every element of A is an element of B, then A isa subset of B. For example, {1, 2, 3, 8} is a subset of {1, 2, 3, 4, 5, 8} , in symbols, {1, 2, 3, 8} {1, 2, 3, 4, 5, 8} . It is sometimes said that “A is contained in B” or even “B contains A”.The same statement about the two sets may also be written as {1, 2, 3, 4, 5, 8} {1, 2, 3, 8}.The union of two sets is the set consisting of everything which is an element of at leastone of the sets, A or B. As an example of the union of two sets {1, 2, 3, 8} {3, 4, 7, 8} {1, 2, 3, 4, 7, 8} because these numbers are those which are in at least one of the two sets. IngeneralA B {x : x A or x B} .Be sure you understand that something which is in both A and B is in the union. It is notan exclusive or.The intersection of two sets, A and B consists of everything which is in both of the sets.Thus {1, 2, 3, 8} {3, 4, 7, 8} {3, 8} because 3 and 8 are those elements the two sets havein common. In general,A B {x : x A and x B} .The symbol [a, b] where a and b are real numbers, denotes the set of real numbers x,such that a x b and [a, b) denotes the set of real numbers such that a x b. (a, b)consists of the set of real numbers x such that a x b and (a, b] indicates the set ofnumbers x such that a x b. [a, ) means the set of all numbers x such that x a and( , a] means the set of all real numbers which are less than o
Linear Algebra, Theory and Applications was written by Dr. Kenneth Kuttler of Brigham Young University for teaching Linear Algebra II. After The Saylor Foundation accepted his submission to Wave I of the Open Textboo
Robert Gerver, Ph.D. North Shore High School 450 Glen Cove Avenue Glen Head, NY 11545 gerverr@northshoreschools.org Rob has been teaching at . Algebra 1 Financial Algebra Geometry Algebra 2 Algebra 1 Geometry Financial Algebra Algebra 2 Algebra 1 Geometry Algebra 2 Financial Algebra ! Concurrently with Geometry, Algebra 2, or Precalculus
INTRODUCTION TO LINEAR ALGEBRA AND S-LINEAR ALGEBRA 1.1 Basic properties of linear algebra 7 1.2 Introduction to s-linear algebra 15 1.3 Some aapplications of S-linear algebra 30 Chapter Two INTRODUCTORY COCEPTS OF BASIC BISTRUCTURES AND S-BISTRUCTU
High-level description of course goals: 1. linear algebra theory; 2. linear algebra computa-tional skills; 3. introduction to abstract math. Today’s topic: introduction to linear algebra. Conceptually, linear algebra is about sets of quantities (a.k.a. vectors
results- rst approach to machine learning, and linear algebra is not the rst step, but perhaps the second or third. Practitioners Study Too Much Linear Algebra When practitioners do circle back to study linear algebra, they learn far more of the eld than is required for or relevant to machine learning. Linear algebra is a large eld of study
Sep 07, 2020 · 06 - Linear Algebra Review De ning Matrices Basic Matrix Operations Special Types of Matrices Matrix Inversion Properties of Matrices Operations of Matrices Simple Linear Regression References OverviewI We wrap up the math topics by reviewing some linear algebra concepts Linear algebra
MTH 210: Intro to Linear Algebra Fall 2019 Course Notes Drew Armstrong Linear algebra is the common denominator of modern mathematics. From the most pure to the most applied, if you use mathematics then you will use linear algebra. It is also a relatively new subject. Linear algebra as we
We have designed Elementary Linear Algebra, Sixth Edition, for the introductory linear algebra course. Students embarking on a linear algebra course should have a thorough knowledge of algebra, and familiarity with analytic geometry and trigonometry. . lay an intuitive foundation for stu
Algebra I – Advanced Linear Algebra (MA251) Lecture Notes Derek Holt and Dmitriy Rumynin year 2009 (revised at the end) Contents 1 Review of Some Linear Algebra 3 1.1 The matrix of a linear m
