A First Course In LINEAR ALGEBRA - Lyryx Learning

CONTENTSv4.11.3 Gram-Schmidt Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2414.11.4 Orthogonal Projections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2444.11.5 Least Squares Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2514.12 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2614.12.1 Vectors and Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2614.12.2 Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2645 Linear Transformations2695.1 Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2695.25.3The Matrix of a Linear Transformation I . . . . . . . . . . . . . . . . . . . . . . . . . . . 272Properties of Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2815.45.5Special Linear Transformations in R2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286One to One and Onto Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2925.65.75.8Isomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298The Kernel And Image Of A Linear Map . . . . . . . . . . . . . . . . . . . . . . . . . . . 310The Matrix of a Linear Transformation II . . . . . . . . . . . . . . . . . . . . . . . . . . 3155.9The General Solution of a Linear System . . . . . . . . . . . . . . . . . . . . . . . . . . . 3216 Complex Numbers3296.1 Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3296.26.3Polar Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336Roots of Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3396.4The Quadratic Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3437 Spectral Theory3477.1 Eigenvalues and Eigenvectors of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 3477.1.17.1.27.1.37.2Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3627.2.1 Similarity and Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3627.2.27.2.37.3Definition of Eigenvectors and Eigenvalues . . . . . . . . . . . . . . . . . . . . . 347Finding Eigenvectors and Eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . 350Eigenvalues and Eigenvectors for Special Types of Matrices . . . . . . . . . . . . 356Diagonalizing a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364Complex Eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369Applications of Spectral Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3727.3.1 Raising a Matrix to a High Power . . . . . . . . . . . . . . . . . . . . . . . . . . 3737.3.27.3.3Raising a Symmetric Matrix to a High Power . . . . . . . . . . . . . . . . . . . . 375Markov Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3787.3.3.1 Eigenvalues of Markov Matrices . . . . . . . . . . . . . . . . . . . . . 384

viCONTENTS7.3.47.3.57.4Dynamical Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384The Matrix Exponential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4017.4.1 Orthogonal Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4017.4.2 The Singular Value Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . 4097.4.3Positive Definite Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4177.4.3.1 The Cholesky Factorization . . . . . . . . . . . . . . . . . . . . . . . . 4207.4.4QR Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4227.4.4.1 The QR Factorization and Eigenvalues . . . . . . . . . . . . . . . . . . 4247.4.57.4.4.2 Power Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4278 Some Curvilinear Coordinate Systems8.18.2439Polar Coordinates and Polar Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439Spherical and Cylindrical Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4499 Vector Spaces4559.1 Algebraic Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4559.29.3Spanning Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471Linear Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4759.49.5Subspaces and Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483Sums and Intersections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4989.69.7Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499Isomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5059.89.7.1 One to One and Onto Transformations . . . . . . . . . . . . . . . . . . . . . . . . 5059.7.2 Isomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509The Kernel And Image Of A Linear Map . . . . . . . . . . . . . . . . . . . . . . . . . . . 5189.9The Matrix of a Linear Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524A Some Prerequisite Topics537A.1 Sets and Set Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537A.2 Well Ordering and Induction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539B Selected Exercise Answers543Index591

PrefaceA First Course in Linear Algebra presents an introduction to the fascinating subject of linear algebra forstudents who have a reasonable understanding of basic algebra. Major topics of linear algebra are presented in detail, with proofs of important theorems provided. Separate sections may be included in whichproofs are examined in further depth and in general these can be excluded without loss of contrinuity.Where possible, applications of key concepts are explored. In an effort to assist those students who areinterested in continuing on in linear algebra connections to additional topics covered in advanced coursesare introduced.Each chapter begins with a list of desired outcomes which a student should be able to achieve uponcompleting the chapter. Throughout the text, examples and diagrams are given to reinforce ideas andprovide guidance on how to approach various problems. Students are encouraged to work through thesuggested exercises provided at the end of each section. Selected solutions to these exercises are given atthe end of the text.As this is an open text, you are encouraged to interact with the textbook through annotating, revising,and reusing to your advantage.1

1. Systems of Equations1.1 Systems of Equations, GeometryOutcomesA. Relate the types of solution sets of a system of two (three) variables to the intersections oflines in a plane (the intersection of planes in three space)As you may remember, linear equations like 2x 3y 6 can be graphed as straight lines in the coordinate plane. We say that this equation is in two variables, in this case x and y. Suppose you have two suchequations, each of which can be graphed as a straight line, and consider the resulting graph of two lines.What would it mean if there exists a point of intersection between the two lines? This point, which lies onboth graphs, gives x and y values for which both equations are true. In other words, this point gives theordered pair (x, y) that satisfy both equations. If the point (x, y) is a point of intersection, we say that (x, y)is a solution to the two equations. In linear algebra, we often are concerned with finding the solution(s)to a system of equations, if such solutions exist. First, we consider graphical representations of solutionsand later we will consider the algebraic methods for finding solutions.When looking for the intersection of two lines in a graph, several situations may arise. The following picture demonstrates the possible situations when considering two equations (two lines in the graph)involving two variables.yyyxOne SolutionxxInfinitely Many SolutionsNo SolutionsIn the first diagram, there is a unique point of intersection, which means that there is only one (unique)solution to the two equations. In the second, there are no points of intersection and no solution. When nosolution exists, this means that the two lines are parallel and they never intersect. The third situation whichcan occur, as demonstrated in diagram three, is that the two lines are really the same line. For example,x y 1 and 2x 2y 2 are equations which when graphed yield the same line. In this case there areinfinitely many points which are solutions of these two equations, as every ordered pair which is on thegraph of the line satisfies both equations. When considering linear systems of equations, there are alwaysthree types of solutions possible; exactly one (unique) solution, infinitely many solutions, or no solution.3

4Systems of EquationsExample 1.1: A Graphical SolutionUse a graph to find the solution to the following system of equationsx y 3y x 5Solution. Through graphing the above equations and identifying the point of intersection, we can find thesolution(s). Remember that we must have either one solution, infinitely many, or no solutions at all. Thefollowing graph shows the two equations, as well as the intersection. Remember, the point of intersectionrepresents the solution of the two equations, or the (x, y) which satisfy both equations. In this case, thereis one point of intersection at ( 1, 4) which means we have one unique solution, x 1, y 4.y6(x, y) ( 1, 4)42x 4 3 2 11 In the above example, we investigated the intersection point of two equations in two variables, x andy. Now we will consider the graphical solutions of three equations in two variables.Consider a system of three equations in two variables. Again, these equations can be graphed asstraight lines in the plane, so that the resulting graph contains three straight lines. Recall the three possibletypes of solutions; no solution, one solution, and infinitely many solutions. There are now more complexways of achieving these situations, due to the presence of the third line. For example, you can imaginethe case of three intersecting lines having no common point of intersection. Perhaps you can also imaginethree intersecting lines which do intersect at a single point. These two situations are illustrated below.yyxNo SolutionxOne Solution

1.1. Systems of Equations, Geometry5Consider the first picture above. While all three lines intersect with one another, there is no commonpoint of intersection where all three lines meet at one point. Hence, there is no solution to the threeequations. Remember, a solution is a point (x, y) which satisfies all three equations. In the case of thesecond picture, the lines intersect at a common point. This means that there is one solution to the threeequations whose graphs are the given lines. You should take a moment now to draw the graph of a systemwhich results in three parallel lines. Next, try the graph of three identical lines. Which type of solution isrepresented in each of these graphs?We have now considered the graphical solutions of systems of two equations in two variables, as wellas three equations in two variables. However, there is no reason to limit our investigation to equations intwo variables. We will now consider equations in three variables.You may recall that equations in three variables, such as 2x 4y 5z 8, form a plane. Above, wewere looking for intersections of lines in order to identify any possible solutions. When graphically solvingsystems of equations in three variables, we look for intersections of planes. These points of intersectiongive the (x, y, z) that satisfy all the equations in the system. What types of solutions are possible whenworking with three variables? Consider the following picture involving two planes, which are given bytwo equations in three variables.Notice how these two planes intersect in a line. This means that the points (x, y, z) on this line satisfyboth equations in the system. Since the line contains infinitely many points, this system has infinitelymany solutions.It could also happen that the two planes fail to intersect. However, is it possible to have two planesintersect at a single point? Take a moment to attempt drawing this situation, and convince yourself that itis not possible! This means that when we have only two equations in three variables, there is no way tohave a unique solution! Hence, the types of solutions possible for two equations in three variables are nosolution or infinitely many solutions.Now imagine adding a third plane. In other words, consider three equations in three variables. Whattypes of solutions are now possible? Consider the following diagram. New PlaneIn this diagram, there is no point which lies in all three planes. There is no intersection between all

6Systems of Equationsplanes so there is no solution. The picture illustrates the situation in which the line of intersection of thenew plane with one of the original planes forms a line parallel to the line of intersection of the first twoplanes. However, in three dimensions, it is possible for two lines to fail to intersect even though they arenot parallel. Such lines are called skew lines.Recall that when working with two equations in three variables, it was not possible to have a uniquesolution. Is it possible when considering three equations in three variables? In fact, it is possible, and wedemonstrate this situation in the following picture.New Plane In this case, the three planes have a single point of intersection. Can you think of other types ofsolutions possible? Another is that the three planes could intersect in a line, resulting in infinitely manysolutions, as in the following diagram.We have now seen how three equations in three variables can have no solution, a unique solution, orintersect in a line resulting in infinitely many solutions. It is also possible that the three equations graphthe same plane, which also leads to infinitely many solutions.You can see that when working with equations in three variables, there are many more ways to achievethe different types of solutions than when working with two variables. It may prove enlightening to spendtime imagining (and drawing) many possible scenarios, and you should take some time to try a few.You should also take some time to imagine (and draw) graphs of systems in more than three variables.Equations like x y 2z 4w 8 with more than three variables are often called hyper-planes. You maysoon realize that it is tricky to draw the graphs of hyper-planes! Through the tools of linear algebra, wecan algebraically examine these types of systems which are difficult to graph. In the following section, wewill consider these algebraic tools.

1.2. Systems Of Equations, Algebraic Procedures7ExercisesExercise 1.1.1 Graphically, find the point (x1 , y1 ) which lies on both lines, x 3y 1 and 4x y 3.That is, graph each line and see where they intersect.Exercise 1.1.2 Graphically, find the point of intersection of the two lines 3x y 3 and x 2y 1. Thatis, graph each line and see where they intersect.Exercise 1.1.3 You have a system of k equations in two variables, k 2. Explain the geometric significance of(a) No solution.(b) A unique solution.(c) An infinite number of solutions.1.2 Systems Of Equations, Algebraic ProceduresOutcomesA. Use elementary operations to find the solution to a linear system of equations.B. Find the row-echelon form and reduced row-echelon form of a matrix.C. Determine whether a system of linear equations has no solution, a unique solution or aninfinite number of solutions from its row-echelon form.D. Solve a system of equations using Gaussian Elimination and Gauss-Jordan Elimination.E. Model a physical system with linear equations and then solve.We have taken an in depth look at graphical representations of systems of equations, as well as how tofind possible solutions graphically. Our attention now turns to working with systems algebraically.

8Systems of EquationsDefinition 1.2: System of Linear EquationsA system of linear equations is a list of equations,a11 x1 a12 x2 · · · a1n xn b1a21 x1 a22 x2 · · · a2n xn b2.am1 x1 am2 x2 · · · amn xn bmwhere ai j and b j are real numbers. The above is a system of m equations in the n variables,x1 , x2 · · · , xn . Written more simply in terms of summation notation, the above can be written inthe formn ai j x j bi, i 1, 2, 3, · · · , mj 1The relative size of m and n is not important here. Notice that we have allowed ai j and b j to be anyreal number. We can also call these numbers scalars . We will use this term throughout the text, so keepin mind that the term scalar just means that we are working with real numbers.Now, suppose we have a system where bi 0 for all i. In other words every equation equals 0. This isa special type of system.Definition 1.3: Homogeneous System of EquationsA system of equations is called homogeneous if each equation in the system is equal to 0. Ahomogeneous system has the forma11 x1 a12 x2 · · · a1n xn 0a21 x1 a22 x2 · · · a2n xn 0.am1 x1 am2 x2 · · · amn xn 0where ai j are scalars and xi are variables.Recall from the previous section that our goal when working with systems of linear equations was tofind the point of intersection of the equations when graphed. In other words, we looked for the solutions tothe system. We now wish to find these solutions algebraically. We want to find values for x1 , · · · , xn whichsolve all of the equations. If such a set of values exists, we call (x1 , · · · , xn ) the solution set.Recall the above discussions about the types of solutions possible. We will see that systems of linearequations will have one unique solution, infinitely many solutions, or no solution. Consider the followingdefinition.Definition 1.4: Consistent and Inconsistent SystemsA system of linear equations is called consistent if there exists at least one solution. It is calledinconsistent if there is no solution.

1.2. Systems Of Equations, Algebraic Procedures9If you think of each equation as a condition which must be satisfied by the variables, consistent wouldmean there is some choice of variables which can satisfy all the conditions. Inconsistent would mean thereis no choice of the variables which can satisfy all of the conditions.The following sections provide methods for determining if a system is consistent or inconsistent, andfinding solutions if they exist.1.2.1. Elementary OperationsWe begin this section with an example. Recall from Example 1.1 that the solution to the given system was(x, y) ( 1, 4).Example 1.5: Verifying an Ordered Pair is a SolutionAlgebraically verify that (x, y) ( 1, 4) is a solution to the following system of equations.x y 3y x 5Solution. By graphing these two equations and identifying the point of intersection, we previously foundthat (x, y) ( 1, 4) is the unique solution.We can verify algebraically by substituting these values into the original equations, and ensuring thatthe equations hold. First, we substitute the values into the first equation and check that it equals 3.x y ( 1) (4) 3This equals 3 as needed, so we see that ( 1, 4) is a solution to the first equation. Substituting the valuesinto the second equation yieldsy x (4) ( 1) 4 1 5which is true. For (x, y) ( 1, 4) each equation is true and therefore, this is a solution to the system. Now, the interesting question is this: If you were not given these numbers to verify, how could youalgebraically determine the solution? Linear algebra gives us the tools needed to answer this question.The following basic operations are important tools that we will utilize.Definition 1.6: Elementary OperationsElementary operations are those operations consisting of the following.1. Interchang

A First Course in Linear Algebra presents an introduction to the fascinating subject of linear algebra for students who have a reasonable understanding of basic algebra. Major topics of linear algebra are pre-sented in detail, with proofs of important theorems provided. Separate sections may be included in which