Ch.4: Computational linear algebra
Chapter 4 of the book "No Bullshit Guide to Linear Algebra" by Ivan Savov Reduced row echelon form Matrix equations Matrix multiplication Determinants Matrix inverse
104 flashcards · Mathematics · by lakesare
Part of Textbooks › No Bullshit Guide to Linear Algebra (Ivan Savov) · 10 courses
[Computational linear algebra] In the system ax+by=cax + by = cax+by=c , dx+ey=fdx + ey = fdx+ey=f , you add α\alphaα times the first equation to the second. What system do you get, and how do its solutions compare?
ax+by=cax + by = cax+by=c , (d+αa)x+(e+αb)y=f+αc\ (d + \alpha a)x + (e + \alpha b)y = f + \alpha c (d+αa)x+(e+αb)y=f+αc , with exactly the same solutions. This is a row operation: it changes the coefficients but not the solution.Example: x+y=3x + y = 3x+y=3 , x−y=1x - y = 1x−y=1 with α=−1\alpha = -1α=−1 gives x+y=3x + y = 3x+y=3 , −2y=−2-2y = -2−2y=−2 ; both systems have the solution x=2,y=1x = 2, y = 1x=2,y=1 .
[Computational linear algebra] A∈Rm×ℓA \in \mathbb{R}^{m \times \ell}A∈Rm×ℓ and B∈Rℓ×nB \in \mathbb{R}^{\ell \times n}B∈Rℓ×n . What is the formula for the entry cijc_{ij}cij of the matrix product C=ABC = ABC=AB , and what size is CCC ?
cij=∑k=1ℓaikbkj\boldsymbol{c_{ij} = \sum_{k=1}^{\ell} a_{ik} b_{kj}}cij=∑k=1ℓaikbkj , and C∈Rm×nC \in \mathbb{R}^{m \times n}C∈Rm×n . cijc_{ij}cij is the dot product of row iii of AAA and column jjj of BBB .Example: [12][34]=[1⋅3+2⋅4]=[11]\begin{bmatrix} 1 & 2 \end{bmatrix} \begin{bmatrix} 3 \\ 4 \end{bmatrix} = \begin{bmatrix} 1 \cdot 3 + 2 \cdot 4 \end{bmatrix} = \begin{bmatrix} 11 \end{bmatrix}[12][34]=[1⋅3+2⋅4]=[11]
[Computational linear algebra] The determinant ∣A∣|A|∣A∣ of a square matrix AAA gives information about which property of the rows of AAA ?
Whether the rows of AAA are linearly independent. The determinant is also connected to the geometry of vectors, solving systems of equations, and whether AAA is invertible.Example: ∣1224∣=1⋅4−2⋅2=0\left| \begin{matrix} 1 & 2 \\ 2 & 4 \end{matrix} \right| = 1 \cdot 4 - 2 \cdot 2 = 0∣∣1224∣∣=1⋅4−2⋅2=0 , and the second row is 2 times the first.
[Reduced row echelon form] What is the augmented matrix of the system 1x1+2x2=51x_1 + 2x_2 = 51x1+2x2=5 , 3x1+9x2=21\ 3x_1 + 9x_2 = 21 3x1+9x2=21 ?
[1253921]\boldsymbol{\left[\begin{array}{cc|c} 1 & 2 & 5 \\ 3 & 9 & 21 \end{array}\right]}[1329521] One row per equation: the coefficients of x1x_1x1 and x2x_2x2 , then (after the vertical line, where the equal signs were) the constant on the right-hand side.
[Reduced row echelon form] Why can a system of linear equations be replaced by its augmented matrix without losing anything?
The solution depends only on the coefficients and the right-hand constants, not on the names of the unknowns. Example: 1x1+2x2=51x_1 + 2x_2 = 51x1+2x2=5 , 3x1+9x2=213x_1 + 9x_2 = 213x1+9x2=21 and 1s+2t=51s + 2t = 51s+2t=5 , 3s+9t=213s + 9t = 213s+9t=21 have the same solution, (x1,x2)=(s,t)=(1,2)(x_1, x_2) = (s, t) = (1, 2)(x1,x2)=(s,t)=(1,2) .
[Reduced row echelon form] Which three kinds of row operations may be applied to an augmented matrix without changing the solutions of its system?
Add a multiple of one row to another row Example: R2←R2−3R1R_2 \leftarrow R_2 - 3R_1R2←R2−3R1 turns [1253921]\left[\begin{array}{cc|c} 1 & 2 & 5 \\ 3 & 9 & 21 \end{array}\right][1329521] into [125036]\left[\begin{array}{cc|c} 1 & 2 & 5 \\ 0 & 3 & 6 \end{array}\right][102356] . Swap two rows Example: [012101]\left[\begin{array}{cc|c} 0 & 1 & 2 \\ 1 & 0 & 1 \end{array}\right][011021] becomes [101012]\left[\begin{array}{cc|c} 1 & 0 & 1 \\ 0 & 1 & 2 \end{array}\right][100112] . Multi…
[Reduced row echelon form] What does the notation R2←R2−3R1R_2 \leftarrow R_2 - 3R_1R2←R2−3R1 mean?
Replace row 2 by (row 2) minus 3 times (row 1). Example: rows R1=(1,2∣5)R_1 = (1, 2 \mid 5)R1=(1,2∣5) , R2=(3,9∣21)R_2 = (3, 9 \mid 21)R2=(3,9∣21) give the new R2=(0,3∣6)R_2 = (0, 3 \mid 6)R2=(0,3∣6) .
[Reduced row echelon form] Solve 1x1+2x2=51x_1 + 2x_2 = 51x1+2x2=5 , 3x1+9x2=21\ 3x_1 + 9x_2 = 21 3x1+9x2=21 with row operations on the augmented matrix. Which operations, and what is the solution?
R2←R2−3R1R_2 \leftarrow R_2 - 3R_1R2←R2−3R1 , R2←13R2\ R_2 \leftarrow \frac{1}{3} R_2 R2←31R2 , R1←R1−2R2\ R_1 \leftarrow R_1 - 2R_2 R1←R1−2R2 , giving x1=1x_1 = 1x1=1 , x2=2x_2 = 2x2=2 . [1253921]→[125036]→[125012]→[101012]\left[\begin{array}{cc|c} 1 & 2 & 5 \\ 3 & 9 & 21 \end{array}\right] \to \left[\begin{array}{cc|c} 1 & 2 & 5 \\ 0 & 3 & 6 \end{array}\right] \to \left[\begin{array}{cc|c} 1 & 2 & 5 \\ 0 & 1 & 2 \end{array}\right] \to \left[\begin{array}{cc|c} 1 & 0 & 1 \\ 0 & 1 &…
[Reduced row echelon form] What is the solution of a system of linear equations in the variables x1,…,xnx_1, \dots, x_nx1,…,xn ?
The set of all (x1,…,xn)(x_1, \dots, x_n)(x1,…,xn) that satisfy all the equations. So it is a set, the solution set: it can be empty, one point, or infinitely many points (a line, a plane, ...).Example: the solution of x1+x2=2x_1 + x_2 = 2x1+x2=2 , x1−x2=0x_1 - x_2 = 0x1−x2=0 is {(1,1)}\{(1, 1)\}{(1,1)} .
[Reduced row echelon form] What is the pivot of row jjj of a matrix, and what is a leading one?
Pivot: the leftmost nonzero entry of the row. A leading one is a pivot equal to 1. Any pivot becomes a leading one when you divide its row by the pivot.Example: in the row (0,6,3∣18)(0, 6, 3 \mid 18)(0,6,3∣18) the pivot is 6; R←16RR \leftarrow \frac{1}{6} RR←61R gives (0,1,12∣3)(0, 1, \tfrac{1}{2} \mid 3)(0,1,21∣3) .
[Reduced row echelon form] When is a matrix in row echelon form (REF)?
All entries below the leading ones are zero. It is obtained by adding or subtracting multiples of the row with the leading one from the rows below it.Example: [1231401020013]\left[\begin{array}{ccc|c} 1 & 2 & 3 & 14 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \end{array}\right]⎣⎡1002103011423⎦⎤
[Reduced row echelon form] When is a matrix in reduced row echelon form (RREF)?
Each pivot is a leading one, and all entries below and above the pivots are zero. Entries in columns without a leading one can be any numbers: [10∗0∗01∗0∗0001∗]\left[\begin{array}{cccc|c} 1 & 0 & * & 0 & * \\ 0 & 1 & * & 0 & * \\ 0 & 0 & 0 & 1 & * \end{array}\right]⎣⎡100010∗∗0001∗∗∗⎦⎤ From REF you get RREF by subtracting multiples of each pivot's row from the rows above it.
[Reduced row echelon form] What is the difference between Gaussian elimination and Gauss–Jordan elimination?
Gaussian elimination brings a matrix into row echelon form; Gauss–Jordan elimination brings it into reduced row echelon form. REF has zeros below the leading ones; RREF has zeros below and above them.
[Reduced row echelon form] What is the rank of a matrix AAA ?
The number of pivots in the RREF of AAA . Example: [1224]\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}[1224] has RREF [1200]\begin{bmatrix} 1 & 2 \\ 0 & 0 \end{bmatrix}[1020] , so its rank is 1.
[Reduced row echelon form] What are the two phases of the Gauss–Jordan elimination algorithm?
Forward phase (left to right) Get a leading one in the leftmost column, subtract multiples of its row to make every entry below it zero, then repeat with the next column. The result is in row echelon form. Example: [1253921]→[125036]→[125012]\left[\begin{array}{cc|c} 1 & 2 & 5 \\ 3 & 9 & 21 \end{array}\right] \to \left[\begin{array}{cc|c} 1 & 2 & 5 \\ 0 & 3 & 6 \end{array}\right] \to \left[\begin{array}{cc|c} 1 & 2 & 5 \\ 0 & 1 & 2 \end{array}\right][1329521]→[102356]→[102152] Backward…
[Reduced row echelon form] In the forward phase of Gauss–Jordan elimination, the entry at the top of the current column is 0. What do you do?
Swap the current row with a row below it that has a nonzero entry in this column. Then divide that row by the entry to get a leading one.Example: [012101]\left[\begin{array}{cc|c} 0 & 1 & 2 \\ 1 & 0 & 1 \end{array}\right][011021] : R1↔R2R_1 \leftrightarrow R_2R1↔R2 gives [101012]\left[\begin{array}{cc|c} 1 & 0 & 1 \\ 0 & 1 & 2 \end{array}\right][100112] .
[Reduced row echelon form] Solve with Gauss–Jordan elimination: x+2y+3z=14x + 2y + 3z = 14x+2y+3z=14 , 2x+5y+6z=30\quad 2x + 5y + 6z = 302x+5y+6z=30 , −x+2y+3z=12\quad -x + 2y + 3z = 12−x+2y+3z=12
x=1, y=2, z=3\boldsymbol{x = 1, \ y = 2, \ z = 3}x=1, y=2, z=3 Forward: R2←R2−2R1R_2 \leftarrow R_2 - 2R_1R2←R2−2R1 , R3←R3+R1R_3 \leftarrow R_3 + R_1R3←R3+R1 , R3←R3−4R2R_3 \leftarrow R_3 - 4R_2R3←R3−4R2 , R3←16R3R_3 \leftarrow \frac{1}{6} R_3R3←61R3 give [1231401020013]\left[\begin{array}{ccc|c} 1 & 2 & 3 & 14 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \end{array}\right]⎣⎡1002103011423⎦⎤ (REF).Backward: R1←R1−3R3R_1 \leftarrow R_1 - 3R_3R1←R1−3R3 , then R1←R1−2R2R_1 \leftarrow R_1…
[Reduced row echelon form] What can the solution set of three linear equations in three variables x1,x2,x3x_1, x_2, x_3x1,x2,x3 be, and how does each case show in the RREF?
One solution Every variable's column has a pivot; the values can be read off. Example: [100c1010c2001c3]\left[\begin{array}{ccc|c} 1 & 0 & 0 & c_1 \\ 0 & 1 & 0 & c_2 \\ 0 & 0 & 1 & c_3 \end{array}\right]⎣⎡100010001c1c2c3⎦⎤ gives (c1,c2,c3)(c_1, c_2, c_3)(c1,c2,c3) . Infinitely many solutions, a line One equation is redundant: a row of zeros appears and one variable is free. Example: [10a1c101a2c20000]\left[\begin{array}{ccc|c} 1 & 0 & a_1 & c_1 \\ 0 & 1 & a_2 & c_2 \\ 0 & 0 & 0 & 0…
[Reduced row echelon form] What is a free variable in the RREF of an augmented matrix?
A variable whose column contains no leading one. It can take any value t∈Rt \in \mathbb{R}t∈R ; the other variables are expressed in terms of ttt .Example: in [102501340000]\left[\begin{array}{ccc|c} 1 & 0 & 2 & 5 \\ 0 & 1 & 3 & 4 \\ 0 & 0 & 0 & 0 \end{array}\right]⎣⎡100010230540⎦⎤ , x3x_3x3 is free.
[Reduced row echelon form] Why does a row of zeros appear in the RREF when one of the equations is a linear combination of the others?
Row operations can subtract that combination of the other rows from it, which leaves 0=00 = 00=0 . Then there are really only two equations in three variables, so one variable is free.Example: in x+y=1x + y = 1x+y=1 , y+z=2y + z = 2y+z=2 , x+2y+z=3x + 2y + z = 3x+2y+z=3 the third equation is the sum of the first two; R3←R3−R1−R2R_3 \leftarrow R_3 - R_1 - R_2R3←R3−R1−R2 gives (0,0,0∣0)(0, 0, 0 \mid 0)(0,0,0∣0) .
[Reduced row echelon form] Find all solutions of the system with RREF [102501340000]\left[\begin{array}{ccc|c} 1 & 0 & 2 & 5 \\ 0 & 1 & 3 & 4 \\ 0 & 0 & 0 & 0 \end{array}\right]⎣⎡100010230540⎦⎤ .
(x1,x2,x3)=(5,4,0)+t (−2,−3,1)\boldsymbol{(x_1, x_2, x_3) = (5, 4, 0) + t\,(-2, -3, 1)}(x1,x2,x3)=(5,4,0)+t(−2,−3,1) , t∈R\ t \in \mathbb{R} t∈R x3=tx_3 = tx3=t is free, x1=5−2tx_1 = 5 - 2tx1=5−2t , x2=4−3tx_2 = 4 - 3tx2=4−3t .Geometrically: the line through (5,4,0)(5, 4, 0)(5,4,0) with direction vector (−2,−3,1)(-2, -3, 1)(−2,−3,1) .
[Reduced row echelon form] Find all solutions of the system with RREF [012700000000]\left[\begin{array}{ccc|c} 0 & 1 & 2 & 7 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right]⎣⎡000100200700⎦⎤ .
(x1,x2,x3)=(0,7,0)+s (1,0,0)+t (0,−2,1)\boldsymbol{(x_1, x_2, x_3) = (0, 7, 0) + s\,(1, 0, 0) + t\,(0, -2, 1)}(x1,x2,x3)=(0,7,0)+s(1,0,0)+t(0,−2,1) , s,t∈R\ s, t \in \mathbb{R} s,t∈R x1=sx_1 = sx1=s and x3=tx_3 = tx3=t are free; x2=7−2tx_2 = 7 - 2tx2=7−2t .Geometrically: a plane, with general equation 0x+1y+2z=70x + 1y + 2z = 70x+1y+2z=7 (the first row).
[Reduced row echelon form] Why does an RREF row (0,0,0∣c3)(0, 0, 0 \mid c_3)(0,0,0∣c3) with c3≠0c_3 \neq 0c3=0 mean the system has no solution?
No (x1,x2,x3)(x_1, x_2, x_3)(x1,x2,x3) satisfies 0x1+0x2+0x3=c3≠00x_1 + 0x_2 + 0x_3 = c_3 \neq 00x1+0x2+0x3=c3=0 . Example: s+t=4s + t = 4s+t=4 , s+t=44s + t = 44s+t=44 ; R2←R2−R1R_2 \leftarrow R_2 - R_1R2←R2−R1 gives the row (0,0∣40)(0, 0 \mid 40)(0,0∣40) .
[Reduced row echelon form] How many parameters describe the points of a line, of a plane, and of R3\mathbb{R}^3R3 ?
Line: 1, plane: 2, R3\mathbb{R}^3R3 : 3. The number of parameters is the dimension: one free variable gives a line {p⃗+tv⃗, t∈R}\{\vec{p} + t\vec{v}, \ t \in \mathbb{R}\}{p+tv, t∈R} , two give a plane {p⃗+sv⃗+tw⃗, s,t∈R}\{\vec{p} + s\vec{v} + t\vec{w}, \ s, t \in \mathbb{R}\}{p+sv+tw, s,t∈R} .
[Reduced row echelon form] Each equation aix+biy=cia_i x + b_i y = c_iaix+biy=ci is a line in R2\mathbb{R}^2R2 . What are the possible solution sets of two such equations, geometrically?
One solution The two lines intersect at a point. Example: x+y=2x + y = 2x+y=2 , x−y=0x - y = 0x−y=0 meet at (1,1)(1, 1)(1,1) . Infinitely many solutions The two lines are the same line. Example: x+y=2x + y = 2x+y=2 , 2x+2y=42x + 2y = 42x+2y=4 . No solution The two lines are parallel and distinct. Example: x+y=2x + y = 2x+y=2 , x+y=3x + y = 3x+y=3 .
[Reduced row echelon form] Each equation aix+biy+ciz=dia_i x + b_i y + c_i z = d_iaix+biy+ciz=di is a plane in R3\mathbb{R}^3R3 . What are the possible solution sets of three such equations, geometrically?
One solution Three non-parallel planes meet in a point. Example: x=1x = 1x=1 , y=2y = 2y=2 , z=3z = 3z=3 meet at (1,2,3)(1, 2, 3)(1,2,3) . A line One equation is redundant; two non-parallel planes meet in a line. Example: x=1x = 1x=1 , y=2y = 2y=2 , x+y=3x + y = 3x+y=3 : the line {(1,2,t)}\{(1, 2, t)\}{(1,2,t)} . A plane Two equations are redundant. Example: z=1z = 1z=1 , 2z=22z = 22z=2 , 3z=33z = 33z=3 : the plane z=1z = 1z=1 . No solution Two (or more) of the planes are parallel and distinct.…
[Reduced row echelon form] Besides solving systems of linear equations, which two later tasks does the book do with Gauss–Jordan elimination?
Computing inverse matrices and finding bases for vector spaces.
[Matrix equations] Write the system x1+2x2=5x_1 + 2x_2 = 5x1+2x2=5 , 3x1+9x2=21\ 3x_1 + 9x_2 = 21 3x1+9x2=21 as a matrix equation Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b .
[1239][x1x2]=[521]\boldsymbol{\begin{bmatrix} 1 & 2 \\ 3 & 9 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 5 \\ 21 \end{bmatrix}}[1329][x1x2]=[521] AAA is the 2×22 \times 22×2 matrix of coefficients, x⃗\vec{x}x the vector of unknowns ( 2×12 \times 12×1 ), b⃗\vec{b}b the vector of constants ( 2×12 \times 12×1 ).
[Matrix equations] AAA is invertible. How do you solve Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b for x⃗\vec{x}x , and why does it work?
Multiply both sides by A−1A^{-1}A−1 on the left: x⃗=A−1b⃗\vec{x} = A^{-1}\vec{b}x=A−1b . A−1A=IA^{-1}A = IA−1A=I , and Ix⃗=x⃗I\vec{x} = \vec{x}Ix=x . A−1Ax⃗=A−1b⃗A^{-1} A \vec{x} = A^{-1}\vec{b}A−1Ax=A−1b
[Matrix equations] What is the identity matrix III (written 𝟙 in the book)?
The square matrix with ones on the diagonal and zeros everywhere else. Ix⃗=x⃗I\vec{x} = \vec{x}Ix=x for every vector x⃗\vec{x}x , and A−1A=AA−1=IA^{-1}A = AA^{-1} = IA−1A=AA−1=I .Example: [1001][47]=[47]\begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} 4 \\ 7 \end{bmatrix} = \begin{bmatrix} 4 \\ 7 \end{bmatrix}[1001][47]=[47]
[Matrix equations] A=[1239]A = \begin{bmatrix} 1 & 2 \\ 3 & 9 \end{bmatrix}A=[1329] has the inverse A−1=[3−23−113]A^{-1} = \begin{bmatrix} 3 & -\frac{2}{3} \\ -1 & \frac{1}{3} \end{bmatrix}A−1=[3−1−3231] .Solve Ax⃗=[521]A\vec{x} = \begin{bmatrix} 5 \\ 21 \end{bmatrix}Ax=[521] .
x⃗=[12]\boldsymbol{\vec{x} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}}x=[12] x⃗=A−1b⃗\vec{x} = A^{-1}\vec{b}x=A−1b : 3⋅5−23⋅21=15−14=13 \cdot 5 - \frac{2}{3} \cdot 21 = 15 - 14 = 13⋅5−32⋅21=15−14=1 and −1⋅5+13⋅21=−5+7=2-1 \cdot 5 + \frac{1}{3} \cdot 21 = -5 + 7 = 2−1⋅5+31⋅21=−5+7=2 .
[Matrix equations] Does writing a system as Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b and solving x⃗=A−1b⃗\vec{x} = A^{-1}\vec{b}x=A−1b save computation compared with Gauss–Jordan elimination on [ A∣b⃗ ][\,A \mid \vec{b}\,][A∣b] ?
No: finding A−1A^{-1}A−1 costs roughly as much as bringing [ A∣b⃗ ][\,A \mid \vec{b}\,][A∣b] to RREF. The gain is in notation: x⃗=A−1b⃗\vec{x} = A^{-1}\vec{b}x=A−1b is the same expression whether AAA is 2×22 \times 22×2 or 1000×10001000 \times 10001000×1000 .
[Matrix equations] AAA , BBB , CCC are matrices with AB=CAB = CAB=C and BBB invertible. Why can't you write A=C/BA = C/BA=C/B , and what do you do instead?
Matrix division is not defined; multiply by B−1B^{-1}B−1 instead. BB−1=B−1B=IBB^{-1} = B^{-1}B = IBB−1=B−1B=I , so multiplying by B−1B^{-1}B−1 undoes multiplying by BBB .
[Matrix equations] What two things must you keep in mind when solving matrix equations, that you don't need for equations with numbers?
The order of the factors matters Matrix multiplication is not commutative: in general AB≠BAAB \neq BAAB=BA . Example: ABCABCABC and BACBACBAC are products of the same three matrices but are in general different. Act from the left or from the right Multiplying both sides by a matrix must be done on the same side of both. Example: from AB=CAB = CAB=C , multiplying by B−1B^{-1}B−1 on the right of both sides gives A=CB−1A = CB^{-1}A=CB−1 .
[Matrix equations] AB=CAB = CAB=C with BBB invertible. Solve for AAA : is it A=CB−1A = CB^{-1}A=CB−1 or A=B−1CA = B^{-1}CA=B−1C ?
A=CB−1\boldsymbol{A = CB^{-1}}A=CB−1 Multiply both sides by B−1B^{-1}B−1 on the right: ABB−1=CB−1ABB^{-1} = CB^{-1}ABB−1=CB−1 , and BB−1=IBB^{-1} = IBB−1=I . B−1B^{-1}B−1 must be placed next to BBB to cancel it.
[Matrix equations] AB=CAB = CAB=C with AAA invertible. Solve for BBB .
B=A−1C\boldsymbol{B = A^{-1}C}B=A−1C Multiply both sides by A−1A^{-1}A−1 on the left: A−1AB=A−1CA^{-1}AB = A^{-1}CA−1AB=A−1C , and A−1A=IA^{-1}A = IA−1A=I .
[Matrix equations] When AAA is wider than it is tall, why can Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b not be solved as x⃗=A−1b⃗\vec{x} = A^{-1}\vec{b}x=A−1b , and what do the solutions look like?
AAA is not invertible (fewer equations than unknowns); the solutions, if any, form a whole subspace such as a line or a plane. Example: [11][x1x2]=2\begin{bmatrix} 1 & 1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = 2[11][x1x2]=2 is solved by every point of the line (2,0)+t(−1,1)(2, 0) + t(-1, 1)(2,0)+t(−1,1) .
[Matrix multiplication] Is [abcd][efgh]=[aebfcgdh]\begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} e & f \\ g & h \end{bmatrix} = \begin{bmatrix} ae & bf \\ cg & dh \end{bmatrix}[acbd][egfh]=[aecgbfdh] ?
No. The product is [ae+bgaf+bhce+dgcf+dh]\boldsymbol{\begin{bmatrix} ae + bg & af + bh \\ ce + dg & cf + dh \end{bmatrix}}[ae+bgce+dgaf+bhcf+dh] . Unlike addition and subtraction, matrix multiplication is not done entry by entry.
[Matrix multiplication] How is the entry in row iii , column jjj of the product ABABAB computed?
As the dot product of row iii of AAA and column jjj of BBB . The entry in row 3, column 4 is r⃗3⋅c⃗4\vec{r}_3 \cdot \vec{c}_4r3⋅c4 .Dot product: v⃗⋅w⃗=∑iviwi\vec{v} \cdot \vec{w} = \sum_i v_i w_iv⋅w=∑iviwi .
[Matrix multiplication] When does the product ABABAB exist, and what size is it for A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n , B∈Rn×kB \in \mathbb{R}^{n \times k}B∈Rn×k ?
When the number of columns of AAA equals the number of rows of BBB ; then AB∈Rm×kAB \in \mathbb{R}^{m \times k}AB∈Rm×k . The rows of AAA must have the same length as the columns of BBB , so that their dot products are defined (the "touching" dimensions agree).Example: A∈R2×3A \in \mathbb{R}^{2 \times 3}A∈R2×3 , B∈R3×2B \in \mathbb{R}^{3 \times 2}B∈R3×2 give AB∈R2×2AB \in \mathbb{R}^{2 \times 2}AB∈R2×2 and BA∈R3×3BA \in \mathbb{R}^{3 \times 3}BA∈R3×3 .
[Matrix multiplication] A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n , B∈Rp×qB \in \mathbb{R}^{p \times q}B∈Rp×q , C∈Rs×tC \in \mathbb{R}^{s \times t}C∈Rs×t . When does the product ABCABCABC exist?
When n=pn = pn=p and q=sq = sq=s ; then ABC∈Rm×tABC \in \mathbb{R}^{m \times t}ABC∈Rm×t . Example: A∈R2×3A \in \mathbb{R}^{2 \times 3}A∈R2×3 , B∈R3×4B \in \mathbb{R}^{3 \times 4}B∈R3×4 , C∈R4×1C \in \mathbb{R}^{4 \times 1}C∈R4×1 give ABC∈R2×1ABC \in \mathbb{R}^{2 \times 1}ABC∈R2×1 .
[Matrix multiplication] Is matrix multiplication associative? Is it commutative?
Associative: yes, (AB)C=A(BC)=ABC(AB)C = A(BC) = ABC(AB)C=A(BC)=ABC . Commutative: no, in general AB≠BAAB \neq BAAB=BA . Example: [1101][1011]=[2111]\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 1 & 1 \end{bmatrix} = \begin{bmatrix} 2 & 1 \\ 1 & 1 \end{bmatrix}[1011][1101]=[2111] , but [1011][1101]=[1112]\begin{bmatrix} 1 & 0 \\ 1 & 1 \end{bmatrix} \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 1 \\ 1 & 2 \end{bmatrix}[1101][1011]=[1112] .
[Matrix multiplication] Compute [123456][123456]\begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix} \begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{bmatrix}[142536]⎣⎡135246⎦⎤ .
[22284964]\boldsymbol{\begin{bmatrix} 22 & 28 \\ 49 & 64 \end{bmatrix}}[22492864] [1+6+152+8+184+15+308+20+36]\begin{bmatrix} 1 + 6 + 15 & 2 + 8 + 18 \\ 4 + 15 + 30 & 8 + 20 + 36 \end{bmatrix}[1+6+154+15+302+8+188+20+36]
[Matrix multiplication] Compute [123456][123456]\begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{bmatrix} \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \end{bmatrix}⎣⎡135246⎦⎤[142536] .
[91215192633294051]\boldsymbol{\begin{bmatrix} 9 & 12 & 15 \\ 19 & 26 & 33 \\ 29 & 40 & 51 \end{bmatrix}}⎣⎡91929122640153351⎦⎤ [1+82+103+123+166+209+245+2410+3015+36]\begin{bmatrix} 1 + 8 & 2 + 10 & 3 + 12 \\ 3 + 16 & 6 + 20 & 9 + 24 \\ 5 + 24 & 10 + 30 & 15 + 36 \end{bmatrix}⎣⎡1+83+165+242+106+2010+303+129+2415+36⎦⎤ The same two matrices multiplied in the other order give a 2×22 \times 22×2 matrix, [22284964]\begin{bmatrix} 22 & 28 \\ 49 & 64 \end{bmatrix}[22492864] .
[Matrix multiplication] A matrix A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n can be seen as a function. Which function?
A:Rn→RmA: \mathbb{R}^n \to \mathbb{R}^mA:Rn→Rm , A(x⃗)=Ax⃗\ A(\vec{x}) = A\vec{x} A(x)=Ax . This function is a linear transformation, and every linear transformation from Rn\mathbb{R}^nRn to Rm\mathbb{R}^mRm is of this form for some A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n .Example: A=[2003]A = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}A=[2003] sends (1,1)(1, 1)(1,1) to (2,3)(2, 3)(2,3) .
[Matrix multiplication] What does the composition g∘fg \circ fg∘f of two functions mean?
(g∘f)(x)=g(f(x))(g \circ f)(x) = g(f(x))(g∘f)(x)=g(f(x)) : first apply fff , then apply ggg to the result. Example: f(x)=x+1f(x) = x + 1f(x)=x+1 , g(y)=2yg(y) = 2yg(y)=2y give (g∘f)(3)=g(4)=8(g \circ f)(3) = g(4) = 8(g∘f)(3)=g(4)=8 .
[Matrix multiplication] Why is the matrix product defined the way it is?
So that applying AAA and then BBB is the same as applying the single matrix BABABA : B(Ax⃗)=(BA)x⃗\ B(A\vec{x}) = (BA)\vec{x} B(Ax)=(BA)x . So composing linear transformations is multiplying their matrices.Example: A=[2001]A = \begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix}A=[2001] , B=[0110]B = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}B=[0110] , x⃗=(1,1)\vec{x} = (1, 1)x=(1,1) : Ax⃗=(2,1)A\vec{x} = (2, 1)Ax=(2,1) , B(Ax⃗)=(1,2)B(A\vec{x}) = (1, 2)B(Ax)=(1,2) , and BA=[0120]BA = \begin{bmat…
[Matrix multiplication] Why is it no surprise that AB≠BAAB \neq BAAB=BA for matrices?
Matrices are linear transformations, and f∘g≠g∘ff \circ g \neq g \circ ff∘g=g∘f for functions in general. Example: f(x)=x+1f(x) = x + 1f(x)=x+1 , g(x)=2xg(x) = 2xg(x)=2x : (g∘f)(0)=2(g \circ f)(0) = 2(g∘f)(0)=2 , but (f∘g)(0)=1(f \circ g)(0) = 1(f∘g)(0)=1 .
[Matrix multiplication] What is the elementary matrix ERE_\mathcal{R}ER of a row operation R\mathcal{R}R ?
The matrix with R(A)=ERA\mathcal{R}(A) = E_\mathcal{R}AR(A)=ERA for every AAA : performing R\mathcal{R}R is multiplying by ERE_\mathcal{R}ER on the left. The three types of row operations give three types of elementary matrices.Example: for R1←R1+mR2R_1 \leftarrow R_1 + mR_2R1←R1+mR2 , [1m01][abcd]=[a+mcb+mdcd]\ \begin{bmatrix} 1 & m \\ 0 & 1 \end{bmatrix} \begin{bmatrix} a & b \\ c & d \end{bmatrix} = \begin{bmatrix} a + mc & b + md \\ c & d \end{bmatrix} [10m1][acbd]=[a+mccb+mdd] .
[Matrix multiplication] How do you find the elementary matrix ERE_\mathcal{R}ER of a row operation R\mathcal{R}R (the matrix with R(A)=ERA\mathcal{R}(A) = E_\mathcal{R}AR(A)=ERA for every AAA )?
Apply R\mathcal{R}R to the identity matrix: ER=R(I)E_\mathcal{R} = \mathcal{R}(I)ER=R(I) . R(I)=ER I=ER\mathcal{R}(I) = E_\mathcal{R}\,I = E_\mathcal{R}R(I)=ERI=ER .Example: swapping the two rows of III gives [0110]\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}[0110] , and [0110][abcd]=[cdab]\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} \begin{bmatrix} a & b \\ c & d \end{bmatrix} = \begin{bmatrix} c & d \\ a & b \end{bmatrix}[0110][acbd]=[cadb] .
[Matrix multiplication] How can you perform column operations on a matrix AAA with elementary matrices?
Multiply AAA by an elementary matrix on the right. Example: [abcd][1m01]=[ab+macd+mc]\begin{bmatrix} a & b \\ c & d \end{bmatrix} \begin{bmatrix} 1 & m \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} a & b + ma \\ c & d + mc \end{bmatrix}[acbd][10m1]=[acb+mad+mc] adds mmm times column 1 to column 2.
[Determinants] What are the inputs and outputs of the determinant operation, and how is it written?
Input: a square matrix; output: a number. det:Rn×n→R\det : \mathbb{R}^{n \times n} \to \mathbb{R}det:Rn×n→R , written det(A)\det(A)det(A) or ∣A∣|A|∣A∣ . Example: det[1234]=−2\det\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} = -2det[1324]=−2 .
[Determinants] What does det(A)\det(A)det(A) measure geometrically, for AAA of size 2×22 \times 22×2 , 3×33 \times 33×3 and d×dd \times dd×d with d>3d > 3d>3 ?
The volume of the shape with the rows of AAA as edges: area of a parallelogram, volume of a parallelepiped, ddd -dimensional hyper-volume. A parallelepiped is a box whose opposite faces are parallel but whose sides may be slanted.
[Determinants] T:R2→R2T : \mathbb{R}^2 \to \mathbb{R}^2T:R2→R2 is the linear transformation T(x⃗)=ATx⃗T(\vec{x}) = A_T\vec{x}T(x)=ATx . What does det(AT)\det(A_T)det(AT) say about TTT ?
det(AT)\det(A_T)det(AT) is the scale factor of TTT : TTT maps the unit square to a parallelogram of area ∣det(AT)∣|\det(A_T)|∣det(AT)∣ , and every area is multiplied by ∣det(AT)∣|\det(A_T)|∣det(AT)∣ . The unit square with sides (1,0)(1, 0)(1,0) , (0,1)(0, 1)(0,1) and its image with sides (a,b)(a, b)(a,b) , (c,d)(c, d)(c,d) .Example: AT=[2003]A_T = \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}AT=[2003] maps the unit square to a 2×32 \times 32×3 rectangle of area 6=det(AT)6 = \det(A_T)6=…
[Determinants] T(x⃗)=ATx⃗T(\vec{x}) = A_T\vec{x}T(x)=ATx is a linear transformation of R2\mathbb{R}^2R2 . Which values of det(AT)\det(A_T)det(AT) make TTT shrink areas, enlarge areas, or preserve areas?
Shrink: ∣det(AT)∣<1|\det(A_T)| < 1∣det(AT)∣<1 . Enlarge: ∣det(AT)∣>1|\det(A_T)| > 1∣det(AT)∣>1 . Area preserving: ∣det(AT)∣=1|\det(A_T)| = 1∣det(AT)∣=1 . Example: a rotation has det=1\det = 1det=1 ; [12001]\begin{bmatrix} \frac12 & 0 \\ 0 & 1 \end{bmatrix}[21001] has det=12\det = \frac12det=21 and halves every area.
[Determinants] What is the formula for the determinant of a 2×22 \times 22×2 matrix?
∣a11a12a21a22∣=a11a22−a12a21\boldsymbol{\begin{vmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{vmatrix} = a_{11}a_{22} - a_{12}a_{21}}∣∣a11a21a12a22∣∣=a11a22−a12a21 Example: ∣1234∣=1⋅4−2⋅3=−2\begin{vmatrix} 1 & 2 \\ 3 & 4 \end{vmatrix} = 1 \cdot 4 - 2 \cdot 3 = -2∣∣1324∣∣=1⋅4−2⋅3=−2 .
[Determinants] How is the determinant of a 3×33 \times 33×3 matrix A=[aij]A = [a_{ij}]A=[aij] defined in terms of 2×22 \times 22×2 determinants?
∣A∣=a11∣a22a23a32a33∣−a12∣a21a23a31a33∣+a13∣a21a22a31a32∣\boldsymbol{|A| = a_{11}\begin{vmatrix} a_{22} & a_{23} \\ a_{32} & a_{33} \end{vmatrix} - a_{12}\begin{vmatrix} a_{21} & a_{23} \\ a_{31} & a_{33} \end{vmatrix} + a_{13}\begin{vmatrix} a_{21} & a_{22} \\ a_{31} & a_{32} \end{vmatrix}}∣A∣=a11∣∣a22a32a23a33∣∣−a12∣∣a21a31a23a33∣∣+a13∣∣a21a31a22a32∣∣ Each entry of the first row times the 2×22 \times 22×2 determinant left after deleting that entry's row and column,…
[Determinants] How does the extended-array trick compute a 3×33 \times 33×3 determinant?
Copy columns 1, 2 as columns 4, 5; add the products on the three down-right diagonals, subtract those on the three down-left diagonals. Solid lines: positive diagonals. Dashed lines: negative diagonals.Example: for [111123121]\begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \\ 1 & 2 & 1 \end{bmatrix}⎣⎡111122131⎦⎤ :positive: 1⋅2⋅1+1⋅3⋅1+1⋅1⋅2=71 \cdot 2 \cdot 1 + 1 \cdot 3 \cdot 1 + 1 \cdot 1 \cdot 2 = 71⋅2⋅1+1⋅3⋅1+1⋅1⋅2=7 negative: 1⋅2⋅1+1⋅3⋅2+1⋅1⋅1=91 \cdot 2 \cdot 1 + 1 \cdot 3 \cdot 2 + 1 \cdot 1…
[Determinants] What is the minor MijM_{ij}Mij of the entry aija_{ij}aij of a square matrix AAA ?
The determinant of the submatrix obtained by removing row iii and column jjj of AAA . Example: for A=[a11a12a13a21a22a23a31a32a33]A = \begin{bmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{bmatrix}A=⎣⎡a11a21a31a12a22a32a13a23a33⎦⎤ , M12=∣a21a23a31a33∣=a21a33−a23a31\; M_{12} = \begin{vmatrix} a_{21} & a_{23} \\ a_{31} & a_{33} \end{vmatrix} = a_{21}a_{33} - a_{23}a_{31}M12=∣∣a21a31a23a33∣∣=a21a33−a23a31 .
[Determinants] What is the general formula for the determinant of an n×nn \times nn×n matrix AAA , expanding along the first row? ( M1jM_{1j}M1j = minor of a1ja_{1j}a1j )
det(A)=∑j=1n(−1)1+ja1jM1j\boldsymbol{\det(A) = \displaystyle\sum_{j=1}^{n} (-1)^{1+j} a_{1j} M_{1j}}det(A)=j=1∑n(−1)1+ja1jM1j The factor (−1)1+j(-1)^{1+j}(−1)1+j alternates between +1+1+1 and −1-1−1 .Example: for a 4×44 \times 44×4 matrix BBB : det(B)=b11M11−b12M12+b13M13−b14M14\det(B) = b_{11}M_{11} - b_{12}M_{12} + b_{13}M_{13} - b_{14}M_{14}det(B)=b11M11−b12M12+b13M13−b14M14 .
[Determinants] Must a determinant be expanded along the first row? What is the formula for expanding det(A)\det(A)det(A) along column jjj ? ( MijM_{ij}Mij = minor of aija_{ij}aij )
No, any row or column works: det(A)=∑i=1n(−1)i+jaijMij\boldsymbol{\det(A) = \displaystyle\sum_{i=1}^{n} (-1)^{i+j} a_{ij} M_{ij}}det(A)=i=1∑n(−1)i+jaijMij Example: along the second column of a 3×33 \times 33×3 matrix: det(A)=−a12M12+a22M22−a32M32\det(A) = -a_{12}M_{12} + a_{22}M_{22} - a_{32}M_{32}det(A)=−a12M12+a22M22−a32M32 .
[Determinants] Along which row or column is it best to expand a determinant, and why?
Along the row or column with the most zeros: each zero entry makes its term of the formula zero. Example: ∣200134512∣=2∣3412∣=2(6−4)=4\begin{vmatrix} 2 & 0 & 0 \\ 1 & 3 & 4 \\ 5 & 1 & 2 \end{vmatrix} = 2\begin{vmatrix} 3 & 4 \\ 1 & 2 \end{vmatrix} = 2(6 - 4) = 4∣∣215031042∣∣=2∣∣3142∣∣=2(6−4)=4 , expanding along row 1.
[Determinants] A square matrix has a row (or a column) consisting entirely of zeros. What is its determinant?
0\boldsymbol{0}0 Expanding along that row, every term contains a factor 0.Example: ∣123000134∣=0\begin{vmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \\ 1 & 3 & 4 \end{vmatrix} = 0∣∣101203304∣∣=0 .
[Determinants] What is the area of the parallelogram with corners (0,0)(0, 0)(0,0) , v⃗=(v1,v2)\vec{v} = (v_1, v_2)v=(v1,v2) , w⃗=(w1,w2)\vec{w} = (w_1, w_2)w=(w1,w2) and v⃗+w⃗\vec{v} + \vec{w}v+w ?
The determinant of the matrix with rows v⃗\vec{v}v and w⃗\vec{w}w (in absolute value): ∣v1v2w1w2∣=v1w2−v2w1\begin{vmatrix} v_1 & v_2 \\ w_1 & w_2 \end{vmatrix} = v_1w_2 - v_2w_1∣∣v1w1v2w2∣∣=v1w2−v2w1 Example: v⃗=(3,0)\vec{v} = (3, 0)v=(3,0) , w⃗=(1,2)\vec{w} = (1, 2)w=(1,2) : area =3⋅2−0⋅1=6= 3 \cdot 2 - 0 \cdot 1 = 6=3⋅2−0⋅1=6 (base 3, height 2).
[Determinants] What is the volume of the parallelepiped with edges u⃗=(u1,u2,u3)\vec{u} = (u_1, u_2, u_3)u=(u1,u2,u3) , v⃗=(v1,v2,v3)\vec{v} = (v_1, v_2, v_3)v=(v1,v2,v3) , w⃗=(w1,w2,w3)\vec{w} = (w_1, w_2, w_3)w=(w1,w2,w3) from the origin?
The determinant of the matrix with rows u⃗,v⃗,w⃗\vec{u}, \vec{v}, \vec{w}u,v,w (in absolute value): ∣u1u2u3v1v2v3w1w2w3∣\begin{vmatrix} u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \\ w_1 & w_2 & w_3 \end{vmatrix}∣∣u1v1w1u2v2w2u3v3w3∣∣ =u1(v2w3−v3w2)−u2(v1w3−v3w1)+u3(v1w2−v2w1)= u_1(v_2w_3 - v_3w_2) - u_2(v_1w_3 - v_3w_1) + u_3(v_1w_2 - v_2w_1)=u1(v2w3−v3w2)−u2(v1w3−v3w1)+u3(v1w2−v2w1) Example: u⃗=(1,0,0)\vec{u} = (1, 0, 0)u=(1,0,0) , v⃗=(0,2,0)\vec{v} = (0, 2, 0)v=(0,2,0) ,…
[Determinants] D=∣v1v2w1w2∣D = \begin{vmatrix} v_1 & v_2 \\ w_1 & w_2 \end{vmatrix}D=∣∣v1w1v2w2∣∣ for v⃗,w⃗∈R2\vec{v}, \vec{w} \in \mathbb{R}^2v,w∈R2 , and θ\thetaθ is the angle measured from v⃗\vec{v}v toward w⃗\vec{w}w (counterclockwise). How does the sign of DDD depend on θ\thetaθ ?
0<θ<π0 < \theta < \pi0<θ<π D>0D > 0D>0 Example: v⃗=(1,0)\vec{v} = (1, 0)v=(1,0) , w⃗=(0,1)\vec{w} = (0, 1)w=(0,1) : θ=π2\theta = \frac{\pi}{2}θ=2π , D=1D = 1D=1 . π<θ<2π\pi < \theta < 2\piπ<θ<2π D<0D < 0D<0 Example: v⃗=(1,0)\vec{v} = (1, 0)v=(1,0) , w⃗=(0,−1)\vec{w} = (0, -1)w=(0,−1) : θ=3π2\theta = \frac{3\pi}{2}θ=23π , D=−1D = -1D=−1 . θ=0\theta = 0θ=0 or θ=π\theta = \piθ=π D=0D = 0D=0 (the vectors point in the same or in opposite directions). Example: v⃗=(1,0)\vec{v} = (1, 0)v=(1,0) , w⃗=(…
[Determinants] v⃗,w⃗∈R2\vec{v}, \vec{w} \in \mathbb{R}^2v,w∈R2 and θ\thetaθ is the angle from v⃗\vec{v}v toward w⃗\vec{w}w . Express D=∣v1v2w1w2∣D = \begin{vmatrix} v_1 & v_2 \\ w_1 & w_2 \end{vmatrix}D=∣∣v1w1v2w2∣∣ using lengths and θ\thetaθ , and say why.
D=∥v⃗∥ ∥w⃗∥sinθ\boldsymbol{D = \|\vec{v}\| \, \|\vec{w}\| \sin\theta}D=∥v∥∥w∥sinθ Parallelogram area = base ×\times× height, with base ∥v⃗∥\|\vec{v}\|∥v∥ and height ∥w⃗∥sinθ\|\vec{w}\|\sin\theta∥w∥sinθ (negative for π<θ<2π\pi < \theta < 2\piπ<θ<2π ).Example: v⃗=(2,0)\vec{v} = (2, 0)v=(2,0) , w⃗=(1,1)\vec{w} = (1, 1)w=(1,1) , θ=π4\theta = \frac{\pi}{4}θ=4π : 2⋅2⋅22=2=2⋅1−0⋅12 \cdot \sqrt{2} \cdot \frac{\sqrt{2}}{2} = 2 = 2 \cdot 1 - 0 \cdot 12⋅2⋅22=2=2⋅1−0⋅1 .
[Determinants] What are the five algebraic properties of the determinant listed by Savov ( AAA , BBB square n×nn \times nn×n matrices)?
Product det(AB)=det(A)det(B)=det(BA)\det(AB) = \det(A)\det(B) = \det(BA)det(AB)=det(A)det(B)=det(BA) Example: det[2001]=2\det\begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix} = 2det[2001]=2 , det[1003]=3\det\begin{bmatrix} 1 & 0 \\ 0 & 3 \end{bmatrix} = 3det[1003]=3 , and their product [2003]\begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix}[2003] has det=6\det = 6det=6 . Inverse If det(A)≠0\det(A) \neq 0det(A)=0 , then AAA is invertible and det(A−1)=1det(A)\det(A^{-1}) = \frac{1}{\det(A)…
[Determinants] Why does det(αA)=αndet(A)\det(\alpha A) = \alpha^n \det(A)det(αA)=αndet(A) for an n×nn \times nn×n matrix AAA , and not αdet(A)\alpha \det(A)αdet(A) ?
αA\alpha AαA multiplies each of the nnn rows by α\alphaα , and each row multiplied by α\alphaα multiplies the determinant by α\alphaα . Example: doubling a 3×33 \times 33×3 matrix doubles all three edges of its parallelepiped, so the volume grows by 23=82^3 = 823=8 .
[Determinants] How does each of the three Gauss–Jordan row operations change the determinant of a matrix?
Add a multiple of one row to another row The determinant does not change. Example: ∣1234∣=−2\begin{vmatrix} 1 & 2 \\ 3 & 4 \end{vmatrix} = -2∣∣1324∣∣=−2 and, after R2←R2−3R1R_2 \leftarrow R_2 - 3R_1R2←R2−3R1 , ∣120−2∣=−2\begin{vmatrix} 1 & 2 \\ 0 & -2 \end{vmatrix} = -2∣∣102−2∣∣=−2 . Swap two rows The determinant changes sign. Example: ∣3412∣=6−4=2=−(−2)\begin{vmatrix} 3 & 4 \\ 1 & 2 \end{vmatrix} = 6 - 4 = 2 = -(-2)∣∣3142∣∣=6−4=2=−(−2) . Multiply a row by a constant α\alphaα The…
[Determinants] Why does adding a multiple of one row to another row, Ri←Ri+αRjR_i \leftarrow R_i + \alpha R_jRi←Ri+αRj , leave the determinant unchanged?
Cavalieri's principle: parallelepipeds with equal base between the same two parallel planes have equal volume. The row operation changes only the slant of the parallelepiped, not its base or height. In 2D: two parallelograms with base bbb between the same two parallel lines at distance hhh both have area b×hb \times hb×h .
[Determinants] Why does swapping two rows change the sign of the determinant?
The determinant measures signed volume, and a swap reverses the relative orientation of the row vectors. Example: rows (1,0),(0,1)(1, 0), (0, 1)(1,0),(0,1) give det=1\det = 1det=1 ; rows (0,1),(1,0)(0, 1), (1, 0)(0,1),(1,0) give det=−1\det = -1det=−1 .
[Determinants] Why does multiplying one row by α\alphaα multiply the determinant by α\alphaα ?
Making one side of the parallelepiped α\alphaα times longer makes its volume α\alphaα times larger.
[Determinants] What does det(A)=0\det(A) = 0det(A)=0 mean geometrically for a 3×33 \times 33×3 matrix AAA with rows u⃗,v⃗,w⃗\vec{u}, \vec{v}, \vec{w}u,v,w ?
At least one row is a linear combination of the others, so the parallelepiped is flattened into a parallelogram (volume 0). If det(A)≠0\det(A) \neq 0det(A)=0 , the rows are linearly independent and form a real parallelepiped.Example: rows (1,2,3)(1, 2, 3)(1,2,3) , (2,4,6)(2, 4, 6)(2,4,6) , (0,0,1)(0, 0, 1)(0,0,1) : the second row is twice the first, so det=0\det = 0det=0 .
[Determinants] Savov calls a square matrix deficient if its determinant is zero, and full if its determinant is nonzero.
[Determinants] Why can row operations never change whether det(A)\det(A)det(A) is zero or nonzero?
Each row operation multiplies the determinant by a nonzero number: 1 (add a multiple of a row), −1-1−1 (swap), or α≠0\alpha \neq 0α=0 (scale a row). So det(A)≠0\det(A) \neq 0det(A)=0 if and only if det(rref(A))≠0\det(\operatorname{rref}(A)) \neq 0det(rref(A))=0 .
[Determinants] What is the rank of a matrix?
The number of nonzero rows in its reduced row echelon form. Example: rref[1224]=[1200]\operatorname{rref}\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix} = \begin{bmatrix} 1 & 2 \\ 0 & 0 \end{bmatrix}rref[1224]=[1020] , so the rank is 1.
[Determinants] When does A∈Rn×nA \in \mathbb{R}^{n \times n}A∈Rn×n have full rank, and how does that relate to det(A)\det(A)det(A) ?
When rref(A)\operatorname{rref}(A)rref(A) has nnn pivots. Then det(A)≠0\det(A) \neq 0det(A)=0 and rref(A)=1 1\operatorname{rref}(A) = \mathrm{1\!\!1}rref(A)=11 ; if rref(A)\operatorname{rref}(A)rref(A) has a row of zeros, det(A)=0\det(A) = 0det(A)=0 . Example: ∣1239∣=3≠0\begin{vmatrix} 1 & 2 \\ 3 & 9 \end{vmatrix} = 3 \neq 0∣∣1329∣∣=3=0 and rref[1239]=1 1\operatorname{rref}\begin{bmatrix} 1 & 2 \\ 3 & 9 \end{bmatrix} = \mathrm{1\!\!1}rref[1329]=11 .
[Determinants] How is the cross product v⃗×w⃗\vec{v} \times \vec{w}v×w of v⃗=(v1,v2,v3)\vec{v} = (v_1, v_2, v_3)v=(v1,v2,v3) and w⃗=(w1,w2,w3)\vec{w} = (w_1, w_2, w_3)w=(w1,w2,w3) written as a determinant?
v⃗×w⃗=∣ı^ȷ^k^v1v2v3w1w2w3∣\boldsymbol{\vec{v} \times \vec{w} = \begin{vmatrix} \hat{\imath} & \hat{\jmath} & \hat{k} \\ v_1 & v_2 & v_3 \\ w_1 & w_2 & w_3 \end{vmatrix}}v×w=∣∣^v1w1^v2w2k^v3w3∣∣ , expanded along the first row. =(v2w3−v3w2, v3w1−v1w3, v1w2−v2w1)= (v_2w_3 - v_3w_2, \; v_3w_1 - v_1w_3, \; v_1w_2 - v_2w_1)=(v2w3−v3w2,v3w1−v1w3,v1w2−v2w1) . The extended-array trick also works here.Example: (1,2,3)×(4,5,6)=(12−15, 12−6, 5−8)=(−3,6,−3)(1, 2, 3) \times (4, 5, 6…
[Determinants] Which property of determinants corresponds to v⃗×w⃗=−w⃗×v⃗\vec{v} \times \vec{w} = -\vec{w} \times \vec{v}v×w=−w×v ?
Swapping two rows changes the sign of the determinant. w⃗×v⃗\vec{w} \times \vec{v}w×v is the determinant with rows ı^ȷ^k^\hat{\imath}\hat{\jmath}\hat{k}^^k^ , w⃗\vec{w}w , v⃗\vec{v}v : the rows of v⃗\vec{v}v and w⃗\vec{w}w are swapped.
[Determinants] Write the 3×33 \times 33×3 determinant with rows u⃗,v⃗,w⃗\vec{u}, \vec{v}, \vec{w}u,v,w using the dot product and the cross product.
∣u1u2u3v1v2v3w1w2w3∣=u⃗⋅(v⃗×w⃗)\boldsymbol{\begin{vmatrix} u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \\ w_1 & w_2 & w_3 \end{vmatrix} = \vec{u} \cdot (\vec{v} \times \vec{w})}∣∣u1v1w1u2v2w2u3v3w3∣∣=u⋅(v×w) Example: u⃗=(1,0,0)\vec{u} = (1, 0, 0)u=(1,0,0) , v⃗=(0,2,0)\vec{v} = (0, 2, 0)v=(0,2,0) , w⃗=(1,1,3)\vec{w} = (1, 1, 3)w=(1,1,3) : v⃗×w⃗=(6,0,−2)\vec{v} \times \vec{w} = (6, 0, -2)v×w=(6,0,−2) , so u⃗⋅(v⃗×w⃗)=6\vec{u} \cdot (\vec{v} \times \vec{w}) = 6u⋅(v×w)=6 , the determinant.
[Determinants] Cramer's rule. Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b with AAA square and det(A)≠0\det(A) \neq 0det(A)=0 . What is the component xjx_jxj of the solution?
xj=det(A with column j replaced by b⃗)det(A)\boldsymbol{x_j = \dfrac{\det(A \text{ with column } j \text{ replaced by } \vec{b})}{\det(A)}}xj=det(A)det(A with column j replaced by b) Example: x1+2x2=5x_1 + 2x_2 = 5x1+2x2=5 , 3x1+4x2=113x_1 + 4x_2 = 113x1+4x2=11 ( det(A)=−2\det(A) = -2det(A)=−2 ): x1=∣52114∣/(−2)=20−22−2=1x_1 = \begin{vmatrix} 5 & 2 \\ 11 & 4 \end{vmatrix} \big/ (-2) = \frac{20 - 22}{-2} = 1x1=∣∣51124∣∣/(−2)=−220−22=1 x2=∣15311∣/(−2)=11−15−2=2x_2 = \begin{vmatrix}…
[Determinants] When is Cramer's rule handier than Gauss–Jordan elimination for solving Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b ?
When you want only one component of x⃗\vec{x}x , without solving for the others. Cramer's rule gives xjx_jxj from just two determinants.
[Determinants] How does the determinant test check whether nnn vectors v⃗1,…,v⃗n∈Rn\vec{v}_1, \dots, \vec{v}_n \in \mathbb{R}^nv1,…,vn∈Rn are linearly independent?
Write them as the rows of a matrix MMM : they are independent if and only if det(M)≠0\det(M) \neq 0det(M)=0 . The slower alternative: compute rref(M)\operatorname{rref}(M)rref(M) ; row operations do not change whether the rows are independent.Example: (1,2),(2,4)(1, 2), (2, 4)(1,2),(2,4) : ∣1224∣=0\begin{vmatrix} 1 & 2 \\ 2 & 4 \end{vmatrix} = 0∣∣1224∣∣=0 , so dependent. (1,2),(3,4)(1, 2), (3, 4)(1,2),(3,4) : det=−2\det = -2det=−2 , so independent.
[Determinants] What is the characteristic polynomial pA(λ)p_A(\lambda)pA(λ) of a square matrix AAA , and what are its roots?
pA(λ)=det(A−λ1 1)\boldsymbol{p_A(\lambda) = \det(A - \lambda\mathrm{1\!\!1})}pA(λ)=det(A−λ11) ; its roots are the eigenvalues of AAA . A−λ1 1A - \lambda\mathrm{1\!\!1}A−λ11 is AAA with λ\lambdaλ subtracted from each diagonal entry.
[Determinants] For a 2×22 \times 22×2 matrix AAA , write pA(λ)=det(A−λ1 1)p_A(\lambda) = \det(A - \lambda\mathrm{1\!\!1})pA(λ)=det(A−λ11) using the trace Tr(A)=a11+a22\operatorname{Tr}(A) = a_{11} + a_{22}Tr(A)=a11+a22 and det(A)\det(A)det(A) .
pA(λ)=λ2−Tr(A) λ+det(A)\boldsymbol{p_A(\lambda) = \lambda^2 - \operatorname{Tr}(A)\,\lambda + \det(A)}pA(λ)=λ2−Tr(A)λ+det(A) (a11−λ)(a22−λ)−a12a21=λ2−(a11+a22)λ+(a11a22−a12a21)(a_{11} - \lambda)(a_{22} - \lambda) - a_{12}a_{21} = \lambda^2 - (a_{11} + a_{22})\lambda + (a_{11}a_{22} - a_{12}a_{21})(a11−λ)(a22−λ)−a12a21=λ2−(a11+a22)λ+(a11a22−a12a21) Example: A=[2112]A = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix}A=[2112] : pA(λ)=λ2−4λ+3=(λ−1)(λ−3)p_A(\lambda) = \lambda^2 - 4\lambd…
[Matrix inverse] Can different methods for computing A−1A^{-1}A−1 give different answers?
No, the inverse of a matrix is unique. So you can check a computed inverse by computing it a second way.
[Matrix inverse] How do you test whether A∈Rn×nA \in \mathbb{R}^{n \times n}A∈Rn×n is invertible?
A−1A^{-1}A−1 exists if and only if det(A)≠0\det(A) \neq 0det(A)=0 . The value of the determinant does not matter (large or small, positive or negative), only whether it is zero.Example: ∣1224∣=0\begin{vmatrix} 1 & 2 \\ 2 & 4 \end{vmatrix} = 0∣∣1224∣∣=0 , so this matrix has no inverse.
[Matrix inverse] What is the inverse of [abcd]\begin{bmatrix} a & b \\ c & d \end{bmatrix}[acbd] ?
1ad−bc[d−b−ca]\boldsymbol{\dfrac{1}{ad - bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}}ad−bc1[d−c−ba] (when ad−bc≠0ad - bc \neq 0ad−bc=0 ) Swap the diagonal entries, negate the off-diagonal entries, divide by the determinant.Example: [1239]−1=13[9−2−31]=[3−23−113]\begin{bmatrix} 1 & 2 \\ 3 & 9 \end{bmatrix}^{-1} = \frac13\begin{bmatrix} 9 & -2 \\ -3 & 1 \end{bmatrix} = \begin{bmatrix} 3 & -\frac23 \\ -1 & \frac13 \end{bmatrix}[1329]−1=31[9−3−21]=[3−1−3231]
[Matrix inverse] What is the sign sign(aij)\operatorname{sign}(a_{ij})sign(aij) of the entry aija_{ij}aij of a matrix, and what pattern does it give for a 3×33 \times 33×3 matrix?
sign(aij)=(−1)i+j\boldsymbol{\operatorname{sign}(a_{ij}) = (-1)^{i+j}}sign(aij)=(−1)i+j , a checkerboard starting with +++ : (+−+−+−+−+)\begin{pmatrix} + & - & + \\ - & + & - \\ + & - & + \end{pmatrix}⎝⎛+−+−+−+−+⎠⎞
[Matrix inverse] What is the cofactor cijc_{ij}cij of the entry aija_{ij}aij of a square matrix?
The sign of the entry times its minor: cij=(−1)i+jMij\boldsymbol{c_{ij} = (-1)^{i+j} M_{ij}}cij=(−1)i+jMij MijM_{ij}Mij is the determinant of the matrix left after removing row iii and column jjj .Example: for [1239]\begin{bmatrix} 1 & 2 \\ 3 & 9 \end{bmatrix}[1329] : c11=9c_{11} = 9c11=9 , c12=−3c_{12} = -3c12=−3 , c21=−2c_{21} = -2c21=−2 , c22=1c_{22} = 1c22=1 .
[Matrix inverse] Write det(A)\det(A)det(A) , expanded along the first row, using cofactors c1jc_{1j}c1j . Where does the name "cofactor" come from?
det(A)=∑j=1na1jc1j\boldsymbol{\det(A) = \displaystyle\sum_{j=1}^{n} a_{1j} c_{1j}}det(A)=j=1∑na1jc1j : c1jc_{1j}c1j is what multiplies the factor a1ja_{1j}a1j . Example: for [1239]\begin{bmatrix} 1 & 2 \\ 3 & 9 \end{bmatrix}[1329] : 1⋅9+2⋅(−3)=3=det1 \cdot 9 + 2 \cdot (-3) = 3 = \det1⋅9+2⋅(−3)=3=det .
[Matrix inverse] What is the adjugate matrix adj(A)\operatorname{adj}(A)adj(A) of a square matrix AAA ?
The transpose of the matrix of cofactors: adj(A)=CT\boldsymbol{\operatorname{adj}(A) = C^{\mathsf{T}}}adj(A)=CT , where CCC has the cofactor cijc_{ij}cij in place of each aija_{ij}aij . Example: A=[1239]A = \begin{bmatrix} 1 & 2 \\ 3 & 9 \end{bmatrix}A=[1329] : C=[9−3−21]C = \begin{bmatrix} 9 & -3 \\ -2 & 1 \end{bmatrix}C=[9−2−31] , adj(A)=[9−2−31]\operatorname{adj}(A) = \begin{bmatrix} 9 & -2 \\ -3 & 1 \end{bmatrix}adj(A)=[9−3−21] .
[Matrix inverse] What is the formula for A−1A^{-1}A−1 using the adjugate matrix, and when does the formula fail?
A−1=1det(A)adj(A)\boldsymbol{A^{-1} = \dfrac{1}{\det(A)}\operatorname{adj}(A)}A−1=det(A)1adj(A) ; it fails when det(A)=0\det(A) = 0det(A)=0 (division by zero). For 2×22 \times 22×2 matrices this is [abcd]−1=1ad−bc[d−b−ca]\begin{bmatrix} a & b \\ c & d \end{bmatrix}^{-1} = \frac{1}{ad - bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}[acbd]−1=ad−bc1[d−c−ba] .
[Matrix inverse] Compute A−1A^{-1}A−1 for A=[111123121]A = \begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 3 \\ 1 & 2 & 1 \end{bmatrix}A=⎣⎡111122131⎦⎤ ( det(A)=−2\det(A) = -2det(A)=−2 ) with the adjugate formula.
A−1=[2−12−12−101012−12]\boldsymbol{A^{-1} = \begin{bmatrix} 2 & -\frac12 & -\frac12 \\ -1 & 0 & 1 \\ 0 & \frac12 & -\frac12 \end{bmatrix}}A−1=⎣⎡2−10−21021−211−21⎦⎤ Cofactors: C=[−42010−11−21]C = \begin{bmatrix} -4 & 2 & 0 \\ 1 & 0 & -1 \\ 1 & -2 & 1 \end{bmatrix}C=⎣⎡−41120−20−11⎦⎤ , e.g. c11=2⋅1−3⋅2=−4c_{11} = 2 \cdot 1 - 3 \cdot 2 = -4c11=2⋅1−3⋅2=−4 , c12=−(1⋅1−3⋅1)=2c_{12} = -(1 \cdot 1 - 3 \cdot 1) = 2c12=−(1⋅1−3⋅1)=2 . adj(A)=CT=[−41120−20−11]\operatorname{adj}(A) = C^{\mat…
[Matrix inverse] How do you compute A−1A^{-1}A−1 with Gauss–Jordan elimination?
Form the n×2nn \times 2nn×2n array [ A∣1 1 ][\,A \mid \mathrm{1\!\!1}\,][A∣11] and run Gauss–Jordan elimination on it; the result is [ 1 1∣A−1 ][\,\mathrm{1\!\!1} \mid A^{-1}\,][11∣A−1] .
[Matrix inverse] To compute A−1A^{-1}A−1 : run Gauss–Jordan elimination on [ A∣1 1 ][\,A \mid \mathrm{1\!\!1}\,][A∣11] to get [ 1 1∣A−1 ][\,\mathrm{1\!\!1} \mid A^{-1}\,][11∣A−1] .Why does the right half end up equal to A−1A^{-1}A−1 ?
The row operations R1,…,Rk\mathcal{R}_1, \dots, \mathcal{R}_kR1,…,Rk turn AAA into 1 1\mathrm{1\!\!1}11 , so together they undo AAA , i.e. they act as A−1A^{-1}A−1 ; applied to 1 1\mathrm{1\!\!1}11 they give A−1A^{-1}A−1 . Rk(…R1(A)… )=1 1 ⇒ Rk(…R1(1 1)… )=A−1\mathcal{R}_k(\dots\mathcal{R}_1(A)\dots) = \mathrm{1\!\!1} \;\Rightarrow\; \mathcal{R}_k(\dots\mathcal{R}_1(\mathrm{1\!\!1})\dots) = A^{-1}Rk(…R1(A)…)=11⇒Rk(…R1(11)…)=A−1 . The right half records the combined effect of t…
[Matrix inverse] Compute the inverse of A=[1239]A = \begin{bmatrix} 1 & 2 \\ 3 & 9 \end{bmatrix}A=[1329] by Gauss–Jordan elimination on [ A∣1 1 ][\,A \mid \mathrm{1\!\!1}\,][A∣11] .
A−1=[3−23−113]\boldsymbol{A^{-1} = \begin{bmatrix} 3 & -\frac23 \\ -1 & \frac13 \end{bmatrix}}A−1=[3−1−3231] R2←R2−3R1R_2 \leftarrow R_2 - 3R_1R2←R2−3R1 : [121003−31]\left[\begin{array}{cc|cc} 1 & 2 & 1 & 0 \\ 0 & 3 & -3 & 1 \end{array}\right][10231−301] R2←13R2R_2 \leftarrow \frac13 R_2R2←31R2 : [121001−113]\left[\begin{array}{cc|cc} 1 & 2 & 1 & 0 \\ 0 & 1 & -1 & \frac13 \end{array}\right][10211−1031] R1←R1−2R2R_1 \leftarrow R_1 - 2R_2R1←R1−2R2 : [103−2301−113]\left[\begi…
[Matrix inverse] What is the elementary matrix ERE_{\mathcal{R}}ER of a row operation R\mathcal{R}R , and how do you find it?
The matrix with R(A)=ERA\mathcal{R}(A) = E_{\mathcal{R}}AR(A)=ERA for every AAA ; it is R\mathcal{R}R applied to the identity matrix 1 1\mathrm{1\!\!1}11 . Example: R2←R2−3R1R_2 \leftarrow R_2 - 3R_1R2←R2−3R1 has E=[10−31]E = \begin{bmatrix} 1 & 0 \\ -3 & 1 \end{bmatrix}E=[1−301] , and E[1239]=[1203]E\begin{bmatrix} 1 & 2 \\ 3 & 9 \end{bmatrix} = \begin{bmatrix} 1 & 2 \\ 0 & 3 \end{bmatrix}E[1329]=[1023] .
[Matrix inverse] What are the 2×22 \times 22×2 elementary matrices of the three kinds of row operations?
Add mmm times row 2 to row 1: R1←R1+mR2R_1 \leftarrow R_1 + mR_2R1←R1+mR2 [1m01]\begin{bmatrix} 1 & m \\ 0 & 1 \end{bmatrix}[10m1] Example: [1m01][abcd]=[a+mcb+mdcd]\begin{bmatrix} 1 & m \\ 0 & 1 \end{bmatrix}\begin{bmatrix} a & b \\ c & d \end{bmatrix} = \begin{bmatrix} a + mc & b + md \\ c & d \end{bmatrix}[10m1][acbd]=[a+mccb+mdd] Swap rows 1 and 2: R1↔R2R_1 \leftrightarrow R_2R1↔R2 [0110]\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}[0110] Example: [0110][abcd]=[cdab]\begin{bma…