Ch.6: Linear transformations

Chapter 6 of the book "No Bullshit Guide to Linear Algebra" by Ivan Savov Finding matrix representations Change of basis for matrices Invertible matrix theorem

73 flashcards · Mathematics · by lakesare

Part of Textbooks › No Bullshit Guide to Linear Algebra (Ivan Savov) · 10 courses

  1. [Linear transformations] What property makes a transformation T:V→WT : V \to WT:V→W linear?

    T(αv⃗1+βv⃗2)=αT(v⃗1)+βT(v⃗2)T(\alpha \vec{v}_1 + \beta \vec{v}_2) = \alpha T(\vec{v}_1) + \beta T(\vec{v}_2)T(αv1​+βv2​)=αT(v1​)+βT(v2​) for all inputs v⃗1,v⃗2∈V\vec{v}_1, \vec{v}_2 \in Vv1​,v2​∈V and all constants α,β\alpha, \betaα,β . A linear combination of inputs gives the same linear combination of outputs. With w⃗1=T(v⃗1)\vec{w}_1 = T(\vec{v}_1)w1​=T(v1​) and w⃗2=T(v⃗2)\vec{w}_2 = T(\vec{v}_2)w2​=T(v2​) , the input 14v⃗1+34v⃗2\frac{1}{4}\vec{v}_1 + \frac{3}{4}\vec{v}_241​v1​+43​v2​ gives…

  2. [Linear transformations] What does the word linear in "linear algebra" refer to, according to Savov?

    Transformations that map a linear combination of inputs to the same linear combination of outputs, not lines. Savov calls this perhaps the most important idea in linear algebra.

  3. [Linear transformations] T:V→WT : V \to WT:V→W is a linear transformation. What is its image space Im⁡(T)\operatorname{Im}(T)Im(T) ?

    The set of all outputs of TTT : Im⁡(T)={w⃗∈W∣w⃗=T(v⃗) for some v⃗∈V}⊆W\operatorname{Im}(T) = \{ \vec{w} \in W \mid \vec{w} = T(\vec{v}) \text{ for some } \vec{v} \in V \} \subseteq WIm(T)={w∈W∣w=T(v) for some v∈V}⊆W . The vector version of the image set of a function f:R→Rf : \mathbb{R} \to \mathbb{R}f:R→R .Example: T(x,y)=(x,0)T(x, y) = (x, 0)T(x,y)=(x,0) has Im⁡(T)={(x,0)∣x∈R}\operatorname{Im}(T) = \{(x, 0) \mid x \in \mathbb{R}\}Im(T)={(x,0)∣x∈R} , the xxx -axis.

  4. [Linear transformations] T:V→WT : V \to WT:V→W is a linear transformation. What is its kernel Ker⁡(T)\operatorname{Ker}(T)Ker(T) ?

    The set of inputs that TTT maps to the zero vector: Ker⁡(T)={v⃗∈V∣T(v⃗)=0⃗}⊆V\operatorname{Ker}(T) = \{ \vec{v} \in V \mid T(\vec{v}) = \vec{0} \} \subseteq VKer(T)={v∈V∣T(v)=0}⊆V . The vector version of the roots {x∈R∣f(x)=0}\{x \in \mathbb{R} \mid f(x) = 0\}{x∈R∣f(x)=0} of a function. Ker⁡(T)\operatorname{Ker}(T)Ker(T) is a subset of the input space VVV ; Im⁡(T)\operatorname{Im}(T)Im(T) is a subset of the output space WWW .Example: T(x,y)=(x,0)T(x, y) = (x, 0)T(x,y)=(x,0) has Ker⁡(T)={(0,y)∣y…

  5. [Linear transformations] T:R2→R3T : \mathbb{R}^2 \to \mathbb{R}^3T:R2→R3 is defined by T(x,y)=(x,y,x+y)T(x, y) = (x, y, x + y)T(x,y)=(x,y,x+y) . Find T(3,4)T(3, 4)T(3,4) , Ker⁡(T)\operatorname{Ker}(T)Ker(T) and Im⁡(T)\operatorname{Im}(T)Im(T) .

    T(3,4)=T(3, 4) = T(3,4)= (3,4,7)(3, 4, 7)(3,4,7) , Ker⁡(T)=\ \operatorname{Ker}(T) = Ker(T)= {0⃗}\{\vec{0}\}{0} , Im⁡(T)=\ \operatorname{Im}(T) = Im(T)= span⁡{(1,0,1),(0,1,1)}\operatorname{span}\{(1, 0, 1), (0, 1, 1)\}span{(1,0,1),(0,1,1)} T(x,y)=0⃗T(x, y) = \vec{0}T(x,y)=0 forces x=y=0x = y = 0x=y=0 . Every output is x(1,0,1)+y(0,1,1)x(1, 0, 1) + y(0, 1, 1)x(1,0,1)+y(0,1,1) : a two-dimensional subspace (a plane) of R3\mathbb{R}^3R3 .

  6. [Linear transformations] A linear transformation T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm is a "black box": you can only feed it inputs and observe the outputs. Which inputs are enough to predict every output, and how?

    The nnn vectors e⃗1,…,e⃗n\vec{e}_1, \dots, \vec{e}_ne1​,…,en​ of a basis of Rn\mathbb{R}^nRn : then T(v⃗)=v1T(e⃗1)+⋯+vnT(e⃗n)T(\vec{v}) = v_1 T(\vec{e}_1) + \dots + v_n T(\vec{e}_n)T(v)=v1​T(e1​)+⋯+vn​T(en​) for v⃗=v1e⃗1+⋯+vne⃗n\vec{v} = v_1 \vec{e}_1 + \dots + v_n \vec{e}_nv=v1​e1​+⋯+vn​en​ . This follows from linearity. A non-linear transformation needs far more probing.Example: if T(1,0)=(2,1)T(1, 0) = (2, 1)T(1,0)=(2,1) and T(0,1)=(0,3)T(0, 1) = (0, 3)T(0,1)=(0,3) , then T(4,5)=4(2,1)+5(0,3…

  7. [Linear transformations] For a linear transformation T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm , what are the domain and the codomain?

    Domain: Rn\mathbb{R}^nRn (the input space). Codomain: Rm\mathbb{R}^mRm (the output space). The image space Im⁡(T)\operatorname{Im}(T)Im(T) is a subset of the codomain, and may be smaller.

  8. [Linear transformations] When is a linear transformation T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm surjective (onto)?

    When Im⁡(T)=Rm\operatorname{Im}(T) = \mathbb{R}^mIm(T)=Rm : the image space equals the codomain. Example: T(x,y,z)=(x,z)T(x, y, z) = (x, z)T(x,y,z)=(x,z) is surjective, since every (a,b)∈R2(a, b) \in \mathbb{R}^2(a,b)∈R2 is T(a,0,b)T(a, 0, b)T(a,0,b) .

  9. [Linear transformations] When is a linear transformation TTT injective, and what is the condition in terms of the kernel?

    TTT maps different inputs to different outputs, which for a linear TTT holds exactly when Ker⁡(T)={0⃗}\operatorname{Ker}(T) = \{\vec{0}\}Ker(T)={0} . If T(v⃗1)=T(v⃗2)T(\vec{v}_1) = T(\vec{v}_2)T(v1​)=T(v2​) , then T(v⃗1−v⃗2)=0⃗T(\vec{v}_1 - \vec{v}_2) = \vec{0}T(v1​−v2​)=0 , so v⃗1−v⃗2∈Ker⁡(T)\vec{v}_1 - \vec{v}_2 \in \operatorname{Ker}(T)v1​−v2​∈Ker(T) .Example: T(x,y)=(x,y,x+y)T(x, y) = (x, y, x + y)T(x,y)=(x,y,x+y) has Ker⁡(T)={0⃗}\operatorname{Ker}(T) = \{\vec{0}\}Ker(T)={0} , so it is inje…

  10. [Linear transformations] When is a linear transformation bijective?

    When it is both injective and surjective. Then TTT is a one-to-one correspondence between the input space and the output space.Example: T(x,y)=(y,x)T(x, y) = (y, x)T(x,y)=(y,x) on R2\mathbb{R}^2R2 is bijective.

  11. [Linear transformations] T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm is a bijective linear transformation. What follows about nnn and mmm , and why?

    n=mn = mn=m . Surjective: the outputs fill Rm\mathbb{R}^mRm , so n≥mn \ge mn≥m . Injective: different inputs give different outputs, so m≥nm \ge nm≥n .

  12. [Linear transformations] T:R3→R2T : \mathbb{R}^3 \to \mathbb{R}^2T:R3→R2 is defined by T(x,y,z)=(x,z)T(x, y, z) = (x, z)T(x,y,z)=(x,z) . Find Ker⁡(T)\operatorname{Ker}(T)Ker(T) and Im⁡(T)\operatorname{Im}(T)Im(T) . Is TTT injective? Surjective?

    Ker⁡(T)=\operatorname{Ker}(T) = Ker(T)= span⁡{(0,1,0)}\operatorname{span}\{(0, 1, 0)\}span{(0,1,0)} , Im⁡(T)=\ \operatorname{Im}(T) = Im(T)= R2\mathbb{R}^2R2 . Not injective, surjective. TTT deletes the yyy -component. T(0,1,0)=T(0,2,0)=(0,0)T(0, 1, 0) = T(0, 2, 0) = (0, 0)T(0,1,0)=T(0,2,0)=(0,0) although (0,1,0)≠(0,2,0)(0, 1, 0) \ne (0, 2, 0)(0,1,0)=(0,2,0) .

  13. [Linear transformations] How do you find the matrix MTM_TMT​ of a linear transformation T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm (with respect to the standard bases)?

    Probe TTT with the standard basis vectors e^1,…,e^n\hat{e}_1, \dots, \hat{e}_ne^1​,…,e^n​ and use the outputs as columns: MT=[T(e^1)T(e^2)⋯T(e^n)]M_T = \begin{bmatrix} T(\hat{e}_1) & T(\hat{e}_2) & \cdots & T(\hat{e}_n) \end{bmatrix}MT​=[T(e^1​)​T(e^2​)​⋯​T(e^n​)​] . MTM_TMT​ is m×nm \times nm×n : nnn probe vectors, each output an mmm -vector. Then T(v⃗)=MTv⃗T(\vec{v}) = M_T \vec{v}T(v)=MT​v .Example: T(x,y)=(x,y,x+y)T(x, y) = (x, y, x + y)T(x,y)=(x,y,x+y) has T(1,0)=(1,0,1)T(1, 0) = (1, 0, 1)T…

  14. [Linear transformations] The matrix of a linear transformation T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm is MT=[T(e^1)⋯T(e^n)]M_T = \begin{bmatrix} T(\hat{e}_1) & \cdots & T(\hat{e}_n) \end{bmatrix}MT​=[T(e^1​)​⋯​T(e^n​)​] , with the outputs for the standard basis vectors as columns.Why does MTv⃗=T(v⃗)M_T \vec{v} = T(\vec{v})MT​v=T(v) hold for every v⃗=(v1,…,vn)\vec{v} = (v_1, \dots, v_n)v=(v1​,…,vn​) ?

    MTv⃗=v1T(e^1)+⋯+vnT(e^n)M_T \vec{v} = v_1 T(\hat{e}_1) + \dots + v_n T(\hat{e}_n)MT​v=v1​T(e^1​)+⋯+vn​T(e^n​) (column picture), and by linearity this is T(v1e^1+⋯+vne^n)=T(v⃗)T(v_1 \hat{e}_1 + \dots + v_n \hat{e}_n) = T(\vec{v})T(v1​e^1​+⋯+vn​e^n​)=T(v) . In particular MTe^jM_T \hat{e}_jMT​e^j​ selects column jjj , which is T(e^j)T(\hat{e}_j)T(e^j​) .

  15. [Linear transformations] ΠPxy\Pi_{P_{xy}}ΠPxy​​ is the orthogonal projection onto the xyxyxy -plane in R3\mathbb{R}^3R3 . What is its matrix?

    [100010000]\boldsymbol{\begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{bmatrix}}⎣⎡​100​010​000​⎦⎤​ The projection sets the zzz -component to zero: (vx,vy,vz)↦(vx,vy,0)(v_x, v_y, v_z) \mapsto (v_x, v_y, 0)(vx​,vy​,vz​)↦(vx​,vy​,0) .

  16. [Linear transformations] MTM_TMT​ is the matrix of a linear transformation TTT . Which subspaces of MTM_TMT​ are Im⁡(T)\operatorname{Im}(T)Im(T) and Ker⁡(T)\operatorname{Ker}(T)Ker(T) ?

    Im⁡(T)=C(MT)\operatorname{Im}(T) = \mathcal{C}(M_T)Im(T)=C(MT​) , the column space; Ker⁡(T)=N(MT)\operatorname{Ker}(T) = \mathcal{N}(M_T)Ker(T)=N(MT​) , the null space. Every output MTv⃗M_T \vec{v}MT​v is a linear combination of the columns of MTM_TMT​ ; and T(v⃗)=0⃗T(\vec{v}) = \vec{0}T(v)=0 means MTv⃗=0⃗M_T \vec{v} = \vec{0}MT​v=0 .Example: MT=[100001]M_T = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix}MT​=[10​00​01​] (for T(x,y,z)=(x,z)T(x, y, z) = (x, z)T(x,y,z)=(x,z) ) has C(MT)=R2\m…

  17. [Linear transformations] T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm has matrix MTM_TMT​ . How does the input space Rn\mathbb{R}^nRn split into two subspaces of MTM_TMT​ , and what dimension formula follows?

    Rn=R(MT)⊕N(MT)\mathbb{R}^n = \mathcal{R}(M_T) \oplus \mathcal{N}(M_T)Rn=R(MT​)⊕N(MT​) , an orthogonal sum of the row space and the null space, so n=dim⁡R(MT)+dim⁡N(MT)n = \dim \mathcal{R}(M_T) + \dim \mathcal{N}(M_T)n=dimR(MT​)+dimN(MT​) . Every vector of N(MT)\mathcal{N}(M_T)N(MT​) has dot product 0 with each row of MTM_TMT​ , so it is orthogonal to the row space.Example: MT=[100001]M_T = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix}MT​=[10​00​01​] : row space span⁡{(1,0,0),(0,0,1)}\ope…

  18. [Linear transformations] T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm has matrix MTM_TMT​ . What does TTT do to the vectors of the row space R(MT)\mathcal{R}(M_T)R(MT​) , and to those of the null space N(MT)\mathcal{N}(M_T)N(MT​) ?

    TTT maps R(MT)\mathcal{R}(M_T)R(MT​) one-to-one onto C(MT)\mathcal{C}(M_T)C(MT​) , and maps every vector of N(MT)\mathcal{N}(M_T)N(MT​) to 0⃗\vec{0}0 . T:R(MT)→C(MT)T : \mathcal{R}(M_T) \to \mathcal{C}(M_T)T:R(MT​)→C(MT​) , T:N(MT)→{0⃗}\ T : \mathcal{N}(M_T) \to \{\vec{0}\} T:N(MT​)→{0}

  19. [Linear transformations] Linear transformations TTT and SSS have matrices MTM_TMT​ and MSM_SMS​ . What is the matrix of S∘TS \circ TS∘T (first TTT , then SSS ), and when is the composition defined?

    MSMTM_S M_TMS​MT​ , defined when the output space of TTT has the dimension of the input space of SSS (the inner dimensions of MSMTM_S M_TMS​MT​ agree). S∘T(v⃗)=S(T(v⃗))=MSMTv⃗S \circ T(\vec{v}) = S(T(\vec{v})) = M_S M_T \vec{v}S∘T(v)=S(T(v))=MS​MT​v : MTM_TMT​ multiplies v⃗\vec{v}v first.Example: T:R3→R2T : \mathbb{R}^3 \to \mathbb{R}^2T:R3→R2 and S:R2→R4S : \mathbb{R}^2 \to \mathbb{R}^4S:R2→R4 give MSMTM_S M_TMS​MT​ of size (4×2)(2×3)=4×3(4 \times 2)(2 \times 3) = 4 \times 3(4×2)(2×3)=4×3 .

  20. [Linear transformations] Why is it wrong to say that a linear transformation is a matrix?

    The entries of the matrix depend on the bases chosen for the input and the output space; other bases give a different matrix for the same TTT . Precise notation: BW[MT]BV_{B_W}[M_T]_{B_V}BW​​[MT​]BV​​ for input basis BVB_VBV​ and output basis BWB_WBW​ . Without bases written, the standard basis BsB_sBs​ is meant: Av⃗A\vec{v}Av means Bs[A]Bs[v⃗]Bs_{B_s}[A]_{B_s} [\vec{v}]_{B_s}Bs​​[A]Bs​​[v]Bs​​ .

  21. [Linear transformations] When is a linear transformation TTT invertible, and which matrices correspond to invertible linear transformations?

    When an inverse transformation T−1T^{-1}T−1 exists with T−1(T(v⃗))=v⃗T^{-1}(T(\vec{v})) = \vec{v}T−1(T(v))=v ; these correspond exactly to the invertible matrices MTM_TMT​ . Example: M=[2001]M = \begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix}M=[20​01​] stretches the xxx -direction by 2; M−1=[12001]M^{-1} = \begin{bmatrix} \frac{1}{2} & 0 \\ 0 & 1 \end{bmatrix}M−1=[21​0​01​] shrinks it by 2.

  22. [Linear transformations] What is an affine transformation A:Rn→RmA : \mathbb{R}^n \to \mathbb{R}^mA:Rn→Rm ?

    A linear transformation followed by a translation by a fixed vector b⃗\vec{b}b : A(x⃗)=MTx⃗+b⃗A(\vec{x}) = M_T \vec{x} + \vec{b}A(x)=MT​x+b . The vector version of f(x)=mx+bf(x) = mx + bf(x)=mx+b . For b⃗≠0⃗\vec{b} \ne \vec{0}b=0 it is not linear, since A(0⃗)=b⃗≠0⃗A(\vec{0}) = \vec{b} \ne \vec{0}A(0)=b=0 .Example: MT=[2001]M_T = \begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix}MT​=[20​01​] , b⃗=(1,1)\vec{b} = (1, 1)b=(1,1) : A(3,4)=(6,4)+(1,1)=(7,5)A(3, 4) = (6, 4) + (1, 1) = (7, 5)A(3,4)=(6,4)+(1…

  23. [Linear transformations] Why can every linear transformation T:R3→R2T : \mathbb{R}^3 \to \mathbb{R}^2T:R3→R2 be written as a matrix product, T(v⃗)=MTv⃗T(\vec{v}) = M_T \vec{v}T(v)=MT​v ?

    By linearity each output coefficient is a linear combination of the input coefficients: w1=α1v1+α2v2+α3v3w_1 = \alpha_1 v_1 + \alpha_2 v_2 + \alpha_3 v_3w1​=α1​v1​+α2​v2​+α3​v3​ , w2=β1v1+β2v2+β3v3\ w_2 = \beta_1 v_1 + \beta_2 v_2 + \beta_3 v_3 w2​=β1​v1​+β2​v2​+β3​v3​ . That is exactly [α1α2α3β1β2β3]v⃗\begin{bmatrix} \alpha_1 & \alpha_2 & \alpha_3 \\ \beta_1 & \beta_2 & \beta_3 \end{bmatrix} \vec{v}[α1​β1​​α2​β2​​α3​β3​​]v . Savov: matrix multiplication is defined as rows-dot-columns precisely…

  24. [Finding matrix representations] How does an image editor rotate a black-and-white image stored as the list of coordinates x⃗1,…,x⃗ℓ\vec{x}_1, \dots, \vec{x}_\ellx1​,…,xℓ​ of its black pixels?

    It multiplies every pixel coordinate by the rotation matrix: y⃗i=MTx⃗i\vec{y}_i = M_T \vec{x}_iy​i​=MT​xi​ . The list y⃗1,…,y⃗ℓ\vec{y}_1, \dots, \vec{y}_\elly​1​,…,y​ℓ​ is the rotated image. Once MTM_TMT​ is known, applying TTT to many vectors is cheap.

  25. [Finding matrix representations] What are the matrices of the projections Πx\Pi_xΠx​ onto the xxx -axis and Πy\Pi_yΠy​ onto the yyy -axis in R2\mathbb{R}^2R2 ?

    MΠx=[1000],MΠy=[0001]\boldsymbol{M_{\Pi_x} = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}}, \quad \boldsymbol{M_{\Pi_y} = \begin{bmatrix} 0 & 0 \\ 0 & 1 \end{bmatrix}}MΠx​​=[10​00​],MΠy​​=[00​01​] Probing: Πx(1,0)=(1,0)\Pi_x(1, 0) = (1, 0)Πx​(1,0)=(1,0) and Πx(0,1)=(0,0)\Pi_x(0, 1) = (0, 0)Πx​(0,1)=(0,0) are the columns of MΠxM_{\Pi_x}MΠx​​ . Πx\Pi_xΠx​ keeps the xxx -coordinate and sets the yyy -coordinate to zero.

  26. [Finding matrix representations] What is the formula for the projection Πa⃗(v⃗)\Pi_{\vec{a}}(\vec{v})Πa​(v) of a vector v⃗\vec{v}v onto the line in the direction a⃗\vec{a}a , and how do you get its matrix?

    Πa⃗(v⃗)=(a⃗⋅v⃗∥a⃗∥2)a⃗\boldsymbol{\Pi_{\vec{a}}(\vec{v}) = \left( \frac{\vec{a} \cdot \vec{v}}{\|\vec{a}\|^2} \right) \vec{a}}Πa​(v)=(∥a∥2a⋅v​)a ; the matrix has the columns Πa⃗(1,0)\Pi_{\vec{a}}(1, 0)Πa​(1,0) and Πa⃗(0,1)\Pi_{\vec{a}}(0, 1)Πa​(0,1) . Example: a⃗=(1,2)\vec{a} = (1, 2)a=(1,2) : Πa⃗(1,0)=15(1,2)\Pi_{\vec{a}}(1, 0) = \frac{1}{5}(1, 2)Πa​(1,0)=51​(1,2) and Πa⃗(0,1)=25(1,2)\Pi_{\vec{a}}(0, 1) = \frac{2}{5}(1, 2)Πa​(0,1)=52​(1,2) , so MΠa⃗=15[1224]M_{\Pi_{\vec{a}}} = \frac{1}{5}\begi…

  27. [Finding matrix representations] How do you compute the projection ΠP(v⃗)\Pi_P(\vec{v})ΠP​(v) of v⃗∈R3\vec{v} \in \mathbb{R}^3v∈R3 onto the plane P:n⃗⋅x⃗=0P : \vec{n} \cdot \vec{x} = 0P:n⋅x=0 , and why does it work?

    ΠP(v⃗)=v⃗−Πn⃗(v⃗)\boldsymbol{\Pi_P(\vec{v}) = \vec{v} - \Pi_{\vec{n}}(\vec{v})}ΠP​(v)=v−Πn​(v) Πn⃗(v⃗)\Pi_{\vec{n}}(\vec{v})Πn​(v) is the part of v⃗\vec{v}v perpendicular to the plane (in the direction n⃗\vec{n}n ); subtracting it leaves the part that lies in the plane.Example: P:z=0P : z = 0P:z=0 , n⃗=(0,0,1)\vec{n} = (0, 0, 1)n=(0,0,1) , v⃗=(1,2,3)\vec{v} = (1, 2, 3)v=(1,2,3) : ΠP(v⃗)=(1,2,3)−(0,0,3)=(1,2,0)\Pi_P(\vec{v}) = (1, 2, 3) - (0, 0, 3) = (1, 2, 0)ΠP​(v)=(1,2,3)−(0,0,3)=(1,2,0) .

  28. [Finding matrix representations] For column vectors u⃗,v⃗∈Rn\vec{u}, \vec{v} \in \mathbb{R}^nu,v∈Rn , what are the inner product and the outer product as matrix products, and what are their sizes?

    Inner: u⃗Tv⃗\vec{u}^{\mathsf{T}} \vec{v}uTv , a 1×11 \times 11×1 matrix (a number). Outer: u⃗v⃗T\vec{u} \vec{v}^{\mathsf{T}}uvT , an n×nn \times nn×n matrix. Example: u⃗=(1,2)\vec{u} = (1, 2)u=(1,2) , v⃗=(3,4)\vec{v} = (3, 4)v=(3,4) : u⃗Tv⃗=11\vec{u}^{\mathsf{T}} \vec{v} = 11uTv=11 , u⃗v⃗T=[3468]\ \vec{u} \vec{v}^{\mathsf{T}} = \begin{bmatrix} 3 & 4 \\ 6 & 8 \end{bmatrix} uvT=[36​48​] .

  29. [Finding matrix representations] How is the matrix of the projection onto the line in the direction a⃗\vec{a}a written as an outer product?

    MΠa⃗=a^a^T\boldsymbol{M_{\Pi_{\vec{a}}} = \hat{a} \hat{a}^{\mathsf{T}}}MΠa​​=a^a^T , where a^=a⃗∥a⃗∥\hat{a} = \frac{\vec{a}}{\|\vec{a}\|}a^=∥a∥a​ is the unit vector in the direction a⃗\vec{a}a . Example: a⃗=(1,2)\vec{a} = (1, 2)a=(1,2) , a^=15(1,2)\hat{a} = \frac{1}{\sqrt{5}}(1, 2)a^=5​1​(1,2) : a^a^T=15[1224]\hat{a}\hat{a}^{\mathsf{T}} = \frac{1}{5}\begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}a^a^T=51​[12​24​] .

  30. [Finding matrix representations] Why is the projection Πa^(v⃗)=(a^⋅v⃗) a^\Pi_{\hat{a}}(\vec{v}) = (\hat{a} \cdot \vec{v})\, \hat{a}Πa^​(v)=(a^⋅v)a^ onto a unit vector a^\hat{a}a^ equal to (a^a^T)v⃗(\hat{a} \hat{a}^{\mathsf{T}}) \vec{v}(a^a^T)v ?

    (a^⋅v⃗) a^=a^ (a^Tv⃗)=(a^a^T) v⃗(\hat{a} \cdot \vec{v})\, \hat{a} = \hat{a}\, (\hat{a}^{\mathsf{T}} \vec{v}) = (\hat{a} \hat{a}^{\mathsf{T}})\, \vec{v}(a^⋅v)a^=a^(a^Tv)=(a^a^T)v Steps: a scalar may be written on either side of a vector; the dot product is the matrix product a^Tv⃗\hat{a}^{\mathsf{T}} \vec{v}a^Tv ; matrix multiplication is associative.

  31. [Finding matrix representations] Find the matrix MdM_dMd​ of the projection Πd\Pi_dΠd​ onto the line y=xy = xy=x in R2\mathbb{R}^2R2 .

    Md=[12121212]\boldsymbol{M_d = \begin{bmatrix} \frac{1}{2} & \frac{1}{2} \\ \frac{1}{2} & \frac{1}{2} \end{bmatrix}}Md​=[21​21​​21​21​​] Direction a⃗=(1,1)\vec{a} = (1, 1)a=(1,1) , a^=(12,12)\hat{a} = (\frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}})a^=(2​1​,2​1​) , and a^a^T\hat{a}\hat{a}^{\mathsf{T}}a^a^T has every entry 12\frac{1}{2}21​ .Probing gives the same: Πd(1,0)=(1,0)⋅(1,1)2(1,1)=(12,12)\Pi_d(1, 0) = \frac{(1, 0) \cdot (1, 1)}{2}(1, 1) = (\frac{1}{2}, \frac{1}{2})Πd​(1,0)=2(1,0)⋅(1,1)​(1,1)=(2…

  32. [Finding matrix representations] A projection matrix MΠM_\PiMΠ​ is idempotent. What equation does that mean, and what does it say?

    MΠMΠ=MΠ\boldsymbol{M_\Pi M_\Pi = M_\Pi}MΠ​MΠ​=MΠ​ : projecting a second time changes nothing. One of the defining properties of projections.Example: [1000][1000]=[1000]\begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}\begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}[10​00​][10​00​]=[10​00​] .

  33. [Finding matrix representations] ΠS\Pi_SΠS​ is the projection onto a subspace SSS of Rn\mathbb{R}^nRn , and S⊥={w⃗∈Rn∣w⃗⋅s⃗=0 for all s⃗∈S}S^\perp = \{\vec{w} \in \mathbb{R}^n \mid \vec{w} \cdot \vec{s} = 0 \text{ for all } \vec{s} \in S\}S⊥={w∈Rn∣w⋅s=0 for all s∈S} . What does ΠS\Pi_SΠS​ do to vectors of SSS and of S⊥S^\perpS⊥ ?

    Every v⃗∈S\vec{v} \in Sv∈S stays unchanged: ΠS(v⃗)=v⃗\Pi_S(\vec{v}) = \vec{v}ΠS​(v)=v . Every w⃗∈S⊥\vec{w} \in S^\perpw∈S⊥ goes to 0⃗\vec{0}0 : ΠS(w⃗)=0⃗\Pi_S(\vec{w}) = \vec{0}ΠS​(w)=0 . So S⊥S^\perpS⊥ is the null space of the matrix MΠSM_{\Pi_S}MΠS​​ .Example: SSS = the xxx -axis in R2\mathbb{R}^2R2 : ΠS(3,0)=(3,0)\Pi_S(3, 0) = (3, 0)ΠS​(3,0)=(3,0) , ΠS(0,5)=(0,0)\ \Pi_S(0, 5) = (0, 0) ΠS​(0,5)=(0,0) .

  34. [Finding matrix representations] What is the matrix of the reflection RxR_xRx​ through the xxx -axis in R2\mathbb{R}^2R2 ?

    MRx=[100−1]\boldsymbol{M_{R_x} = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}}MRx​​=[10​0−1​] RxR_xRx​ keeps the xxx -coordinate and changes the sign of the yyy -coordinate: (x,y)↦(x,−y)(x, y) \mapsto (x, -y)(x,y)↦(x,−y) .

  35. [Finding matrix representations] What is the matrix of the reflection RyR_yRy​ through the yyy -axis in R2\mathbb{R}^2R2 ?

    MRy=[−1001]\boldsymbol{M_{R_y} = \begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix}}MRy​​=[−10​01​] RyR_yRy​ changes the sign of the xxx -coordinate and keeps the yyy -coordinate: (x,y)↦(−x,y)(x, y) \mapsto (-x, y)(x,y)↦(−x,y) .

  36. [Finding matrix representations] What is the matrix of the reflection RdR_dRd​ through the line y=xy = xy=x in R2\mathbb{R}^2R2 ?

    MRd=[0110]\boldsymbol{M_{R_d} = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}}MRd​​=[01​10​] RdR_dRd​ swaps the coordinates: (x,y)↦(y,x)(x, y) \mapsto (y, x)(x,y)↦(y,x) . Probing: Rd(1,0)=(0,1)R_d(1, 0) = (0, 1)Rd​(1,0)=(0,1) , Rd(0,1)=(1,0)R_d(0, 1) = (1, 0)Rd​(0,1)=(1,0) .

  37. [Finding matrix representations] Every reflection acts on vectors in one of two simple ways. Which two, and what are they for the reflection RyR_yRy​ through the yyy -axis?

    Some vectors stay unchanged, others become their exact negatives. For RyR_yRy​ : multiples of (0,1)(0, 1)(0,1) stay unchanged, multiples of (1,0)(1, 0)(1,0) are negated. On any other vector, RyR_yRy​ acts by the linear combination of these two actions.Example: (2,3)=2(1,0)+3(0,1)(2, 3) = 2(1, 0) + 3(0, 1)(2,3)=2(1,0)+3(0,1) , so Ry(2,3)=−2(1,0)+3(0,1)=(−2,3)R_y(2, 3) = -2(1, 0) + 3(0, 1) = (-2, 3)Ry​(2,3)=−2(1,0)+3(0,1)=(−2,3) .

  38. [Finding matrix representations] What is the formula for the reflection Ra⃗(v⃗)R_{\vec{a}}(\vec{v})Ra​(v) through the line ℓ:{ta⃗∣t∈R}\ell : \{ t\vec{a} \mid t \in \mathbb{R} \}ℓ:{ta∣t∈R} , and why does it work?

    Ra⃗(v⃗)=2Πa⃗(v⃗)−v⃗\boldsymbol{R_{\vec{a}}(\vec{v}) = 2\Pi_{\vec{a}}(\vec{v}) - \vec{v}}Ra​(v)=2Πa​(v)−v Going from v⃗\vec{v}v to Πa⃗(v⃗)\Pi_{\vec{a}}(\vec{v})Πa​(v) reaches the line; going the same displacement Πa⃗(v⃗)−v⃗\Pi_{\vec{a}}(\vec{v}) - \vec{v}Πa​(v)−v once more reaches the mirror point: v⃗+2(Πa⃗(v⃗)−v⃗)\vec{v} + 2(\Pi_{\vec{a}}(\vec{v}) - \vec{v})v+2(Πa​(v)−v) . Example: a⃗=(1,1)\vec{a} = (1, 1)a=(1,1) , v⃗=(1,0)\vec{v} = (1, 0)v=(1,0) : Πa⃗(v⃗)=(12,12)\Pi_{\vec{a}}(\vec{v}) = (\frac…

  39. [Finding matrix representations] What are the two forms of the formula for the reflection RP(v⃗)R_P(\vec{v})RP​(v) through the plane P:n⃗⋅x⃗=0P : \vec{n} \cdot \vec{x} = 0P:n⋅x=0 ?

    RP(v⃗)=2ΠP(v⃗)−v⃗=v⃗−2Πn⃗(v⃗)\boldsymbol{R_P(\vec{v}) = 2\Pi_P(\vec{v}) - \vec{v} = \vec{v} - 2\Pi_{\vec{n}}(\vec{v})}RP​(v)=2ΠP​(v)−v=v−2Πn​(v) Second form: Πn⃗(v⃗)\Pi_{\vec{n}}(\vec{v})Πn​(v) is the shortest vector from the plane to v⃗\vec{v}v ; subtracting it once reaches the plane, subtracting it a second time reaches the point on the other side. Example: P:z=0P : z = 0P:z=0 , v⃗=(1,2,3)\vec{v} = (1, 2, 3)v=(1,2,3) : RP(v⃗)=(1,2,3)−2(0,0,3)=(1,2,−3)R_P(\vec{v}) = (1, 2, 3) - 2(0, 0, 3) = (1…

  40. [Finding matrix representations] What is the matrix of the counterclockwise rotation RθR_\thetaRθ​ by the angle θ\thetaθ in R2\mathbb{R}^2R2 ?

    MRθ=[cos⁡θ−sin⁡θsin⁡θcos⁡θ]\boldsymbol{M_{R_\theta} = \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix}}MRθ​​=[cosθsinθ​−sinθcosθ​] Probing: (1,0)(1, 0)(1,0) rotates to (cos⁡θ,sin⁡θ)(\cos\theta, \sin\theta)(cosθ,sinθ) , and (0,1)(0, 1)(0,1) , at angle π2\frac{\pi}{2}2π​ , rotates to the angle π2+θ\frac{\pi}{2} + \theta2π​+θ , i.e. to (−sin⁡θ,cos⁡θ)(-\sin\theta, \cos\theta)(−sinθ,cosθ) .

  41. [Finding matrix representations] TTT is the counterclockwise rotation of the plane by 30∘30^\circ30∘ . Find MTM_TMT​ .

    MT=[32−121232]\boldsymbol{M_T = \begin{bmatrix} \frac{\sqrt{3}}{2} & -\frac{1}{2} \\ \frac{1}{2} & \frac{\sqrt{3}}{2} \end{bmatrix}}MT​=[23​​21​​−21​23​​​] cos⁡30∘=32\cos 30^\circ = \frac{\sqrt{3}}{2}cos30∘=23​​ , sin⁡30∘=12\ \sin 30^\circ = \frac{1}{2} sin30∘=21​ , put into [cos⁡θ−sin⁡θsin⁡θcos⁡θ]\begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix}[cosθsinθ​−sinθcosθ​] .

  42. [Finding matrix representations] What is the inverse of the rotation matrix MRθ=[cos⁡θ−sin⁡θsin⁡θcos⁡θ]M_{R_\theta} = \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix}MRθ​​=[cosθsinθ​−sinθcosθ​] , found without calculation?

    The rotation by −θ-\theta−θ : MR−θ=[cos⁡θsin⁡θ−sin⁡θcos⁡θ]\boldsymbol{M_{R_{-\theta}} = \begin{bmatrix} \cos\theta & \sin\theta \\ -\sin\theta & \cos\theta \end{bmatrix}}MR−θ​​=[cosθ−sinθ​sinθcosθ​] cos⁡(−θ)=cos⁡θ\cos(-\theta) = \cos\thetacos(−θ)=cosθ and sin⁡(−θ)=−sin⁡θ\sin(-\theta) = -\sin\thetasin(−θ)=−sinθ . And MR−θMRθ=1 ⁣ ⁣1=MRθMR−θM_{R_{-\theta}} M_{R_\theta} = \mathrm{1\!\!1} = M_{R_\theta} M_{R_{-\theta}}MR−θ​​MRθ​​=11=MRθ​​MR−θ​​ : composing transformations is a matrix product.

  43. [Finding matrix representations] What is the inverse of the reflection RxR_xRx​ through the xxx -axis?

    RxR_xRx​ itself: RxR_xRx​ is self-inverse. MRxMRx=[100−1]2=[1001]M_{R_x} M_{R_x} = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}^2 = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}MRx​​MRx​​=[10​0−1​]2=[10​01​]

  44. [Finding matrix representations] ΠS\Pi_SΠS​ is the projection onto a subspace S≠RnS \ne \mathbb{R}^nS=Rn . What is the inverse of ΠS\Pi_SΠS​ ?

    There is none: projections are not invertible. ΠS\Pi_SΠS​ maps every vector of S⊥S^\perpS⊥ (vectors orthogonal to SSS ) to 0⃗\vec{0}0 , so many inputs have the same output.Example: Πx(0,1)=Πx(0,2)=(0,0)\Pi_x(0, 1) = \Pi_x(0, 2) = (0, 0)Πx​(0,1)=Πx​(0,2)=(0,0) .

  45. [Finding matrix representations] You know T:R2→R2T : \mathbb{R}^2 \to \mathbb{R}^2T:R2→R2 only on a nonstandard basis: T(v⃗1)=t⃗1T(\vec{v}_1) = \vec{t}_1T(v1​)=t1​ and T(v⃗2)=t⃗2T(\vec{v}_2) = \vec{t}_2T(v2​)=t2​ . How do you find MT=[m11m12m21m22]M_T = \begin{bmatrix} m_{11} & m_{12} \\ m_{21} & m_{22} \end{bmatrix}MT​=[m11​m21​​m12​m22​​] ?

    Write MTv⃗1=t⃗1M_T \vec{v}_1 = \vec{t}_1MT​v1​=t1​ and MTv⃗2=t⃗2M_T \vec{v}_2 = \vec{t}_2MT​v2​=t2​ as 4 linear equations in the 4 unknowns m11,m12,m21,m22m_{11}, m_{12}, m_{21}, m_{22}m11​,m12​,m21​,m22​ , and solve. As a matrix equation Am⃗=t⃗A\vec{m} = \vec{t}Am=t with m⃗=(m11,m12,m21,m22)\vec{m} = (m_{11}, m_{12}, m_{21}, m_{22})m=(m11​,m12​,m21​,m22​) : [v1xv1y0000v1xv1yv2xv2y0000v2xv2y]m⃗=[t1xt1yt2xt2y]\ \begin{bmatrix} v_{1x} & v_{1y} & 0 & 0 \\ 0 & 0 & v_{1x} & v_{1y} \\ v_{2x} & v_{2y}…

  46. [Finding matrix representations] T:R2→R2T : \mathbb{R}^2 \to \mathbb{R}^2T:R2→R2 is linear with T(1,1)=(2,0)T(1, 1) = (2, 0)T(1,1)=(2,0) and T(1,−1)=(0,2)T(1, -1) = (0, 2)T(1,−1)=(0,2) . Find MTM_TMT​ .

    MT=[111−1]\boldsymbol{M_T = \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}}MT​=[11​1−1​] First row: m11+m12=2m_{11} + m_{12} = 2m11​+m12​=2 , m11−m12=0m_{11} - m_{12} = 0m11​−m12​=0 , so m11=m12=1m_{11} = m_{12} = 1m11​=m12​=1 . Second row: m21+m22=0m_{21} + m_{22} = 0m21​+m22​=0 , m21−m22=2m_{21} - m_{22} = 2m21​−m22​=2 , so m21=1m_{21} = 1m21​=1 , m22=−1m_{22} = -1m22​=−1 .Check: MT(1,1)=(2,0)M_T(1, 1) = (2, 0)MT​(1,1)=(2,0) and MT(1,−1)=(0,2)M_T(1, -1) = (0, 2)MT​(1,−1)=(0,2) .

  47. [Finding matrix representations] What are the eigenspaces of a linear transformation TTT ?

    The subspaces of inputs on which TTT acts as multiplication by a constant λ\lambdaλ : T(e⃗λ)=λe⃗λT(\vec{e}_\lambda) = \lambda \vec{e}_\lambdaT(eλ​)=λeλ​ . From German eigen, "own". A projection has the eigenspaces for λ=1\lambda = 1λ=1 (vectors unchanged) and λ=0\lambda = 0λ=0 (vectors sent to 0⃗\vec{0}0 ).Example: the reflection RxR_xRx​ through the xxx -axis has the eigenspace span⁡{(1,0)}\operatorname{span}\{(1, 0)\}span{(1,0)} for λ=1\lambda = 1λ=1 and span⁡{(0,1)}\operatorname{span}\{(0, 1…

  48. [Finding matrix representations] Why is a basis of eigenvectors (the natural basis) the best basis for understanding a linear transformation TTT ?

    On each basis vector, TTT acts as multiplication by a number λ\lambdaλ , so TTT is a scaling along each basis direction. Example: in the basis {(1,0),(0,1)}\{(1, 0), (0, 1)\}{(1,0),(0,1)} , the reflection RxR_xRx​ multiplies the first vector by 111 and the second by −1-1−1 .

  49. [Change of basis for matrices] BBB and B′B'B′ are two bases of a vector space VVV . What does the change-of-basis matrix B′[1 ⁣ ⁣1]B_{B'}[\mathrm{1\!\!1}]_BB′​[11]B​ do, and what is its inverse?

    It converts coordinates with respect to BBB into coordinates with respect to B′B'B′ : [v⃗]B′=B′[1 ⁣ ⁣1]B[v⃗]B[\vec{v}]_{B'} = {}_{B'}[\mathrm{1\!\!1}]_B [\vec{v}]_B[v]B′​=B′​[11]B​[v]B​ . Its inverse is B[1 ⁣ ⁣1]B′_B[\mathrm{1\!\!1}]_{B'}B​[11]B′​ . Example: BBB standard, B′={(1,1),(1,−1)}B' = \{(1, 1), (1, -1)\}B′={(1,1),(1,−1)} : B[1 ⁣ ⁣1]B′=[111−1]_B[\mathrm{1\!\!1}]_{B'} = \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}B​[11]B′​=[11​1−1​] (columns: the vectors of…

  50. [Change of basis for matrices] The change-of-basis matrix B′[1 ⁣ ⁣1]B_{B'}[\mathrm{1\!\!1}]_BB′​[11]B​ is not the identity matrix. Why is it written with 1 ⁣ ⁣1\mathrm{1\!\!1}11 ?

    As a transformation it is the identity: the vector v⃗\vec{v}v does not change, only its coefficients do.

  51. [Change of basis for matrices] T:V→WT : V \to WT:V→W is linear, BV={e^1,…,e^n}B_V = \{\hat{e}_1, \dots, \hat{e}_n\}BV​={e^1​,…,e^n​} is a basis of VVV and BW={b⃗1,…,b⃗m}B_W = \{\vec{b}_1, \dots, \vec{b}_m\}BW​={b1​,…,bm​} a basis of WWW . What is the entry cijc_{ij}cij​ of the matrix BW[MT]BV_{B_W}[M_T]_{B_V}BW​​[MT​]BV​​ ?

    The iii -th coefficient of the output T(e^j)T(\hat{e}_j)T(e^j​) with respect to BWB_WBW​ : T(e^j)=c1jb⃗1+⋯+cmjb⃗mT(\hat{e}_j) = c_{1j}\vec{b}_1 + \dots + c_{mj}\vec{b}_mT(e^j​)=c1j​b1​+⋯+cmj​bm​ . So column jjj is T(e^j)T(\hat{e}_j)T(e^j​) written in BWB_WBW​ coordinates. Then [T(v⃗)]BW=BW[MT]BV[v⃗]BV[T(\vec{v})]_{B_W} = {}_{B_W}[M_T]_{B_V} [\vec{v}]_{B_V}[T(v)]BW​​=BW​​[MT​]BV​​[v]BV​​ .Example: [v⃗]BV=(5,6,0,…,0)[\vec{v}]_{B_V} = (5, 6, 0, \dots, 0)[v]BV​​=(5,6,0,…,0) gives 5⋅(column 1)+6⋅(co…

  52. [Change of basis for matrices] T:V→VT : V \to VT:V→V has the matrix B[MT]B_B[M_T]_BB​[MT​]B​ with respect to the basis BBB . What is its matrix with respect to another basis B′B'B′ ?

    B′[MT]B′=B′[1 ⁣ ⁣1]B B[MT]B B[1 ⁣ ⁣1]B′\boldsymbol{{}_{B'}[M_T]_{B'} = {}_{B'}[\mathrm{1\!\!1}]_B \; {}_B[M_T]_B \; {}_B[\mathrm{1\!\!1}]_{B'}}B′​[MT​]B′​=B′​[11]B​B​[MT​]B​B​[11]B′​ Applied to [v⃗]B′[\vec{v}]_{B'}[v]B′​ , right to left: B[1 ⁣ ⁣1]B′_B[\mathrm{1\!\!1}]_{B'}B​[11]B′​ converts the input to BBB coordinates, B[MT]B_B[M_T]_BB​[MT​]B​ applies TTT , and B′[1 ⁣ ⁣1]B_{B'}[\mathrm{1\!\!1}]_BB′​[11]B​ converts the output to B′B'B′ coordinates.Adjacen…

  53. [Change of basis for matrices] RdR_dRd​ , the reflection through the line y=xy = xy=x , has the standard matrix [0110]\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}[01​10​] . What is its matrix with respect to the basis B′={(1,1),(1,−1)}B' = \{(1, 1), (1, -1)\}B′={(1,1),(1,−1)} ?

    B′[MRd]B′=[100−1]\boldsymbol{{}_{B'}[M_{R_d}]_{B'} = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}}B′​[MRd​​]B′​=[10​0−1​] Probing in B′B'B′ : Rd(1,1)=(1,1)=1⋅(1,1)R_d(1, 1) = (1, 1) = 1 \cdot (1, 1)Rd​(1,1)=(1,1)=1⋅(1,1) and Rd(1,−1)=(−1,1)=−1⋅(1,−1)R_d(1, -1) = (-1, 1) = -1 \cdot (1, -1)Rd​(1,−1)=(−1,1)=−1⋅(1,−1) .Same result from P−1MRdPP^{-1} M_{R_d} PP−1MRd​​P with P=B[1 ⁣ ⁣1]B′=[111−1]P = {}_B[\mathrm{1\!\!1}]_{B'} = \begin{bmatrix} 1 & 1 \\ 1 & -1 \end{bmatrix}P=B​[11…

  54. [Change of basis for matrices] N∈Rn×nN \in \mathbb{R}^{n \times n}N∈Rn×n and P∈Rn×nP \in \mathbb{R}^{n \times n}P∈Rn×n is invertible. When are NNN and MMM related by a similarity transformation?

    When M=PNP−1M = PNP^{-1}M=PNP−1 . The columns of PPP form a basis of Rn\mathbb{R}^nRn , and MMM represents the same linear transformation as NNN , written with respect to another basis.Example: N=[1002]N = \begin{bmatrix} 1 & 0 \\ 0 & 2 \end{bmatrix}N=[10​02​] , P=[1101]P = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}P=[10​11​] : M=PNP−1=[1202][1−101]=[1102]M = PNP^{-1} = \begin{bmatrix} 1 & 2 \\ 0 & 2 \end{bmatrix}\begin{bmatrix} 1 & -1 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 1 \\ 0 & 2 \…

  55. [Change of basis for matrices] M=PNP−1M = PNP^{-1}M=PNP−1 for an invertible PPP . Which four properties of a matrix are the same for MMM and NNN ? (The examples use N=[1002]N = \begin{bmatrix} 1 & 0 \\ 0 & 2 \end{bmatrix}N=[10​02​] , P=[1101]P = \begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix}P=[10​11​] , M=PNP−1=[1102]M = PNP^{-1} = \begin{bmatrix} 1 & 1 \\ 0 & 2 \end{bmatrix}M=PNP−1=[10​12​] .)

    Trace Tr⁡(M)=Tr⁡(N)\operatorname{Tr}(M) = \operatorname{Tr}(N)Tr(M)=Tr(N) Example: 1+2=31 + 2 = 31+2=3 for both. Determinant det⁡(M)=det⁡(N)\det(M) = \det(N)det(M)=det(N) Example: 1⋅2−1⋅0=2=1⋅2−0⋅01 \cdot 2 - 1 \cdot 0 = 2 = 1 \cdot 2 - 0 \cdot 01⋅2−1⋅0=2=1⋅2−0⋅0 . Rank rank⁡(M)=rank⁡(N)\operatorname{rank}(M) = \operatorname{rank}(N)rank(M)=rank(N) Example: both have rank 2. Eigenvalues eig⁡(M)=eig⁡(N)\operatorname{eig}(M) = \operatorname{eig}(N)eig(M)=eig(N) Example: both have the eigenvalues…

  56. [Change of basis for matrices] The trace, determinant, rank and eigenvalues of a matrix are called its invariant properties, because they do not depend on the choice of basis.

    Example: [1002]\begin{bmatrix} 1 & 0 \\ 0 & 2 \end{bmatrix}[10​02​] and [1102]\begin{bmatrix} 1 & 1 \\ 0 & 2 \end{bmatrix}[10​12​] represent the same transformation in two bases; both have trace 3 and determinant 2.

  57. [Invertible matrix theorem] Invertible matrix theorem. For an n×nn \times nn×n matrix AAA , which ten statements are equivalent? (The examples use A=[2111]A = \begin{bmatrix} 2 & 1 \\ 1 & 1 \end{bmatrix}A=[21​11​] .)

    (1) Invertible AAA is invertible. Example: A−1=[1−1−12]A^{-1} = \begin{bmatrix} 1 & -1 \\ -1 & 2 \end{bmatrix}A−1=[1−1​−12​] . (2) Unique solutions Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b has exactly one solution for each b⃗∈Rn\vec{b} \in \mathbb{R}^nb∈Rn . Example: Ax⃗=(3,2)A\vec{x} = (3, 2)Ax=(3,2) has only the solution x⃗=A−1(3,2)=(1,1)\vec{x} = A^{-1}(3, 2) = (1, 1)x=A−1(3,2)=(1,1) . (3) Trivial null space N(A)={0⃗}\mathcal{N}(A) = \{\vec{0}\}N(A)={0} . Example: 2x+y=02x + y = 02x+y=0 and x+y=0x + y =…

  58. [Invertible matrix theorem] Is A=[1224]A = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}A=[12​24​] invertible? Which statements of the invertible matrix theorem fail?

    Not invertible; all ten statements fail. det⁡(A)=4−4=0\det(A) = 4 - 4 = 0det(A)=4−4=0 ; the second column is twice the first; (2,−1)∈N(A)(2, -1) \in \mathcal{N}(A)(2,−1)∈N(A) ; rank⁡(A)=1\operatorname{rank}(A) = 1rank(A)=1 ; Ax⃗=(1,0)A\vec{x} = (1, 0)Ax=(1,0) has no solution.

  59. [Invertible matrix theorem] What does a proof by contradiction of a statement do?

    It assumes the opposite of the statement and derives a contradiction, so the opposite is false and the statement is true. Closely related: to show (A)⇒(B)(A) \Rightarrow (B)(A)⇒(B) , show not-(B)⇒not-(A)\text{not-}(B) \Rightarrow \text{not-}(A)not-(B)⇒not-(A) (the contrapositive).Example: 2∉Q\sqrt{2} \notin \mathbb{Q}2​∈/Q is proved by contradiction.

  60. [Invertible matrix theorem] What must you prove to establish (A)⇔(B)(A) \Leftrightarrow (B)(A)⇔(B) rather than (A)⇒(B)(A) \Rightarrow (B)(A)⇒(B) ?

    Both directions: (A)⇒(B)(A) \Rightarrow (B)(A)⇒(B) and (B)⇒(A)(B) \Rightarrow (A)(B)⇒(A) .

  61. [Invertible matrix theorem] In what order does Savov's proof of the invertible matrix theorem connect its ten statements? (1) invertible, (2) unique solutions of Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b , (3) N(A)={0⃗}\mathcal{N}(A) = \{\vec{0}\}N(A)={0} , (4) Ax⃗=0⃗A\vec{x} = \vec{0}Ax=0 only for x⃗=0⃗\vec{x} = \vec{0}x=0 , (5) columns form a basis, (6) rank nnn , (7) rref⁡(A)=1 ⁣ ⁣1n\operatorname{rref}(A) = \mathrm{1\!\!1}_nrref(A)=11n​ , (8) ATA^{\mathsf{T}}AT invertible, (9) rows form a basis, (10) det…

    A closed chain (1)⇒(2)⇒⋯⇒(7)⇒(1)(1) \Rightarrow (2) \Rightarrow \dots \Rightarrow (7) \Rightarrow (1)(1)⇒(2)⇒⋯⇒(7)⇒(1) , plus (1)⇔(8)⇔(9)(1) \Leftrightarrow (8) \Leftrightarrow (9)(1)⇔(8)⇔(9) and (5)⇔(10)(5) \Leftrightarrow (10)(5)⇔(10) . In a closed chain, any one true statement makes all of them true (follow the arrows), and any one false statement makes all of them false (follow them backwards).

  62. [Invertible matrix theorem] AAA is an invertible n×nn \times nn×n matrix. Why does Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b have exactly one solution for each b⃗\vec{b}b ?

    x⃗=A−1b⃗\vec{x} = A^{-1}\vec{b}x=A−1b is a solution, and any solution y⃗\vec{y}y​ satisfies y⃗=A−1Ay⃗=A−1b⃗\vec{y} = A^{-1}A\vec{y} = A^{-1}\vec{b}y​=A−1Ay​=A−1b . Invertible matrix theorem, (1)⇒(2)(1) \Rightarrow (2)(1)⇒(2) .

  63. [Invertible matrix theorem] Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b has exactly one solution for each b⃗\vec{b}b . Why does this force N(A)={0⃗}\mathcal{N}(A) = \{\vec{0}\}N(A)={0} ?

    A nonzero y⃗∈N(A)\vec{y} \in \mathcal{N}(A)y​∈N(A) would give a second solution x⃗+y⃗\vec{x} + \vec{y}x+y​ , since A(x⃗+y⃗)=b⃗+0⃗=b⃗A(\vec{x} + \vec{y}) = \vec{b} + \vec{0} = \vec{b}A(x+y​)=b+0=b . Invertible matrix theorem, (2)⇒(3)(2) \Rightarrow (3)(2)⇒(3) .

  64. [Invertible matrix theorem] AAA is n×nn \times nn×n and Av⃗=0⃗A\vec{v} = \vec{0}Av=0 has only the solution v⃗=0⃗\vec{v} = \vec{0}v=0 . Why do the columns c⃗1,…,c⃗n\vec{c}_1, \dots, \vec{c}_nc1​,…,cn​ of AAA form a basis of Rn\mathbb{R}^nRn ?

    v1c⃗1+⋯+vnc⃗n=Av⃗=0⃗v_1\vec{c}_1 + \dots + v_n\vec{c}_n = A\vec{v} = \vec{0}v1​c1​+⋯+vn​cn​=Av=0 only for v⃗=0⃗\vec{v} = \vec{0}v=0 , so the columns are linearly independent, and nnn such vectors in Rn\mathbb{R}^nRn form a basis. This uses the column picture of Av⃗A\vec{v}Av . Invertible matrix theorem, (4)⇒(5)(4) \Rightarrow (5)(4)⇒(5) .

  65. [Invertible matrix theorem] The nnn columns of an n×nn \times nn×n matrix AAA are linearly independent. Why is rref⁡(A)=1 ⁣ ⁣1n\operatorname{rref}(A) = \mathrm{1\!\!1}_nrref(A)=11n​ ?

    rank⁡(A)=n\operatorname{rank}(A) = nrank(A)=n , so the RREF has nnn pivots, and an n×nn \times nn×n RREF with nnn pivots is 1 ⁣ ⁣1n\mathrm{1\!\!1}_n11n​ . Rank = number of linearly independent columns = number of pivots. Invertible matrix theorem, (5)⇒(6)⇒(7)(5) \Rightarrow (6) \Rightarrow (7)(5)⇒(6)⇒(7) .

  66. [Invertible matrix theorem] Row operations R1,…,RkR_1, \dots, R_kR1​,…,Rk​ turn AAA into 1 ⁣ ⁣1n\mathrm{1\!\!1}_n11n​ . Why is AAA invertible, and what is A−1A^{-1}A−1 ?

    A−1=Ek⋯E2E1A^{-1} = E_k \cdots E_2 E_1A−1=Ek​⋯E2​E1​ , where EiE_iEi​ is the elementary matrix of the row operation RiR_iRi​ . The row operations give Ek⋯E2E1A=1 ⁣ ⁣1nE_k \cdots E_2 E_1 A = \mathrm{1\!\!1}_nEk​⋯E2​E1​A=11n​ . Invertible matrix theorem, (7)⇒(1)(7) \Rightarrow (1)(7)⇒(1) .Example: A=[1021]A = \begin{bmatrix} 1 & 0 \\ 2 & 1 \end{bmatrix}A=[12​01​] , R1R_1R1​ : subtract 2 times row 1 from row 2, so E1=[10−21]=A−1E_1 = \begin{bmatrix} 1 & 0 \\ -2 & 1 \end{bmatrix} = A^{-1}E1​=[1−2​0…

  67. [Invertible matrix theorem] Why is AAA invertible if and only if ATA^{\mathsf{T}}AT is invertible?

    Transposing AA−1=1 ⁣ ⁣1nAA^{-1} = \mathrm{1\!\!1}_nAA−1=11n​ gives (A−1)TAT=1 ⁣ ⁣1n(A^{-1})^{\mathsf{T}} A^{\mathsf{T}} = \mathrm{1\!\!1}_n(A−1)TAT=11n​ , so (AT)−1=(A−1)T(A^{\mathsf{T}})^{-1} = (A^{-1})^{\mathsf{T}}(AT)−1=(A−1)T . The same argument applied to ATA^{\mathsf{T}}AT gives the other direction. Invertible matrix theorem, (1)⇔(8)(1) \Leftrightarrow (8)(1)⇔(8) .

  68. [Invertible matrix theorem] AAA is n×nn \times nn×n . Why do the rows of AAA form a basis of Rn\mathbb{R}^nRn exactly when ATA^{\mathsf{T}}AT is invertible?

    The rows of AAA are the columns of ATA^{\mathsf{T}}AT , and a square matrix is invertible exactly when its columns form a basis. Invertible matrix theorem, (8)⇔(9)(8) \Leftrightarrow (9)(8)⇔(9) , using (1)⇔(5)(1) \Leftrightarrow (5)(1)⇔(5) for ATA^{\mathsf{T}}AT .

  69. [Invertible matrix theorem] Why do the columns of an n×nn \times nn×n matrix AAA form a basis of Rn\mathbb{R}^nRn exactly when det⁡(A)≠0\det(A) \ne 0det(A)=0 ?

    det⁡(A)≠0\det(A) \ne 0det(A)=0 if and only if the columns of AAA are linearly independent, and nnn independent vectors in Rn\mathbb{R}^nRn form a basis. ∣det⁡(A)∣|\det(A)|∣det(A)∣ is the factor by which x⃗↦Ax⃗\vec{x} \mapsto A\vec{x}x↦Ax scales nnn -dimensional volume; dependent columns give volume 0. Invertible matrix theorem, (5)⇔(10)(5) \Leftrightarrow (10)(5)⇔(10) .

  70. [Invertible matrix theorem] What condition on the kernel says whether a linear transformation T:Rn→RnT : \mathbb{R}^n \to \mathbb{R}^nT:Rn→Rn is invertible?

    TTT is invertible if and only if Ker⁡(T)={0⃗}\operatorname{Ker}(T) = \{\vec{0}\}Ker(T)={0} . Ker⁡(T)=N(MT)\operatorname{Ker}(T) = \mathcal{N}(M_T)Ker(T)=N(MT​) , so this is statement (3) of the invertible matrix theorem. Equivalently: TTT is invertible if and only if MTM_TMT​ is.

  71. [Invertible matrix theorem] Why can a linear transformation SSS with a nonzero vector in Ker⁡(S)\operatorname{Ker}(S)Ker(S) not be undone?

    SSS maps every vector of Ker⁡(S)\operatorname{Ker}(S)Ker(S) to 0⃗\vec{0}0 , so from the output 0⃗\vec{0}0 the input cannot be determined. Example: S(x,y)=(x,0)S(x, y) = (x, 0)S(x,y)=(x,0) gives S(0,1)=S(0,2)=S(0,0)=(0,0)S(0, 1) = S(0, 2) = S(0, 0) = (0, 0)S(0,1)=S(0,2)=S(0,0)=(0,0) .

  72. [Invertible matrix theorem] When is a linear transformation TTT bijective, in terms of the equation T(x⃗)=y⃗T(\vec{x}) = \vec{y}T(x)=y​ ?

    When T(x⃗)=y⃗T(\vec{x}) = \vec{y}T(x)=y​ has exactly one solution x⃗\vec{x}x for each y⃗\vec{y}y​ in the output space. Then TTT is invertible: T−1(y⃗)T^{-1}(\vec{y})T−1(y​) is that solution.

  73. [Invertible matrix theorem] For a linear transformation T:Rn→RnT : \mathbb{R}^n \to \mathbb{R}^nT:Rn→Rn (same input and output space), which single property is already enough for TTT to be invertible?

    Either injective or surjective alone. Injective: Ker⁡(T)={0⃗}\operatorname{Ker}(T) = \{\vec{0}\}Ker(T)={0} . Surjective: rank⁡(MT)=n\operatorname{rank}(M_T) = nrank(MT​)=n , so by rank–nullity nullity⁡(MT)=n−n=0\operatorname{nullity}(M_T) = n - n = 0nullity(MT​)=n−n=0 , i.e. Ker⁡(T)={0⃗}\operatorname{Ker}(T) = \{\vec{0}\}Ker(T)={0} .For T:R3→R2T : \mathbb{R}^3 \to \mathbb{R}^2T:R3→R2 this fails: T(x,y,z)=(x,z)T(x, y, z) = (x, z)T(x,y,z)=(x,z) is surjective but not invertible.