Ch.3: Intro to linear algebra

Chapter 3 of the book "No Bullshit Guide to Linear Algebra" by Ivan Savov Introduction Review of vector operations Matrix operations Linearity Overview of linear algebra

75 flashcards · Mathematics · by lakesare

Part of Textbooks › No Bullshit Guide to Linear Algebra (Ivan Savov) · 10 courses

1. [Introduction] What is a vector v⃗∈Rn\vec{v} \in \mathbb{R}^nv∈Rn ?

   An nnn -tuple of real numbers (v1,v2,…,vn)(v_1, v_2, \dots, v_n)(v1​,v2​,…,vn​) . To specify v⃗\vec{v}v , you specify its nnn components.Example: v⃗=(v1,v2,v3)∈(R,R,R)≡R3\vec{v} = (v_1, v_2, v_3) \in (\mathbb{R}, \mathbb{R}, \mathbb{R}) \equiv \mathbb{R}^3v=(v1​,v2​,v3​)∈(R,R,R)≡R3 .

2. [Introduction] What is a matrix A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n ?

   A rectangular array of real numbers with mmm rows and nnn columns. Example: a 3×23 \times 23×2 matrix A=[a11a12a21a22a31a32]A = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \\ a_{31} & a_{32} \end{bmatrix}A=⎣⎡​a11​a21​a31​​a12​a22​a32​​⎦⎤​ is specified by its six entries a11,a12,…,a32a_{11}, a_{12}, \dots, a_{32}a11​,a12​,…,a32​ .

3. [Introduction] A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n and x⃗∈Rn\vec{x} \in \mathbb{R}^nx∈Rn . What is the matrix-vector product Ax⃗A\vec{x}Ax in the column picture?

   The linear combination of the columns of AAA with coefficients x1,…,xnx_1, \dots, x_nx1​,…,xn​ : Ax⃗=x1A[:,1]+⋯+xnA[:,n]A\vec{x} = x_1 A_{[:,1]} + \dots + x_n A_{[:,n]}Ax=x1​A[:,1]​+⋯+xn​A[:,n]​ . A[:,j]A_{[:,j]}A[:,j]​ denotes the jjj -th column of AAA .Example: [123456][21]=2[135]+1[246]=[41016]\begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{bmatrix} \begin{bmatrix} 2 \\ 1 \end{bmatrix} = 2 \begin{bmatrix} 1 \\ 3 \\ 5 \end{bmatrix} + 1 \begin{bmatrix} 2 \\ 4 \\ 6 \end{bmatrix} = \begin{bmatrix}…

4. [Introduction] AAA has two columns. Which vector x⃗\vec{x}x gives Ax⃗=3⋅(first column of A)+4⋅(second column of A)A\vec{x} = 3 \cdot (\text{first column of } A) + 4 \cdot (\text{second column of } A)Ax=3⋅(first column of A)+4⋅(second column of A) ?

   x⃗=(3,4)\boldsymbol{\vec{x} = (3, 4)}x=(3,4) In the column picture, Ax⃗=x1A[:,1]+x2A[:,2]A\vec{x} = x_1 A_{[:,1]} + x_2 A_{[:,2]}Ax=x1​A[:,1]​+x2​A[:,2]​ .

5. [Introduction] y⃗=αe⃗1+βe⃗2\vec{y} = \alpha \vec{e}_1 + \beta \vec{e}_2y​=αe1​+βe2​ with e⃗1,e⃗2,y⃗∈Rn\vec{e}_1, \vec{e}_2, \vec{y} \in \mathbb{R}^ne1​,e2​,y​∈Rn . How do you write y⃗\vec{y}y​ as a matrix-vector product?

   y⃗=E[αβ]\boldsymbol{\vec{y} = E \begin{bmatrix} \alpha \\ \beta \end{bmatrix}}y​=E[αβ​] , where EEE is the n×2n \times 2n×2 matrix with columns e⃗1\vec{e}_1e1​ and e⃗2\vec{e}_2e2​ . The numbers α,β\alpha, \betaα,β are the coefficients of the linear combination. The matrix-vector product is defined for the purpose of writing linear combinations.

6. [Introduction] A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n . What is the linear transformation TAT_ATA​ : its input space, output space and action?

   TA:Rn→Rm\boldsymbol{T_A : \mathbb{R}^n \to \mathbb{R}^m}TA​:Rn→Rm , TA(x⃗)=Ax⃗\ \boldsymbol{T_A(\vec{x}) = A\vec{x}} TA​(x)=Ax AAA has mmm rows, so Ax⃗A\vec{x}Ax is an mmm -vector. We say TAT_ATA​ is represented by the matrix AAA .Example: A=[3005]A = \begin{bmatrix} 3 & 0 \\ 0 & 5 \end{bmatrix}A=[30​05​] gives TA(x1,x2)=(3x1,5x2)T_A(x_1, x_2) = (3x_1, 5x_2)TA​(x1​,x2​)=(3x1​,5x2​) .

7. [Introduction] What does the inverse matrix A−1A^{-1}A−1 of a square invertible matrix AAA do to Ax⃗A\vec{x}Ax ?

   It gives back the input: A−1(Ax⃗)=x⃗\boldsymbol{A^{-1}(A\vec{x}) = \vec{x}}A−1(Ax)=x . Like an inverse function: f−1(f(x))=xf^{-1}(f(x)) = xf−1(f(x))=x .Example: A=[3005]A = \begin{bmatrix} 3 & 0 \\ 0 & 5 \end{bmatrix}A=[30​05​] multiplies the components by 3 and 5; A−1=[130015]A^{-1} = \begin{bmatrix} \frac{1}{3} & 0 \\ 0 & \frac{1}{5} \end{bmatrix}A−1=[31​0​051​​] multiplies them by 13\frac{1}{3}31​ and 15\frac{1}{5}51​ .

8. [Introduction] B=[1203]B = \begin{bmatrix} 1 & 2 \\ 0 & 3 \end{bmatrix}B=[10​23​] . Compute Bx⃗B\vec{x}Bx for x⃗=(x1,x2)\vec{x} = (x_1, x_2)x=(x1​,x2​) , and find B−1B^{-1}B−1 .

   Bx⃗=(x1+2x2, 3x2)\boldsymbol{B\vec{x} = (x_1 + 2x_2, \ 3x_2)}Bx=(x1​+2x2​, 3x2​) , B−1=[1−23013]\ \boldsymbol{B^{-1} = \begin{bmatrix} 1 & -\frac{2}{3} \\ 0 & \frac{1}{3} \end{bmatrix}} B−1=[10​−32​31​​] B−1B^{-1}B−1 must recover x2=13(3x2)x_2 = \frac{1}{3}(3x_2)x2​=31​(3x2​) and x1=(x1+2x2)−23(3x2)x_1 = (x_1 + 2x_2) - \frac{2}{3}(3x_2)x1​=(x1​+2x2​)−32​(3x2​) .Check: B−1B=[12−201]=[1001]B^{-1}B = \begin{bmatrix} 1 & 2 - 2 \\ 0 & 1 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}B−1…

9. [Introduction] What is the identity matrix, and what does A−1AA^{-1}AA−1A equal?

   The square matrix with 1s on the diagonal and 0s everywhere else; A−1A=I\boldsymbol{A^{-1}A = I}A−1A=I . Ix⃗=x⃗I\vec{x} = \vec{x}Ix=x for every vector x⃗\vec{x}x . (The book writes the identity matrix as a double-struck 1.)Example: in R2×2\mathbb{R}^{2 \times 2}R2×2 , I=[1001]I = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}I=[10​01​] .

10. [Introduction] What is the fundamental idea of linear algebra, relating matrices and linear transformations?

    Matrices and linear transformations correspond: every linear T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm is T(x⃗)=ATx⃗T(\vec{x}) = A_T\vec{x}T(x)=AT​x for a matrix ATA_TAT​ , and every matrix AAA defines TA(x⃗)=Ax⃗T_A(\vec{x}) = A\vec{x}TA​(x)=Ax . So "linear algebra is about vectors and matrices" can be read as "linear algebra is about vectors and linear transformations", just as high school maths is about numbers and functions.

11. [Introduction] A linear transformation T:Rn→RmT : \mathbb{R}^n \to \mathbb{R}^mT:Rn→Rm is represented by a matrix ATA_TAT​ . On what do the entries of ATA_TAT​ depend?

    On the choice of basis for the input space and the basis for the output space. ATA_TAT​ is one particular representation of the abstract transformation TTT .

12. [Introduction] For a function f:R→Rf : \mathbb{R} \to \mathbb{R}f:R→R there are its zeros and its image. What are the corresponding notions for TA(x⃗)=Ax⃗T_A(\vec{x}) = A\vec{x}TA​(x)=Ax ?

    Zeros of fff ↔ kernel of TAT_ATA​ = null space N(A)\mathcal{N}(A)N(A) ; image of fff ↔ image of TAT_ATA​ = column space C(A)\mathcal{C}(A)C(A) . The kernel is the set of inputs x⃗\vec{x}x with Ax⃗=0⃗A\vec{x} = \vec{0}Ax=0 . The column space is the set of linear combinations of the columns of AAA , which are exactly the possible outputs Ax⃗A\vec{x}Ax .Example: for A=[1000]A = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}A=[10​00​] , N(A)\mathcal{N}(A)N(A) is the yyy -axis and C(A)\mathcal{C}(A)C(…

13. [Introduction] A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n , B∈Rk×mB \in \mathbb{R}^{k \times m}B∈Rk×m . Which matrix represents applying TAT_ATA​ first and then TBT_BTB​ , i.e. TB(TA(x⃗))T_B(T_A(\vec{x}))TB​(TA​(x)) ?

    BA\boldsymbol{BA}BA , since TB(TA(x⃗))=B(Ax⃗)=BAx⃗T_B(T_A(\vec{x})) = B(A\vec{x}) = BA\vec{x}TB​(TA​(x))=B(Ax)=BAx . This corresponds to composing functions, g∘f(x)=g(f(x))g \circ f(x) = g(f(x))g∘f(x)=g(f(x)) . The matrix applied first stands on the right.

14. [Review of vector operations] How are u⃗+v⃗\vec{u} + \vec{v}u+v and u⃗−v⃗\vec{u} - \vec{v}u−v computed for u⃗,v⃗∈Rn\vec{u}, \vec{v} \in \mathbb{R}^nu,v∈Rn ?

    Component-wise: wi=ui±vi\boldsymbol{w_i = u_i \pm v_i}wi​=ui​±vi​ for all i∈[1,…,n]i \in [1, \dots, n]i∈[1,…,n] . Both are maps Rn×Rn→Rn\mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}^nRn×Rn→Rn .Example: (1,2,3)+(4,0,−1)=(5,2,2)(1, 2, 3) + (4, 0, -1) = (5, 2, 2)(1,2,3)+(4,0,−1)=(5,2,2) and (1,2,3)−(4,0,−1)=(−3,2,4)(1, 2, 3) - (4, 0, -1) = (-3, 2, 4)(1,2,3)−(4,0,−1)=(−3,2,4) .

15. [Review of vector operations] How is a vector u⃗∈Rn\vec{u} \in \mathbb{R}^nu∈Rn scaled by a constant α∈R\alpha \in \mathbb{R}α∈R ?

    Each component is multiplied by α\alphaα : αu⃗=(αu1,…,αun)\boldsymbol{\alpha\vec{u} = (\alpha u_1, \dots, \alpha u_n)}αu=(αu1​,…,αun​) . Example: u⃗=(1.5,1)\vec{u} = (1.5, 1)u=(1.5,1) and α=2\alpha = 2α=2 give w⃗=2u⃗=(3,2)\vec{w} = 2\vec{u} = (3, 2)w=2u=(3,2) , a vector in the same direction, twice as long.

16. [Review of vector operations] What is the dot product u⃗⋅v⃗\vec{u} \cdot \vec{v}u⋅v of u⃗,v⃗∈Rn\vec{u}, \vec{v} \in \mathbb{R}^nu,v∈Rn ?

    u⃗⋅v⃗=∑i=1nuivi\boldsymbol{\vec{u} \cdot \vec{v} = \sum_{i=1}^{n} u_i v_i}u⋅v=∑i=1n​ui​vi​ , a real number. Defined in any dimension, for two vectors of the same length. Also called scalar product or inner product, written ⟨u⃗,v⃗⟩\langle \vec{u}, \vec{v} \rangle⟨u,v⟩ .Example: (1,2,3)⋅(4,5,6)=4+10+18=32(1, 2, 3) \cdot (4, 5, 6) = 4 + 10 + 18 = 32(1,2,3)⋅(4,5,6)=4+10+18=32 .

17. [Review of vector operations] What is the dot product of two orthogonal vectors, and why?

    Zero. The dot product measures how similar two vectors are: how much of one vector goes in the direction of the other. For orthogonal vectors that part is zero.Example: ı^⋅ȷ^=(1,0,0)⋅(0,1,0)=0\hat{\imath} \cdot \hat{\jmath} = (1, 0, 0) \cdot (0, 1, 0) = 0^⋅^​=(1,0,0)⋅(0,1,0)=0 .

18. [Review of vector operations] What is the cross product u⃗×v⃗\vec{u} \times \vec{v}u×v of u⃗,v⃗∈R3\vec{u}, \vec{v} \in \mathbb{R}^3u,v∈R3 ?

    u⃗×v⃗=(u2v3−u3v2, u3v1−u1v3, u1v2−u2v1)\boldsymbol{\vec{u} \times \vec{v} = (u_2v_3 - u_3v_2, \ u_3v_1 - u_1v_3, \ u_1v_2 - u_2v_1)}u×v=(u2​v3​−u3​v2​, u3​v1​−u1​v3​, u1​v2​−u2​v1​) Also called the vector product. Defined only for three-dimensional vectors.Example: ı^×ȷ^=(1,0,0)×(0,1,0)=(0,0,1)=k^\hat{\imath} \times \hat{\jmath} = (1, 0, 0) \times (0, 1, 0) = (0, 0, 1) = \hat{k}^×^​=(1,0,0)×(0,1,0)=(0,0,1)=k^ .

19. [Review of vector operations] What geometric property does w⃗=u⃗×v⃗\vec{w} = \vec{u} \times \vec{v}w=u×v have, for u⃗,v⃗∈R3\vec{u}, \vec{v} \in \mathbb{R}^3u,v∈R3 ?

    w⃗\vec{w}w is perpendicular to both u⃗\vec{u}u and v⃗\vec{v}v . So the cross product answers: given u⃗\vec{u}u and v⃗\vec{v}v , find a vector perpendicular to both.Example: (1,2,3)×(4,5,6)=(12−15, 12−6, 5−8)=(−3,6,−3)(1, 2, 3) \times (4, 5, 6) = (12 - 15, \ 12 - 6, \ 5 - 8) = (-3, 6, -3)(1,2,3)×(4,5,6)=(12−15, 12−6, 5−8)=(−3,6,−3) , and (1,2,3)⋅(−3,6,−3)=−3+12−9=0(1, 2, 3) \cdot (-3, 6, -3) = -3 + 12 - 9 = 0(1,2,3)⋅(−3,6,−3)=−3+12−9=0 .

20. [Review of vector operations] What is the length ∥u⃗∥\|\vec{u}\|∥u∥ of u⃗∈Rn\vec{u} \in \mathbb{R}^nu∈Rn ?

    ∥u⃗∥=u12+u22+⋯+un2=u⃗⋅u⃗\boldsymbol{\|\vec{u}\| = \sqrt{u_1^2 + u_2^2 + \dots + u_n^2} = \sqrt{\vec{u} \cdot \vec{u}}}∥u∥=u12​+u22​+⋯+un2​​=u⋅u​ The nnn -dimensional version of Pythagoras' formula for the hypotenuse. Also called magnitude or norm, written ∥u⃗∥\|\vec{u}\|∥u∥ , ∣u⃗∣2|\vec{u}|_2∣u∣2​ or uuu .Example: ∥(3,4)∥=9+16=5\|(3, 4)\| = \sqrt{9 + 16} = 5∥(3,4)∥=9+16​=5 .

21. [Review of vector operations] Why is the Euclidean length u12+⋯+un2\sqrt{u_1^2 + \dots + u_n^2}u12​+⋯+un2​​ also called the ℓ2\ell^2ℓ2 -norm?

    It squares each component and takes the square root of the sum. Pronounced "ell-two norm". Likewise the ℓ4\ell^4ℓ4 -norm is ∥u⃗∥4=u14+⋯+un44\|\vec{u}\|_4 = \sqrt[4]{u_1^4 + \dots + u_n^4}∥u∥4​=4u14​+⋯+un4​​ .Example: ∥(1,1)∥2=2\|(1, 1)\|_2 = \sqrt{2}∥(1,1)∥2​=2​ and ∥(1,1)∥4=24\|(1, 1)\|_4 = \sqrt[4]{2}∥(1,1)∥4​=42​ .

22. [Review of vector operations] How do you build the unit vector v^\hat{v}v^ in the direction of a nonzero vector v⃗\vec{v}v ?

    v^=v⃗∥v⃗∥\boldsymbol{\hat{v} = \dfrac{\vec{v}}{\|\vec{v}\|}}v^=∥v∥v​ A unit vector has length 1 and is used to specify a direction in space.Example: v⃗=(3,4)\vec{v} = (3, 4)v=(3,4) , ∥v⃗∥=5\|\vec{v}\| = 5∥v∥=5 , v^=(0.6,0.8)\hat{v} = (0.6, 0.8)v^=(0.6,0.8) .

23. [Review of vector operations] d^\hat{d}d^ is a unit vector and θ\thetaθ the angle between v⃗\vec{v}v and d^\hat{d}d^ . How much of v⃗\vec{v}v is in the direction d^\hat{d}d^ ?

    vd=d^⋅v⃗=∥v⃗∥cos⁡θ\boldsymbol{v_d = \hat{d} \cdot \vec{v} = \|\vec{v}\| \cos\theta}vd​=d^⋅v=∥v∥cosθ Since ∥d^∥=1\|\hat{d}\| = 1∥d^∥=1 , d^⋅v⃗=∥d^∥∥v⃗∥cos⁡θ=∥v⃗∥cos⁡θ\hat{d} \cdot \vec{v} = \|\hat{d}\| \|\vec{v}\| \cos\theta = \|\vec{v}\| \cos\thetad^⋅v=∥d^∥∥v∥cosθ=∥v∥cosθ .Example: the xxx -component of a force is Fx=F⃗⋅ı^=∥F⃗∥cos⁡θF_x = \vec{F} \cdot \hat{\imath} = \|\vec{F}\| \cos\thetaFx​=F⋅^=∥F∥cosθ .

24. [Review of vector operations] What is the projection Πd^(v⃗)\Pi_{\hat{d}}(\vec{v})Πd^​(v) of v⃗\vec{v}v in the direction of a unit vector d^\hat{d}d^ ?

    Πd^(v⃗)=(d^⋅v⃗) d^\boldsymbol{\Pi_{\hat{d}}(\vec{v}) = (\hat{d} \cdot \vec{v}) \, \hat{d}}Πd^​(v)=(d^⋅v)d^ The length vd=d^⋅v⃗v_d = \hat{d} \cdot \vec{v}vd​=d^⋅v of the part of v⃗\vec{v}v in direction d^\hat{d}d^ , times the direction d^\hat{d}d^ . All parts of v⃗\vec{v}v not in the d^\hat{d}d^ direction are ignored.Example: Πı^((2,5))=2ı^=(2,0)\Pi_{\hat{\imath}}((2, 5)) = 2\hat{\imath} = (2, 0)Π^​((2,5))=2^=(2,0) .

25. [Review of vector operations] What is the projection Πd⃗(v⃗)\Pi_{\vec{d}}(\vec{v})Πd​(v) of v⃗\vec{v}v in the direction of a vector d⃗\vec{d}d that is not of unit length?

    Πd⃗(v⃗)=(d⃗⋅v⃗∥d⃗∥2)d⃗\boldsymbol{\Pi_{\vec{d}}(\vec{v}) = \left( \dfrac{\vec{d} \cdot \vec{v}}{\|\vec{d}\|^2} \right) \vec{d}}Πd​(v)=(∥d∥2d⋅v​)d Example: v⃗=(2,3)\vec{v} = (2, 3)v=(2,3) , d⃗=(1,1)\vec{d} = (1, 1)d=(1,1) : d⃗⋅v⃗=5\vec{d} \cdot \vec{v} = 5d⋅v=5 , ∥d⃗∥2=2\|\vec{d}\|^2 = 2∥d∥2=2 , so Πd⃗(v⃗)=52(1,1)=(2.5,2.5)\Pi_{\vec{d}}(\vec{v}) = \frac{5}{2}(1, 1) = (2.5, 2.5)Πd​(v)=25​(1,1)=(2.5,2.5) .

26. [Review of vector operations] Why does the projection formula onto a non-unit vector d⃗\vec{d}d divide by ∥d⃗∥2\|\vec{d}\|^2∥d∥2 and not by ∥d⃗∥\|\vec{d}\|∥d∥ ?

    d⃗\vec{d}d appears twice and each appearance must become the unit vector d^=d⃗/∥d⃗∥\hat{d} = \vec{d} / \|\vec{d}\|d^=d/∥d∥ . Π(v⃗)=(v⃗⋅d^) d^=(v⃗⋅d⃗∥d⃗∥)d⃗∥d⃗∥=(v⃗⋅d⃗∥d⃗∥2)d⃗\Pi(\vec{v}) = (\vec{v} \cdot \hat{d}) \, \hat{d} = \left( \vec{v} \cdot \dfrac{\vec{d}}{\|\vec{d}\|} \right) \dfrac{\vec{d}}{\|\vec{d}\|} = \left( \dfrac{\vec{v} \cdot \vec{d}}{\|\vec{d}\|^2} \right) \vec{d}Π(v)=(v⋅d^)d^=(v⋅∥d∥d​)∥d∥d​=(∥d∥2v⋅d​)d

27. [Matrix operations] A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n . What does the entry aija_{ij}aij​ denote?

    The entry in row iii and column jjj of AAA . Example: in A=[1234]A = \begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix}A=[13​24​] , a21=3a_{21} = 3a21​=3 .

28. [Matrix operations] How are A+BA + BA+B and A−BA - BA−B computed, and when are they defined?

    Entry-wise, cij=aij±bijc_{ij} = a_{ij} \pm b_{ij}cij​=aij​±bij​ ; only when AAA and BBB have the same dimensions. Both are maps Rm×n×Rm×n→Rm×n\mathbb{R}^{m \times n} \times \mathbb{R}^{m \times n} \to \mathbb{R}^{m \times n}Rm×n×Rm×n→Rm×n .Example: [1234]+[0110]=[1344]\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} + \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} = \begin{bmatrix} 1 & 3 \\ 4 & 4 \end{bmatrix}[13​24​]+[01​10​]=[14​34​] .

29. [Matrix operations] How is a matrix AAA scaled by a number α\alphaα ?

    Every entry is multiplied by α\alphaα : (αA)ij=αaij\boldsymbol{(\alpha A)_{ij} = \alpha a_{ij}}(αA)ij​=αaij​ . Example: 2[103−1]=[206−2]2 \begin{bmatrix} 1 & 0 \\ 3 & -1 \end{bmatrix} = \begin{bmatrix} 2 & 0 \\ 6 & -2 \end{bmatrix}2[13​0−1​]=[26​0−2​] .

30. [Matrix operations] A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n , v⃗∈Rn\vec{v} \in \mathbb{R}^nv∈Rn . What is the formula for the components of w⃗=Av⃗\vec{w} = A\vec{v}w=Av , and in which space does w⃗\vec{w}w lie?

    wi=∑j=1naijvj\boldsymbol{w_i = \sum_{j=1}^{n} a_{ij} v_j}wi​=∑j=1n​aij​vj​ for i∈[1,…,m]i \in [1, \dots, m]i∈[1,…,m] , so w⃗∈Rm\boldsymbol{\vec{w} \in \mathbb{R}^m}w∈Rm . Example: [123456][1−1]=[1−23−45−6]=[−1−1−1]∈R3×1\begin{bmatrix} 1 & 2 \\ 3 & 4 \\ 5 & 6 \end{bmatrix} \begin{bmatrix} 1 \\ -1 \end{bmatrix} = \begin{bmatrix} 1 - 2 \\ 3 - 4 \\ 5 - 6 \end{bmatrix} = \begin{bmatrix} -1 \\ -1 \\ -1 \end{bmatrix} \in \mathbb{R}^{3 \times 1}⎣⎡​135​246​⎦⎤​[1−1​]=⎣⎡​1−23−45−6​⎦⎤​=⎣⎡​−1−1−1​⎦⎤​∈R3×1 .

31. [Matrix operations] What are the two ways to understand the matrix-vector product Av⃗A\vec{v}Av ?

    Column picture Av⃗A\vec{v}Av is a linear combination of the columns of AAA , with coefficients v1,…,vnv_1, \dots, v_nv1​,…,vn​ . Example: [1234][1−1]=1[13]−1[24]=[−1−1]\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} 1 \\ -1 \end{bmatrix} = 1 \begin{bmatrix} 1 \\ 3 \end{bmatrix} - 1 \begin{bmatrix} 2 \\ 4 \end{bmatrix} = \begin{bmatrix} -1 \\ -1 \end{bmatrix}[13​24​][1−1​]=1[13​]−1[24​]=[−1−1​] Row picture The iii -th component of Av⃗A\vec{v}Av is the dot product of row iii of AAA w…

32. [Matrix operations] For which sizes of AAA and BBB is the product ABABAB defined, and what size is ABABAB ?

    A∈Rm×ℓA \in \mathbb{R}^{m \times \ell}A∈Rm×ℓ and B∈Rℓ×nB \in \mathbb{R}^{\ell \times n}B∈Rℓ×n (columns of AAA = rows of BBB ) give AB∈Rm×nAB \in \mathbb{R}^{m \times n}AB∈Rm×n . Example: a 3×23 \times 23×2 matrix times a 2×22 \times 22×2 matrix is a 3×23 \times 23×2 matrix.

33. [Matrix operations] A∈Rm×ℓA \in \mathbb{R}^{m \times \ell}A∈Rm×ℓ , B∈Rℓ×nB \in \mathbb{R}^{\ell \times n}B∈Rℓ×n . What is the entry cijc_{ij}cij​ of C=ABC = ABC=AB ?

    cij=∑k=1ℓaikbkj\boldsymbol{c_{ij} = \sum_{k=1}^{\ell} a_{ik} b_{kj}}cij​=∑k=1ℓ​aik​bkj​ : the dot product of row iii of AAA with column jjj of BBB . Example: [1234][5678]=[5+146+1615+2818+32]=[19224350]\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} = \begin{bmatrix} 5 + 14 & 6 + 16 \\ 15 + 28 & 18 + 32 \end{bmatrix} = \begin{bmatrix} 19 & 22 \\ 43 & 50 \end{bmatrix}[13​24​][57​68​]=[5+1415+28​6+1618+32​]=[1943​2250​] .

34. [Matrix operations] What is the transpose ATA^{\mathsf{T}}AT of A∈Rm×nA \in \mathbb{R}^{m \times n}A∈Rm×n ?

    The n×mn \times mn×m matrix with aijT=aji\boldsymbol{a^{\mathsf{T}}_{ij} = a_{ji}}aijT​=aji​ : AAA flipped through its diagonal. Entries on the diagonal are not changed.Example: [α1α2α3β1β2β3]T=[α1β1α2β2α3β3]\begin{bmatrix} \alpha_1 & \alpha_2 & \alpha_3 \\ \beta_1 & \beta_2 & \beta_3 \end{bmatrix}^{\mathsf{T}} = \begin{bmatrix} \alpha_1 & \beta_1 \\ \alpha_2 & \beta_2 \\ \alpha_3 & \beta_3 \end{bmatrix}[α1​β1​​α2​β2​​α3​β3​​]T=⎣⎡​α1​α2​α3​​β1​β2​β3​​⎦⎤​ .

35. [Matrix operations] What are the properties of the transpose operation?

    Sum (A+B)T=AT+BT(A + B)^{\mathsf{T}} = A^{\mathsf{T}} + B^{\mathsf{T}}(A+B)T=AT+BT Example: ([1200]+[0030])T=[1320]=[1020]+[0300]\left( \begin{bmatrix} 1 & 2 \\ 0 & 0 \end{bmatrix} + \begin{bmatrix} 0 & 0 \\ 3 & 0 \end{bmatrix} \right)^{\mathsf{T}} = \begin{bmatrix} 1 & 3 \\ 2 & 0 \end{bmatrix} = \begin{bmatrix} 1 & 0 \\ 2 & 0 \end{bmatrix} + \begin{bmatrix} 0 & 3 \\ 0 & 0 \end{bmatrix}([10​20​]+[03​00​])T=[12​30​]=[12​00​]+[00​30​] Product (AB)T=BTAT(AB)^{\mathsf{T}} = B^{\mathsf{T}} A^{\maths…

36. [Matrix operations] Why is (AB)T=BTAT(AB)^{\mathsf{T}} = B^{\mathsf{T}} A^{\mathsf{T}}(AB)T=BTAT , with the order reversed?

    Entry (i,j)(i, j)(i,j) of both sides is ∑kajkbki\sum_k a_{jk} b_{ki}∑k​ajk​bki​ . (AB)ijT=(AB)ji=∑kajkbki=∑k(BT)ik(AT)kj=(BTAT)ij(AB)^{\mathsf{T}}_{ij} = (AB)_{ji} = \sum_k a_{jk} b_{ki} = \sum_k (B^{\mathsf{T}})_{ik} (A^{\mathsf{T}})_{kj} = (B^{\mathsf{T}} A^{\mathsf{T}})_{ij}(AB)ijT​=(AB)ji​=∑k​ajk​bki​=∑k​(BT)ik​(AT)kj​=(BTAT)ij​ . Also, only the reversed order has matching sizes when AAA and BBB are not square.

37. [Matrix operations] How can a vector v⃗∈Rn\vec{v} \in \mathbb{R}^nv∈Rn be treated as a matrix?

    As a column vector ( n×1n \times 1n×1 matrix) or as a row vector ( 1×n1 \times n1×n matrix). Example: (1,2,3)(1, 2, 3)(1,2,3) as a column vector is [123]\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}⎣⎡​123​⎦⎤​ , as a row vector [123]\begin{bmatrix} 1 & 2 & 3 \end{bmatrix}[1​2​3​] .

38. [Matrix operations] u⃗,v⃗\vec{u}, \vec{v}u,v are column vectors. How is the dot product u⃗⋅v⃗\vec{u} \cdot \vec{v}u⋅v written as a matrix product?

    u⃗⋅v⃗=u⃗Tv⃗\boldsymbol{\vec{u} \cdot \vec{v} = \vec{u}^{\mathsf{T}} \vec{v}}u⋅v=uTv : a 1×n1 \times n1×n matrix times an n×1n \times 1n×1 matrix. So the dot product is a special case of matrix multiplication; equally, matrix multiplication is defined by dot products.Example: [123][456]=4+10+18=32\begin{bmatrix} 1 & 2 & 3 \end{bmatrix} \begin{bmatrix} 4 \\ 5 \\ 6 \end{bmatrix} = 4 + 10 + 18 = 32[1​2​3​]⎣⎡​456​⎦⎤​=4+10+18=32 .

39. [Matrix operations] What is the outer product of two column vectors u⃗,v⃗∈Rn\vec{u}, \vec{v} \in \mathbb{R}^nu,v∈Rn ?

    u⃗v⃗T∈Rn×n\boldsymbol{\vec{u}\vec{v}^{\mathsf{T}} \in \mathbb{R}^{n \times n}}uvT∈Rn×n , the matrix with entries uivju_i v_jui​vj​ . Transposing the first vector gives the inner product u⃗Tv⃗\vec{u}^{\mathsf{T}}\vec{v}uTv (a number); transposing the second gives the outer product (an n×1n \times 1n×1 times a 1×n1 \times n1×n matrix).Example: [12][34]=[3468]\begin{bmatrix} 1 \\ 2 \end{bmatrix} \begin{bmatrix} 3 & 4 \end{bmatrix} = \begin{bmatrix} 3 & 4 \\ 6 & 8 \end{bmatrix}[12​][3​4​]=[36​48​] .

40. [Matrix operations] Which matrix Mx∈R3×3M_x \in \mathbb{R}^{3 \times 3}Mx​∈R3×3 computes the projection of any v⃗∈R3\vec{v} \in \mathbb{R}^3v∈R3 onto the xxx -axis, and why?

    Mx=ı^ı^T=[100000000]\boldsymbol{M_x = \hat{\imath}\hat{\imath}^{\mathsf{T}} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}}Mx​=^^T=⎣⎡​100​000​000​⎦⎤​ Mxv⃗=ı^(ı^Tv⃗)=ı^(ı^⋅v⃗)=vxı^M_x\vec{v} = \hat{\imath}(\hat{\imath}^{\mathsf{T}}\vec{v}) = \hat{\imath}(\hat{\imath} \cdot \vec{v}) = v_x\hat{\imath}Mx​v=^(^Tv)=^(^⋅v)=vx​^ , the projection formula (d^⋅v⃗)d^(\hat{d} \cdot \vec{v})\hat{d}(d^⋅v)d^ with d^=ı^\hat{d} = \hat{\imath}d^=^ .Example: Mx(2,5,7)=(2,0,0)M_x (2, 5,…

41. [Matrix operations] What is the inverse A−1A^{-1}A−1 of a matrix AAA , and does every matrix have one?

    The matrix with AA−1=I=A−1A\boldsymbol{AA^{-1} = I = A^{-1}A}AA−1=I=A−1A ; only invertible (square) matrices have one. A−1A^{-1}A−1 undoes what AAA did: A−1(Av⃗)=(A−1A)v⃗=Iv⃗=v⃗A^{-1}(A\vec{v}) = (A^{-1}A)\vec{v} = I\vec{v} = \vec{v}A−1(Av)=(A−1A)v=Iv=v . It plays the role of "division by AAA " in matrix equations.Example: [1000]\begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}[10​00​] is not invertible: it sends both (0,1)(0, 1)(0,1) and (0,2)(0, 2)(0,2) to 0⃗\vec{0}0 , so no matrix can recover the…

42. [Matrix operations] A,BA, BA,B are invertible. What is (AB)−1(AB)^{-1}(AB)−1 , and why?

    (AB)−1=B−1A−1\boldsymbol{(AB)^{-1} = B^{-1}A^{-1}}(AB)−1=B−1A−1 , in reverse order. (AB)(B−1A−1)=A(BB−1)A−1=AA−1=I(AB)(B^{-1}A^{-1}) = A(BB^{-1})A^{-1} = AA^{-1} = I(AB)(B−1A−1)=A(BB−1)A−1=AA−1=I . Likewise (ABC)−1=C−1B−1A−1(ABC)^{-1} = C^{-1}B^{-1}A^{-1}(ABC)−1=C−1B−1A−1 .To undo "first BBB , then AAA ", first undo AAA , then undo BBB .

43. [Matrix operations] AAA is invertible. What is (AT)−1(A^{\mathsf{T}})^{-1}(AT)−1 , and why?

    (AT)−1=(A−1)T\boldsymbol{(A^{\mathsf{T}})^{-1} = (A^{-1})^{\mathsf{T}}}(AT)−1=(A−1)T AT(A−1)T=(A−1A)T=IT=IA^{\mathsf{T}} (A^{-1})^{\mathsf{T}} = (A^{-1}A)^{\mathsf{T}} = I^{\mathsf{T}} = IAT(A−1)T=(A−1A)T=IT=I , using (XY)T=YTXT(XY)^{\mathsf{T}} = Y^{\mathsf{T}}X^{\mathsf{T}}(XY)T=YTXT .

44. [Matrix operations] AAA , BBB and A+BA + BA+B are invertible. Is (A+B)−1=A−1+B−1(A + B)^{-1} = A^{-1} + B^{-1}(A+B)−1=A−1+B−1 ?

    No, not in general. Example: A=B=IA = B = IA=B=I : (A+B)−1=(2I)−1=12I(A + B)^{-1} = (2I)^{-1} = \frac{1}{2}I(A+B)−1=(2I)−1=21​I , but A−1+B−1=2IA^{-1} + B^{-1} = 2IA−1+B−1=2I .

45. [Matrix operations] What is the trace Tr⁡[A]\operatorname{Tr}[A]Tr[A] of A∈Rn×nA \in \mathbb{R}^{n \times n}A∈Rn×n ?

    The sum of the nnn diagonal entries: Tr⁡[A]=∑i=1naii\boldsymbol{\operatorname{Tr}[A] = \sum_{i=1}^{n} a_{ii}}Tr[A]=∑i=1n​aii​ . Example: Tr⁡[1234]=1+4=5\operatorname{Tr}\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} = 1 + 4 = 5Tr[13​24​]=1+4=5 .

46. [Matrix operations] What are the properties of the trace?

    Linear property Tr⁡[αA+βB]=αTr⁡[A]+βTr⁡[B]\operatorname{Tr}[\alpha A + \beta B] = \alpha \operatorname{Tr}[A] + \beta \operatorname{Tr}[B]Tr[αA+βB]=αTr[A]+βTr[B] Example: Tr⁡[2I+3I]=Tr⁡[5I]=10=2⋅2+3⋅2\operatorname{Tr}[2I + 3I] = \operatorname{Tr}[5I] = 10 = 2 \cdot 2 + 3 \cdot 2Tr[2I+3I]=Tr[5I]=10=2⋅2+3⋅2 for I∈R2×2I \in \mathbb{R}^{2 \times 2}I∈R2×2 . Product Tr⁡[AB]=Tr⁡[BA]\operatorname{Tr}[AB] = \operatorname{Tr}[BA]Tr[AB]=Tr[BA] Example: [1234][5678]=[19224350]\begin{bmatrix} 1 & 2 \\ 3 & 4…

47. [Matrix operations] What is the determinant of a 2×22 \times 22×2 matrix [abcd]\begin{bmatrix} a & b \\ c & d \end{bmatrix}[ac​bd​] ?

    ad−bc\boldsymbol{ad - bc}ad−bc The determinant is a map det⁡:Rn×n→R\det : \mathbb{R}^{n \times n} \to \mathbb{R}det:Rn×n→R , written det⁡(A)\det(A)det(A) or ∣A∣|A|∣A∣ .Example: ∣1234∣=4−6=−2\begin{vmatrix} 1 & 2 \\ 3 & 4 \end{vmatrix} = 4 - 6 = -2∣∣​13​24​∣∣​=4−6=−2 .

48. [Matrix operations] What does the determinant of AAA tell you geometrically?

    The volume of the box whose sides are the rows of AAA . For 2×22 \times 22×2 : ad−bcad - bcad−bc is the area of the parallelogram formed by (a,b)(a, b)(a,b) and (c,d)(c, d)(c,d) (up to sign).Example: rows (2,0)(2, 0)(2,0) and (0,3)(0, 3)(0,3) form a 2×32 \times 32×3 rectangle, and 2⋅3−0⋅0=62 \cdot 3 - 0 \cdot 0 = 62⋅3−0⋅0=6 .

49. [Matrix operations] Why is det⁡[abcd]=0\det \begin{bmatrix} a & b \\ c & d \end{bmatrix} = 0det[ac​bd​]=0 when the rows point in the same direction, (a,b)=α(c,d)(a, b) = \alpha(c, d)(a,b)=α(c,d) ?

    The parallelogram formed by the rows is flat, so its area is 0. Algebraically: ad−bc=(αc)d−(αd)c=0ad - bc = (\alpha c)d - (\alpha d)c = 0ad−bc=(αc)d−(αd)c=0 .Example: ∣2412∣=4−4=0\begin{vmatrix} 2 & 4 \\ 1 & 2 \end{vmatrix} = 4 - 4 = 0∣∣​21​42​∣∣​=4−4=0 .

50. [Matrix operations] What are the properties of determinants?

    Product det⁡(AB)=det⁡(A)det⁡(B)\det(AB) = \det(A)\det(B)det(AB)=det(A)det(B) Example: det⁡[1234]=−2\det\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} = -2det[13​24​]=−2 , det⁡[5678]=−2\det\begin{bmatrix} 5 & 6 \\ 7 & 8 \end{bmatrix} = -2det[57​68​]=−2 , and their product [19224350]\begin{bmatrix} 19 & 22 \\ 43 & 50 \end{bmatrix}[1943​2250​] has determinant 950−946=4950 - 946 = 4950−946=4 . Eigenvalues det⁡(A)=∏i=1nλi\det(A) = \prod_{i=1}^{n} \lambda_idet(A)=∏i=1n​λi​ , where λ1,…,λn\lambda_1, \do…

51. [Linearity] In f(x)=ax+b+mx+qx2+cx3f(x) = \dfrac{a}{x} + b + mx + qx^2 + cx^3f(x)=xa​+b+mx+qx2+cx3 , which term is linear, and what are the others called?

    mxmxmx is the linear term (it contains xxx to the first power); the others are nonlinear. ax\frac{a}{x}xa​ : one-over- xxx term, bbb : constant term, qx2qx^2qx2 : quadratic term, cx3cx^3cx3 : cubic term.

52. [Linearity] What is the most general linear function f:R→Rf : \mathbb{R} \to \mathbb{R}f:R→R ?

    f(x)=mx\boldsymbol{f(x) = mx}f(x)=mx , where m∈Rm \in \mathbb{R}m∈R is the coefficient of xxx . A linear function multiplies its input by the constant mmm .Example: f(x)=2xf(x) = 2xf(x)=2x .

53. [Linearity] f(x)=2xf(x) = 2xf(x)=2x and g(y)=3yg(y) = 3yg(y)=3y . What is h(x)=g(f(x))h(x) = g(f(x))h(x)=g(f(x)) ?

    h(x)=6x\boldsymbol{h(x) = 6x}h(x)=6x The composition of two linear functions is linear, and its coefficient is the product of their coefficients: 3⋅2=63 \cdot 2 = 63⋅2=6 .

54. [Linearity] When is a function fff linear (the general definition)?

    When f(αx1+βx2)=αf(x1)+βf(x2)\boldsymbol{f(\alpha x_1 + \beta x_2) = \alpha f(x_1) + \beta f(x_2)}f(αx1​+βx2​)=αf(x1​)+βf(x2​) for all inputs x1,x2x_1, x_2x1​,x2​ and all constants α,β\alpha, \betaα,β . A linear function maps a linear combination of inputs to the same linear combination of outputs.Example: f(x)=2xf(x) = 2xf(x)=2x : f(αx1+βx2)=2αx1+2βx2=αf(x1)+βf(x2)f(\alpha x_1 + \beta x_2) = 2\alpha x_1 + 2\beta x_2 = \alpha f(x_1) + \beta f(x_2)f(αx1​+βx2​)=2αx1​+2βx2​=αf(x1​)+βf(x2​) .

55. [Linearity] Is the line l(x)=mx+bl(x) = mx + bl(x)=mx+b with b≠0b \neq 0b=0 a linear function? Why?

    No. l(αx1+βx2)=m(αx1+βx2)+bl(\alpha x_1 + \beta x_2) = m(\alpha x_1 + \beta x_2) + bl(αx1​+βx2​)=m(αx1​+βx2​)+b , but αl(x1)+βl(x2)=m(αx1+βx2)+(α+β)b\alpha l(x_1) + \beta l(x_2) = m(\alpha x_1 + \beta x_2) + (\alpha + \beta)bαl(x1​)+βl(x2​)=m(αx1​+βx2​)+(α+β)b . These differ whenever α+β≠1\alpha + \beta \neq 1α+β=1 .Example: l(x)=x+1l(x) = x + 1l(x)=x+1 : l(2⋅1)=3l(2 \cdot 1) = 3l(2⋅1)=3 , but 2 l(1)=42\,l(1) = 42l(1)=4 .

56. [Linearity] A function with a linear part plus a constant term, such as l(x)=mx+bl(x) = mx + bl(x)=mx+b , is called an affine transformation.

    Example: l(x)=3x+2l(x) = 3x + 2l(x)=3x+2 is affine, not linear, since l(0)=2≠0l(0) = 2 \neq 0l(0)=2=0 .

57. [Linearity] What is the most general linear function f:R×R→Rf : \mathbb{R} \times \mathbb{R} \to \mathbb{R}f:R×R→R of two variables?

    f(x,y)=mxx+myy\boldsymbol{f(x, y) = m_x x + m_y y}f(x,y)=mx​x+my​y mxm_xmx​ is the xxx -slope ( xxx -coefficient), mym_ymy​ the yyy -slope ( yyy -coefficient).Example: f(x,y)=2x−5yf(x, y) = 2x - 5yf(x,y)=2x−5y .

58. [Linearity] What is a linear expression in the variables x1,x2,x3x_1, x_2, x_3x1​,x2​,x3​ ?

    An expression a1x1+a2x2+a3x3\boldsymbol{a_1x_1 + a_2x_2 + a_3x_3}a1​x1​+a2​x2​+a3​x3​ with arbitrary constants a1,a2,a3a_1, a_2, a_3a1​,a2​,a3​ . "expr is linear in vvv " means the variable vvv appears only to the first power.Example: 1ax1+b6x2+c x3\frac{1}{a}x_1 + b^6x_2 + \sqrt{c}\,x_3a1​x1​+b6x2​+c​x3​ is linear in x1,x2,x3x_1, x_2, x_3x1​,x2​,x3​ , although the constants 1a\frac{1}{a}a1​ , b6b^6b6 , c\sqrt{c}c​ are nonlinear in a,b,ca, b, ca,b,c .

59. [Linearity] What is a linear equation in the variables x1,x2,x3x_1, x_2, x_3x1​,x2​,x3​ ?

    An equation a1x1+a2x2+a3x3=c\boldsymbol{a_1x_1 + a_2x_2 + a_3x_3 = c}a1​x1​+a2​x2​+a3​x3​=c with constants a1,a2,a3,ca_1, a_2, a_3, ca1​,a2​,a3​,c . It contains only linear terms in the variables.Example: 4k−2m+8p=104k - 2m + 8p = 104k−2m+8p=10 is linear in k,m,pk, m, pk,m,p .

60. [Linearity] 4k−2m+8p=104k - 2m + 8p = 104k−2m+8p=10 . Write mmm as a function of kkk and ppp , and kkk as a function of mmm and ppp .

    m(k,p)=2k+4p−5\boldsymbol{m(k, p) = 2k + 4p - 5}m(k,p)=2k+4p−5 , k(m,p)=104+m2−2p\ \boldsymbol{k(m, p) = \frac{10}{4} + \frac{m}{2} - 2p} k(m,p)=410​+2m​−2p 2m=4k+8p−102m = 4k + 8p - 102m=4k+8p−10 , and 4k=10+2m−8p4k = 10 + 2m - 8p4k=10+2m−8p . The first form predicts mmm from known k,pk, pk,p ; the second predicts kkk from m,pm, pm,p .

61. [Linearity] What does the linear equation ax+by=cax + by = cax+by=c describe in the Cartesian plane?

    A line: for b≠0b \neq 0b=0 , y=−abx+cb\boldsymbol{y = -\frac{a}{b}x + \frac{c}{b}}y=−ba​x+bc​ (slope −ab-\frac{a}{b}−ba​ , yyy -intercept cb\frac{c}{b}bc​ ); for b=0b = 0b=0 , the vertical line x=cax = \frac{c}{a}x=ac​ . Example: 2x+4y=82x + 4y = 82x+4y=8 is the line y=−12x+2y = -\frac{1}{2}x + 2y=−21​x+2 .

62. [Linearity] What does the linear equation ax+by+cz=dax + by + cz = dax+by+cz=d describe in R3\mathbb{R}^3R3 , and what are its slopes when c≠0c \neq 0c=0 ?

    A plane: z=dc−acx−bcy\boldsymbol{z = \frac{d}{c} - \frac{a}{c}x - \frac{b}{c}y}z=cd​−ca​x−cb​y , with xxx -slope −ac-\frac{a}{c}−ca​ , yyy -slope −bc-\frac{b}{c}−cb​ and zzz -intercept dc\frac{d}{c}cd​ . Example: x+2y+2z=4x + 2y + 2z = 4x+2y+2z=4 is the plane z=2−12x−yz = 2 - \frac{1}{2}x - yz=2−21​x−y .

63. [Linearity] What is the first-order approximation (linear model) of f(x)f(x)f(x) near x=0x = 0x=0 ?

    f(x)≈f(0)+f′(0) x\boldsymbol{f(x) \approx f(0) + f'(0)\,x}f(x)≈f(0)+f′(0)x The Taylor series f(x)=∑n=0∞anxnf(x) = \sum_{n=0}^{\infty} a_n x^nf(x)=∑n=0∞​an​xn cut after the linear term: the best approximation near x=0x = 0x=0 by a line mx+bmx + bmx+b , with b=f(0)b = f(0)b=f(0) , m=f′(0)m = f'(0)m=f′(0) .Example: ex≈1+xe^x \approx 1 + xex≈1+x .

64. [Linearity] Why are linear models easy to parametrize from measurements of a process fff ?

    Only two measurements are needed: the initial value b=f(0)b = f(0)b=f(0) , and the rate of change m=ΔfΔxm = \frac{\Delta f}{\Delta x}m=ΔxΔf​ . Input x=0x = 0x=0 to get bbb ; then change the input by Δx\Delta xΔx and observe the change Δf\Delta fΔf in the output. Parametrizing nonlinear models is more complicated.Example: f(0)=3f(0) = 3f(0)=3 and f(2)=7f(2) = 7f(2)=7 give the model f(x)=3+2xf(x) = 3 + 2xf(x)=3+2x .

65. [Linearity] What is the first-order approximation of a function F(x1,…,xn)F(x_1, \dots, x_n)F(x1​,…,xn​) , and how are its parameters measured?

    F(x1,…,xn)≈b+m1x1+⋯+mnxn\boldsymbol{F(x_1, \dots, x_n) \approx b + m_1x_1 + \dots + m_nx_n}F(x1​,…,xn​)≈b+m1​x1​+⋯+mn​xn​ , with b=F(0,…,0)b = F(0, \dots, 0)b=F(0,…,0) and mi=ΔFΔxim_i = \frac{\Delta F}{\Delta x_i}mi​=Δxi​ΔF​ . mim_imi​ is measured by changing only the input xix_ixi​ by a small Δxi\Delta x_iΔxi​ and observing the change ΔF\Delta FΔF .Example: F(0,0)=1F(0, 0) = 1F(0,0)=1 , F(1,0)=3F(1, 0) = 3F(1,0)=3 , F(0,1)=0F(0, 1) = 0F(0,1)=0 give F(x,y)≈1+2x−yF(x, y) \approx 1 + 2x - yF(x,y)…

66. [Linearity] What is the core assumption of linear algebra about the operations it studies?

    They are linear: f(α obj1+β obj2)=αf(obj1)+βf(obj2)\boldsymbol{f(\alpha\,\mathbf{obj}_1 + \beta\,\mathbf{obj}_2) = \alpha f(\mathbf{obj}_1) + \beta f(\mathbf{obj}_2)}f(αobj1​+βobj2​)=αf(obj1​)+βf(obj2​) . The objects can be vectors, matrices, or other vector-like objects.Example: TA(αx⃗+βy⃗)=A(αx⃗+βy⃗)=αAx⃗+βAy⃗T_A(\alpha\vec{x} + \beta\vec{y}) = A(\alpha\vec{x} + \beta\vec{y}) = \alpha A\vec{x} + \beta A\vec{y}TA​(αx+βy​)=A(αx+βy​)=αAx+βAy​ .

67. [Overview of linear algebra] Which algebraic procedure finds the line where two planes in R3\mathbb{R}^3R3 intersect?

    Gauss–Jordan elimination, applied to the system formed by the two plane equations. Example: z=0z = 0z=0 and y=0y = 0y=0 intersect in the xxx -axis.

68. [Overview of linear algebra] What does det⁡(A)=0\det(A) = 0det(A)=0 say about the rows of AAA ?

    They are not linearly independent: at least one row can be written as a combination of the other rows. The determinant is a convenient test for linear independence of a set of vectors.Example: ∣1224∣=0\begin{vmatrix} 1 & 2 \\ 2 & 4 \end{vmatrix} = 0∣∣​12​24​∣∣​=0 , and row 2 is 2⋅2 \cdot2⋅ row 1.

69. [Overview of linear algebra] What is a basis for R2\mathbb{R}^2R2 ?

    A set of two vectors {e^1,e^2}\{\hat{e}_1, \hat{e}_2\}{e^1​,e^2​} such that every v⃗∈R2\vec{v} \in \mathbb{R}^2v∈R2 is a linear combination v1e^1+v2e^2v_1\hat{e}_1 + v_2\hat{e}_2v1​e^1​+v2​e^2​ . v⃗=(vx,vy)\vec{v} = (v_x, v_y)v=(vx​,vy​) means v⃗=vxı^+vyȷ^\vec{v} = v_x\hat{\imath} + v_y\hat{\jmath}v=vx​^+vy​^​ in the standard basis {ı^,ȷ^}\{\hat{\imath}, \hat{\jmath}\}{^,^​} .Example: {ı^,ȷ^}\{\hat{\imath}, \hat{\jmath}\}{^,^​} with ı^=(1,0)\hat{\imath} = (1, 0)^=(1,0) , ȷ^=(0,1)\hat{\jm…

70. [Overview of linear algebra] Does a vector v⃗∈R2\vec{v} \in \mathbb{R}^2v∈R2 have the same coordinates in every basis?

    No: the same vector has a different coordinate pair in each basis. Example: v⃗=(3,1)\vec{v} = (3, 1)v=(3,1) in {ı^,ȷ^}\{\hat{\imath}, \hat{\jmath}\}{^,^​} . With e^1=(1,1)\hat{e}_1 = (1, 1)e^1​=(1,1) , e^2=(1,−1)\hat{e}_2 = (1, -1)e^2​=(1,−1) : v⃗=2e^1+1e^2\vec{v} = 2\hat{e}_1 + 1\hat{e}_2v=2e^1​+1e^2​ , so v⃗=(2,1)\vec{v} = (2, 1)v=(2,1) in the basis {e^1,e^2}\{\hat{e}_1, \hat{e}_2\}{e^1​,e^2​} .

71. [Overview of linear algebra] What is an eigenvector of a matrix AAA , and what is its eigenvalue?

    A nonzero input vector v⃗\vec{v}v on which AAA acts as a scaling: Av⃗=λv⃗\boldsymbol{A\vec{v} = \lambda\vec{v}}Av=λv . The scaling constant λ\lambdaλ is the eigenvalue. "Eigenvalue" means "own value". Knowing all eigenvectors and eigenvalues of a matrix can give a complete description of what the matrix does.Example: [1203][11]=[33]\begin{bmatrix} 1 & 2 \\ 0 & 3 \end{bmatrix} \begin{bmatrix} 1 \\ 1 \end{bmatrix} = \begin{bmatrix} 3 \\ 3 \end{bmatrix}[10​23​][11​]=[33​] , so (1,1)(1, 1)(1,1) is…

72. [Overview of linear algebra] Why are the polynomials of degree at most two, P2(x)={a0+a1x+a2x2}P_2(x) = \{a_0 + a_1x + a_2x^2\}P2​(x)={a0​+a1​x+a2​x2} , "vector-like"?

    Each polynomial is described by its coefficient vector (a0,a1,a2)(a_0, a_1, a_2)(a0​,a1​,a2​) , and sums and scalings of polynomials are computed on those coefficients. So concepts like linear independence, dimension and basis apply to polynomials. A vector space is the abstract concept of a "vector-like" class of objects.Example: (1+2x)+3x2=1+2x+3x2(1 + 2x) + 3x^2 = 1 + 2x + 3x^2(1+2x)+3x2=1+2x+3x2 corresponds to (1,2,0)+(0,0,3)=(1,2,3)(1, 2, 0) + (0, 0, 3) = (1, 2, 3)(1,2,0)+(0,0,3)=(1,2,3) .

73. [Overview of linear algebra] What are two ways to solve a system of nnn linear equations in nnn unknowns with linear algebra?

    Gauss–Jordan elimination Write the coefficients as an augmented matrix and apply the Gauss–Jordan elimination algorithm. You work only with the coefficients, not the variable names. Example: x+2y=5x + 2y = 5x+2y=5 , 3y=63y = 63y=6 has augmented matrix [125036]\left[\begin{array}{cc|c} 1 & 2 & 5 \\ 0 & 3 & 6 \end{array}\right][10​23​56​] . Matrix inverse Write the system as a matrix equation Ax⃗=b⃗A\vec{x} = \vec{b}Ax=b and compute x⃗=A−1b⃗\vec{x} = A^{-1}\vec{b}x=A−1b . Example: [1203]−1=[1−230…

74. [Overview of linear algebra] What is a matrix decomposition, and why is it used?

    Writing a matrix as a product of simpler matrices. Some problems are easier to solve on a computer with the simpler factors; the decomposition into eigenvalues and eigenvectors also shows the properties of the matrix.Example: [2006]=[2002][1003]\begin{bmatrix} 2 & 0 \\ 0 & 6 \end{bmatrix} = \begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix} \begin{bmatrix} 1 & 0 \\ 0 & 3 \end{bmatrix}[20​06​]=[20​02​][10​03​] .

75. [Overview of linear algebra] How is Google's original PageRank algorithm, which ranks webpages by importance, explained with linear algebra?

    As the search for an eigenvector of a matrix that contains all hyperlinks between webpages. The components of the eigenvector give the relative importance of each page.