Linear Algebra

91 flashcards · Mathematics · by Athena Papkouf

1. An indexed set of vectors {v1,…,vp} in Rn\left\{\mathbf{v}_{1}, \ldots, \mathbf{v}_{p}\right\} \text { in } \mathbb{R}^{n}{v1​,…,vp​} in Rn is said to be linearly independent

   if the vector equation x1v1+x2v2+⋯+xpvp=0x_{1} \mathbf{v}_{1}+x_{2} \mathbf{v}_{2}+\cdots+x_{p} \mathbf{v}_{p}=\mathbf{0}x1​v1​+x2​v2​+⋯+xp​vp​=0 has only the trivial solution.

2. An indexed set of vectors {v1,…,vp} in Rn\left\{\mathbf{v}_{1}, \ldots, \mathbf{v}_{p}\right\} \text { in } \mathbb{R}^{n}{v1​,…,vp​} in Rn is said to be linearly dependent

   if there exist weights c1,...,cpc_1,...,c_pc1​,...,cp​ , not all zero, such that c1v1+c2v2+⋯+cpvp=0c_{1} \mathbf{v}_{1}+c_{2} \mathbf{v}_{2}+\cdots+c_{p} \mathbf{v}_{p}=\mathbf{0}c1​v1​+c2​v2​+⋯+cp​vp​=0

3. If a set S={v1,…,vp} in RnS=\left\{\mathbf{v}_{1}, \ldots, \mathbf{v}_{p}\right\} \text { in } \mathbb{R}^{n}S={v1​,…,vp​} in Rn contains the zero vector, then the set is linearly dependent.

4. If a set contains more vectors than there are entries in each vector, then the set is linearly dependent

5. Any set S={v1,…,vp} in RnS=\left\{\mathbf{v}_{1}, \ldots, \mathbf{v}_{p}\right\} \text { in } \mathbb{R}^{n}S={v1​,…,vp​} in Rn is linearly dependent if n"> p>np>np>n .

6. A transformation (or function or mapping) TTT from Rn\mathbb{R}^{n}Rn to Rm\mathbb{R}^{m}Rm is a rule that assigns to each vector x\mathbf{x}x in Rn\mathbb{R}^{n}Rn a vector T(x)T(\mathbf{x})T(x) in Rm\mathbb{R}^{m}Rm . The set Rn\mathbb{R}^{n}Rn is called the domain of TTT , and Rm\mathbb{R}^{m}Rm is called the codomain of TTT The notation T:Rn→RmT: \mathbb{R}^{n} \rightarrow \mathbb{R}^{m}T:Rn→Rm indicates that the domain of TTT is Rn\mathbb{R}^{n}Rn and the codomain is Rm\mathbb{R}^{m}Rm . F…

7. A transformation (or mapping) TTT is linear if:

   (i) T(u+v)=T(u)+T(v)T(\mathbf{u}+\mathbf{v})=T(\mathbf{u})+T(\mathbf{v}) \quadT(u+v)=T(u)+T(v) for all u,v\mathbf{u}, \mathbf{v}u,v in the domain of TTT (ii) T(cu)=cT(u)T(c \mathbf{u})=c T(\mathbf{u}) \quadT(cu)=cT(u) for all scalars ccc and all u\mathbf{u}u in the domain of TTT

8. Surjective transformation

   A mapping T:Rn→RmT: \mathbb{R}^{n} \rightarrow \mathbb{R}^{m} T:Rn→Rm is said to be onto Rm\mathbb{R}^{m}Rm if each b\mathbf{b}b in Rm\mathbb{R}^{m}Rm is the image of at least one x\mathbf{x}x in Rn\mathbb{R}^{n}Rn . Range is all of Rm\mathbb{R}^{m}Rm ∀b∈Rm\forall b \in \mathbb{R}^{m}∀b∈Rm are in the rage of T(x)T(x)T(x) , i.e. T(x‾)=bT(\underline{x})=bT(x​)=b has at least one solution ∀b∈Rm\forall b \in \mathbb{R}^{m}∀b∈Rm the columns of its matrix A span Rm\mathbb{R}^{m}Rm

9. Injective transformation

   A mapping T:Rn→RmT: \mathbb{R}^{n} \rightarrow \mathbb{R}^{m} T:Rn→Rm is said to be one-to-one if each b\mathbf{b}b in Rm\mathbb{R}^{m}Rm is the image of at most one x\mathbf{x}x in Rn\mathbb{R}^{n}Rn . Any two different points x1≠x2x_{1} \neq x_{2}x1​̸​=x2​ in the domain have T(x1)≠T(x2)2T\left(x_{1}\right) \neq T\left(x_{2}\right)^{2}T(x1​)̸​=T(x2​)2 if T(x)=bT(x)=bT(x)=b has a solution (i.e. if bbb is in the range of TTT ), then the solution is unique: T(x1)=bT(x_1)=bT(x1​)=b and T(x2)=bT(x_…

10. (AB)ik(A B)_{i k}(AB)ik​

    ∑j=1naijbjk\sum_{j=1}^{n} a_{i j} b_{j k}∑j=1n​aij​bjk​

11. (AB)T(AB)^T(AB)T

    BTATB^TA^TBTAT

12. (A+B)T(A+B)^{T}(A+B)T

    AT+BTA^{T}+B^{T}AT+BT

13. Let A=[abcd]A=\left[\begin{array}{ll} a & b \\ c & d \end{array}\right]A=[ac​bd​] A−1A^{-1}A−1 ===

    A−1=1ad−bc[d−b−ca]A^{-1}=\frac{1}{a d-b c}\left[\begin{array}{rr} d & -b \\ -c & a \end{array}\right]A−1=ad−bc1​[d−c​−ba​] if ad−bc≠0ad-bc \neq 0ad−bc̸​=0

14. find the i-th column of A−1A^{-1}A−1

    [Ae‾i]∼[IA−1ei]\left[A \underline{e}_{i}\right] \sim\left[I A^{-1} e_{i}\right][Ae​i​]∼[IA−1ei​]

15. inner/scalar product of u and v: u⋅vu \cdot vu⋅v

    uTvu^TvuTv

16. outer product of u and v :

    uvTuv^TuvT

17. row-column rule for computing ABABAB

    (AB)ij=ai1b1j+ai2b2j+⋯+ainbnj(A B)_{i j}=a_{i 1} b_{1 j}+a_{i 2} b_{2 j}+\cdots+a_{i n} b_{n j}(AB)ij​=ai1​b1j​+ai2​b2j​+⋯+ain​bnj​

18. column-row rule for computing ABABAB

    AB=[col⁡1(A)col⁡2(A)⋯col⁡n(A)][row⁡1(B)row⁡2(B)⋮row⁡n(B)]=col⁡1(A)row⁡1(B)+⋯+col⁡n(A)row⁡n(B)\begin{aligned} A B &=\left[\begin{array}{ccc} \operatorname{col}_{1}(A) & \operatorname{col}_{2}(A) & \cdots & \operatorname{col}_{n}(A) \end{array}\right]\left[\begin{array}{c} \operatorname{row}_{1}(B) \\ \operatorname{row}_{2}(B) \\ \vdots \\ \operatorname{row}_{n}(B) \end{array}\right] \\ &=\operatorname{col}_{1}(A) \operatorname{row}_{1}(B)+\cdots+\operatorname{col}_{n}(A) \ope…

19. LU factorisation of A

    row reduce A create E1,E2E_1, E_2E1​,E2​ etc from row operations L=(En⋅En−1⋅...⋅E2⋅E1)−1L=(E_n \cdot E_{n-1} \cdot ... \cdot E_2 \cdot E_1)^{-1}L=(En​⋅En−1​⋅...⋅E2​⋅E1​)−1 U=En⋅En−1⋅...⋅E2⋅E1⋅AU = E_n \cdot E_{n-1} \cdot ... \cdot E_2 \cdot E1 \cdot AU=En​⋅En−1​⋅...⋅E2​⋅E1⋅A

20. If T is a linear transformation, then T(0)=T(\mathbf{0})=T(0)= and T(cu+dv)=T(c \mathbf{u}+d \mathbf{v})=T(cu+dv)= for all vectors u, v in the domain of T and all scalars c; d

    T(0)=0T(cu+dv)=cT(u)+dT(v)\begin{array}{c} T(\mathbf{0})=\mathbf{0} \\ T(c \mathbf{u}+d \mathbf{v})=c T(\mathbf{u})+d T(\mathbf{v}) \end{array}T(0)=0T(cu+dv)=cT(u)+dT(v)​

21. (rA)T(r A)^{T}(rA)T

    rATr A^{T}rAT

22. (AB)−1(A B)^{-1}(AB)−1

    B−1A−1B^{-1} A^{-1}B−1A−1

23. (AT)−1\left(A^{T}\right)^{-1}(AT)−1

    (A−1)T\left(A^{-1}\right)^{T}(A−1)T

24. The Invertible Matrix Theorem (for and nxn matrix) 12 points

    A is an invertible matrix. A is row equivalent to the n x n identity matrix. A has n pivot positions. The equation Ax=0A \mathbf{x}=\mathbf{0}Ax=0 has only the trivial solution. The columns of A form a linearly independent set. The linear transformation x↦Ax\mathbf{x} \mapsto A \mathbf{x}x↦Ax is one-to-one. The equation Ax=bA \mathbf{x}=\mathbf{b}Ax=b has at least one solution for each b in Rn . The columns of A span Rn . The linear transformation x↦Ax\mathbf{x} \mapsto A \mathbf{x}x↦Ax i maps…

25. Determinant definition

    For n≥2,n \geq 2,n≥2, the determinant of an n×nn \times nn×n matrix A=[aij]A=\left[a_{i j}\right]A=[aij​] is the sum of n terms of the form ±a1j\pm a_{1 j}±a1j​ det A1j,A_{1 j},A1j​, with plus and minus signs alternating, where the entries a11,a12,…,a1na_{11}, a_{12}, \ldots, a_{1 n}a11​,a12​,…,a1n​ are from the first row of A . In symbols, det⁡A=a11det⁡A11−a12det⁡A12+⋯+(−1)1+na1ndet⁡A1n=∑j=1n(−1)1+ja1jdet⁡A1j\begin{aligned} \operatorname{det} A &=a_{11} \operatorname{det} A_{11}-a_{12} \op…

26. Let A be a square matrix. a. If a multiple of one row of A is added to another row to produce a matrix B , b. If two rows of A are interchanged to produce B . c. If one row of A is multiplied by k to produce B

    a. detB = det A . b. det B = -det A c. det B = k
    det A

27. If A is an n x n matrix, then det AT = det A .

28. Multiplicative Property of determinants

    If A and B are n x n matrices, then det AB= det A ⋅\cdot⋅ det B

29. Cramer's Rule

    Let AAA be an invertible n×nn \times nn×n matrix. For any b\mathbf{b}b in Rn,\mathbb{R}^{n},Rn, the unique solution x\mathbf{x}x of Ax=bA \mathbf{x}=\mathbf{b}Ax=b has entries given by xi=det⁡Ai(b)det⁡A,i=1,2,…,n x_{i}=\frac{\operatorname{det} A_{i}(\mathbf{b})}{\operatorname{det} A}, \quad i=1,2, \ldots, nxi​=detAdetAi​(b)​,i=1,2,…,n

30. An alternative formula for A-1

    Let A be an invertible n×nn \times nn×n matrix. Then A−1=1det⁡Aadj⁡A A^{-1}=\frac{1}{\operatorname{det} A} \operatorname{adj} AA−1=detA1​adjA

31. Determinants as Area or Volume

    If A is a 2 x 2 matrix, the area of the parallelogram determined by the columns of A is |det A| . If A is a 3 x 3 matrix, the volume of the parallelepiped determined by the columns of A is |det A|.

32. Vector subspace

    A subspace of a vector space V is a subset H of V that has three properties: a. The zero vector of V is in H . b. H is closed under vector addition. That is, for each u and v in H , the sum u + v is in H . c. H is closed under multiplication by scalars. That is, for each u in H and each scalar c , the vector cu is in H .

33. Subspace Spanned by a Set

    If v1,…,vp are in a vector space V, then Span⁡{v1,…,vp} is a subspace of V\text { If } \mathbf{v}_{1}, \ldots, \mathbf{v}_{p} \text { are in a vector space } V, \text { then } \operatorname{Span}\left\{\mathbf{v}_{1}, \ldots, \mathbf{v}_{p}\right\} \text { is a subspace of } V If v1​,…,vp​ are in a vector space V, then Span{v1​,…,vp​} is a subspace of V

34. null space of an m x n matrix A

    the set of all solutions of the homogeneous equation Ax=0. Nul A={x:x is in Rn and Ax=0}\text { Nul } A=\left\{\mathbf{x}: \mathbf{x} \text { is in } \mathbb{R}^{n} \text { and } A \mathbf{x}=\mathbf{0}\right\} Nul A={x:x is in Rn and Ax=0} The null space of an m x n matrix A is a subspace of Rn

35. column space of an m x n matrix A

    Set of all linear combinations of the columns of A col⁡A=Span⁡{a1,…,an}\operatorname{col} A=\operatorname{Span}\left\{\mathbf{a}_{1}, \ldots, \mathbf{a}_{n}\right\}colA=Span{a1​,…,an​} The column space of an m x n matrix A is a subspace of Rm. col⁡A={b:b=Ax for some x in Rn}\operatorname{col} A=\left\{\mathbf{b}: \mathbf{b}=A \mathbf{x} \text { for some } \mathbf{x} \text { in } \mathbb{R}^{n}\right\}colA={b:b=Ax for some x in Rn} The column space of an m x n matrix A is all of Rm if and only…

36. Basis

    Let HHH be a subspace of a vector space VVV . An indexed set of vectors B={b1,…,bp}\mathcal{B}=\left\{\mathbf{b}_{1}, \ldots, \mathbf{b}_{p}\right\}B={b1​,…,bp​} in VVV is a basis for HHH if (i) B\mathcal{B}B is a linearly independent set, and (ii) the subspace spanned by B\mathcal{B}B coincides with HH H ; that is, H=Span⁡{b1,…,bp}H=\operatorname{Span}\left\{\mathbf{b}_{1}, \ldots, \mathbf{b}_{p}\right\}H=Span{b1​,…,bp​}

37. The Spanning Set Theorem

    Let S={v1,…,vp}S=\left\{\mathbf{v}_{1}, \ldots, \mathbf{v}_{p}\right\}S={v1​,…,vp​} be a set in VVV , and let H=Span⁡{v1,…,vp}H=\operatorname{Span}\left\{\mathbf{v}_{1}, \ldots, \mathbf{v}_{p}\right\}H=Span{v1​,…,vp​} a. If one of the vectors in SSS - say, vk\mathbf{v}_{k}vk​ - is a linear combination of the remaining vectors in SSS , then the set formed from SSS by removing vk\mathbf{v}_{k}vk​ still spans HHH b. If H≠{0},H \neq\{\mathbf{0}\},H̸​={0}, some subset of SSS is a basis for HHH .

38. basis for Col A

    The pivot columns of a matrix A form a basis for Col A .

39. The Unique Representation Theorem

    Let B={b1,…,bn}\mathcal{B}=\left\{\mathbf{b}_{1}, \ldots, \mathbf{b}_{n}\right\}B={b1​,…,bn​} be a basis for a vector space VVV . Then for each x\mathbf{x}x in VVV , there exists a unique set of scalars c1,…,cnc_{1}, \ldots, c_{n}c1​,…,cn​ such that x=c1b1+⋯+cnbn\mathbf{x}=c_{1} \mathbf{b}_{1}+\cdots+c_{n} \mathbf{b}_{n}x=c1​b1​+⋯+cn​bn​

40. coordinates of x relative to the basis B

    Suppose B={b1,…,bn}\mathcal{B}=\left\{\mathbf{b}_{1}, \ldots, \mathbf{b}_{n}\right\}B={b1​,…,bn​} is a basis for VVV and x\mathbf{x}x is in VVV . The coordinates of x\mathbf{x}x relative to the basis B\mathcal{B}B (or the B\mathcal{B}B -coordinates of x\mathbf{x}x ) are the weights c1,…,cnc_{1}, \ldots, c_{n}c1​,…,cn​ such that x=c1b1+⋯+cnbn\mathbf{x}=c_{1} \mathbf{b}_{1}+\cdots+c_{n} \mathbf{b}_{n}x=c1​b1​+⋯+cn​bn​

41. coordinate mapping

    Let B={b1,…,bn}\mathcal{B}=\left\{\mathbf{b}_{1}, \ldots, \mathbf{b}_{n}\right\}B={b1​,…,bn​} be a basis for a vector space VVV . Then the coordinate mapping x↦[x]B\mathbf{x} \mapsto[\mathbf{x}]_{\mathcal{B}}x↦[x]B​ is a one-to-one linear transformation from VVV onto Rn\mathbb{R}^{n}Rn .

42. If a vector space VVV has a basis B={b1,…,bn},\mathcal{B}=\left\{\mathbf{b}_{1}, \ldots, \mathbf{b}_{n}\right\},B={b1​,…,bn​}, then any set in VVV containing more than nnn vectors must be linearly dependent.

43. If a vector space V has a basis of n vectors, then every basis of V must consist of exactly n vectors.

44. If V is spanned by a finite set, then V is said to be finite-dimensional, and the dimension of V , written as dim V , is the number of vectors in a basis for V . The dimension of the zero vector space {0} is defined to be zero. If V is not spanned by a finite set, then V is said to be infinite-dimensional.

45. Let H be a subspace of a finite-dimensional vector space V . Any linearly independent set in H can be expanded, if necessary, to a basis for H . Also, H is finite-dimensional and dimH≤dimVdim H \leq dim VdimH≤dimV

46. The Basis Theorem

    Let V be a p-dimensional vector space, p≥1p \geq1p≥1 . Any linearly independent set of exactly p elements in V is automatically a basis for V . Any set of exactly p elements that spans V is automatically a basis for V .

47. dimension of Nul A and Col A

    The dimension of Nul A is the number of free variables in the equation Ax=0, and the dimension of Col A is the number of pivot columns in A

48. Rank

    The rank of A is the dimension of the column space of A.

49. The Rank Theorem

    The dimensions of the column space and the row space of an m x n matrix A are equal. This common dimension, the rank of A , also equals the number of pivot positions in A and satisfies the equation rank A + dim Nul A = n

50. Let B={b1,…,bn}\mathcal{B}=\left\{\mathbf{b}_{1}, \ldots, \mathbf{b}_{n}\right\}B={b1​,…,bn​} and C={c1,…,cn}\mathcal{C}=\left\{\mathbf{c}_{1}, \ldots, \mathbf{c}_{n}\right\}C={c1​,…,cn​} be bases of a vector space VVV . Then there is a unique n×nn \times nn×n matrix PPP C←B_{\mathcal{C} \leftarrow \mathcal{B}}C←B​ such that

    [x]C[x]_C[x]C​ === PPP C←B_{\mathcal{C} \leftarrow \mathcal{B}}C←B​ [x]B[x]_B[x]B​ PPP C←B_{\mathcal{C} \leftarrow \mathcal{B}}C←B​ === [[b1]C[b2]C⋯[bn]C]\left[\begin{array}{lll} {\left[\mathbf{b}_{1}\right]_{\mathcal{C}}} & {\left[\mathbf{b}_{2}\right]_{C}} & \cdots & {\left[\mathbf{b}_{n}\right]_{C}} \end{array}\right][[b1​]C​​[b2​]C​​⋯​[bn​]C​​]

51. Change of basis formula

    [c1c2b1b2]∼[I\left[\begin{array}{llll} \mathbf{c}_{1} & \mathbf{c}_{2} & \mathbf{b}_{1} & \mathbf{b}_{2} \end{array}\right] \sim\left[\begin{array}{l} I \end{array}\right.[c1​​c2​​b1​​b2​​]∼[I​ ∣P| P∣P CB_{CB}CB​ ]]]

52. eigenvector

    An eigenvector of an n x n matrix A is a nonzero vector x such that Ax = λx for some scalar λ.

53. eigenvalue

    A scalar λ is called an eigenvalue of A if there is a nontrivial solution x of Ax = λx; such an x is called an eigenvector corresponding to λ.

54. eigenvalues of a triangular matrix

    The eigenvalues of a triangular matrix are the entries on its main diagonal.

55. If v1,…,vr\mathbf{v}_{1}, \ldots, \mathbf{v}_{r}v1​,…,vr​ are eigenvectors that correspond to distinct eigenvalues λ1,…,λr\lambda_{1}, \ldots, \lambda_{r}λ1​,…,λr​ of an n×nn \times nn×n matrix AAA ,

    then the set {v1,…,vr}\left\{\mathbf{v}_{1}, \ldots, \mathbf{v}_{r}\right\}{v1​,…,vr​} is linearly independent.

56. A scalar λ is an eigenvalue of an n x n matrix A if and only if λ satisfies the characteristic equation

    det⁡(A−λI)=0\operatorname{det}(A-\lambda I)=0det(A−λI)=0

57. similarity of matrices

    If n x n matrices A and B are similar, then they have the same characteristic polynomial and hence the same eigenvalues (with the same multiplicities).

58. The Diagonalization Theorem

    An n×nn \times nn×n matrix AAA is diagonalizable if and only if AAA has n linearly independent eigenvectors.In fact, A=PDP−1A=P D P^{-1}A=PDP−1 , with DDD a diagonal matrix, if and only if the columns of PPP are n linearly independent eigenvectors of A . In this case, the diagonal entries of D are eigenvalues of A that correspond, respectively, to the eigenvectors in P.

59. An n x n matrix with n distinct eigenvalues is diagonalizable.

60. Let A be an n x n matrix whose distinct eigenvalues are λ1\lambda_1λ1​ ,...,λp,...,\lambda_p,...,λp​ a. For 1 ≤\leq≤ k ≤\leq≤ p , the dimension of the eigenspace for λk\lambda_kλk​ ... b. The matrix A is diagonalizable if c. If A is diagonalizable and Bk is a basis for the eigenspace corresponding to λk\lambda_kλk​ for each k , then the total collection of vectors in the sets B1,...,BpB_1,...,B_pB1​,...,Bp​ ...

    a. is less than or equal to the multiplicity of the eigenvalue λk\lambda_kλk​ . b. and only if the sum of the dimensions of the eigenspaces equals n , and this happens if and only if (i) the characteristic polynomial factors completely into linear factors and (ii) the dimension of the eigenspace for each λk\lambda_kλk​ equals the multiplicity of λk\lambda_kλk​ . c. forms an eigenvector basis for Rn .

61. [T]B←B[T]_{\mathcal{B} \leftarrow \mathcal{B}}[T]B←B​

    [T]B←B=B−1AB[T]_{\mathcal{B} \leftarrow \mathcal{B}}=B^{-1} A B[T]B←B​=B−1AB (B∣AB)→(B | AB) \to(B∣AB)→ (I∣(I |(I∣ [T]B←B[T]_{\mathcal{B} \leftarrow \mathcal{B}}[T]B←B​ )))

62. [T]CB[T]_{CB}[T]CB​

    (C∣AB)→(I∣[T]CB)(C | AB) \to (I | [T]_{CB})(C∣AB)→(I∣[T]CB​) C−1ABC^{-1}ABC−1AB

63. Diagonal Matrix Representation

    Suppose A=PDP−1A=P D P^{-1}A=PDP−1 where DDD is a diagonal n×nn \times nn×n matrix. If B\mathcal{B}B is the basis for Rn\mathbb{R}^{n}Rn formed from the columns of PPP then DDD is the B\mathcal{B}B -matrix for the transformation x↦Ax\mathbf{x} \mapsto A \mathbf{x}x↦Ax

64. norm ∥v∥\|\mathbf{v}\|∥v∥

    v⋅v=v12+v22+⋯+vn2, and ∥v∥2=v⋅v\sqrt{\mathbf{v} \cdot \mathbf{v}}=\sqrt{v_{1}^{2}+v_{2}^{2}+\cdots+v_{n}^{2}}, \quad \text { and } \quad\|\mathbf{v}\|^{2}=\mathbf{v} \cdot \mathbf{v}v⋅v ​=v12​+v22​+⋯+vn2​ ​, and ∥v∥2=v⋅v

65. ∥cv∥\|c \mathbf{v}\|∥cv∥

    ∣c∣∥v∥|c|\|\mathbf{v}\|∣c∣∥v∥

66. dist⁡(u,v)\operatorname{dist}(\mathbf{u}, \mathbf{v})dist(u,v)

    ∥u−v∥\|\mathbf{u}-\mathbf{v}\|∥u−v∥

67. two vectors are orthogonal if

    u⋅v=0\mathbf{u} \cdot \mathbf{v}=0u⋅v=0

68. Two vectors u and v are orthogonal if and only if

    ∥u+v∥2=∥u∥2+∥v∥2\|\mathbf{u}+\mathbf{v}\|^{2}=\|\mathbf{u}\|^{2}+\|\mathbf{v}\|^{2}∥u+v∥2=∥u∥2+∥v∥2

69. 1. A vector x is in W⊥W^{\perp}W⊥ if and only if 2. W⊥W^{\perp}W⊥ is a subspace of.. .

    x is orthogonal to every vector in a set that spans W . Rn

70. (Row⁡A)⊥(\operatorname{Row} A)^{\perp}(RowA)⊥

    Nul⁡A\operatorname{Nul} A \quadNulA

71. (Col⁡A)⊥\quad(\operatorname{Col} A)^{\perp}(ColA)⊥

    Nul⁡AT\operatorname{Nul} A^{T}NulAT

72. u⋅v\mathbf{u} \cdot \mathbf{v}u⋅v

    ∥u∥∥v∥cos⁡ϑ\|\mathbf{u}\|\|\mathbf{v}\| \cos \vartheta∥u∥∥v∥cosϑ

73. If S={u1,…,up}S=\left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}S={u1​,…,up​} is an orthogonal set of nonzero vectors in Rn,\mathbb{R}^{n},Rn,

    then SSS is linearly independent and hence is a basis for the subspace spanned by S.

74. An orthogonal basis for a subspace W of Rn is

    a basis for W that is also an orthogonal set.

75. Let {u1,…,up}\left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}{u1​,…,up​} be an orthogonal basis for a subspace WWW of Rn.\mathbb{R}^{n} .Rn. For each y\mathbf{y}y in WWW the weights in the linear combination y=c1u1+⋯+cpup\mathbf{y}=c_{1} \mathbf{u}_{1}+\cdots+c_{p} \mathbf{u}_{p}y=c1​u1​+⋯+cp​up​ are given by

    cj=y⋅ujuj⋅uj(j=1,…,p) c_{j}=\frac{\mathbf{y} \cdot \mathbf{u}_{j}}{\mathbf{u}_{j} \cdot \mathbf{u}_{j}} \quad(j=1, \ldots, p) cj​=uj​⋅uj​y⋅uj​​(j=1,…,p)

76. y^=proj⁡Ly\hat{\mathbf{y}}=\operatorname{proj}_{L} \mathbf{y}y^​=projL​y

    y⋅uu⋅uu\frac{\mathbf{y} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}} \mathbf{u}u⋅uy⋅u​u

77. An mxn matrix U has orthonormal columns .

    if and only if UT U=I

78. Let UUU be an m×nm \times nm×n matrix with orthonormal columns, and let x\mathbf{x}x and y\mathbf{y}y be in Rn\mathbb{R}^{n}Rn . Then a. ∥Ux∥\|U \mathbf{x}\|∥Ux∥ b. (Ux)⋅(Uy)(U \mathbf{x}) \cdot(U \mathbf{y})(Ux)⋅(Uy) c. (Ux)⋅(Uy)=0(U \mathbf{x}) \cdot(U \mathbf{y})=0(Ux)⋅(Uy)=0 if and only if

    a. ∥x∥\|\mathbf{x}\|∥x∥ b. x⋅y\mathbf{x} \cdot \mathbf{y}x⋅y c. x⋅y=0\mathbf{x} \cdot \mathbf{y}=0x⋅y=0

79. The Orthogonal Decomposition Theorem

    Let WWW be a subspace of Rn\mathbb{R}^{n}Rn . Then each y\mathbf{y}y in Rn\mathbb{R}^{n}Rn can be written uniquely in the form y=y^+z\mathbf{y}=\hat{\mathbf{y}}+\mathbf{z}y=y^​+z where y^\hat{\mathbf{y}}y^​ is in WWW and z\mathbf{z}z is in W⊥.W^{\perp} .W⊥. In fact, if {u1,…,up}\left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}{u1​,…,up​} is any orthogonal basis of WWW then y^=y⋅u1u1⋅u1u1+⋯+y⋅upup⋅upup\hat{\mathbf{y}}=\frac{\mathbf{y} \cdot \mathbf{u}_{1}}{\mathbf{u}_{1} \cdot \mathbf{u}_{1}…

80. If y is in W=Span⁡{u1,…,up}, then \text { If } \mathbf{y} \text { is in } W=\operatorname{Span}\left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}, \text { then } If y is in W=Span{u1​,…,up​}, then

    proj⁡Wy=y\operatorname{proj}_{W} \mathbf{y}=\mathbf{y}projW​y=y

81. The Best Approximation Theorem

    Let WWW be a subspace of Rn,\mathbb{R}^{n},Rn, let y\mathbf{y}y be any vector in Rn,\mathbb{R}^{n},Rn, and let y^\hat{\mathbf{y}}y^​ be the orthogonal projection of y\mathbf{y}y onto WWW . Then y^\hat{\mathbf{y}}y^​ is the closest point in WWW to y,\mathbf{y},y, in the sense that ∥y−y^∥&lt;∥y−v∥\|\mathbf{y}-\hat{\mathbf{y}}\|&lt;\|\mathbf{y}-\mathbf{v}\|∥y−y^​∥<∥y−v∥ for all v\mathbf{v}v in WWW distinct from y^\hat{\mathbf{y}}y^​ .

82. If {u1,…,up}\left\{\mathbf{u}_{1}, \ldots, \mathbf{u}_{p}\right\}{u1​,…,up​} is an orthonormal basis for a subspace WWW of Rn,\mathbb{R}^{n},Rn, then If U=[u1u2⋯up],U=\left[\begin{array}{llll}\mathbf{u}_{1} &amp; \mathbf{u}_{2} &amp; \cdots &amp; \mathbf{u}_{p}\end{array}\right],U=[u1​​u2​​⋯​up​​], then

    proj⁡Wy=(y⋅u1)u1+(y⋅u2)u2+⋯+(y⋅up)up\operatorname{proj}_{W} \mathbf{y}=\left(\mathbf{y} \cdot \mathbf{u}_{1}\right) \mathbf{u}_{1}+\left(\mathbf{y} \cdot \mathbf{u}_{2}\right) \mathbf{u}_{2}+\cdots+\left(\mathbf{y} \cdot \mathbf{u}_{p}\right) \mathbf{u}_{p}projW​y=(y⋅u1​)u1​+(y⋅u2​)u2​+⋯+(y⋅up​)up​ proj⁡Wy=UUTy\operatorname{proj}_{W} \mathbf{y}=U U^{T} \mathbf{y} \quadprojW​y=UUTy for all y\mathbf{y}y in Rn\mathbb{R}^{n}Rn

83. The Gram--Schmidt Process

    Given a basis {x1,…,xp}\left\{\mathbf{x}_{1}, \ldots, \mathbf{x}_{p}\right\}{x1​,…,xp​} for a nonzero subspace WWW of Rn\mathbb{R}^{n}Rn define v1=x1v2=x2−x2⋅v1v1⋅v1v1v3=x3−x3⋅v1v1⋅v1v1−x3⋅v2v2⋅v2v2\begin{array}{l} \mathbf{v}_{1}=\mathbf{x}_{1} \\ \mathbf{v}_{2}=\mathbf{x}_{2}-\frac{\mathbf{x}_{2} \cdot \mathbf{v}_{1}}{\mathbf{v}_{1} \cdot \mathbf{v}_{1}} \mathbf{v}_{1} \\ \mathbf{v}_{3}=\mathbf{x}_{3}-\frac{\mathbf{x}_{3} \cdot \mathbf{v}_{1}}{\mathbf{v}_{1} \cdot \mathbf{v}_{1}} \mathbf{v}_{1…

84. least-squares solution of Ax=b

    If AAA is m×nm \times nm×n and b\mathbf{b}b is in Rm,\mathbb{R}^{m},Rm, a least-squares solution of Ax=bA \mathbf{x}=\mathbf{b}Ax=b is an x^\hat{\mathbf{x}}x^ in Rn\mathbb{R}^{n}Rn such that ∥b−Ax^∥≤∥b−Ax∥\|\mathbf{b}-A \hat{\mathbf{x}}\| \leq\|\mathbf{b}-A \mathbf{x}\|∥b−Ax^∥≤∥b−Ax∥ for all x\mathbf{x}x in Rn\mathbb{R}^{n}Rn

85. each least-squares solution of Ax=b satisfies the equation

    ATAx=ATbA^{T} A \mathbf{x}=A^{T} \mathbf{b}ATAx=ATb

86. The set of least-squares solutions of Ax=bA \mathbf{x}=\mathbf{b}Ax=b coincides with

    the nonempty set of solutions of the normal equations ATAx=ATbA^{T} A \mathbf{x}=A^{T} \mathbf{b}ATAx=ATb .

87. Let $A$ be an m×nm \times nm×n matrix. The following statements are logically equivalent: a. The equation Ax=bA \mathbf{x}=\mathbf{b}Ax=b has b. The columns of AAA are linearly . c. The matrix ATAA^{T} AATA is . When these statements are true, the least-squares solution x^\hat{\mathbf{x}}x^ is given by

    a. a unique least-squares solution for each b\mathbf{b}b in Rm\mathbb{R}^{m}Rm b. independent c. invertible x^=(ATA)−1ATb\hat{\mathbf{x}}=\left(A^{T} A\right)^{-1} A^{T} \mathbf{b}x^=(ATA)−1ATb

88. If A is symmetric, then any two eigenvectors from different eigenspaces are orthogonal.

89. An nxn matrix A is orthogonally diagonalizable if and only if A is a symmetric matrix

90. The Spectral Theorem for Symmetric Matrices

    An n×nn \times nn×n symmetric matrix A has the following properties: a. A has n real eigenvalues, counting multiplicities. b. The dimension of the eigenspace for each eigenvalue λ\lambdaλ equals the multiplicity of λ\lambdaλ as a root of the characteristic equation. c. The eigenspaces are mutually orthogonal, in the sense that eigenvectors corresponding to different eigenvalues are orthogonal. d. A is orthogonally diagonalizable.

91. spectral decomposition of A

    A=λ1u1u1T+λ2u2u2T+⋯+λnununTA=\lambda_{1} \mathbf{u}_{1} \mathbf{u}_{1}^{T}+\lambda_{2} \mathbf{u}_{2} \mathbf{u}_{2}^{T}+\cdots+\lambda_{n} \mathbf{u}_{n} \mathbf{u}_{n}^{T}A=λ1​u1​u1T​+λ2​u2​u2T​+⋯+λn​un​unT​