Section 2.10 Matrix Algebra
Theorem 2.10.2. Additive identity - the zero matrix.
The zero matrix \(\boldsymbol{0}_{m \times n},\) is the unique additive identity for matrix addition in \(\R^{m \times n}.\)Proof.
Theorem 2.10.3.
Let \(m,n,p\in\N\) and suppose \(\A\in\R^{m\times n}.\) Then \(\A\boldsymbol{0}_{n\times p}=\boldsymbol{0}_{m\times p}\) and \(\boldsymbol{0}_{p\times m}\A=\boldsymbol{0}_{p \times n}.\)Proof.
Definition 2.10.4. Matrix subtraction.
Suppose \(\A,\B\in\R^{m\times n}.\) Then \(\A-\B\) is defined as \(\A-\B=\A+(-1)\B.\) That is,
\begin{equation*}
(\A-\B)_{ij}=A_{ij}-B_{ij}
\end{equation*}
Matrix subtraction inverts matrix addition in the following sense: for \(\A,\B\in\R^{m\times n},\,\A+\B-\B=\A.\)
Theorem 2.10.5.
Suppose \(\A,\B\in\R^{m\times n}.\) Then \(\A-\B=\boldsymbol{0}_{m\times n}\Leftrightarrow \A=\B.\)Proof.
Theorem 2.10.6. Matrix multiplication distributive laws.
Whenever the expressions in the following are defined,
\begin{align*}
\A(\C\pm \D)\amp =\A\C\pm \A\D\qquad\text{and}\\
(\A\pm \B)\C\amp =\A\C\pm \B\C
\end{align*}
Proof.
Theorem 2.10.7. Left multiplication is well-defined.
Given a matrix \(\A\) and equal matrices \(\C\) and \(\D\) for which the products \(\A\C\) and \(\A\D\) are defined, one has \(\A\C=\A\D.\)Proof.
Theorem 2.10.8. Right multiplication is well-defined.
Given a matrix \(\A\) and equal matrices \(\C\) and \(\D\) for which the products \(\C\A\) and \(\D\A\) are defined, one has \(\C\A=\D\A.\)Proof.
Theorem 2.10.9. Matrix multiplication is associative.
Whenever both matrix products are defined, \(\A(\B\C)=(\A\B)\C.\)Proof.
Remark 2.10.10.
For real number multiplication, there is one identity \((1)\) and for any nonzero real number \(x,\) there is a unique inverse \(\displaystyle{\frac{1}{x}}\) for which \(\displaystyle{x\cdot\frac{1}{x}=\frac{1}{x}\cdot x=1.}\)The non-commutativity of matrix multiplication prevents such simplicities in linear algebra.
Indeed for a given nonsquare matrix, identities are different from each side and inverses are defined from only one side and are highly non-unique. (Square matrices always have a unique two-sided identity and have a unique two-sided inverse if they are invertible from either side.)
Definition 2.10.11. Left- and right identities for matrix multiplication.
Let \(\A\in\R^{m\times n}.\) A left-identity \(\I_{L\A}\) for \(\A\) satisfies \(\I_{L\A}\A=\A\) and a right-identity \(\I_{R\A}\) for \(\A\) satisfies \(\A\I_{R\A}=\A.\)Theorem 2.10.12. Properties of left- and right-identities.
Let \(\A\in\R^{m\times n}.\) If \(m\gt n\) then \(\A\) has an infinite number of left-identities \(\I_{L\A}\in\R^{m\times m}\) and a unique right-identity \(\I_n.\) \({ }^{ }\)If \(m\lt n\) then \(\A\) has an infinite number of right-identities \(\I_{R\A}\in\R^{n\times n}\) and a unique left-identity \(\I_m.\)Proof.
Example 2.10.13. Left-identities of \(\A\in\R^{3\times2}\).
\begin{equation*}
\left(\begin{array}{rrr}a\amp b\amp c\\d\amp e\amp f\\g\amp h\amp i\end{array}\right)\left(\begin{array}{rr}1\amp 2\\3\amp 5\\2\amp 7\end{array}\right)=\left(\begin{array}{rr}1\amp 2\\3\amp 5\\2\amp 7\end{array}\right).
\end{equation*}
Expressing this more explicitly we have
\begin{align*}
a\,\,+\,\,3b\,\,+\,\,2c\amp\,\,\,=\,\,\,1\\
2a\,\,+\,\,5b\,\,+\,\,7c\amp\,\,\,=\,\,\,2\\
d\,\,+\,\,3e\,\,+\,\,2f\amp\,\,\,=\,\,\,3\\
2d\,\,+\,\,5e\,\,+\,\,7f\amp\,\,\,=\,\,\,5\\
g\,\,+\,\,3h\,\,+\,\,2i\amp\,\,\,=\,\,\,2\\
2g\,\,+\,\,5h\,\,+\,\,7i\amp\,\,\,=\,\,\,7
\end{align*}
which is a system of six \((m\cdot n)\) equations in nine \((m\cdot m)\) unknowns so three of the entries of \(\I_{L\A}\) will be arbitrary. We choose \(c,f,i\) to be arbitrary and express the other variables \(a,b,d,e,g,h\) in terms of \(c,f,i.\) After some algebra we find
\begin{equation*}
\I_{L\A}=\left(\begin{array}{ccc}1-11c\amp 3c\amp c\\-11f\amp 1+3f\amp f\\11-11i\amp -3+3i\amp i\end{array}\right).
\end{equation*}
We may express the set of left identities for \(\A\) as
\begin{equation*}
\left\{\left.\left(\begin{array}{ccc}1-11c\amp 3c\amp c\\-11f\amp 1+3f\amp f\\11-11i\amp -3+3i\amp i\end{array}\right)\,\right|\,c,f,i\in\R\right\}.
\end{equation*}
By Theorem 2.10.12 the unique right-identity for \(\A\) is \(\I_2.\)Given \(\A\in\R^{m\times n}\) with \(m\gt n,\) the number of equations defining \(\I_{L\A}\) will always be fewer than the number of entries in \(\I_{L\A}\) so there will always be an infinite number of left-identities when defined. The analogous statement holds for right-identities.
This fact makes general left- and right-identities less useful than we might hope, so we will not dwell on left- nor right-identities though we do offer one homework problem which asks us to prove Theorem 2.10.12 and another which asks us to compute left-and right-identities for a few matrices.
We will focus generally on the case when \(\A\) has both a left-identity and a right-identity. This requires 1) that \(\A\) is square and 2) that \(\I_{L\A}\) and \(\I_{R\A}\) are equal, and also determines completely the entries of the (now unique) inverse.
Definition 2.10.14. Matrix multiplicative identity.
Let \(n\in\N.\) An identity for matrix multiplication in \(\R^{n\times n}\) is a matrix \(\I\in\R^{n \times n}\) such that \(\A\I=\I\A=\A\) for all \(\A \in \R^{n \times n}.\) Note also that \(\fa\b\in\R^n,\,\I\b=\b.\)Definition 2.10.15.
Let \(n\in\N.\) Define \(\I_n\) as follows:
\begin{align*}
\I_n=\left(\begin{array}{cccc}1\amp 0\amp \cdots\amp 0\\
0\amp 1\amp \cdots\amp 0\\
\vdots\amp \amp \ddots\amp \vdots\\
0\amp 0\amp \cdots\amp 1\end{array}\right)
\end{align*}
Theorem 2.10.16.
\(\I_n\) is the unique multiplicative identity for matrix multiplication in \(\R^{n\times n}.\)Proof.
When the size is unambiguous we will just write \(\I\) instead of \(\I_n\) and when the context is clear we will use the term identity instead of additive identity or multiplicative identity.
We solve \(3x=6\) by multiplying each side by \(\displaystyle{3^{-1}=\frac{1}{3}}\) which yields \(\displaystyle{x=3^{-1}\cdot6=\frac{6}{3}=2}.\) It is natural to ask if there is such a thing as an inverse under matrix multiplication for a given \(\A\in\R^{n\times n}.\) Though we provide a definition which works for any size square matrices, we will focus for now on \(\R^{2\times 2},\) determining the condition under which \(\A\in\R^{2\times2}\) is invertible and finding a formula for \(\A^{-1}\) when the condition is met. We will later extend our treatment to \(\R^{n\times n}.\)
Definition 2.10.17. Left- and right-inverses.
Let \(\A\in\R^{m\times n}.\) A matrix \(\B\) is a left-inverse of \(\A\) if \(\B\A=\I,\) and if such a \(\B\) exists we denote it \(\A^{-1}_L.\) A matrix \(\C\) is a right-inverse of \(\A\) if \(\A\C=\I,\) and if such a \(\C\) exists we denote it \(\A^{-1}_R.\)Theorem 2.10.18. Sizes of left- and right-inverses.
Let \(\A\in\R^{m\times n}.\) Any left- or right-inverse of \(\A\) necessarily lies in \(\R^{n\times m}.\) A left-inverse \(\B\) exists only if \(m\ge n.\) Similarly, a right-inverse \(\C\) exists only if \(m\le n.\)Proof.
Example 2.10.19. Left-inverses of \(\A\in\R^{3\times2}\).
\begin{equation*}
\left(\begin{array}{rrr}a\amp b\amp c\\d\amp e\amp f\end{array}\right)\left(\begin{array}{rr}1\amp 2\\3\amp 5\\2\amp 7\end{array}\right)=\left(\begin{array}{rr}1\amp 0\\0\amp 1\end{array}\right).
\end{equation*}
Expressing this more explicitly we have
\begin{align*}
a\,\,+\,\,3b\,\,+\,\,2c\amp\,\,\,=\,\,\,1\\
2a\,\,+\,\,5b\,\,+\,\,7c\amp\,\,\,=\,\,\,0\\
d\,\,+\,\,3e\,\,+\,\,2f\amp\,\,\,=\,\,\,0\\
2d\,\,+\,\,5e\,\,+\,\,7f\amp\,\,\,=\,\,\,1
\end{align*}
which is a system of four equations in six unknowns so two of the entries of \(\A^{-1}_L\) will be arbitrary. We choose \(c,f\) to be arbitrary and express the other variables \(a,b,d,e\) in terms of \(c,f.\) After some algebra we find
\begin{equation*}
\A^{-1}_L=\left(\begin{array}{ccc}-5-11c\amp 2+3c\amp c\\3-11f\amp -1+3f\amp f\end{array}\right).
\end{equation*}
We may express the set of left-inverses of \(\A\) as
\begin{equation*}
\left\{\left.\left(\begin{array}{rrr}-5-11c\amp 2+3c\amp c\\3-11f\amp -1+3f\amp f\end{array}\right)\,\right|\,c,f\in\R\right\}.
\end{equation*}
The matrix \(\A\) has no right-inverse but a matrix such as \(\B=\left(\begin{array}{rrr}1\amp 5\amp -4\\2\amp 7\amp -6\end{array}\right)\) has a right-inverse \(\B^{-1}_R\) which may be found in essentially the same manner as \(\A^{-1}_L\) above.Given \(\A\in\R^{m\times n}\) with \(m\gt n,\) the number of equations defining \(\A^{-1}_L\) will always be fewer than the number of entries in \(\A^{-1}_L\) so there will always be an infinite number of left-identities when defined. The analogous statement holds for right-identities.
Though they are more useful than left- and right-identities, we will not focus much on left- nor right-inverses though we do offer one homework problem which asks us to compute a few left-and right-inverses and another which introduces a specific left- and right-inverse.
We will focus generally on the case when \(\A\) has both a left-inverse and a right-inverse. Then 1) \(\A\) must be square and 2) \(\A^{-1}_L\) and \(\A^{-1}_R\) must be equal, and also determines completely the entries of the (now unique) inverse.
Theorem 2.10.20. For square matrices, left- and right-inverses coincide.
Let \(\A,\B\in\R^{n\times n}\) and suppose \(\A\) is invertible and that \(\A\B=\I.\) Then \(\B\A=\I.\)Proof.
Definition 2.10.21. Matrix inverse.
Let \(\A\in\R^{n\times n}.\) If there exists a matrix \(\B\in\R^{n\times n}\) satisfying \(\A\B=\I\) then we say that \(\A\) is invertible (or nonsingular) with \(\B=\A^{-1},\) and we call \(\A^{-1}\) the multiplicative inverse (or just inverse) of \(\A\) and say “\(\A\)-inverse.”Square matrices which are not invertible are called singular.
Theorem 2.10.22. Matrix inverses are unique.
Let \(\A\in\R^{n\times n}.\) If \(\A^{-1}\) exists, it is unique.Proof.
Theorem 2.10.23. \(\boldsymbol{2\times2}\) matrix inverse.
Let \(\A=\begin{pmatrix}a\amp b\\c\amp d \end{pmatrix} \in\R^{2\times 2}.\) Then if \(ad-bc\ne0,\) we have
\begin{equation*}
\A^{-1}=\frac{1}{ad-bc}\left(\begin{array}{rr}d\amp -b\\-c\amp a \end{array} \right).
\end{equation*}
Further, if \(ad-bc=0\) then \(\A\) is singular.Proof.
Theorem 2.10.24. Inverse of a diagonal matrix.
Let \(\D=\left(\begin{array}{cccc}D_{11}\amp 0\amp \cdots\amp 0\\0\amp D_{22}\amp \cdots\amp 0\\\vdots\amp \amp \ddots\amp \vdots\\0\amp \cdots\amp \cdots\amp D_{nn}\end{array}\right).\) Then if all \(D_{ii}\ne0,\) we have \(\D^{-1}=\left(\begin{array}{cccc}D_{11}^{-1}\amp 0\amp \cdots\amp 0\\0\amp D_{22}^{-1}\amp \cdots\amp 0\\\vdots\amp \amp \ddots\amp \vdots\\0\amp \cdots\amp \cdots\amp D_{nn}^{-1}\end{array}\right).\) If any \(D_{ii}=0\) then \(\D^{-1}\) does not exist.So, when it exists we can find the matrix inverse for \(2\times2\) matrices and for diagonal matrices by easy computations.
For \(3\times3\) and larger matrices the process of finding the inverse, when it exists, is more complicated and addressed in Section 3.10.
Theorem 2.10.25.
Let \(\A\in\R^{n\times n},\,\b\in\R^n.\) If \(\A\) is nonsingular then \(\x=\A^{-1}\b\) is the unique solution to the equation \(\A\x=\b.\)Proof.
Using the matrix inverse. Although Theorem 2.10.25 yields an elegant solution to \(\A\x=\b,\) it turns out to be more efficient to solve \(\A\x=\b\) by a different method called Gaussian elimination, discussed in Section 3.1. Matrix inverses do have important real-world applications in 3D animation and beam deflection and in many theoretical applications of which we will enounter several.
Theorem 2.10.26.
Let \(\A\in \R^{n \times n}\) be invertible. Then \((\A^{-1})^{-1}=\A.\)Proof.
Multiplication by the matrix inverse, when it exists, inverts matrix multiplication in the following senses:
\begin{equation*}
\B^{-1}\B\A=\I\A=\A=\A\I=\A\B\B^{-1}.
\end{equation*}
However, since matrix multiplication is not commutative we should expect neither \(\B\A\B^{-1}=\A\) nor \(\B^{-1}\A\B=\A.\) Indeed, we can construct a counterexample pair \(\A,\B\) as follows.
For \(\A=\left(\begin{array}{rrr}1\amp 1\amp 0\\0\amp 1\amp 0\\0\amp 0\amp 1\end{array}\right)\) and \(\B=\left(\begin{array}{rrr}1\amp 0\amp 0\\1\amp 1\amp 0\\0\amp 0\amp 1\end{array}\right)\) with (as the reader can check) \(\B^{-1}=\left(\begin{array}{rrr}1\amp 0\amp 0\\-1\amp 1\amp 0\\0\amp 0\amp 1\end{array}\right),\) we have (as the reader can check)
\begin{equation*}
\B\A\B^{-1}=\left(\begin{array}{rrr}0\amp 1\amp 0\\-1\amp 2\amp 0\\0\amp 0\amp 1\end{array}\right)
\end{equation*}
while
\begin{equation*}
\B^{-1}\A\B=\left(\begin{array}{rrr}2\amp 1\amp 0\\-1\amp 0\amp 0\\0\amp 0\amp 1\end{array}\right).
\end{equation*}
Theorem 2.10.27. Matrix product inverse - two factors.
If \(\A,\B\in\R^{n\times n}\) are invertible, then \(\A\B\) is invertible and \((\A\B)^{-1}=\B^{-1}\A^{-1}.\)Proof.
Corollary 2.10.28. Matrix product inverse - \(\boldsymbol{k}\) factors.
Let \(\A_1,\A_2,...,\A_k \in \R^{n\times n}\) be invertible. Then \(\A_1\A_2\cdots \A_k\) is invertible and \((\A_1\A_2\cdots \A_k)^{-1} = \A_k^{-1}\A_{k-1}^{-1}\cdots \A_2^{-1}\A_1^{-1}.\)Proof.
Definition 2.10.29. Matrix transpose.
Let \(\A=(A_{ij})\in\R^{m\times n}.\) Then the transpose of \(\A\) is \(\A^T:=(A_{ji})\in\R^{n\times m}.\) The operation of taking the transpose of \(\A\) is called the transposition of of \(\A.\)We may think of the transpose of a matrix as the reflection across the diagonal \(i=j,\) even if part of that diagonal lies outside the matrix.
Transposing a column-vector \(\x\in\R^{n\times1}\) results in a row vector \(\x^T\in\R^{1\times n}\text{,}\) and vice-versa.
Example 2.10.30.
We have \(\left(\begin{array}{rrr}3\amp -2\amp -4\\6\amp -6\amp -9 \end{array} \right)^T=\left(\begin{array}{rr}3\amp 6\\-2\amp -6\\-4\amp -9 \end{array} \right).\)
Theorem 2.10.31. Transposing twice.
For any \(\A\in\R^{m\times n}\) we have \((\A^{T})^T=\A.\)Proof.
Theorem 2.10.32. Transpose of a matrix sum.
For any \(\A,\B\in\R^{m\times n},\) we have
\begin{equation*}
(\A+\B)^T=\A^T+\B^T.
\end{equation*}
Proof.
Theorem 2.10.33. Transpose of a matrix product.
For any \(\A\in\R^{m\times n},\B\in\R^{n\times p}\) we have
\begin{equation*}
(\A\B)^T=\B^T\A^T.
\end{equation*}
Proof.
Theorem 2.10.34. The inverse of the transpose.
\(\A\in\R^{n\times n}\) is invertible if and only if \(\A^T\) is invertible, in which case \(\left(\A^T\right)^{-1}=\left(\A^{-1}\right)^T.\)Proof.
\(\left(\Leftarrow\right):\) Letting \(\C=A^T\) and applying Theorem 2.10.31 as appropriate in the above argument yields the reverse implication.
Definition 2.10.35. Symmetric matrices.
Let \(\A\in\R^{n\times n}.\) We say that \(\A\) is symmetric if \(\A^T=\A.\)
Example 2.10.36. Symmetric and non-symmetric matrices.
\begin{equation*}
\left(\begin{array}{rr}8\amp-3\\-3\amp5\end{array}\right)
\end{equation*}
and
\begin{equation*}
\left(\begin{array}{rr}2\amp-1\amp7\\-1\amp4\amp-6\\7\amp-6\amp0\end{array}\right)
\end{equation*}
are symmetric, while
\begin{equation*}
\left(\begin{array}{rr}8\amp-4\\-3\amp5\end{array}\right)
\end{equation*}
and
\begin{equation*}
\left(\begin{array}{rr}2\amp-1\amp8\\-1\amp4\amp-6\\7\amp-6\amp0\end{array}\right)
\end{equation*}
are not symmetric.
Example 2.10.37. The Gram matrix.
\begin{equation*}
\A^T\A=\left(\dpr{\A_{i\ast}}{\A_{\ast j}}\right)
\end{equation*}
is called the Gram matrix. It is symmetric since
\begin{align*}
\qquad\qquad\left(\A^T\A\right)^T=\amp\,\A^T\A^{TT}\qquad\qquad\amp\amp\text{ by } \knowl{./knowl/xref/producttranspose.html}{\text{Theorem 2.10.33}}\\
\qquad\qquad=\amp\A^T\A\qquad\qquad\amp\amp\text{ by }\knowl{./knowl/xref/transposetranspose.html}{\text{Theorem 2.10.31}}.
\end{align*}
Theorem 2.10.38. What kind of matrix has \(\boldsymbol{\A^T\A=\I}\text{?}\)
Let \(\A\in\R^{m\times n}.\) If \(\A^T\A=\I\) the set of columns of \(\A\) is an orthonormal set. If \(\A\) is square then \(\A^{-1}=\A^T.\)Proof.
Definition 2.10.39. Skew-symmetric matrices.
Let \(\A\in\R^{n\times n}.\) We say that \(\A\) is skew-symmetric if \(\A^T=-\A.\)Theorem 2.10.40.
The diagonal of a skew-symmetric matrix contains only \(0\)’s.Proof.
Theorem 2.10.41.
Let \(\A\in\R^{n\times n}.\) Then- \(\B:=\A+\A^T\) is symmetric, and
- \(\C:=\A-\A^T\) is skew-symmetric.
Proof.
Let \(\A\in\R^{n\times n}\text{;}\) we have
\begin{align*}
\qquad\qquad\B^T=\amp\,\A^T+\A^{TT}\qquad\qquad\amp\amp\text{ by } \knowl{./knowl/xref/sumtranspose.html}{\text{Theorem 2.10.32}}\\
\qquad\qquad=\amp\,\A^T+\A\qquad\qquad\amp\amp\text{ by }\knowl{./knowl/xref/transposetranspose.html}{\text{Theorem 2.10.31}}\\
\qquad\qquad=\amp\,\A+\A^T=\B\qquad\qquad\amp\amp
\end{align*}
and
\begin{align*}
\qquad\qquad\C^T=\amp\,\A^T-\A^{TT}\qquad\qquad\amp\amp\text{ by } \knowl{./knowl/xref/sumtranspose.html}{\text{Theorem 2.10.32}}\\
\qquad\qquad=\amp\,\A^T-\A\qquad\qquad\amp\amp\text{ by }\knowl{./knowl/xref/transposetranspose.html}{\text{Theorem 2.10.31}}\\
\qquad\qquad=\amp\,-(\A-\A^T)=-\C.\qquad\qquad\amp\amp
\end{align*}
Since \(\B=\A+\A^T\) satisfies Definition 2.10.35 and \(\C=\A-\A^T\) satisfies Definition 2.10.39, the theorem is proved.
