\[ \newcommand{R}{\mathbb{R}} \newcommand{C}{\mathbb{C}} \newcommand{N}{\mathbb{N}} \newcommand{defarrow}{\quad \stackrel{\text{def}}{\Longleftrightarrow} \quad} \]
Spectral theory
Let \(A \in M_{n\times n}(\C)\) be the realization of a bounded linear transformation \(\C^n \to \C^n\) (standard basis assumed), and let \(|\cdot|\) denote the standard unitary norm on \(\C^n\), \[ |(z_1, \ldots, z_n)| = \big( \sum_{j=1}^n |z_j|^2 \big)^{1/2}; \qquad |z_j|^2 = |x_j + i y_j|^2 = |x_j|^2 + |y_j|^2. \] In this section, most entities considered will be complex. You can think of \(A\) as real, but, if so, still describing a bounded linear map \(\C^n \to \C^n\).
Existence theory for constant-coefficient linear ODE's
Consider \[ \dot u = A u,\qquad u(0) = u_{0} \in \C^{n}. \qquad \text{(1)} \] (The choice \(t_0 = 0\) is irrelevant, since \(u(\cdot-t_0)\) is a solution exactly if \(u\) is.) Note that the right-hand side \(f(u) = Au\) is uniformly Lipschitz with \[|Au - Av| \leq \|A\| |u-v|, \qquad \|A\| = \sup_{|u| =1} |Au|,\] so that this problem is locally and uniquely solvable. As we shall see, any solution can be globally continued on \(\R\), and even explicitly constructed.
The spectrum of an operator
Let \(T \in B(X)\) be a bounded linear transformation \(X \to X\) (for example, \(T \colon \C^n \to \C^n\) given by \(A\)).
- \(\lambda \in \C\) is called an eigenvalue of \(T\) if there exists a nonzero \(v \in X\) such that \[Tv = \lambda v.\] The vector \(v\) is called an eigenvector corresponding to the eigenvalue \(\lambda\).
- The set of values \(\lambda \in \C\) for which \((T - \lambda I)\) is invertible with a bounded inverse \((T - \lambda I)^{-1} \in B(X)\) is called the resolvent set of \(T\). Its complement in \(\C\), denoted \(\sigma(T)\), is called the spectrum of \(T\).
> For matrices, the spectrum consists only of eigenvalues
For \(A \in M_{n \times n}(\C)\), \[\sigma(A) = \{\lambda \in \C \colon \mathrm{det}(A - \lambda I) = 0\}\] consists of the roots \((\lambda_1, \ldots, \lambda_n)\) of the characteristic polynomial \(p_A(\lambda) \stackrel{\text{def.}}{=} \mathrm{det}(A - \lambda I) \); these are identical with the eigenvalues of \(A\).
N.b. Defining properties of the determinant are not treated in this course; the determinant of a square matrix is the product of the final diagonal pivots in its reduced row echelon form (at the end of the Gauss–Jordan elimination).
Multiplicity
- The multiplicity of a root \(\lambda\) of \(p_A(\lambda)\) is the algebraic multiplicity of the the eigenvalue \(\lambda\), denoted \(\mathrm{mult}(\lambda)\).
- The eigenvectors corresponding to an eigenvalue \(\lambda\) span a subspace of \(\C^n\), \[ \mathrm{ker}(A- \lambda I), \] called the eigenspace of \(\lambda\). The dimension of this space is the geometric multiplicity of \(\lambda\).
- An eigenvalue \(\lambda\) is called simple if \(\lambda\) is simple as a root of \(p_A(\lambda)\), \[ \lambda \text{ simple } \defarrow \mathrm{mult}(\lambda) = 1;\] it is semi-simple if the geometric and algebraic multiplicity coincide, \[\lambda \text{ semi-simple} \defarrow \mathrm{mult}(\lambda) = \mathrm{dim}\, \mathrm{ker}(A-\lambda I). \]
As we will see, if all eigenvalues of \(A\) are semi-simple, then \(A\) can be diagonalized.
› Characterization of solution spaces
The solution set of \(\dot u = Au\) is a vector space isomorphic to \(\C^n\). If \(A\) is real and only real initial data \(u_0 \in \R^n\) is considered, then the solution space is isomorphic to \(\R^n\).
Fundamental matrix
- A basis \(\{u_{j}\}_{j=1}^{n}\) of solutions is called a fundamental system for \(\dot u = Au\); the corresponding matrix \((u_{j})_{j}\) is a fundamental matrix.
N.b. According to the above characterization, a set of solutions \(\{u_{j}\}_{j}\) is a fundamental system exactly if \(\{u_{j}(0)\}_j\) is a basis for \(\C^{n}\) (or \(\R^n\) if we are considering real solutions).
The exponential map for square matrices
- The map \[ \mathrm{exp}(A) \:\stackrel{\text{def.}}{=}\: \sum_{j=0}^{\infty} \frac{A^{j}}{j!} = I + A+ \frac{A^2}{2} + \frac{A^3}{3!} + \ldots, \qquad A \in L(\C^{n}),\] also written \(e^A\), is called the exponential of \(A\).
The exponential map is well defined
Let \(A \in L(\C^n)\). Then \(\exp(A) \in L(\C^n)\) (in particular, \(\exp(A)\) is a matrix).
Properties of the exponential map
- If \(AB = BA\), then
\[ B \, \exp(A) = \exp(A)\, B \qquad\text{ and }\qquad \exp(A+B) = \exp(A) \exp(B). \]
- If \(T \in M_{n\times n}(\C^n)\) is invertible, then \[T \exp(A) T^{-1} = \exp(T A T^{-1}).\]
- \(\exp(A)\) is invertible with \[\big(\exp(A)\big)^{-1} = \exp(-A).\]
- \([t \mapsto \exp(tA)]\) is continuously differentiable with \[ \frac{d}{dt} \exp(tA) = A \exp(tA).\]
Solution formula
The unique solution of (1) is \[ u(t;u_{0}) = \exp(tA) u_{0}, \] and \(\exp(tA)\) is a fundamental matrix with \(\exp(tA)|_{t=0} = I\).
Spectral decompositions
If \(A\) is nilpotent, i.e., if \[ A^{n_0} = 0 \quad\text{ for some } n_0 \in \N, \] then \(\exp(A)\)—and therefore \(\exp(tA)\)—is a finite sum: \[ \exp(A) = \sum_{j=0}^{\infty} \frac{A^{j}}{j!} = \sum_{j=0}^{n_{0}-1} \frac{A^{j}}{j!} = I + A + \ldots + \frac{A^{n_{0}-1}}{(n_{0}-1)!}. \] In general, other methods must be employed.
Cayley–Hamilton
A matrix satisfies its characteristic polynomial: \(p_A(A) = 0\).
N.b. Since \(p_A\) is a polynomial of degree \(n\), this implies that \(A^n\) can be replaced with a polynomial of degree at most \(n-1\). Hence \(\exp(A)\) can be reduced to a polynomial in \(A\) of degree at most \(n-1\). This is the basis for the spectral decomposition below.
Algebraic description of the solution space
Let \(\lambda \in \C\) denote an eigenvalue of \(A\).
- A vector \(v \neq 0\) is called a generalized eigenvector if \((A-\lambda I)^{k} v = 0\) for some \(k \in \N\); we call
\[ N_{k} :=\ker \left( ( A- \lambda I )^{k} \right) \] a generalized eigenspace corresponding to the eigenvalue \(\lambda\).
Each eigenvalue has a maximal generalized eigenspace
There exists a minimal integer, \(k_\lambda \in \N\), such that \(N_{k} = N_{k_{\lambda}}\) for all \(k \geq k_{\lambda}\).
Let \[ R_{k} := \mathrm{ran}((A-\lambda I)^{k}), \qquad k \in \N. \] The Riesz index \(m_\lambda\) is the minimal natural number that ends the chain \(\{R_k\}_k \): \[ m_{\lambda} \stackrel{\text{def.}}{=} \min \{ k \in \N \colon R_{k} = R_{k+1}\}. \] The vector spaces \(R_{k}\) satisfy \[ \{0\} \subset R_{k+1} \subset R_{k} \subset \C^{n}, \qquad\text{ with } \qquad R^{k} = R^{m_{\lambda}} \quad\text{ for all } k \geq m_{\lambda}. \]
Riesz decomposition
We have \(k_{\lambda} = m_{\lambda}\), and \[ \C^{n} = N(\lambda) \oplus R(\lambda) := \mathrm{ker} \left((A - \lambda I)^{m_{\lambda}} \right) \oplus \mathrm{ran} \left((A - \lambda I)^{m_{\lambda}} \right). \]
Spectral decomposition
\(\C^n\) can be decomposed into maximal generalized eigenspaces \(N(\lambda_k)\) with \(\mathrm{dim}(N(\lambda_k)) = \mathrm{mult}(\lambda_k)\): \[ \C^{n} = \oplus_{k=1}^{m} N(\lambda_{k}). \] Here \(m\) is the number of different eigenvalues (not counted with multiplicity).
The matrix form of the spectral decomposition
According to the above, \(\C^{n} = \oplus_{k=1}^{m} N(\lambda_{k})\) has a basis of generalized eigenvectors. Let \[ A_{k} := A|_{N(\lambda_{k})}, \qquad I_{k} := I|_{N(\lambda_{k})}, \qquad \tilde N_{k} := A_{k} - \lambda_{k} I_{k}, \quad k = 1, \ldots, m, \] be the restrictions of the mappings \(A\), \(I\) and \(A - \lambda_k I\) onto the eigenspaces \(N(\lambda_k)\) (meaning that they act only on the basis vectors of the corresponding eigenspaces). Then \(\tilde N_{k}\) is nilpotent, since \[ \tilde N_{k}^{m_{\lambda_{k}}} = 0 \] on the generalized eigenspace \(N(\lambda_k)\) (this is the definition of \(m_{\lambda_k}\)). In our basis of generalized eigenvectors \(A\) takes the form \[ \begin{bmatrix} [A_{1}] & 0 & 0 &\ldots & 0\\ 0 & [A_{2}] & 0 & \ldots & 0\\ \vdots & & \ddots & &\vdots\\ 0 & & \ldots & 0 & [A_{m}] \end{bmatrix}_{n \times n} = \begin{bmatrix} [\lambda_{1} I_{1} + \tilde N_{1}] & 0 & 0 &\ldots & 0\\ 0 & [\lambda_{2} I_{2} + \tilde N_{2}] & 0 & \ldots & 0\\ \vdots & & \ddots & &\vdots\\ 0 &\ldots & & 0 & [\lambda_{m} I_{m} + \tilde N_{m}] \end{bmatrix}_{n \times n}. \] Because \(\tilde N_{k} I_{k} = I_{k} \tilde N_{k}\), \(\exp(t\lambda_{k} I_{k}) = e^{t \lambda} I_k\), and \(\tilde N_{k}^{m_{\lambda_k}} = 0\) one has \[ \exp(t A_k) = \exp\big(t(\lambda_{k} I_{k} + \tilde N_{k})\big) = \exp\big(t\lambda_{k} I_{k}\big) \exp\left(t \tilde N_{k}\right) = e^{t \lambda_{k}} \big( I_{k} + t \tilde N_{k} + \ldots + \frac{(t \tilde N_{k})^{m_{\lambda_{k}}-1}}{(m_{\lambda_{k}}-1)!}\big), \] and then \[ \exp(t \underbrace{T [A_{k}]_{k} T^{-1}}_{tA \text{ in original basis}}) = T \exp(t[A_{k}]_{k}) T^{-1} \qquad (T \text{ change-of-basis matrix}). \] One only needs to find suitable bases for \(N(\lambda_{k})\), \(k = 1, \ldots, m\).
The matrix \[ A := \begin{bmatrix} 0 & -8 &4\\ 0 & 2 & 0\\ 2 & 3 & -2 \end{bmatrix} \] has eigenvalues \(\lambda_{1,2} = 2\) and \(\lambda_{3} = -4\). Its generalized eigenvectors solve \[ (A-2I)^{2} v = \begin{bmatrix} 12 & 28 & -24\\ 0 & 0 & 0\\ -12 & -28 & 24 \end{bmatrix} \, v = 0 \quad \Leftrightarrow \quad v = s\, \begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix} + t \, \begin{bmatrix} 0\\ 6 \\ 7 \end{bmatrix} \qquad s,t \in \C, \] and \[ (A+4I)\, v = 0 \quad \Leftrightarrow\quad v = s \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix} \qquad s \in \C. \] Let \[ T := \begin{bmatrix} 2 & 0 & -1\\ 0 & 6 & 0\\ 1 & 7 & 1 \end{bmatrix} \quad\text{ so that }\quad T^{-1} = \frac{1}{18} \begin{bmatrix} 6 & -7 & 6\\ 0 & 3 & 0\\ -6 & -14 & 12 \end{bmatrix}, \quad T^{-1} A T = \begin{bmatrix} 2 & -10 & 0\\ 0 & 2 & 0\\ 0 & 0 & -4 \end{bmatrix}. \] In the basis given by \(T\) we have \[ I_{1} = \begin{bmatrix} 1 & 0\\ 0 & 1 \end{bmatrix}, \quad \tilde N_{1} = \begin{bmatrix} 0 & -10\\ 0 & 0 \end{bmatrix} \quad\text{ with } \quad A_{1} = 2 I_{1} + \tilde N_{1}, \] \[ \exp(tA_{1}) = \exp(2t I_{1}) \exp(t \tilde N_{1}) = e^{2t}(I_{1} + t \tilde N_{1}) = e^{2t} \begin{bmatrix} 1 & -10 t\\ 0 & 1 \end{bmatrix}, \] and \[ \exp(t T^{-1} A T) = \begin{bmatrix} e^{2t} & -10 t e^{2t} & 0\\ 0 & e^{2t} & 0\\ 0 & 0 & e^{-4t} \end{bmatrix}. \] Expressed in the original basis, \[ \exp(tA) = T \exp(t T^{-1} A T) T^{-1} = \frac{1}{9} e^{2t} \begin{bmatrix} 6 & -7 & 6\\ 0 & 9 & 0\\ 3 & 7 & 3 \end{bmatrix} - \frac{1}{3} t e^{2t} \begin{bmatrix} 0 & 10 & 0\\ 0 & 0 & 0\\ 0 & 5 & 0 \end{bmatrix} + \frac{1}{9} e^{-4t} \begin{bmatrix} 3 & 7 & -6\\ 0 & 0 & 0\\ -3 & -7 & 6 \end{bmatrix}. \]
Applications: the Jordan normal form and finite-dimensional spectral theorem
The Jordan normal form
The Jordan normal form corresponds to a spectral decomposition in which the bases for \(N(\lambda_k)\) are chosen such that the nilpotent matrices \(\tilde N_k\) have the special form \[ \tilde N_{\lambda_k} = \begin{bmatrix} 0 & j_1 & 0 & \ldots & 0 \\ 0 & 0 & j_2 & \ddots & \vdots\\ \vdots & & \ddots & \ddots & 0\\ 0 & 0 & \ldots & 0 & j_{n_k -1}\\ 0 & 0 & \ldots & 0 & 0 \end{bmatrix}, \qquad j_l \in \{0,1\}, \quad l = 1, \ldots, n_k -1, \] with \(n_k = \mathrm{mult}(\lambda_k)\), and \[ A_k = \lambda_k I_k + \tilde N_{\lambda_k} = \begin{bmatrix} \lambda_k & j_1 & 0 & \ldots & 0 \\ 0 & \lambda_k & j_2 & \ddots & \vdots\\ \vdots & & \ddots & \ddots & 0\\ 0 & 0 & \ldots & \lambda_k & j_{n_k -1}\\ 0 & 0 & \ldots & 0 & \lambda_k \end{bmatrix}. \] To obtain this, given an eigenvalue \(\lambda\), pick a generalized eigenvector \[ v_{m_\lambda} \in \ker(A-\lambda I)^{m_\lambda}, \quad v_{m_\lambda} \not\in \ker(A-\lambda I)^{m_\lambda-1} \] and set \[ v_{m_\lambda -1 } := (A-\lambda I)v_{m_\lambda}, \quad \ldots \quad, v_{1} := (A-\lambda I)^{m_\lambda -1} v_{m_\lambda}, \] so that \[ v_j \in \ker\big( (A-\lambda I)^{j} \big), \quad v_j \not\in \ker\big( (A-\lambda I)^{j-1} \big), \qquad j = 1, \ldots, m_\lambda. \] The Jordan chain \(\{v_1, \ldots, v_{m_\lambda}\}\) is a basis for a subspace of \(N(\lambda)\), on which \[ \tilde N v_j = (A - \lambda I)v_j = v_{j-1}, \qquad j = 1, \ldots, m_\lambda, \] if we let \(v_0 := 0\). Hence, the \(j\):th column of \(\tilde N\) is \(v_{j-1}\). This gives the nilpotent part of a so-called Jordan block (with ones above the diagonal, all other elements zero). If \(m_\lambda < n_k\) additional Jordan chains need to be added. Each chain gives rise to a Jordan block; adding the different chains into a basis for \(N(\lambda)\) gives the form of \(\tilde N\) above.
The eigenvalues of \[ A := \begin{bmatrix} 0 & -8 &4\\ 0 & 2 & 0\\ 2 & 3 & -2 \end{bmatrix} \] are \(\lambda_{1} = 2\) (double) and \(\lambda_{2} = -4\) (simple).
Since \(\mathrm{mult(2)} = 2\), we have \(k_2 = m_2 \leq 2\), so we can start the Jordan chain by looking for a vector in \(N_2\) (which equals the maximal generalized eigenspace \(N(2)\)). Candidates are (cf. above): \[ u = \begin{bmatrix} 2 \\ 0 \\ 1 \end{bmatrix} \quad\text{ and }\quad w = \begin{bmatrix} 0\\ 6 \\ 7 \end{bmatrix}. \] Since \((A-2I)u = 0\) we have \(u \in N_1\), whereas \[(A-2I)w = \begin{bmatrix} -20 \\ 0 \\ -10\end{bmatrix} = -10 u\) implies that \[ v_2 := w \in N_2 \setminus N_1 \quad\text{ whereas }\quad v_1 := (A-2I)w = -10 u \in N_1. \] The Jordan block corresponding to the simple eigenvalue \(-4\) consists of just the eigenvalue itself, and the eigenvector spanning the one-dimensional eigenspace \(N(-4)\) is \(\tilde v_1 := (-1,0,1)\), as calculated above.
The change-of-basis matrix is thus given by \[ T := [ v_1 \; v_2 \; \tilde v_1 ] = \begin{bmatrix} -20 & 0 & -1\\ 0 & 6 & 0\\ -10 & 7 & 1 \end{bmatrix}, \] in which the linear transformation expressed by \(A\) in the original basis takes the Jordan normal form1) \[ \begin{bmatrix} 2 & 1 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & -4 \end{bmatrix}. \]
The spectral theorem for Hermitian matrices
- For \(A \in M_{n \times n}(\C)\) the matrix \(A^*\) defined by \(a^*_{ij} := \overline{a_{ji}}\) is called its adjoint or conjugate transpose. Equivalently, \[ A^* = \overline{A^t},\] where \(A^t\) is the transpose of \(A\).
- \(A\) is said to be Hermitian (or self-adjoint) if \(A = A^*\).
The spectral theorem says that any Hermitian matrix admits a basis of eigenvectors in which \(A\) can be diagonalized, and that this basis can be chosen to be orthonormal, meaning that the basis vectors are of unit length and perpendicular to each other.2)
N.b. If a matrix can be diagonalized, this can always be achieved using the spectral (or Jordan) decomposition (though the corresponding basis need not be orthonormal).