\[ \newcommand{R}{\mathbb{R}} \newcommand{N}{\mathbb{N}} \newcommand{defarrow}{\quad \stackrel{\text{def}}{\Longleftrightarrow} \quad} \]

Bounded linear transformations

Let \(X\) and \(Y\) be normed spaces (both real, or both complex), and \(T \in L(X,Y)\) a linear mapping between them.

Boundedness

A linear mapping \(T\colon X \to Y\) is called bounded, \(T \in B(X,Y)\), if \(T\) maps bounded sets into bounded sets: \[ T \in B(X,Y) \defarrow \exists C; \quad \|Tx\|_Y \leq C \|x\|_X \quad \text{ for all } x \in X. \] Thus, if \(T\) is bounded, the number \[ \|T\| \stackrel{\text{def.}}{=} \sup_{x \neq 0} \frac{\| Tx \|_{Y}}{\|x\|_X} \] is finite; it is the (operator) norm of \(T\).

N.b. In some sources \(B(X,Y)\) is denoted by \(L(X,Y)\); to us, \(L(X,Y)\) is the space of linear transformations between two (not necessarily normed) vector spaces; if \(X\) and \(Y\) are normed spaces, then \(B(X,Y) \subset L(X,Y)\).

From now on, we will not always write out the indices for the norms; just recall that \( x \in X\) and \(Lx \in Y\).

Ex.
  • The integral \(\int_0^t f(s)\,ds\) defines a linear transformation on the space of bounded and continuous functions \(f\colon [0,1] \to \R\), \[T \colon BC([0,1],\R) \to BC([0,1],\R), \qquad (Tf)(t) = \int_0^t f(s)\,ds. \] This transformation is bounded, since \[ \|Tf\|_{BC([0,1],\R)} = \sup_{t \in [0,1]} \Big| \int_0^t f(s)\,ds \Big| \leq \int_0^1 \max_{s \in [0,1]}|f(s)|\,ds = \|f\|_{BC([0,1],\R)},\] so that \(\|T\| \leq 1\). In fact, \(\|T\| =1\) (can you see why?).
  • If \(g \in BC([0,1],\R)\), a similar argument yields that \[T \colon BC([0,1],\R) \to BC([0,1],\R), \qquad (Tf)(t) = \int_0^t f(s) g(s)\,ds \] is bounded too, with \(\|T\| \leq \max_{t \in [0,1]} |g(t)| = \|g\|_{BC([0,1],\R)}\) (see if you can make this better).
  • By the same argument, with \(t=1\) and \(g \in BC([0,1],\R)\), the definite integral \(\int_0^1 f(s) g(s) \,ds\) defines a bounded linear functional 1) \[T \colon BC([0,1],\R) \to \R, \qquad f \mapsto \int_0^1 f(s)g(s)\,ds.\] Remember the form of this functional – it is the inner product for the Hilbert space \(L_2((0,1),\R)\).
  • The derivative \(\frac{d}{dx}\) is in general not a bounded operator.2) To see why, consider a sequence of functions like \[ f_n(x) := \sin(nx).\] These functions are uniformly bounded, but not their derivatives. This indicates that, to solve differential equations, it is better to reformulate them as integral operators.

Equivalence of norm expressions

For \(T \in B(X,Y)\), \[ \|T\| = \sup_{x \neq 0} \frac{\| Tx \|}{\|x\|} = \sup_{\|x\| = 1} \|Tx\| = \sup_{\|x\| \leq 1} \|Tx\|\] all describe the least possible bound on \(C\) such that \(\|Tx\| \leq C\|x\|\) for all \(x \in X\).

Proof

Proof

Since \(T\) is linear, and since norms are positively homogeneous, we get \[ \frac{\|Tx\|}{\|x\|} = \Big\| \frac{1}{\|x\|} Tx \Big\| = \Big\| T \big( \frac{x}{\|x\|} \big) \Big\|, \qquad x \neq 0. \] Note that the mapping \(S_\lambda \to S_1\), \(x \mapsto \frac{x}{\|x\|}\), is bijective. Thus, if \(\lambda > 0\), we have \[ \sup_{\|x\| = \lambda} \| Tx \| = \lambda \sup_{\|x\| = 1} \|Tx\|. \] By considering all fixed, but different, \(\lambda > 0\), we see that \[ \sup_{\|x\| = \lambda > 0} \frac{\| Tx \|}{\|x\|} = \sup_{\|x\| = 1} \| Tx \|, \qquad\text{ so that }\qquad \sup_{x \neq 0} \frac{\| Tx \|}{\|x\|} = \sup_{\|x\| = 1} \| Tx \|. \] Then, consider \(\lambda \leq 1\) to see that \[ \sup_{\|x\| \leq 1} \| Tx \| \stackrel{\lambda \leq 1}{\leq} \sup_{\|x\| = 1} \| Tx \| \leq \sup_{\|x\| \leq 1} \| Tx \|, \] where the last inequality follows from the definition of the supremum. This proves the assertion.


› B(X,Y) is a normed space

\[ B(X,Y) = \{T \in L(X,Y) \colon T \text{ is bounded}\}\] is a normed space when equipped with the operator norm \(\|\cdot\|\).

Proof

Proof

We already know that \(L(X,Y)\) is a linear space, so it remains to show that \(B(X,Y)\) is a subspace and \(\|\cdot\|\) a norm on \(B(X,Y)\).

Subspace property. take \(T,S \in B(X,Y)\) and \(\mu,\lambda\) scalars. Then \[ \begin{align*} \sup_{\|x\| \leq 1} \|(\mu T + \lambda S)(x)\|_Y &\leq \sup_{\|x\| \leq 1} (|\mu| \|Tx\|_Y + |\lambda| \|Sx\|_Y)\\ &\leq |\mu| \sup_{\|x\| \leq 1} \|Tx\|_Y + |\lambda| \sup_{\|x\| \leq 1} \|Sx\|_Y = |\mu| \|T\| + |\lambda| \|S\| \end{align*} \] is finite by choice of \(S,T\). Thus \(\mu T + \lambda S\) is bounded if \(T\) and \(S\) are bounded, so that \(B(X,Y)\) is a subspace of \(L(X,Y)\).

Norm properties. These are consequences of that the operator norm \(\|\cdot\|\) is defined using (primarily) the norm of \(Y\).

Positive definiteness: \[ \|T\| = 0 \quad\Longleftrightarrow\quad \|Tx\|_Y = 0 \quad \forall x \in X \quad\Longleftrightarrow\quad T \equiv 0 \quad\text{ in }\: L(X,Y). \]

Positive homogeneity: \[ \|\lambda T\| = \sup_{\|x \|_X \leq 1} \|\lambda Tx\|_Y = |\lambda| \sup_{\|x \|_X\leq 1} \| Tx\|_Y = |\lambda| \|T\|. \]

Triangle inequality: \[ \begin{align*} \|T + S\| = \sup_{\|x \|_X \leq 1}\|(T+S)x\|_Y &\leq \sup_{\|x \|_X \leq 1}\big( \|Tx\|_Y + \|Sx\|_Y \big)\\ &\leq \sup_{\|x \|_X \leq 1} \|Tx\|_Y + \sup_{\|x \|_X \leq 1} \|Sx\|_Y = \|T\| + \|S\|. \end{align*} \]


Ex.
  • As we shall see, \(B(\R^n,\R^m) = L(\R^n,\R^m)\) (as sets and linear spaces): given bases for \(\R^n, \R^m\) there is a bijective correspondence between matrices \(A \in M_{m \times n}(\R)\) and bounded linear transformations \(T \in B(\R^n, \R^m)\).
  • Let \(X\) be a real normed space. The space \(X' := B(X,\R)\) is called the dual of \(X\); its elements are bounded linear functionals on \(X\). If \(X\) is complex, \(B(X,\mathbb{C})\) is its dual.3)
  • The dual of \(\R\) is \(\R\): each bounded linear functional \( T \in B(\R,\R) \) is realized by multiplication with a real constant: \[ T \in B(\R,\R) \quad\Longleftrightarrow\quad Tx = \lambda x, \quad \lambda \in \R. \]
  • Riesz representation theorem: Let \(L_2(I,\R)\) be the space of real-valued square-integrable functions on an open interval \(I \subset \R\), with norm \[ \|f\|_{L_2(I,\R)} = \big(\int_I |f(t)|^2\,dt\big)^{1/2}. \] The Riesz representation theorem asserts that each bounded linear functional \(T\) on \(L_2(I,\R)\) can be identified with an element \(g \in L_2(I,\R)\), via \[ T f = \int_I f(t)g(t)\,dt.\] Thus \( L_2(I,\R) \cong B(L_2(I,\R),\R)\) is its own dual.4)

› B(X,Y) is Banach for Y Banach

If \(Y\) is complete, so is \(B(X,Y)\).

N.b. Note that \(X\) has no role in the completeness of \(B(X,Y)\). 5)

Proof

Proof

Pointwise convergence. Let \(\{T_n\}_{n \in \N}\) be a Cauchy sequence in \(B(X,Y)\). Then, for each fixed \(x \in X\), \[ \|(T_n - T_m)x\|_Y \leq \|T_m - T_n\| \|x\|_X \stackrel{m,n \to \infty}{\to} 0, \] so that \(\{T_n x\}_{n \in \N}\) is Cauchy in \(Y\). By assumption, \(Y\) is complete, so \(\{T_n x\}_{n \in \N}\) is convergent. Define \[ Tx := \lim_{n \to \infty} T_n x, \qquad x \in X. \] The pointwise limit defines a linear and bounded transformation. With this construction \(T\colon X \to Y\) is linear, \[ T(\lambda x+ \mu y) = \lim_{n \to \infty} T_n (\lambda x + \mu y) = \lim_{n \to \infty} ( \lambda T_n x + \mu T_n y) = \lambda \lim_{n \to \infty} T_n x + \mu \lim_{n \to \infty} T_n y = \lambda Tx + \mu Ty, \] and for each fixed \(x \in X\) there exists \(n_\varepsilon\) (depending also on \(x\)), such that, for all \(n \geq n_\varepsilon\), \[ \begin{align*} \|Tx\|_Y \leq \| (T-T_n)x\|_Y + \| T_n x\|_Y &\leq \varepsilon + \| T_n x\|_Y \\ &\leq \varepsilon + \underbrace{\sup_{n \in \N}\| T_n \|}_{\text{finite}} \|x\|_X, \end{align*} \] where we have used that Cauchy sequences are bounded (so that \(\|T_n\|\) is bounded, uniformly for \( n \in \N\)). By taking the supremum over all \(x\) with \(\|x\|_X = 1\), we obtain that \(T\) is bounded.

Convergence in \(B(X,Y)\). It remains to show that \(T_n \to T\) in \(B(X,Y)\). Similarly to the above argument, if \(m\geq n_{\varepsilon/2}\) (depending also on \(x\)), we have \[ \begin{align*} \|(T-T_n) x\|_Y \leq \| (T-T_m)x\|_Y + \| (T_m - T_n) x\|_Y &\leq {\textstyle \frac{\varepsilon}{2}} + \| (T_m -T_n) x\|_Y \\ &< {\textstyle \frac{\varepsilon}{2}} + \| T_m - T_n \| \|x\|_X. \end{align*} \] Since \(\{T_n\}_{n\in \N}\) is Cauchy, there exists \(N_{\varepsilon/2}\) such that \[ \|T_n - T_m \| < \frac{\varepsilon}{2} \quad\text{ for }\quad m,n \geq N_{\varepsilon/2}. \] Choose \(n \geq N_{\varepsilon/2}\) and, for each \(x\), an appropriate \(m \geq \max\{n_{\varepsilon/2}, N_{\varepsilon/2}\}\) By taking the supremum over all \(x\) with \(\|x\|_X = 1\) we thus find \[ \|T-T_n\| < \varepsilon \quad\text{ for }\quad n \geq N_{\varepsilon/2}. \] Hence, \(\lim_{n\to \infty} T_n = T\) in \(B(X,Y)\).


Boundedness and continuity

Continuity

A mapping \(f \colon X \to Y\) between to metric spaces is said to be continuous at \(x_0\) if \[ f(x_n) \to f(x_0) \:\text{ in } Y \qquad\text{ as }\qquad x_n \to x_0 \:\text{ in } X. \] Since continuous and sequential limits agree, this is the same as \[ \forall\, \varepsilon > 0 \quad \exists\, \delta > 0; \qquad d_Y(f(x),f(x_0)) < \varepsilon \quad\text{ for }\quad d_X(x,x_0) < \delta. \] A mapping that is continuous at all points in \(X\) is called continuous.

Ex.
  • In a normed space, \((X,\|\cdot\|)\), the norm is a continuous function \(X \to \R\): if \(x_n \to x_0\) in \(X\), then \[ d_\R(\|x_n\|,\|x_0\|) = \big| \|x_n\| - \|x_0\| \big| \leq \|x_n - x_0\| = d_X(x_n,x_0) \to 0, \] by the reverse triangle inequality.

› For linear operators, continuity means boundedness

Let \(T \in L(X,Y)\). Then the following statements are equivalent:

  • \(T\) is everywhere continuous.
  • \(T\) is continuous at \(x = 0\).
  • \(T\) is bounded.

Proof

Proof

First, note that for any fixed \(x_0 \in X\), \[ \begin{align*} Tx_n \to Tx_0 \quad\text{ as }\quad x_n \to x_0 \quad&\Longleftrightarrow\quad T(x_n - x_0) \to {\mathbf 0}_Y \quad\text{ as }\quad (x_n - x_0) \to \mathbf{0}_X\\ &\stackrel{z_n = x_n - x_0}{\Longleftrightarrow}\quad T z_n \to \mathbf{0}_Y \quad\text{ as }\quad z_n \to \mathbf{0}_X, \end{align*} \] so that, for linear operators, continuity at the origin is the same as continuity everywhere (\(x_0\) is arbitrary).

To see that boundedness and continuity at the origin are equivalent, assume first that \(T\) is bounded. Then \[ \|Tx\|_Y \leq \|T\| \|x\|_X \to 0 \quad\text{ as }\quad \|x\|_X \to 0, \] so that \(T\) is also continuous. Contrariwise, assume that \(T\) is continuous at the origin. Then \[ \| Tx \|_Y = \| Tx - T\mathbf{0}\|_Y \leq \varepsilon \quad\text{ for }\quad \|x\|_X = \|x - \mathbf{0}\|_X \leq \delta. \] But \(T\) is linear, so by scaling \(x\) (replace \(x\) with \(\delta x\)) we obtain \[ \|T x\|_Y \leq \frac{\varepsilon}{\delta} \quad\text{ for }\quad \|x\|_X \leq 1. \] Thus \(T\) is bounded.


Ex.
  • Any linear operator \(T \in L(X,Y)\) defined on a finite-dimensional normed space \(X\) is continuous. Reason: identify \(X \cong \R^n\) and note that \[\mathrm{ran}(T) = \mathrm{span}\{Te_1, \ldots, Te_n\} \cong \R^m \quad\text{ for some } m \leq n,\] where \(\{e_1, \ldots, e_n\}\) is a basis for \(\R^n\). Hence, \[ T \colon X \cong \R^n \to \R^m \cong \tilde Y \subset Y\] is a linear transformation onto a finite-dimensional subspace \(\tilde Y\) of \(Y\), and, as such, has a matrix representation \[ T \colon x \mapsto Ax = \big(\sum_{j=1}^n a_{ij} x_j\big)_{i=1}^m. \] All norms on a finite-dimensional vector space are equivalent, so whatever the norms of \(X\) and \(Y\), we can consider any suitable norms for \(\R^n \cong X\) and \(\R^m \cong \tilde Y\). Choose, for example, the \(l_\infty\)-norm: then \[ \|Ax\|_{l_\infty} = \max_{1 \leq i \leq m} \big| \sum_{j=1}^n a_{ij} x_j \big| \leq n \max_{i,j} |a_{ij}| \max_j |x_j| = n \max_{i,j} |a_{ij}| \|x\|_{l_\infty}. \] This means that \(T\) is bounded with \(\|T\| \leq n \max_{i,j} |a_{ij}|\), and therefore also continuous.

N.b. Equivalent norms yield the same open and closed sets, the same convergence, but not the same constants in the estimates – in particular, the exact value of \(\|T\|\) depends on the norms for \(X\) and \(Y\).

› The kernel of a bounded operator is closed

Let \(T \in B(X,Y)\). Then \(\mathrm{ker}(T)\) is a closed subspace of \(X\). In particular, if \(X\) is a Banach space, so is \(\mathrm{ker}(T)\).

Proof

Proof

Take \[ \{x_n\}_{n\in N} \subset \mathrm{ker}(T); \qquad \lim_{n \to \infty} x_n = x_0 \in X. \] We want to show that \(x_0 \in \mathrm{ker}(T)\). But this follows from the continuity of \(T\): \[ \|Tx_0\|_Y = \|T x_0 - T x_n\|_Y \leq \|T\| \|x_0 - x_n\|_X \to 0 \quad\text{ as }\quad x_n \to x_0 \qquad \Longrightarrow \qquad T x_0 = 0. \] If, furthermore, \(X\) is complete, so is the closed subspace \(\mathrm{ker}(T) \subset X\).


Ex.
  • The null space of a matrix \(A \in M_{m\times n}(\R)\) is a closed subspace of \(\R^n\).
  • In \(L_2((-\pi,\pi),\R)\), the kernel of the bounded linear functional \[T \colon f \mapsto \frac{1}{\pi} \int_{-\pi}^{\pi} f(t) \sin(t)\,dt \] is a closed subspace; it consists of all functions with zero Fourier coefficient before \(\sin(t)\) in its Fourier expansion.
1)
A functional is a function from a vector space to its field of scalars (\(\R\) or \(\mathbb C\)).
2)
Unless we consider it on some special space of functions, as the finite-dimensional space of real polynomials of degree less than \(n\).
3)
This notion of dual coincides with that of a continuous dual; it is possible to define more general duals.
4)
So far, this identification is in terms of linear spaces, but we will see later that it extends to inner products (and therefore to norms).
5)
If, however, \(X\) is non-trivial, i.e., if \(X \neq \{\bf 0\}\), then the converse also holds, so that \(B(X,Y)\) is complete if and only if \(Y\) is complete.
2017-03-24, Hallvard Norheim Bø