Functional analysis takes the geometry of ordinary vectors, such as length, angle and projection, and applies it to spaces whose "vectors" are functions or infinite sequences. It is the language of quantum mechanics, signal processing, partial differential equations and machine learning. This guide covers normed and Hilbert spaces, the two pillars of a first course at BS or MS level, with fully worked examples.
Normed Spaces
A norm on a real or complex vector space \(X\) is a function \(\|\cdot\|: X \to [0, \infty)\) such that for all \(x, y \in X\) and scalars \(\alpha\):
- \(\|x\| = 0\) if and only if \(x = 0\),
- \(\|\alpha x\| = |\alpha|\,\|x\|\),
- \(\|x + y\| \le \|x\| + \|y\|\) (the triangle inequality).
A norm gives a distance \(d(x, y) = \|x - y\|\), so we can talk about convergence and continuity.
Standard Examples
- On \(\mathbb{R}^n\), the p-norms \(\|x\|_p = \big(\sum |x_i|^p\big)^{1/p}\) for \(1 \le p < \infty\), and \(\|x\|_\infty = \max |x_i|\).
- The sequence space \(\ell^p\) of sequences with \(\sum |x_n|^p < \infty\).
- \(C[a, b]\), continuous functions with the sup norm \(\|f\|_\infty = \max_{t\in[a,b]} |f(t)|\).
- \(L^p[a, b]\), functions with \(\|f\|_p = \big(\int_a^b |f|^p\big)^{1/p} < \infty\) (identified when equal almost everywhere).
Banach Spaces: Completeness Matters
A sequence \((x_n)\) is Cauchy if \(\|x_n - x_m\| \to 0\) as \(n, m \to \infty\). A normed space in which every Cauchy sequence converges is a Banach space. All the examples above are Banach spaces, but \(C[0,1]\) with the integral norm \(\|f\|_1 = \int_0^1 |f|\) is not: continuous functions can converge in this norm to a step function, which has left the space.
Completeness is what makes the big theorems work: the Banach fixed-point theorem, the open mapping theorem, the uniform boundedness principle and the Hahn–Banach theorem.
Worked Example 1: Comparing Norms and Testing the Parallelogram Law
Part (a). For \(x = (3, -4) \in \mathbb{R}^2\), compute \(\|x\|_1\), \(\|x\|_2\) and \(\|x\|_\infty\).
Step 1. \(\|x\|_1 = |3| + |-4| = 7\).
Step 2. \(\|x\|_2 = \sqrt{3^2 + (-4)^2} = \sqrt{25} = 5\).
Step 3. \(\|x\|_\infty = \max(3, 4) = 4\).
Notice \(\|x\|_\infty \le \|x\|_2 \le \|x\|_1\), which holds for every vector. In finite dimensions all norms are equivalent; in infinite dimensions they are not.
Part (b). Decide whether the 1-norm comes from an inner product.
A norm comes from an inner product exactly when it satisfies the parallelogram law
$$ \|x + y\|^2 + \|x - y\|^2 = 2\|x\|^2 + 2\|y\|^2. $$
Step 1. Take \(x = (1, 0)\) and \(y = (0, 1)\). Then \(x + y = (1, 1)\) and \(x - y = (1, -1)\).
Step 2. In the 1-norm: \(\|x+y\|_1 = 2\), \(\|x-y\|_1 = 2\), \(\|x\|_1 = \|y\|_1 = 1\).
Step 3. Left side: \(4 + 4 = 8\). Right side: \(2 + 2 = 4\).
Step 4. Since \(8 \ne 4\), the 1-norm does not come from any inner product. In the 2-norm both sides equal \(4\), as expected.
Inner Product and Hilbert Spaces
An inner product \(\langle x, y \rangle\) is linear in one argument, conjugate symmetric \(\big(\langle x, y\rangle = \overline{\langle y, x \rangle}\big)\) and positive definite. It defines a norm \(\|x\| = \sqrt{\langle x, x\rangle}\) and the Cauchy–Schwarz inequality
$$ |\langle x, y \rangle| \le \|x\|\,\|y\|. $$
A complete inner product space is a Hilbert space. The prime examples are \(\mathbb{C}^n\), \(\ell^2\) with \(\langle x, y\rangle = \sum x_n \overline{y_n}\), and \(L^2[a,b]\) with
$$ \langle f, g \rangle = \int_a^b f(t)\,\overline{g(t)}\,dt. $$
Orthogonality and Projection
Vectors are orthogonal if \(\langle x, y\rangle = 0\). The Projection Theorem says that for every closed subspace \(M\) of a Hilbert space \(H\), each \(x \in H\) has a unique closest point \(Px \in M\), and \(x - Px\) is orthogonal to \(M\). So \(H = M \oplus M^\perp\).
If \(\{e_n\}\) is an orthonormal basis, every \(x\) expands as
$$ x = \sum_n \langle x, e_n \rangle e_n, \qquad \|x\|^2 = \sum_n |\langle x, e_n\rangle|^2, $$
the second identity being Parseval's identity. Fourier series are exactly this expansion in \(L^2[-\pi, \pi]\).
Worked Example 2: Best Approximation in \(L^2[0,1]\)
Find the linear function \(a + bx\) closest to \(f(x) = x^2\) in the \(L^2[0,1]\) norm.
Step 1: Orthonormalise \(\{1, x\}\) by Gram–Schmidt. Since \(\int_0^1 1^2\,dx = 1\), take \(e_1 = 1\). Next, \(\langle x, e_1\rangle = \int_0^1 x\,dx = \tfrac12\), so \(u_2 = x - \tfrac12\). Its norm squared is
$$ \int_0^1 \left(x - \tfrac12\right)^2 dx = \left[\frac{(x - \frac12)^3}{3}\right]_0^1 = \frac{1}{24} + \frac{1}{24} = \frac{1}{12}. $$
So \(e_2 = \sqrt{12}\,(x - \tfrac12) = 2\sqrt3\,(x - \tfrac12)\).
Step 2: Compute the coefficients.
$$ \langle x^2, e_1\rangle = \int_0^1 x^2\,dx = \frac13, $$
$$ \langle x^2, e_2\rangle = 2\sqrt3\int_0^1 \left(x^3 - \tfrac12 x^2\right)dx = 2\sqrt3\left(\frac14 - \frac16\right) = \frac{2\sqrt3}{12} = \frac{\sqrt3}{6}. $$
Step 3: Project.
$$ Pf = \frac13 \cdot 1 + \frac{\sqrt3}{6}\cdot 2\sqrt3\left(x - \frac12\right) = \frac13 + \left(x - \frac12\right) = x - \frac16. $$
Step 4: Check orthogonality of the error. The error is \(x^2 - x + \tfrac16\). Then \(\int_0^1 (x^2 - x + \tfrac16)\,dx = \tfrac13 - \tfrac12 + \tfrac16 = 0\) and \(\int_0^1 x(x^2 - x + \tfrac16)\,dx = \tfrac14 - \tfrac13 + \tfrac1{12} = 0\). The error is orthogonal to both \(1\) and \(x\), confirming the answer: the best line is \(x - \tfrac16\). This is the continuous version of least-squares regression.
Bounded Linear Operators
A linear map \(T: X \to Y\) between normed spaces is bounded if \(\|Tx\| \le C\|x\|\) for some constant \(C\); for linear maps, bounded is equivalent to continuous. The operator norm is
$$ \|T\| = \sup_{\|x\| \le 1} \|Tx\|. $$
Quick example. On \(C[0,1]\) with the sup norm, let \((Tf)(x) = \int_0^x f(t)\,dt\). Then \(|Tf(x)| \le x\|f\|_\infty \le \|f\|_\infty\), so \(\|T\| \le 1\). Taking \(f \equiv 1\) gives \(Tf(x) = x\) with sup norm \(1\), so \(\|T\| = 1\).
The Riesz Representation Theorem says every bounded linear functional \(\phi\) on a Hilbert space has the form \(\phi(x) = \langle x, y\rangle\) for a unique \(y\), with \(\|\phi\| = \|y\|\). This result is the backbone of the weak formulation of PDEs and the finite element method.
Did You Know?
- Stefan Banach's 1932 book "Théorie des opérations linéaires" established normed spaces as a subject; many ideas were reportedly discussed at the Scottish Café in Lwów.
- David Hilbert studied \(\ell^2\)-type spaces around 1904–1910, and John von Neumann later axiomatised abstract Hilbert spaces for quantum mechanics.
- In infinite dimensions, the closed unit ball is never compact; this single fact explains much of what makes functional analysis harder than linear algebra.
Key Takeaways
- A norm measures length; a complete normed space is a Banach space.
- A norm comes from an inner product exactly when the parallelogram law holds.
- In a Hilbert space, orthogonal projection gives the unique best approximation from a closed subspace.
- Bounded linear operators are the continuous linear maps, measured by the operator norm.
Need help with proofs about Banach spaces, Hilbert spaces or operators? Amathify's experts and PhD scholars offer step-by-step solutions and assignment help via WhatsApp or the Ask a Question form.
Stuck on a problem or assignment?
Our experts and PhD scholars solve maths, physics and chemistry — step by step.