10  Inner Product Spaces

10.1 The Geometric Deficiency of Vector Spaces

A vector space provides algebraic structure: we can add vectors and scale them. The theory developed thus far—spanning, linear independence, dimension, linear maps, determinants, eigenvalues—proceeds entirely without reference to geometric notions like distance, angle, or orthogonality.

Consider \mathbb{R}^2 as an abstract vector space over \mathbb{R}. We have bases, we have linear maps, we can solve systems of equations. But we cannot measure the length of a vector. We cannot determine whether two vectors are perpendicular. We cannot speak of “the closest point in a subspace to a given vector.” These are not deficiencies of our theory—they are features we have deliberately omitted.

Euclidean space \mathbb{R}^n admits additional structure: the dot product \langle x, y \rangle = \sum_{i=1}^{n} x_i y_i. This operation measures both magnitude and angle. The length of x is \|x\| = \sqrt{\langle x, x \rangle}. Two vectors are orthogonal (perpendicular) when \langle x, y \rangle = 0. The angle \theta between nonzero vectors satisfies \cos \theta = \frac{\langle x, y \rangle}{\|x\| \|y\|}.

These notions are not purely algebraic—they impose metric structure, converting the vector space into a geometric object where we can measure and compare.

Many vector spaces naturally admit analogous structure. The space C([a,b]) of continuous functions on [a,b] has an inner product \langle f, g \rangle = \int_a^b f(x) g(x) \, dx. This measures “alignment” between functions. The norm \|f\| = \sqrt{\langle f, f \rangle} is the L^2 norm, essential in Fourier analysis and quantum mechanics. Two functions are orthogonal when \int f g = 0, a condition arising naturally in the study of orthogonal polynomials, Fourier series, and approximation theory.

The complex vector space \mathbb{C}^n requires modification: the standard inner product is \langle z, w \rangle = \sum_{i=1}^{n} z_i \overline{w_i}, where \overline{w_i} denotes complex conjugation. This conjugate-linearity is necessary for \langle z, z \rangle to be real and positive, ensuring \|z\|^2 = \langle z, z \rangle defines a genuine norm.

This chapter develops the theory of inner product spaces: vector spaces equipped with an operation measuring magnitude and angle. We establish basic properties, introduce the induced norm and metric, study the adjoint of a linear operator, and explore connections to bilinear and sesquilinear forms. The theory unifies the geometry of Euclidean space with the analysis of function spaces, providing the foundation for orthogonal projections, spectral theory, and quadratic forms in subsequent chapters.

10.2 Inner Products: Definition and Examples

We work over \mathbb{R} or \mathbb{C}. The definition differs slightly between real and complex cases due to conjugate symmetry.

Definition 10.1 (Inner product (real case)) Let \mathcal{V} be a vector space over \mathbb{R}. An inner product on \mathcal{V} is a function \langle \cdot, \cdot \rangle : \mathcal{V} \times \mathcal{V} \to \mathbb{R} satisfying, for all u, v, w \in \mathcal{V} and c \in \mathbb{R}:

  1. (Linearity in first argument) \langle cu + dv, w \rangle = c\langle u, w \rangle + d\langle v, w \rangle

  2. (Symmetry) \langle u, v \rangle = \langle v, u \rangle

  3. (Positive definiteness) \langle v, v \rangle > 0 for all v \neq 0

Linearity in the first argument combined with symmetry implies linearity in the second argument: \langle u, cv + dw \rangle = \langle cv + dw, u \rangle = c\langle v, u \rangle + d\langle w, u \rangle = c\langle u, v \rangle + d\langle u, w \rangle.

An inner product is thus a symmetric bilinear form that is positive definite.

Definition 10.2 (Inner product (complex case)) Let \mathcal{V} be a vector space over \mathbb{C}. An inner product on \mathcal{V} is a function \langle \cdot, \cdot \rangle : \mathcal{V} \times \mathcal{V} \to \mathbb{C} satisfying:

  1. (Linearity in first argument) \langle cu + dv, w \rangle = c\langle u, w \rangle + d\langle v, w \rangle

  2. (Conjugate symmetry) \langle u, v \rangle = \overline{\langle v, u \rangle}

  3. (Positive definiteness) \langle v, v \rangle > 0 for all v \neq 0

Conjugate symmetry forces \langle v, v \rangle = \overline{\langle v, v \rangle}, so \langle v, v \rangle is real. Positive definiteness then makes sense.

Conjugate symmetry implies conjugate linearity in the second argument: \langle u, cv + dw \rangle = \overline{\langle cv + dw, u \rangle} = \overline{c}\langle v, u \rangle + \overline{d}\langle w, u \rangle = \overline{c}\langle u, v \rangle + \overline{d}\langle u, w \rangle.

An inner product on a complex vector space is a sesquilinear form (linear in the first argument, conjugate-linear in the second) that is positive definite.

Definition 10.3 (Inner product space) An inner product space is a vector space \mathcal{V} equipped with an inner product \langle \cdot, \cdot \rangle.

When the field is clear from context, we write simply “inner product” and “inner product space.” When necessary, we specify “real inner product space” or “complex inner product space.”

10.2.1 Examples

1. Euclidean space \mathbb{R}^n. The standard inner product is \langle x, y \rangle = \sum_{i=1}^{n} x_i y_i = x^T y. This is the dot product. Linearity and symmetry are immediate. Positive definiteness: \langle x, x \rangle = \sum x_i^2 > 0 for x \neq 0.

2. Complex Euclidean space \mathbb{C}^n. The standard inner product is \langle z, w \rangle = \sum_{i=1}^{n} z_i \overline{w_i}. Conjugate symmetry: \langle z, w \rangle = \sum z_i \overline{w_i} = \overline{\sum \overline{z_i} w_i} = \overline{\langle w, z \rangle}. Positive definiteness: \langle z, z \rangle = \sum |z_i|^2 > 0 for z \neq 0.

3. Weighted inner products on \mathbb{R}^n. Given positive weights w_1, \ldots, w_n > 0, define \langle x, y \rangle_w = \sum_{i=1}^{n} w_i x_i y_i. This generalizes the standard inner product, emphasizing certain coordinates.

4. L^2 inner product on C([a,b]). For continuous functions f, g : [a,b] \to \mathbb{R}, define \langle f, g \rangle = \int_a^b f(x) g(x) \, dx. Linearity and symmetry follow from properties of the integral. Positive definiteness: if f \neq 0, then f is nonzero on some interval, so \int f^2 > 0.

5. Weighted L^2 inner product. Given a positive continuous weight function w : [a,b] \to (0,\infty), define \langle f, g \rangle_w = \int_a^b f(x) g(x) w(x) \, dx. This appears in the theory of orthogonal polynomials (Legendre, Chebyshev, Hermite).

6. Complex L^2 inner product on C([a,b], \mathbb{C}). For complex-valued continuous functions, \langle f, g \rangle = \int_a^b f(x) \overline{g(x)} \, dx. This is essential in Fourier analysis and quantum mechanics.

7. Frobenius inner product on M_{m \times n}(\mathbb{R}). For matrices A, B \in M_{m \times n}(\mathbb{R}), define \langle A, B \rangle = \operatorname{tr}(A^T B) = \sum_{i=1}^{m} \sum_{j=1}^{n} a_{ij} b_{ij}. This treats the matrix as a vector in \mathbb{R}^{mn}, summing all componentwise products. For complex matrices, use \langle A, B \rangle = \operatorname{tr}(A^* B) where A^* is the conjugate transpose.

10.2.2 Non-Examples

Not every symmetric bilinear form is an inner product. Positive definiteness is the essential constraint.

1. Minkowski inner product on \mathbb{R}^4. In special relativity, spacetime events are modeled by (t, x, y, z) \in \mathbb{R}^4 with “inner product” \langle u, v \rangle = u_0 v_0 - u_1 v_1 - u_2 v_2 - u_3 v_3. This is symmetric and bilinear but not positive definite: \langle (1,1,0,0), (1,1,0,0) \rangle = 1 - 1 = 0 despite (1,1,0,0) \neq 0. This structure is called a Lorentzian metric or pseudo-inner product. It is central to the geometry of spacetime but does not satisfy our definition.

2. Degenerate bilinear forms. On \mathbb{R}^2, define \langle (x_1, y_1), (x_2, y_2) \rangle = x_1 x_2. This is symmetric and bilinear but not positive definite: \langle (0,1), (0,1) \rangle = 0 yet (0,1) \neq 0. Such forms are degenerate—they annihilate nonzero vectors.

Positive definiteness is the condition that distinguishes genuine inner products from more general bilinear or sesquilinear forms. It ensures that \langle v, v \rangle measures the “size” of v in a meaningful way.

10.3 Basic Properties

Theorem 10.1 Let \mathcal{V} be an inner product space. For all u, v, w \in \mathcal{V} and c \in \mathbb{F}:

  1. \langle 0, v \rangle = \langle v, 0 \rangle = 0

  2. \langle u, v + w \rangle = \langle u, v \rangle + \langle u, w \rangle (additivity in second argument)

  3. In the real case, \langle u, cv \rangle = c\langle u, v \rangle (linearity in second argument)

  4. In the complex case, \langle u, cv \rangle = \overline{c}\langle u, v \rangle (conjugate linearity in second argument)

  5. If \langle u, v \rangle = 0 for all v \in \mathcal{V}, then u = 0

Proof.

  1. By linearity in the first argument, \langle 0, v \rangle = \langle 0 \cdot 0, v \rangle = 0 \langle 0, v \rangle = 0. By symmetry (or conjugate symmetry), \langle v, 0 \rangle = \overline{\langle 0, v \rangle} = \overline{0} = 0.

  2. In the real case, \langle u, v + w \rangle = \langle v + w, u \rangle = \langle v, u \rangle + \langle w, u \rangle = \langle u, v \rangle + \langle u, w \rangle by symmetry and linearity in the first argument. In the complex case, use conjugate symmetry: \langle u, v + w \rangle = \overline{\langle v + w, u \rangle} = \overline{\langle v, u \rangle + \langle w, u \rangle} = \overline{\langle v, u \rangle} + \overline{\langle w, u \rangle} = \langle u, v \rangle + \langle u, w \rangle.

  3. and (d) follow similarly from symmetry/conjugate symmetry and linearity in the first argument.

  4. Taking v = u gives \langle u, u \rangle = 0. By positive definiteness, u = 0. \square

Property (e) shows that the inner product is non-degenerate: the only vector orthogonal to everything is the zero vector. This distinguishes inner products from degenerate bilinear forms.

10.4 The Induced Norm

An inner product induces a notion of length.

Definition 10.4 (Norm induced by inner product) For v \in \mathcal{V}, the norm of v is \|v\| = \sqrt{\langle v, v \rangle}.

Since \langle v, v \rangle > 0 for v \neq 0 and \langle 0, 0 \rangle = 0, the square root is always defined and \|v\| \ge 0.

Theorem 10.2 The norm satisfies:

  1. \|v\| \ge 0 with equality if and only if v = 0

  2. \|cv\| = |c| \|v\| for all c \in \mathbb{F}

  3. (Cauchy–Schwarz inequality) |\langle u, v \rangle| \le \|u\| \|v\|

  4. (Triangle inequality) \|u + v\| \le \|u\| + \|v\|

Proof.

  1. This follows from positive definiteness of the inner product.

  2. In the real case, \|cv\|^2 = \langle cv, cv \rangle = c^2 \langle v, v \rangle = c^2 \|v\|^2, so \|cv\| = |c| \|v\|. In the complex case, \|cv\|^2 = \langle cv, cv \rangle = c\overline{c} \langle v, v \rangle = |c|^2 \|v\|^2, giving the same result.

  3. We prove the Cauchy–Schwarz inequality below in Theorem 10.3.

  4. The triangle inequality follows from Cauchy–Schwarz. We have \begin{align*} \|u + v\|^2 &= \langle u + v, u + v \rangle \\ &= \langle u, u \rangle + \langle u, v \rangle + \langle v, u \rangle + \langle v, v \rangle \\ &= \|u\|^2 + 2\operatorname{Re}(\langle u, v \rangle) + \|v\|^2 \\ &\le \|u\|^2 + 2|\langle u, v \rangle| + \|v\|^2 \\ &\le \|u\|^2 + 2\|u\|\|v\| + \|v\|^2 \\ &= (\|u\| + \|v\|)^2. \end{align*} Taking square roots gives \|u + v\| \le \|u\| + \|v\|. \square

The Cauchy–Schwarz inequality is fundamental. We prove it now.

Theorem 10.3 (Cauchy–Schwarz inequality) For all u, v \in \mathcal{V}, |\langle u, v \rangle| \le \|u\| \|v\|, with equality if and only if u and v are linearly dependent.

Proof. If v = 0, both sides are zero and equality holds. Assume v \neq 0.

For any t \in \mathbb{R}, consider \|u - tv\|^2 \ge 0. Expanding, \begin{align*} 0 &\le \langle u - tv, u - tv \rangle \\ &= \langle u, u \rangle - t\langle u, v \rangle - t\langle v, u \rangle + t^2 \langle v, v \rangle \\ &= \|u\|^2 - 2t\operatorname{Re}(\langle u, v \rangle) + t^2 \|v\|^2. \end{align*}

This is a quadratic in t that is nonnegative for all t. Its discriminant must be nonpositive: (2\operatorname{Re}(\langle u, v \rangle))^2 - 4\|u\|^2 \|v\|^2 \le 0, giving \operatorname{Re}(\langle u, v \rangle)^2 \le \|u\|^2 \|v\|^2.

In the complex case, we choose t more carefully. Let \langle u, v \rangle = re^{i\theta} where r = |\langle u, v \rangle| and \theta \in \mathbb{R}. Consider w = e^{-i\theta}u. Then \langle w, v \rangle = e^{-i\theta}\langle u, v \rangle = e^{-i\theta} r e^{i\theta} = r \in \mathbb{R}.

Applying the argument above to w and v (for which the inner product is real), we obtain r^2 \le \|w\|^2 \|v\|^2 = \|u\|^2 \|v\|^2, so |\langle u, v \rangle|^2 \le \|u\|^2 \|v\|^2.

Equality holds if and only if the quadratic \|u - tv\|^2 has a zero, equivalently u - tv = 0 for some t, i.e., u and v are linearly dependent. \square

The Cauchy–Schwarz inequality is one of the most important inequalities in mathematics. It appears in probability (Cauchy–Schwarz for random variables), analysis (Hölder’s inequality generalizes it), and countless other contexts.

10.5 The Induced Metric and Convergence

The norm induces a metric, converting the inner product space into a metric space where we can speak of convergence, continuity, and completeness.

Definition 10.5 (Metric induced by inner product) For u, v \in \mathcal{V}, the distance between u and v is d(u, v) = \|u - v\|.

Theorem 10.4 The function d : \mathcal{V} \times \mathcal{V} \to [0, \infty) is a metric:

  1. d(u, v) \ge 0 with equality if and only if u = v

  2. d(u, v) = d(v, u)

  3. d(u, w) \le d(u, v) + d(v, w)

Proof. Property (a) follows from \|u - v\| \ge 0 with equality if and only if u - v = 0. Property (b) holds since \|u - v\| = \|-(v - u)\| = |-1| \|v - u\| = \|v - u\|. Property (c) is the triangle inequality: \|u - w\| = \|(u - v) + (v - w)\| \le \|u - v\| + \|v - w\|. \square

A sequence (v_n) in \mathcal{V} converges to v \in \mathcal{V} if \|v_n - v\| \to 0 as n \to \infty. A sequence is Cauchy if \|v_n - v_m\| \to 0 as n, m \to \infty. The space \mathcal{V} is complete if every Cauchy sequence converges.

Finite-dimensional inner product spaces are always complete (by equivalence of norms in finite dimensions). Infinite-dimensional spaces may or may not be complete. The space C([a,b]) with the L^2 inner product is not complete—the completion is L^2([a,b]), the space of square-integrable functions.

Definition 10.6 (Hilbert space) A Hilbert space is a complete inner product space.

Hilbert spaces are the proper setting for infinite-dimensional analysis. Quantum mechanics models physical states as vectors in a Hilbert space; observables are self-adjoint operators on that space. Fourier analysis studies decomposition of functions in L^2 Hilbert spaces. Partial differential equations are analyzed using Sobolev spaces, which are Hilbert spaces with weighted norms.

We work primarily with finite-dimensional spaces, where completeness is automatic. The theory extends to infinite dimensions with appropriate care regarding convergence and completeness.

10.6 The Parallelogram Law and Polarization Identity

Inner products satisfy special identities not shared by arbitrary norms.

Theorem 10.5 (Parallelogram law) For all u, v \in \mathcal{V}, \|u + v\|^2 + \|u - v\|^2 = 2\|u\|^2 + 2\|v\|^2.

Proof. Expand using the definition of norm: \begin{align*} \|u + v\|^2 + \|u - v\|^2 &= \langle u + v, u + v \rangle + \langle u - v, u - v \rangle \\ &= (\|u\|^2 + 2\operatorname{Re}(\langle u, v \rangle) + \|v\|^2) + (\|u\|^2 - 2\operatorname{Re}(\langle u, v \rangle) + \|v\|^2) \\ &= 2\|u\|^2 + 2\|v\|^2. \quad \square \end{align*}

Geometrically, in \mathbb{R}^2, this states: the sum of the squares of the diagonals of a parallelogram equals the sum of the squares of all four sides.

The parallelogram law characterizes norms arising from inner products.

Theorem 10.6 A norm \|\cdot\| on a vector space \mathcal{V} arises from an inner product if and only if it satisfies the parallelogram law.

We omit the proof, which constructs the inner product from the norm using the polarization identity below.

Theorem 10.7 (Polarization identity) In a real inner product space, \langle u, v \rangle = \frac{1}{4}(\|u + v\|^2 - \|u - v\|^2). In a complex inner product space, \langle u, v \rangle = \frac{1}{4}(\|u + v\|^2 - \|u - v\|^2 + i\|u + iv\|^2 - i\|u - iv\|^2).

Proof. In the real case, expand \|u + v\|^2 = \|u\|^2 + 2\langle u, v \rangle + \|v\|^2 and \|u - v\|^2 = \|u\|^2 - 2\langle u, v \rangle + \|v\|^2. Subtracting gives \|u + v\|^2 - \|u - v\|^2 = 4\langle u, v \rangle.

The complex case requires additional terms to extract the real and imaginary parts of \langle u, v \rangle. \square

The polarization identity shows that the inner product is completely determined by the norm. This is remarkable: angle information is encoded in length information.

10.7 Bilinear and Sesquilinear Forms

Inner products are special cases of more general structures.

Definition 10.7 (Bilinear form) Let \mathcal{V} be a vector space over \mathbb{R}. A bilinear form on \mathcal{V} is a function B : \mathcal{V} \times \mathcal{V} \to \mathbb{R} that is linear in each argument: B(cu + dv, w) = cB(u, w) + dB(v, w), \quad B(u, cv + dw) = cB(u, v) + dB(u, w).

A bilinear form is symmetric if B(u, v) = B(v, u) for all u, v. It is positive definite if B(v, v) > 0 for all v \neq 0. An inner product on a real vector space is precisely a symmetric positive definite bilinear form.

Definition 10.8 (Sesquilinear form) Let \mathcal{V} be a vector space over \mathbb{C}. A sesquilinear form on \mathcal{V} is a function S : \mathcal{V} \times \mathcal{V} \to \mathbb{C} that is linear in the first argument and conjugate-linear in the second: S(cu + dv, w) = cS(u, w) + dS(v, w), \quad S(u, cv + dw) = \overline{c}S(u, v) + \overline{d}S(u, w).

A sesquilinear form is Hermitian if S(u, v) = \overline{S(v, u)}. It is positive definite if S(v, v) > 0 for all v \neq 0. An inner product on a complex vector space is precisely a Hermitian positive definite sesquilinear form.

Bilinear and sesquilinear forms without positive definiteness arise naturally. The Minkowski metric in special relativity is a symmetric bilinear form that is not positive definite. Symplectic forms in Hamiltonian mechanics are skew-symmetric bilinear forms (B(u, v) = -B(v, u)). These structures are central to differential geometry and physics but lie outside the scope of inner product theory.

10.8 Matrix Representation of Inner Products

In finite dimensions, inner products can be represented by matrices.

Let \mathcal{V} be a finite-dimensional inner product space with basis \mathcal{B} = \{v_1, \ldots, v_n\}. For vectors u = \sum a_i v_i and w = \sum b_j v_j, linearity gives \langle u, w \rangle = \left\langle \sum_i a_i v_i, \sum_j b_j v_j \right\rangle = \sum_{i,j} a_i \overline{b_j} \langle v_i, v_j \rangle.

Define the Gram matrix G \in M_{n \times n}(\mathbb{F}) by g_{ij} = \langle v_i, v_j \rangle. Then \langle u, w \rangle = [u]_{\mathcal{B}}^* G [w]_{\mathcal{B}}, where [u]_{\mathcal{B}}^* denotes the conjugate transpose (in the complex case) or transpose (in the real case) of the coordinate vector [u]_{\mathcal{B}}.

In \mathbb{R}^n with the standard inner product and standard basis, G = I_n, so \langle x, y \rangle = x^T y.

The Gram matrix encodes the inner product. It is Hermitian (G^* = G in the complex case, G^T = G in the real case) and positive definite (all eigenvalues are positive). Conversely, every Hermitian positive definite matrix defines an inner product via the formula above.

Theorem 10.8 The Gram matrix G is positive definite: for all nonzero c \in \mathbb{F}^n, c^* G c > 0.

Proof. Let v = \sum c_i v_i where c = (c_1, \ldots, c_n)^T. Then c^* G c = \langle v, v \rangle. If c \neq 0, then v \neq 0 (since \{v_1, \ldots, v_n\} is a basis), so \langle v, v \rangle > 0 by positive definiteness of the inner product. \square

The Gram matrix also encodes geometric information about the configuration of basis vectors. The determinant \det(G) measures the squared volume of the parallelepiped spanned by v_1, \ldots, v_n in the inner product space.

10.9 Orthogonality and Angles

The inner product measures both magnitude and relative orientation. We formalize the notion of perpendicularity and angular separation.

Definition 10.9 (Orthogonal vectors) Vectors u, v \in \mathcal{V} are orthogonal, written u \perp v, if \langle u, v \rangle = 0.

Orthogonality generalizes perpendicularity from Euclidean geometry. In \mathbb{R}^n with the standard inner product, u \perp v if and only if the vectors meet at a right angle. In function spaces, f \perp g when \int f g = 0—the functions are “independent” in the L^2 sense.

The zero vector is orthogonal to every vector: \langle 0, v \rangle = 0 for all v. The zero vector is the only vector orthogonal to itself.

Theorem 10.9 (Pythagorean theorem) If u \perp v, then \|u + v\|^2 = \|u\|^2 + \|v\|^2.

Proof. Expand using the definition of norm: \|u + v\|^2 = \langle u + v, u + v \rangle = \|u\|^2 + \langle u, v \rangle + \langle v, u \rangle + \|v\|^2. Since \langle u, v \rangle = 0, both cross terms vanish. In the real case, \langle v, u \rangle = \langle u, v \rangle = 0. In the complex case, \langle v, u \rangle = \overline{\langle u, v \rangle} = \overline{0} = 0. \square

This is the algebraic expression of the Pythagorean theorem: in a right triangle, the square of the hypotenuse equals the sum of the squares of the legs. The theorem extends to finite sums: if v_1, \ldots, v_k are pairwise orthogonal, then \left\|\sum_{i=1}^k v_i\right\|^2 = \sum_{i=1}^k \|v_i\|^2.

The converse fails: \|u + v\|^2 = \|u\|^2 + \|v\|^2 does not imply u \perp v in general (consider u = v with \|u\| = 0). However, in a real inner product space, if \|u + v\|^2 = \|u\|^2 + \|v\|^2 and both u, v are nonzero, orthogonality follows by expanding and canceling: the cross term 2\langle u, v \rangle must vanish. In the complex case, the expansion gives 2\operatorname{Re}\langle u, v \rangle = 0, which only forces the real part of \langle u, v \rangle to vanish—not \langle u, v \rangle itself.

For nonzero vectors, orthogonality is equivalent to extremal behavior.

Theorem 10.10 Among all vectors v with \|v\| = r for fixed r > 0, the sum \|u + v\| is minimized when v \perp u.

Proof. By the Pythagorean theorem, if v \perp u, then \|u + v\|^2 = \|u\|^2 + \|v\|^2 = \|u\|^2 + r^2. For arbitrary v with \|v\| = r, \|u + v\|^2 = \|u\|^2 + 2\operatorname{Re}(\langle u, v \rangle) + r^2 \ge \|u\|^2 + r^2, with equality if and only if \operatorname{Re}(\langle u, v \rangle) = 0. In the real case, this forces \langle u, v \rangle = 0. In the complex case, further analysis shows \langle u, v \rangle = 0 minimizes \|u + v\| globally among all v with \|v\| = r. \square

Orthogonality thus characterizes perpendicular directions geometrically: adding an orthogonal displacement to a vector increases length minimally.

10.9.1 Angles Between Vectors

For nonzero vectors u, v \in \mathcal{V}, the Cauchy–Schwarz inequality ensures -1 \le \frac{\langle u, v \rangle}{\|u\| \|v\|} \le 1. In a real inner product space, this ratio equals \cos \theta for a unique angle \theta \in [0, \pi].

Definition 10.10 (Angle between vectors (real case)) Let \mathcal{V} be a real inner product space and u, v \in \mathcal{V} nonzero. The angle \theta \in [0, \pi] between u and v satisfies \cos \theta = \frac{\langle u, v \rangle}{\|u\| \|v\|}.

When \theta = 0, the vectors are parallel and point in the same direction: v = cu with c > 0. When \theta = \pi, they point in opposite directions: v = cu with c < 0. When \theta = \pi/2, they are orthogonal.

In complex inner product spaces, the angle is less canonical since \langle u, v \rangle is complex. One convention defines \theta by \cos \theta = \frac{|\langle u, v \rangle|}{\|u\| \|v\|} \in [0, 1], yielding \theta \in [0, \pi/2]. This measures the magnitude of the “projection” but discards phase information. For most purposes in complex spaces, orthogonality (\langle u, v \rangle = 0) suffices without reference to angles.

The Cauchy–Schwarz inequality acquires geometric meaning: |\langle u, v \rangle| = \|u\| \|v\| if and only if \theta = 0 (vectors are collinear), and \langle u, v \rangle = 0 if and only if \theta = \pi/2 (vectors are orthogonal).

10.10 The Riesz Representation Theorem

Every linear functional on a finite-dimensional inner product space arises from the inner product with a fixed vector. This fundamental result collapses the distinction between vectors and linear functionals in the presence of an inner product.

Theorem 10.11 (Riesz representation theorem) Let \mathcal{V} be a finite-dimensional inner product space over \mathbb{F}. For every linear functional \varphi : \mathcal{V} \to \mathbb{F}, there exists a unique vector u \in \mathcal{V} such that \varphi(v) = \langle v, u \rangle \quad \text{for all } v \in \mathcal{V}.

Proof. Let \{e_1, \ldots, e_n\} be an orthonormal basis of \mathcal{V}. Define u = \sum_{i=1}^n \overline{\varphi(e_i)} \, e_i. For any v = \sum_{j=1}^n a_j e_j, \langle v, u \rangle = \sum_{j=1}^n a_j \left\langle e_j, \sum_{i=1}^n \overline{\varphi(e_i)} \, e_i \right\rangle = \sum_{j=1}^n a_j \varphi(e_j) = \varphi\!\left(\sum_{j=1}^n a_j e_j\right) = \varphi(v), using orthonormality \langle e_j, e_i \rangle = \delta_{ji} and linearity of \varphi.

For uniqueness, suppose \langle v, u \rangle = \langle v, u' \rangle for all v. Then \langle v, u - u' \rangle = 0 for all v. Setting v = u - u' gives \|u - u'\|^2 = 0, hence u = u'. \square

The Riesz representation theorem extends to infinite-dimensional Hilbert spaces, where it identifies every continuous linear functional with an inner product. In finite dimensions, all linear functionals are continuous, so no continuity hypothesis is needed.

10.11 Closing Remarks

This chapter introduced inner products as the bridge between algebraic and geometric structures on vector spaces. An inner product \langle \cdot, \cdot \rangle induces a norm \|\cdot\|, which induces a metric d(\cdot, \cdot), converting the vector space into a metric space where convergence, continuity, and completeness become meaningful. The Cauchy–Schwarz inequality |\langle u, v \rangle| \le \|u\| \|v\| underlies the triangle inequality and all estimates involving lengths and angles.

We established that inner products on real spaces are symmetric positive definite bilinear forms, while on complex spaces they are Hermitian positive definite sesquilinear forms. The parallelogram law \|u+v\|^2 + \|u-v\|^2 = 2\|u\|^2 + 2\|v\|^2 characterizes which norms arise from inner products, and the polarization identity reconstructs the inner product from the norm.

In finite dimensions, inner products admit matrix representations via the Gram matrix G with g_{ij} = \langle v_i, v_j \rangle for basis vectors v_i. The Gram matrix is Hermitian and positive definite, and every such matrix defines an inner product. Orthogonality u \perp v generalizes perpendicularity, satisfying the Pythagorean theorem \|u+v\|^2 = \|u\|^2 + \|v\|^2 and admitting interpretation as extremal separation.

The theory developed here provides the foundation for Chapter 11, where we study orthogonal decompositions, projections onto subspaces, and the Gram–Schmidt process for constructing orthonormal bases. The projection theorem—that every finite-dimensional subspace \mathcal{W} of an inner product space admits a direct sum decomposition \mathcal{V} = \mathcal{W} \oplus \mathcal{W}^\perp into orthogonal complements—depends critically on the positive definiteness established in this chapter.

Chapter 12 introduces the adjoint operator T^* : \mathcal{W} \to \mathcal{V} for T : \mathcal{V} \to \mathcal{W}, characterized by \langle T(v), w \rangle = \langle v, T^*(w) \rangle. Self-adjoint operators T = T^* on finite-dimensional inner product spaces admit orthonormal eigenbases by the spectral theorem, reducing T to diagonal form in coordinates adapted to the geometry. This spectral decomposition underpins quadratic form theory, principal component analysis, and quantum mechanics.

The machinery of inner product spaces extends far beyond finite dimensions. Hilbert spaces—complete infinite-dimensional inner product spaces—are the proper setting for Fourier analysis, where functions decompose into orthogonal series of sines and cosines. Sobolev spaces, essential in partial differential equations, are Hilbert spaces with weighted inner products incorporating derivatives. The Riesz representation theorem identifies every continuous linear functional on a Hilbert space with an inner product, collapsing the distinction between vectors and covectors that persists in non-metric settings.

Inner products impose rigidity: not every vector space admits one, and those that do acquire special structure distinguishing them from arbitrary normed spaces. The interplay between algebra (linear maps, bases, coordinates) and geometry (lengths, angles, orthogonality) governs the remainder of this text.