# 3.39. Linear Algebra¶

## 3.39.1. Scalar Product¶

Virtually all spaces used in physics are Hilbert spaces (treated in the weak sense, i.e. equipped with distributions), which means that they have a scalar product and a norm.

The braket in Dirac notation is a scalar product and we are free to define it anyway we like, as long as it satisfies the following properties:

where if, and only if, . Scalar product induces the norm:

Any norm has to satisfy the following properties:

where if, and only if, . Those properties are automatically satisfied by the induced norm.

In general, any of these properties can be weakened, one can study spaces that have a norm, but not a scalar product, or spaces, that have objects resembling a norm (or a scalar product), that only satisfy some of the properties of the norm (or a scalar product). Those are not very important in physics. On the other hand, it is very important to understand how to work with Hilbert spaces (in the weak sense). Dirac notation makes it very easy to understand and remember how to work with such spaces.

### Examples¶

Some examples of frequently used spaces and scalar products follows.

Finite dimensional spaces:

Euclidean scalar product:

Infinite dimensional spaces:

scalar product:

Function spaces:

scalar product:

scalar product:

scalar product:

Energy scalar product:

All of these scalar products automatically satisfy all of the properties of the scalar product, only the energy scalar product doesn’t automatically satisfy , which imposes some conditions on the parameters and .

## 3.39.2. Projections¶

Projection is a linear idempotent operator :

It takes a vector from and projects it onto a vector from . Further application of the operator gains nothing: . It decomposes the space into a direct sum of the projection subspace and its complement . If is from then its complement is from .

Orthogonal projection is a projection that is Hermitean:

The complement of an orthogonal projection is orthogonal to any vector from :

In other words, orthogonal projection projects a vector from the space into an orthogonal subspace (projection subspace) . The two spaces and are orthogonal, because any vector from is orthogonal to all vectors from . Given the space , the operator is unique.

The complement of non-orthogonal projection is not orthogonal to any vector from :

And the two spaces and are not orthogonal, because any vector from is not orthogonal to any vector from . Given both spaces and , the operator is unique.

If we choose any orthonormal basis , , , …, of the subspace , then the orthogonal projection is:

(3.39.2.1)

because:

and

is independent of the basis, i.e , as long as span the same subspace as , because the operator is unique.

To find the closest vector from to the vector from , we need to minimize the norm . So we write for some vector from and simplify the norm:

which is minimal for , so we found out that the closest vector is . We used the fact that , because is from the orthogonal complement to the subspace . In other words, orthogonal projection finds the closest vector from a subspace onto which it projects.

### Projection Coefficients¶

Given the basis (orthogonal or non-orthogonal), we would like to find a formula for the projection coefficients defined by:

(3.39.2.2)

This holds, because belongs to the space and every vector from it can be expressed as a linear combination of .

#### Projecting to Orthogonal Basis¶

For orthogonal projections we simply substitute the equation (3.39.2.1) into (3.39.2.2) and get:

from which the projection coefficients are given by

(3.39.2.3)

#### Projecting to Nonorthogonal Basis¶

In order to project onto a nonorthogonal basis (for example a finite element basis), we multiply (3.39.2.2) by from the left and simplify:

so we need to solve a linear system for the coefficients :

(3.39.2.4)

where

This works for any basis, it doesn’t have to be normalized nor orthogonal. In the special case of a (normalized) orthogonal basis, we get and from (3.39.2.4) we get

so we recovered the equation (3.39.2.3) as expected.

### Examples¶

orthogonal projection. Orthogonal basis:

Different orthogonal basis:

non-orthogonal (oblique) projection ():

Because the projection is not orthogonal (in the Euclidean scalar product), the projected point is not the closest point (in the induced Euclidean norm) to . For the projection becomes orthogonal and indeed the projected point then becomes the closest point to .

Lagrange interpolation projection onto the space :

projection onto the space . Orthogonal basis:

Different orthogonal basis:

Nonorthogonal basis:

projection. Nonorthogonal basis: