Properties of orthonormal systems

Inner Product Spaces: Orthonormal systems

Properties of orthonormal systems

We discuss some properties of orthonormal systems of vectors.

Properties of orthonormal systems of vectors
Let #V# be an inner product space and #\vec{x}# a vector of #V#.

Orthonormal systems in #V# are linearly independent.
If #\basis{\vec{a}_1,\ldots ,\vec{a}_n}# is an orthonormal basis of #V#, then the coordinates of #\vec{x}# with respect to this basis are, successively, #\dotprod{\vec{x}}{\vec{a}_1}, \ldots , \dotprod{\vec{x}}{\vec{a}_n}#: \[\vec{x}=(\dotprod{\vec{x}}{\vec{a}_1})\vec{a}_1+\cdots+(\dotprod{\vec{x}}{\vec{a}_n})\vec{a}_n\]
The length of #\vec{x}# is equal to the length of the coordinate vector of #\vec{x}# relative to the standard inner product:
\[\norm{\vec{x}}^2 =\left(\dotprod{\vec{x}}{\vec{a}_1}\right)^2 + \cdots + \left(\dotprod{\vec{x}}{\vec{a}_n}\right)^2\]

1. Suppose that #\vec{a}_1,\ldots ,\vec{a}_n# is an orthonormal system in #V#. To demonstrate its independence, we start with the equation
\[\lambda_1 \vec{a}_1 + \cdots + \lambda_n \vec{a}_n =\vec{0}\] with unknowns #\lambda_1,\ldots , \lambda_n#. By taking the inner product of each side of the equality with the vector #\vec{a}_j# with #1\leq j\leq n# we find
\[
\dotprod{(\lambda_1 \vec{a}_1 + \cdots + \lambda_n \vec{a}_n)}{ \vec{a}_j} =\dotprod{\vec{0}}{\vec{a}_j}=0
\] The left-hand side can be simplified as follows:
\[\begin{array}{rcl}\dotprod{(\lambda_1 \vec{a}_1 + \cdots + \lambda_n \vec{a}_n)}{ \vec{a}_j}&=&\lambda_1(\dotprod{\vec{a}_1}{\vec{a}_j})+\cdots + \lambda_n(\dotprod{\vec{a}_n}{\vec{a}_j})\\&&\phantom{xx}\color{blue}{\text{linearity of the inner product}}\\ &=&\lambda_j (\dotprod{\vec{a}_j}{\vec{a}_j})\\&&\phantom{xx}\color{blue}{\text{orthogonality of the system}}\\&=&\lambda_j\\ &&\phantom{xx}\color{blue}{\text{orthonormality of the system}}\\\end{array}\]So the only solution to #\lambda_1 \vec{a}_1 + \cdots + \lambda_n \vec{a}_n =\vec{0}# is #\lambda_j=0# for all #j#, showing the independence of #\vec{a}_1,\ldots ,\vec{a}_n#.

2. Because #\vec{a}_1,\ldots ,\vec{a}_n# is a basis of #V#, there are scalars #\mu_1 , \ldots ,\mu_n# such that #\vec{x}=\mu_1 \vec{a}_1 +\cdots + \mu_n \vec{a}_n#. In order to determine #\mu_j#, we again take the inner product of each side of this equality with #\vec{a}_j#:
\[
\dotprod{\vec{x}}{\vec{a}_j}=\dotprod{(\mu_1 \vec{a}_1 +\cdots + \mu_n \vec{a}_n)}{\vec{a}_j}
\] After working out the right-hand side in a similar manner as above we find #\dotprod{\vec{x}}{\vec{a}_j}=\mu_j#.

3. We now know that we can write #\vec{x}# as #\vec{x}=(\dotprod{\vec{x}}{\vec{a}_1})\vec{a}_1+\cdots+(\dotprod{\vec{x}}{\vec{a}_n})\vec{a}_n#. Since all of these terms are perpendicular to each other, we can apply the Pythagorean theorem, which gives \[\begin{array}{rcl}\norm{\vec{x}}^2&=&\norm{(\dotprod{\vec{x}}{\vec{a}_1})\vec{a}_1+\cdots+(\dotprod{\vec{x}}{\vec{a}_n})\vec{a}_n}^2\\&&\phantom{xx}\color{blue}{\text{above expression for }\vec{x}}\\&=&\norm{(\dotprod{\vec{x}}{\vec{a}_1})\vec{a}_1}^2 + \cdots + \norm{(\dotprod{\vec{x}}{\vec{a}_n})\vec{a}_n}^2\\&&\phantom{xx}\color{blue}{\text{Pythagoras}}\\&=&(\dotprod{\vec{x}}{\vec{a}_1})^2\norm{\vec{a}_1}^2 + \cdots + (\dotprod{\vec{x}}{\vec{a}_n})^2\norm{ \vec{a}_n}^2\\&&\phantom{xx}\color{blue}{\text{linearity of the inner product}}\\&=&(\dotprod{\vec{x}}{\vec{a}_1})^2+\cdots+(\dotprod{\vec{x}}{\vec{a}_n})^2\\&&\phantom{xx}\color{blue}{\norm{\vec{a}_i}= 1 \text{ thanks to orthonormality of the system}}\end{array}\]

Basis

From the mutual independence of the vectors in an orthonormal system it follows that, if we have an orthonormal system of #n# vectors in an #n#-dimensional inner product space, this system forms a basis for this space.

Later we will show that an orthonormal basis can be found for any finite-dimensional inner product space.

Example 3D The system \[
\frac{1}{\sqrt{2}}\rv{1,-1,0}, \, \frac{1}{\sqrt{2}}\rv{1,1,0},\,\rv{0,0,1}
\] forms an orthonormal system, and so is independent. The three vectors form a basis of #\mathbb{R}^3# because the dimension of the space equals #3#. The coordinates of #\vec{a}=\rv{2,2,2}# with respect to this basis are
\[
\dotprod{\vec{a}}{\frac{1}{\sqrt{2}}\rv{1,-1,0}}=0, \phantom{xx}
\dotprod{\vec{a}}{ \frac{1}{\sqrt{2}}\rv{1,1,0}}=2\sqrt{2},\phantom{xx} \dotprod{\vec{a}}{\rv{0,0,1}}=2
\] Consequently,
\[
\rv{2,2,2}=2\sqrt{2}\cdot\frac{1}{\sqrt{2}}\,\rv{1,1,0} + 2\,\cdot \rv{0,0,1}
\]

The following statement is of interest for computing inner products if we have an orthonormal basis at our disposal.

From inner product to standard inner productLet #\basis{\vec{e}_1,\ldots ,\vec{e}_n}# be an orthonormal basis for an inner product space #V#, and let

\[\vec{x}=\sum_{i=1}^n x_i\vec{e}_i,\qquad \vec{y}=\sum_{j=1}^n y_j\vec{e}_j \] be vectors of #V#, written as linear combinations of the basis vectors. Then the inner product of #\vec{x}# and #\vec{y}# can be expressed as follows:\[\dotprod{\vec{x}}{\vec{y}}=\sum_{i=1}^n x_i\cdot y_i\]

Because of the definition of a basis we can write the vectors #\vec{x}# and #\vec{y}# as
\[\vec{x}=\sum_{i=1}^n x_i\vec{e}_i,\qquad \vec{y}=\sum_{i=1}^n y_i\vec{e}_i\] We now work out the inner product: \[\begin{array}{rcl}\dotprod{\vec{x}}{\vec{y}}&=&\displaystyle\dotprod{\left(\sum_{i=1}^n x_i\vec{e}_i\right)}{\left(\sum_{i=1}^n y_i\vec{e}_i\right)}\\&&\phantom{xx}\color{blue}{\text{written in terms of the basis}}\\&=&\displaystyle\sum_{i=1}^n\sum_{j=1}^n x_i y_j \cdot(\dotprod{\vec{e}_i}{ \vec{e}_j})\\&&\phantom{xx}\color{blue}{\text{linearity of the inner product}}\\&=&\displaystyle \sum_{i=1}^nx_iy_i\cdot(\dotprod{\vec{e}_i}{\vec{e}_i})\\&&\phantom{xx}\color{blue}{\dotprod{\vec{e}_i}{\vec{e}_j}=0\text{ if }i\neq j}\\&=&\displaystyle\sum_{i=1}^n x_i y_i\\&&\phantom{xx}\color{blue}{\dotprod{\vec{e}_i}{\vec{e}_j}=1\text{ if }i=j}\end{array}\]

Matrix form Just as for the standard inner product, the inner product can be written as a product of a #(1\times n)#-matrix and an #(n\times 1)#-matrix. We write #\vec{c}_x# for coordinate vector #\vec{x}# with respect to the basis #\vec{e}_1,\ldots ,\vec{e}_n#, viewed as a column vector. Similarly, we write #\vec{c}_y# for the coordinate vector #\vec{y}# written as a column vector. Thus, these vectors are #(n\times1)#-matrices. Now we can write the inner product #\dotprod{\vec{x}}{\vec{y}}# as \[\dotprod{\vec{x}}{\vec{y}} = \vec{c}_x^{\top}\,\vec{c}_y\] For example,\[\dotprod{\rv{2,3,1}}{\rv{-3,5,2}} = 2\cdot(-3)+3\cdot 5 +1\cdot 2= \matrix{2&3&1}\,\matrix{-3\\ 5\\2} \]

BasisThe theorem tells us that if we start calculating in #V# with coordinates with respect to an orthonormal basis, we can express the inner product of two vectors in #V# in terms of the standard inner product on #{\mathbb{R}^n}#.

We consider the vector space #{\mathbb{R}^3}# with the standard inner product. Suppose that the following orthonormal basis is given.
\[\begin{array}{rll}
\displaystyle \vec{v}_1&=&\displaystyle \rv{ {{2}\over{\sqrt{6}}} , {{1}\over{\sqrt{6}}} , -{{1}\over{\sqrt{6}}} } \\
\vec{v}_2&=&\displaystyle \rv{ 0 , {{1}\over{\sqrt{2}}} , {{1}\over{\sqrt{2}}} } \\
\vec{v}_3&=&\displaystyle \rv{ {{1}\over{\sqrt{3}}} , -{{1}\over{\sqrt{3}}} , {{1}\over{\sqrt{3}}} }
\end{array}\]
Determine the coordinate vector #\vec{c}=\rv{c_1,c_2,c_3}# with respect to the given basis of the vector
\[
\vec{x}=\rv{ 3 , 2 , 2 }
\]

#\vec{c}=# #\rv{\sqrt{6},2\sqrt{2},\sqrt{3}}#

The given basis is orthonormal. According to the Properties of orthonormal systems of vectors, the coordinates of a vector #\vec{x}# with respect to an orthonormal basis equal the dot products of #\vec{x}# with the basis vectors. We calculate each of these inner products individually.
\[\begin{array}{rcl}
c_1&=&\displaystyle\dotprod{\vec{v}_1}{\vec{x}}\\
&&\phantom{xx}\color{blue}{\text{theorem}}\\
&=&\displaystyle\dotprod{\rv{ {{2}\over{\sqrt{6}}} , {{1}\over{\sqrt{6}}} , -{{1}\over{\sqrt{6}}} } }{\rv{ 3 , 2 , 2 } }\\
&&\phantom{xx}\color{blue}{\text{explicit vectors filled in}}\\
&=&\displaystyle\left({{2}\over{\sqrt{6}}}\cdot3\right)+\left({{1}\over{\sqrt{6}}}\cdot 2\right)+\left(-{{1}\over{\sqrt{6}}}\cdot2\right)\\
&&\phantom{xx}\color{blue}{\text{definition standard inner product}}\\
&=&\displaystyle\sqrt{6}\\
&&\phantom{xx}\color{blue}{\text{simplified}}\\
\end{array}\]
\[\begin{array}{rcl}
c_2&=&\displaystyle\dotprod{\vec{v}_2}{\vec{x}}\\
&&\phantom{xx}\color{blue}{\text{theorem}}\\
&=&\displaystyle\dotprod{\rv{ 0 , {{1}\over{\sqrt{2}}} , {{1}\over{\sqrt{2}}} } }{\rv{ 3 , 2 , 2 } }\\
&&\phantom{xx}\color{blue}{\text{explicit vectors filled in}}\\
&=&\displaystyle\left(0\cdot3\right)+\left({{1}\over{\sqrt{2}}}\cdot 2\right)+\left({{1}\over{\sqrt{2}}}\cdot2\right)\\
&&\phantom{xx}\color{blue}{\text{definition standard inner product}}\\
&=&\displaystyle2\sqrt{2}\\
&&\phantom{xx}\color{blue}{\text{simplified}}\\
\end{array}\]
\[\begin{array}{rcl}
c_3&=&\displaystyle\dotprod{\vec{v}_3}{\vec{x}}\\
&&\phantom{xx}\color{blue}{\text{theorem}}\\
&=&\displaystyle \dotprod{\rv{ {{1}\over{\sqrt{3}}} , -{{1}\over{\sqrt{3}}} , {{1}\over{\sqrt{3}}} } }{\rv{ 3 , 2 , 2 } }\\
&&\phantom{xx}\color{blue}{\text{explicit vectors filled in}}\\
&=&\displaystyle\left({{1}\over{\sqrt{3}}}\cdot3\right)+\left(-{{1}\over{\sqrt{3}}}\cdot 2\right)+\left({{1}\over{\sqrt{3}}}\cdot2\right)\\
&&\phantom{xx}\color{blue}{\text{definition standard inner product}}\\
&=&\displaystyle\sqrt{3}\\
&&\phantom{xx}\color{blue}{\text{simplified}}\\
\end{array}\]
So the coordinate vector #\vec{c}# is given by #\rv{\sqrt{6},2\sqrt{2},\sqrt{3}}#.

New example