Show that the set of even functions, \(f(x) = f(-x) \) is a subspace of the vector space of all function \(f(\Re ) \)
Answer:
(a) if \(f\) is an even function, then \(f(x) -f(-x)=0 \)
let \(w(x) =f(x) +g(k) \) where \(f,g\,\ \)are even functions. To show closure under addition, We need to show that \(w( x) \) is also an even function. \begin{align*} w( x) -w( -x) & = \{ f(x)+g(x) \} - \{ f(-x)+g(-x) \}\\ & = \{ f(x)-f(-x) \} + \{ g(x) -g(-x) \}\\ & = 0+0\\ & = 0 \end{align*}
Hence \(w(x)\) is closed under addition. To show closure under scalar multiplication. Let \(c\in \Re \).\(\,\) we need to show that \(cf(x)\) is even function when \(f(x)\) is even function. Let \(g(x)=cf(x)\) \begin{align*} g(x)-g(-x) & =cf(x)-cf(-x)\\ & =c\left \{ f(x)-f(-x)\right \} \\ & =c\left ( 0\right ) \\ & =0 \end{align*}
Hence closed under scalar multiplication.
And since the ”zero” function is also even (and odd as well), Hence even functions are subspace of the vector space of all function \(f( \Re ) \)
Show that the set of odd functions, \(g( -x) =-g( x) \) form a complementary subspace with the set of even functions (i.e. two subspaces \(W,Z\) of \(V\) are complementary if
(i) \(W\cap Z=\left \{ \vec{0}\right \} \)
(ii) \(W\cup Z=V\), i.e. every \(\vec{v}\in V\) can be written as \(\vec{v}=\vec{w}+\vec{z}\) where \(\vec{w}\in W,\vec{z}\in Z\)
solution: Let the set of odd functions be \(W\) and let the set of even functions be \(Z\). Let the set of all functions be \(V\).
To show that \(W,Z\) are complementary, we need to show that the above 2 properties are met.
Looking at property (i). This property says that the function \(\vec{v}\in V\) can be decomposed into the sum of an odd function and even function in one and only one way. i.e. \(\vec{v}=\vec{w}+\vec{z}\) where \(\vec{w}\in W,\vec{z}\in Z\) is a unique decomposition of \(\vec{v}\).
To show this, apply proof by contradiction. Assume the function \(\vec{v}\in V\) can be written as the sum of even and odd functions in 2 different ways. \(\vec{v}=\vec{w}_{1}+\vec{z}_{1}\) and also \(\vec{v}=\vec{w}_{2}+\vec{z}_{2}\) where \(\vec{w}_{1},\vec{w}_{2}\in W\) and \(\vec{z}_{1},\vec{z}_{2}\in Z\). But this means that \(\vec{w}_{1}+\vec{z}_{1}=\vec{w}_{2}+\vec{z}_{2}\). Which implies that \(\vec{w}_{1}-\vec{w}_{2}=\vec{z}_{2}-\vec{z}_{1}\).
Since the difference between 2 even functions is an even function (This can be easily shown from properties of even functions if needed), and the difference between 2 odd function is an odd function, then we have that an even function is identically equal to an odd function. Which is not possible unless both are zero. Hence \(\vec{w}_{1}-\vec{w}_{2}=\vec{z}_{2}-\vec{z}_{1}=0\) which means that \(\vec{w}_{1}=\vec{w}_{2}\) and \(\vec{z}_{2}=\vec{z}_{1}\), therefor the decomposition of \(\vec{v}\) must be unique. This proofs property (i).
Now we need to proof property (ii). This means that any function can be written as the sum of an odd and even function.
answer: Let \(f( x) \in V\) be any arbitrary function. Write it as follows
\[ f( x) =\frac{1}{2}f( x) +\frac{1}{2}f( x) \]
Now add and subtract from the RHS \(\frac{1}{2}f( -x) \), This will not change anything
\[ f( x) =\frac{1}{2}f( x) +\frac{1}{2}f( x) +\left \{ \frac{1}{2}f( -x) -\frac{1}{2}f( -x) \right \} \]
regroup as follows
\begin{align*} f( x) & =\left \{ \frac{1}{2}f( x) +\frac{1}{2}f( -x) \right \} +\left \{ \frac{1}{2}f( x) -\frac{1}{2}f( -x) \right \} \\ & =\frac{1}{2}\left \{ f( x) +f( -x) \right \} +\frac{1}{2}\left \{ f( x) -f( -x) \right \} \end{align*}
Now let \(g( x) =\left \{ f( x) +f( -x) \right \} \), then to show that \(g( x) \) is even, i.e. \(g( x) \in W\), need to show that \(g( x) -g( -x) =0\) \begin{align*} g( x) -g( -x) & =\left \{ f( x) +f( -x) \right \} -\left \{ f( -x) +f( -(-x)) \right \} \\ & =\left \{ f( x) +f( -x) \right \} -\left \{ f( -x) +f( x) \right \} \\ & =f( x) -f( x) +f( -x) -f( -x)\\ & =0 \end{align*}
Hence \(g( x) \) is even.
Now let \(h( x) =\left \{ f( x) -f( -x) \right \} \), to show that \(h( x) \) is odd, i.e. \(h( x) \in Z\), we need to show that \(h( -x) =-h( x) \) or \(h( -x) +h( x) =0\)
\begin{align*} h( -x) +h( x) & =\left \{ f( -x) -f( -(-x)) \right \} +\left \{ f( x) -f(-x)\right \} \\ & =\left \{ f( -x) -f(x)\right \} +\left \{ f( x) -f(-x)\right \} \\ & =f( -x) -f(-x)-f(x)+f( x)\\ & =0 \end{align*}
Hence \(h( x) \) is odd.
Hence we showed that \(f( x) =\frac{1}{2}\)even function \(+\frac{1}{2}\)odd function. Hence \(f( x) =f_{e}( x) +f_{o}( x) \) where \(f_{e}( x) \) is the even part of \(f( x) \) and \(f_{o}( x) \) is the odd part of \(f( x) \).
side note: Let the basis of the subspace \(W\) be \(\left \{ w_{1},w_{2},\cdots ,w_{n}\right \} \), and let the basis of the subspace \(Z\) be \(\left \{ z_{1},z_{2},\cdots ,z_{n}\right \} \). Property (ii) implies that a basis of \(V\) can be taken as the union of these 2 sets of bases, i.e. basis for \(V=\left \{ w_{1},w_{2},\cdots ,w_{n}\right \} \cup \left \{ z_{1},z_{2},\cdots ,z_{n}\right \} =\left \{ w_{1},w_{2},\cdots ,w_{n},z_{1},z_{2},\cdots ,z_{n}\right \} \)
Problem: Show that every function can be uniquely written as the sum of even and odd function.
Solution: From part(b), since we showed that the subspaces of even and odd functions are complementary, hence this follows from the property of such subspaces.
Problem: Prove that a linear system \(Ax=b\) of \(m\) linear equations in \(n\) unknowns has either
answer:
What I have to show is that if more than one solution exist, then there is infinite number of solutions. In other words, one can not have finitely countable number of solutions other than zero or 1.
Assume there exist 2 solutions. \(\mathbf{x}_{1}\mathbf{,x}_{2}\), hence \(A\mathbf{x}_{1}=\mathbf{b}\), and \(A\mathbf{x}_{2}=\mathbf{b}\).
We can show that any point on the line joining the vectors \(\mathbf{x}_{1}\mathbf{,x}_{2}\) is also a solution.
Vector \(\mathbf{v}\) can be parameterized by scalar \(t\) where \[ \mathbf{v}=\mathbf{x}_{1}+t( \mathbf{x}_{2}-\mathbf{x}_{1}) \] By changing \(t\) we can obtain new vector \(\mathbf{v}\). There are infinitely many such vectors as \(t\) can have infinitely many values.
\begin{align*} A\mathbf{v} & =A( \mathbf{x}_{1}+t( \mathbf{x}_{2}-\mathbf{x}_{1}) )\\ & =A( \mathbf{x}_{1}) +A( t( \mathbf{x}_{2}-\mathbf{x}_{1}) ) \text{ \ \ \ \ by linearity of A }\\ & =A( \mathbf{x}_{1}) +tA( \mathbf{x}_{2}-\mathbf{x}_{1}) \ \ \ \ \ \ \ \ \ \ \text{by linearity of A }\\ & =A( \mathbf{x}_{1}) +t( A( \mathbf{x}_{2}) -A( \mathbf{x}_{1}) ) \text{\ \ \ \ by linearity of A } \end{align*}
But \(A( \mathbf{x}_{1}) =\mathbf{b}\), and \(A( \mathbf{x}_{2}) =\mathbf{b}\), hence the above becomes
\begin{align*} A\mathbf{v} & =\mathbf{b}+t( \mathbf{b}-\mathbf{b})\\ & =\mathbf{b} \end{align*}
Therefor, \(\mathbf{v}\), which is different than \(\mathbf{x}_{1}\) and \(\mathbf{x}_{2}\) is also a solution. Hence if there are 2 solutions, then we can always find an arbitrary new solution from these 2 solutions, Hence there are infinitely many solutions. QED
Problem: Prove that the inner product defined by \(\left \langle f,g\right \rangle =\int _{a}^{b}f( x) g( x) +f^{\prime }( x) g^{\prime }( x) dx\) satisfy the conditions of an inner product on the space on continuously differentiable functions on the interval \(\left [ a,b\right ] \) Answer:
An inner product must satisfy the following properties. Let \(f,g,w\) be continuously differentiable functions on \([a,b]\) and let \(t\) be scalar.
1. \(\left \langle f,g\right \rangle =\left \langle g,f\right \rangle \)
2. \(\left \langle tf,g\right \rangle =t\left \langle f,g\right \rangle \)
3. \(\left \langle f+g,w\right \rangle =\left \langle f,w\right \rangle +\left \langle g,w\right \rangle \)
4. \(\left \langle f,f\right \rangle >0\) if \(f\neq 0\) or \(\left \langle f,f\right \rangle =0\) iff \(f=0\)
To show property 1. Since \[ \left \langle f,g\right \rangle =\int _{a}^{b}f( x) g( x) +f^{\prime }( x) g^{\prime }( x) dx \]
Now, since real valued functions are commutative under multiplication (i.e. \(f(x)g(x)=g(x)f(x))\) and similarly for the derivatives, we can exchange the order of multiplication \begin{align*} \left \langle f,g\right \rangle & =\int _{a}^{b}g(x)f(x)+g^{\prime }(x)f^{\prime }(x)dx\\ & =\left \langle g,f\right \rangle \end{align*}
To show property 2:
\begin{align*} \left \langle tf,g\right \rangle & =\int _{a}^{b}tf( x) g( x) +( tf( x) ) ^{\prime }g^{\prime }( x) dx\\ & =\int _{a}^{b}tf( x) g( x) +tf( x) ^{\prime }g^{\prime }( x) dx\text{ \ \ \ since t is constant}\\ & =\int _{a}^{b}t( f( x) g( x) +f( x) ^{\prime }g^{\prime }( x) ) dx\\ & =t\int _{a}^{b}f( x) g( x) +f( x) ^{\prime }g^{\prime }( x) dx\\ & =t\left \langle f,g\right \rangle \end{align*}
To show property 3:
\begin{align*} \left \langle f+g,w\right \rangle & =\int _{a}^{b}( f+g) ( x) \ w( x) +\frac{d}{dx}( f+g) ( x) \ w^{\prime }( x) dx\\ & =\int _{a}^{b}( f( x) +g( x) ) \ w( x) +( f^{\prime }( x) +g^{\prime }( x) ) \ w^{\prime }( x) dx \end{align*}
Now, since we can distribute multiplication over addition for real valued functions, i.e. \((f+g)w=fw+gw\,\ \)(because function multiplications is a point-by-point multiplication) the above becomes
\[ \left \langle f+g,w\right \rangle =\int _{a}^{b}\left \{ f( x) \ w( x) +g( x) \ w( x) \right \} +\left \{ f^{\prime }( x) w^{\prime }( x) +g^{\prime }( x) w^{\prime }( x) \right \} \ dx \]
By linearity of integration operation we can break above integral into the sum of two integrals
\begin{align*} \left \langle f+g,w\right \rangle & =\int _{a}^{b}f( x) \ w( x) +f^{\prime }( x) w^{\prime }( x) dx+\int _{a}^{b}g( x) \ w( x) +g^{\prime }( x) w^{\prime }( x) \ dx\ \ \\ & =\ \left \langle f,w\right \rangle +\left \langle g,w\right \rangle \end{align*}
To show property 4:
\begin{align*} \left \langle f,f\right \rangle & =\int _{a}^{b}f( x) \ f( x) +f^{\prime }( x) f^{\prime }( x) dx\\ & =\int _{a}^{b}\left [ f( x) \right ] ^{2}+\left [ f^{\prime }( x) \right ] ^{2}dx\\ & =\int _{a}^{b}\left [ f( x) \right ] ^{2}dx+\int _{a}^{b}\left [ f^{\prime }( x) \right ] ^{2}dx \end{align*}
Consider \(\int _{a}^{b}\left [ f( x) \right ] ^{2}dx\). Since \(\left [ f( x) \right ] ^{2}\) can only be positive or zero\(,\)This is the same as \(\int _{a}^{b}g(x)dx\) where \(g( x) \geq 0\) over \([a,b],\) Hence \(\int _{a}^{b}g(x)dx=0\) only if \(g(x)\) is identically zero over \(\lbrack a,b]\), but if \(g( x) =0\), then \(\left [ f( x) \right ] ^{2}=0\) or \(f( x) =0\), which means \(\int _{a}^{b}\left [ f( x) \right ] ^{2}dx=0\).
Now if \(f( x) =0\), then the second integral \(\int _{a}^{b}\left [ f^{\prime }( x) \right ] ^{2}dx=0\) as well.
Hence \(\left \langle f,f\right \rangle =0\) only if \(f( x) \) is identically zero over \([a,b]\)
Hence we showed the 4 properties for this definition of the inner product.
problem: \(L_{2}\) norm on the interval \([a,b]\) is defined as \(\left \langle f,f\right \rangle =\int _{b}^{a}\left [ f( x) \right ] ^{2}dx\)
Find the cubic polynomial that best approximates the function \(e^{x}\) on the interval \(\left [ 0,1\right ] \) by minimizing the \(L_{2}\) error.
solution:
Let \(p( x) =a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3}\), hence we need to have 4 equations to solve for \(a_{0},a_{1},a_{2},a_{3}\)
Let the \(g( x) =p( x) -e^{x}\), which is the error function.
From the definition, the square of norm of this error is \begin{align*} \left \vert E\right \vert ^{2} & =\Vert p( x) -e^{x}\Vert ^{2}\\ & =\Vert g( x) \Vert ^{2}\\ & =\left \langle g( x) ,g( x) \right \rangle \\ & =\int _{0}^{1}\left [ g( x) \right ] ^{2}dx\\ & =\int _{0}^{1}\left [ p( x) -e^{x}\right ] ^{2}dx \end{align*}
\begin{align*} \left \vert E\right \vert ^{2} & =\int _{0}^{1}\left [ p( x) -e^{x}\right ] ^{2}dx=\int _{0}^{1}\left [ a_{0}+a_{1}x+a_{2}x^{2}+a_{3}x^{3}-e^{x}\right ] ^{2}dx\\ & =-\frac{1}{2}+\frac{e^{2}}{2}+2a_{0}+a_{0}^{2}+a_{0}a_{1}+\frac{a_{1}^{2}}{3}+4a_{2}+\\ & \frac{2a_{0}a_{2}}{3}+\frac{a_{2}^{2}}{5}+a_{1}( -2+\frac{a_{2}}{2}+\frac{2a_{3}}{5}) -12a_{3}+\\ & \frac{a_{0}a_{3}}{2}+\frac{a_{2}a_{3}}{3}+\frac{a_{3}^{2}}{7}+e( -2a_{0}-2a_{2}+4a_{3}) \end{align*}
Now minimize this error with respect to each of the coefficients in turn to generate 4 equations to solve.
\begin{align*} \frac{d\left \vert E\right \vert ^{2}}{da_{0}} & =0=2-2e+2a_{0}+a_{1}+\frac{2a_{2}}{3}+\frac{a_{3}}{2}\\ \frac{d\left \vert E\right \vert ^{2}}{da_{1}} & =0=-2+a_{0}+\frac{2a_{1}}{3}+\frac{a_{2}}{2}+\frac{2a_{3}}{5}\\ \frac{d\left \vert E\right \vert ^{2}}{da_{2}} & =0=4-2e+\frac{2a_{0}}{3}+\frac{a_{1}}{2}+\frac{2a_{2}}{5}+\frac{a_{3}}{3}\\ \frac{d\left \vert E\right \vert ^{2}}{da_{4}} & =0=-12+4e+\frac{a_{0}}{2}+\frac{2a_{1}}{5}+\frac{a_{2}}{3}+\frac{2a_{3}}{7} \end{align*}
Hence, set up the above 4 equations in matrix form, we obtain \[\begin{bmatrix} 2 & 1 & \frac{2}{3} & \frac{1}{2}\\ 1 & \frac{2}{3} & \frac{1}{2} & \frac{2}{5}\\ \frac{2}{3} & \frac{1}{2} & \frac{2}{5} & \frac{1}{3}\\ \frac{1}{2} & \frac{2}{5} & \frac{1}{3} & \frac{2}{7}\end{bmatrix}\begin{bmatrix} a_{0}\\ a_{1}\\ a_{2}\\ a_{3}\end{bmatrix} =\begin{bmatrix} 2e-2\\ 2\\ 2e-4\\ 12-4e \end{bmatrix} \]
Solving for \(a^{\prime }s\) using Gaussian elimination leads to solution
\begin{align*} a_{0} & =0.9906\\ a_{1} & =1.0183\\ a_{2} & =0.421246\\ a_{3} & =0.278625 \end{align*}
Hence the best fit cubic polynomial that minimize the error to \(e^{x}\) between \(0\) and \(1\) is
\[ p( x) =0.9906+1.0183x+0.421246x^{2}+0.278625x^{3}\]
This is a table of values to compare \(e^{x}\) and \(p( x) \)
Problem: A Hilbert space is a function space with a norm. If we consider the space of continuous functions on \(\left [ a,b\right ] \) with \(L_{2}\) norm, it is Hilbert space \(H\). A key step in showing that functions on this space can be approximated using a countable (i.e. indexed by integers) orthonormal set is the Bessel Inequality \[ \sum _{i=1}^{n}\left \langle f,\phi _{i}\right \rangle ^{2}\leq \Vert f\Vert ^{2}<\infty \]
where \(\phi _{i}\) is an element of the orthonormal set and \(f\) is the element of the Hilbert space being approximated.
If we approximate \(f( x) \) by \(\sum _{i=1}^{n}\alpha _{i}\phi _{i}( x) \) with \(\alpha _{i}=\left \langle f,\phi _{i}\right \rangle \). Start by stating the error in the approximation to prove the Bessel inequality.
solution
In this solution, I use the analogy to the normal Euclidean space just as a guideline.
\(\alpha _{i}=\left \langle f,\phi _{i}\right \rangle \) is the projection of the function \(f\) onto the basis \(\phi _{i}\). This is similar to extracting the \(i^{th}\) coordinate of a vector. The expression \(\alpha _{i}\phi _{i}( x) \) is then a vector along the direction of the base \(\phi _{i}\), whose length is the projection of \(f\) in the direction of the \(i^{th}\) basis. Hence in general, \[ f( x) =\sum _{i=1}^{Number\ of\ Basis}\alpha _{i}\phi _{i}( x) \] This is similar to the Euclidean coordinate system where we write \(\vec{v}=x\vec{i}+y\vec{j}+z\vec{k}\) where \(\vec{i},\vec{j},\vec{k}\) are the basis in this space and \(x,y,z\,\ \)are the coordinates of the vector. A vector coordinate is the length of the projection of the vector onto each specific basis. The expression for \(f(x)\) above is a generalization of this concept to the function space and to an arbitrary number of basis.
And similarly to what we do in the Euclidean space, the ’length’ of the vector using \(L_{2}\) norm is \(\Vert \vec{v}\Vert _{2}=\sqrt{x^{2}+y^{2}+z^{2}}\), hence \(\Vert \vec{v}\Vert _{2}^{2}=x^{2}+y^{2}+z^{2}\). This is generalized to the \(H\) space by saying\begin{align*} \Vert f\Vert ^{2} & ={\displaystyle \sum \limits _{i}^{Number\ of\ Basis}} ( \alpha _{i}) ^{2}\\ & ={\displaystyle \sum \limits _{i}^{Number\ of\ Basis}} \left \langle f,\phi _{i}\right \rangle ^{2} \end{align*}
If the number of basis is infinite, then we write
\[ \Vert f\Vert ^{2}={\displaystyle \sum \limits _{i}^{\infty }} \left \langle f,\phi _{i}\right \rangle ^{2}\]
Therefore, if the number of basis is infinite, and we sum for some finite number of basis less than infinite, say \(n\), hence the resulting norm must be less than the actual norm we would get if we had added over all the basis. Hence it is obvious that \(\Vert f\Vert ^{2}\geq{\displaystyle \sum \limits _{i}^{n}} \left \langle f,\phi _{i}\right \rangle ^{2}\) since we terminated the sum earlier, and since each quantity being summed is positive, then the partial sum must be less than the limit, which is \(\Vert f\Vert ^{2}.\)
Now we just need to show that the norm finite. If the function itself is finite (meaning its value, or range, is finite) then each of its projections must be finite (\(\left \vert \cos \alpha \right \vert \leq 1)\). Hence given a function which does not ”blow” up, then all its components must be finite. Since we are adding finite number of quantities, each of which is finite in its own, hence the sum must be finite as well. Hence \(\Vert f\Vert <\infty \), or \(\Vert f\Vert ^{2}<\infty \)
Therefore \[{\displaystyle \sum \limits _{i}^{n}} \left \langle f,\phi _{i}\right \rangle ^{2}\leq \Vert f\Vert ^{2}<\infty \]