Solution\begin{align*} x^{\prime } & =a_{1}x-b_{1}x^{2}-c_{1}xy\\ y^{\prime } & =a_{2}y-b_{2}y^{2}-c_{2}xy \end{align*}
Using the values given in the problem, the above equations become\begin{align} x^{\prime } & =60x-3x^{2}-4xy\tag{1A}\\ y^{\prime } & =42y-3y^{2}-2xy\tag{1B} \end{align}
Or\begin{align*} x^{\prime } & =f\left ( x,y\right ) \\ y^{\prime } & =g\left ( x,y\right ) \end{align*}
Equilibrium points are found by setting \(f\left ( x,y\right ) =0\) and \(g\left ( x,y\right ) \). This results in the following two equations to solve for \(x,y\)\begin{align} 60x-3x^{2}-4xy & =0\tag{1}\\ 42y-3y^{2}-2xy & =0\tag{2} \end{align}
The first equation (1A) becomes \(x\left ( 60-3x-4y\right ) =0\) which then gives one solution as\begin{equation} x=0\tag{3} \end{equation} And \(60-3x-4y=0\) gives another solution as\begin{equation} x=\frac{60-4y}{3}\tag{4} \end{equation} The second equation (1B) becomes \(y\left ( 42-3y-2x\right ) =0\) which gives one solution as\begin{equation} y=0\tag{5} \end{equation} And \(42-3y-2x=0\) gives another solution as \begin{equation} y=\frac{42-2x}{3}\tag{6} \end{equation} When \(x=0\) then (6) results in \(y=\frac{42}{3}=14.\) When \(x=\frac{60-4y}{3}\) then (6) results in \(y=\frac{42-2\left ( \frac{60-4y}{3}\right ) }{3}=\frac{8}{9}y+\frac{2}{3}\), or \(y=6\). Hence in this case \(x=\) \(\frac{60-4\left ( 6\right ) }{3}=12\).
Similarly, when \(y=0\) then from (4) \(x=\frac{60-4\left ( 0\right ) }{3}=20\). The above shows that there are \(4\) equilibrium points. These are\begin{align*} x & =0,y=0\\ x & =0,y=14\\ x & =12,y=6\\ x & =20,y=0 \end{align*}
To determine the type of stability of each equilibrium point, and since this is a nonlinear system, we must first linearize the system around each equilibrium point in order to determine the Jacobian matrix.
Once the system is linearized, then the eigenvalues of the Jacobian matrix are found in each case. From the values of eigenvalues we can then determine if the system is stable or not at each one of the above four equilibrium points.
The first step is then to linearize \(f\left ( x,y\right ) \) and \(g\left ( x,y\right ) \) around each of the equilibrium points. If we assume the equilibrium point is given by \(x_{0},y_{0}\) then expanding \(f\left ( x,y\right ) \) in Taylor series around this point gives\[ f\left ( x_{0}+\Delta x,y_{0}+\Delta y\right ) =f\left ( x_{0},y_{0}\right ) +\left . \frac{\partial f\left ( x,y\right ) }{\partial x}\right \vert _{x_{0},y_{0}}\left ( \Delta x+\Delta y\right ) +\left . \frac{\partial f\left ( x,y\right ) }{\partial y}\right \vert _{x_{0},y_{0}}\left ( \Delta x+\Delta y\right ) +\cdots \] But \(f\left ( x_{0},y_{0}\right ) =0\) since it is what defines an equilibrium point, the above becomes, after ignoring higher order terms since we are assuming small \(\Delta x,\Delta y\)\[ f\left ( x_{0}+\Delta x,y_{0}+\Delta y\right ) =\left . \frac{\partial f\left ( x,y\right ) }{\partial x}\right \vert _{x_{0},y_{0}}\left ( \Delta x+\Delta y\right ) +\left . \frac{\partial f\left ( x,y\right ) }{\partial y}\right \vert _{x_{0},y_{0}}\left ( \Delta x+\Delta y\right ) \] Similarly for \(g\left ( x,y\right ) \) we obtain the following\[ g\left ( x_{0}+\Delta x,y_{0}+\Delta y\right ) =\left . \frac{\partial g\left ( x,y\right ) }{\partial x}\right \vert _{x_{0},y_{0}}\left ( \Delta x+\Delta y\right ) +\left . \frac{\partial g\left ( x,y\right ) }{\partial y}\right \vert _{x_{0},y_{0}}\left ( \Delta x+\Delta y\right ) \] Therefore a linearized \(f,g\) functions at the equilibrium point become\[\begin{pmatrix} f\left ( x_{0}+\Delta x,y_{0}+\Delta y\right ) \\ g\left ( x_{0}+\Delta x,y_{0}+\Delta y\right ) \end{pmatrix} =\begin{pmatrix} \left . \frac{\partial f}{\partial x}\right \vert _{x_{0},y_{0}} & \left . \frac{\partial f}{\partial y}\right \vert _{x_{0},y_{0}}\\ \left . \frac{\partial g}{\partial x}\right \vert _{x_{0},y_{0}} & \left . \frac{\partial g}{\partial y}\right \vert _{x_{0},y_{0}}\end{pmatrix}\begin{pmatrix} \Delta x+\Delta y\\ \Delta x+\Delta y \end{pmatrix} \] Replacing the original nonlinear \(f\left ( x,y\right ) ,g\left ( x,y\right ) \) by the above linearized (approximation), the system can now be written as \[\begin{pmatrix} x^{\prime }\\ y^{\prime }\end{pmatrix} =\begin{pmatrix} \frac{\partial f}{\partial x} & \frac{\partial f}{\partial y}\\ \frac{\partial g}{\partial x} & \frac{\partial g}{\partial x}\end{pmatrix} _{\substack{x=x_{0}\\y=y_{0}}}\begin{pmatrix} x\\ y \end{pmatrix} \] Where \(\begin{pmatrix} \frac{\partial f}{\partial x} & \frac{\partial f}{\partial y}\\ \frac{\partial g}{\partial x} & \frac{\partial g}{\partial x}\end{pmatrix} \) is called the the Jacobian \(J\) matrix. Hence the system now can be written as\[ \vec{x}^{\prime }=\left [ J\right ] \vec{x}\] Now \(J\) is determined. From\begin{align*} \frac{\partial f}{\partial x} & =\frac{\partial }{\partial x}\left ( 60x-3x^{2}-4xy\right ) =60-6x-4y\\ \frac{\partial f}{\partial y} & =\frac{\partial }{\partial y}\left ( 60x-3x^{2}-4xy\right ) =-4x\\ \frac{\partial g}{\partial x} & =\frac{\partial }{\partial x}\left ( 42y-3y^{2}-2xy\right ) =-2y\\ \frac{\partial g}{\partial y} & =\frac{\partial }{\partial y}\left ( 42y-3y^{2}-2xy\right ) =42-6y-2x \end{align*}
The Jacobian matrix becomes \[ J=\begin{pmatrix} 60-6x-4y & -4x\\ -2y & 42-6y-2x \end{pmatrix} _{x=x_{0},y=y_{0}}\] And the linearized system is\[\begin{pmatrix} x^{\prime }\\ y^{\prime }\end{pmatrix} =\begin{pmatrix} 60-6x-4y & -4x\\ -2y & 42-6y-2x \end{pmatrix} _{x=x_{0},y=y_{0}}\begin{pmatrix} x\\ y \end{pmatrix} \] Now each equilibrium point is examined using the above linearized system to determine the type of stability a that point.
case \(x_{0}=0,y_{0}=0\)\[ J=\begin{pmatrix} 60-6\left ( 0\right ) -4\left ( 0\right ) & -4\left ( 0\right ) \\ -2\left ( 0\right ) & 42-6\left ( 0\right ) -2\left ( 0\right ) \end{pmatrix} =\begin{pmatrix} 60 & 0\\ 0 & 42 \end{pmatrix} \] Hence the linearized system at this specific equilibrium point is\[\begin{pmatrix} x^{\prime }\\ y^{\prime }\end{pmatrix} =\begin{pmatrix} 60 & 0\\ 0 & 42 \end{pmatrix}\begin{pmatrix} x\\ y \end{pmatrix} \] Since \(J\) is a diagonal matrix, its eigenvalues are the values on the diagonal. Therefore \(\lambda _{1}=60,\lambda _{2}=42\). Since the eigenvalues are positive, then this equilibrium point is not stable.
case \(x_{0}=0,y_{0}=14\)\[ J=\begin{pmatrix} 60-6\left ( 0\right ) -4\left ( 14\right ) & -4\left ( 0\right ) \\ -2\left ( 14\right ) & 42-6\left ( 14\right ) -2\left ( 0\right ) \end{pmatrix} =\begin{pmatrix} 4 & 0\\ -28 & -42 \end{pmatrix} \] Therefore linearized system at this specific equilibrium point is\[\begin{pmatrix} x^{\prime }\\ y^{\prime }\end{pmatrix} =\begin{pmatrix} 4 & 0\\ -28 & -42 \end{pmatrix}\begin{pmatrix} x\\ y \end{pmatrix} \] The eigenvalues can be found by solving \(\begin{vmatrix} 4-\lambda & 0\\ -28 & -42-\lambda \end{vmatrix} =0\) to be \(\lambda _{1}=4,\lambda _{2}=-42\). Because one of the eigenvalues is positive, then this equilibrium point is not stable.
case \(x_{0}=12,y_{0}=6\)\[ J=\begin{pmatrix} 60-6\left ( 12\right ) -4\left ( 6\right ) & -4\left ( 12\right ) \\ -2\left ( 6\right ) & 42-6\left ( 6\right ) -2\left ( 12\right ) \end{pmatrix} =\begin{pmatrix} -36 & -48\\ -12 & -18 \end{pmatrix} \] The linearized system at this specific equilibrium point is\[\begin{pmatrix} x^{\prime }\\ y^{\prime }\end{pmatrix} =\begin{pmatrix} -36 & -48\\ -12 & -18 \end{pmatrix}\begin{pmatrix} x\\ y \end{pmatrix} \] The eigenvalues can be found to be \(\lambda _{1}=-52.632,\lambda _{2}=\) \(-1.368\). Since both eigenvalues are now negative, then this equilibrium point is stable.
case \(x_{0}=20,y_{0}=0\)\[ J=\begin{pmatrix} 60-6\left ( 20\right ) -4\left ( 0\right ) & -4\left ( 20\right ) \\ -2\left ( 0\right ) & 42-6\left ( 0\right ) -2\left ( 20\right ) \end{pmatrix} =\begin{pmatrix} -60 & -80\\ 0 & 2 \end{pmatrix} \] The linearized system at this specific equilibrium point is\[\begin{pmatrix} x^{\prime }\\ y^{\prime }\end{pmatrix} =\begin{pmatrix} -60 & -80\\ 0 & 2 \end{pmatrix}\begin{pmatrix} x\\ y \end{pmatrix} \] The eigenvalues are \(\lambda _{1}=2,\lambda _{2}=\) \(-60\). Since one of the eigenvalues is positive, then this equilibrium point is not stable.
Summary of results obtained so far
| equilibrium point | eigenvalues | type of stability |
| \(x=0,y=0\) | \(\lambda _{1}=60,\lambda _{2}=42\) | not stable (nodal source) |
| \(x=0,y=14\) | \(\lambda _{1}=4,\lambda _{2}=-42\) | not stable (Saddle point) |
| \(x=12,y=6\) | \(\lambda _{1}=-52.632,\lambda _{2}=\) \(-1.368\) | stable (Nodal sink) |
| \(x=20,y=0\) | \(\lambda _{1}=2,\lambda _{2}=\) \(-60\) | not stable (Saddle point) |
To verify the above result, the phase plot for the original nonlinear system was plotted on the computer and the equilibrium points locations highlighted. The plot below agrees with the above result when looking at direction of arrows around each point. We see that the direction field arrows are all moving toward the stable point from any location near it. The stable equilibrium point was colored as green while the unstable ones colored in red.
Interpretation of results Since the solution of the linearized system can be written as linear combination of solutions made up of terms that look like \(c_{i}e^{\lambda _{i}t}\) where \(c_{i}\) are constants of integration and \(\lambda _{i}\) are the eigenvalues found above, then this implies when the real part of the eigenvalue is positive the solution will increase with time, moving away from the equilibrium point. Similarly, if the eigenvalue has a negative real part, it means it is a stable solution because solution will decay with time when perturbed slightly from the equilibrium.
Since this is second order system, there are two eigenvalues. Even if one eigenvalue is stable (i.e. negative), if the other eigenvalue is positive, then the system is unstable since one part of the solution will keep growing with time.
In terms of the dynamics of species, it means if the populations \(x=12\) and population \(y=6\), (this is the stable equilibrium) then these population will remain the same in long term even when one population becomes a little more or less than the other population. But for all other equilibrium populations sizes, such as \(x=20,y=0\), then if the population \(y\) were to change slightly to become say \(y=1\) (may be by external influence) then this will cause both population to start changing, moving it away from \(x=20,y=0\) as time increases, hence \(x=20,y=0\) is not stable population size.
This seems to be sensitive to the parameters \(a_{i},b_{i},c_{i}\) given in the problem. It is not easy to give a more physical reasoning as why some population values is stable while other are not, other than to also note that all the unstable ones had at least one population at zero.
Solution
The ODE can be written as\begin{align*} y^{\prime \prime }-5y^{\prime }+6y & =4e^{t}\left ( U\left ( t\right ) -U\left ( t-2\right ) \right ) \\ & =4\left ( e^{t}U\left ( t\right ) -e^{t}U\left ( t-2\right ) \right ) \end{align*}
Where \(U\left ( t\right ) \) is the unit step function. In the following solutions, these Laplace transform relations obtained from table are used
\begin{align*} U\left ( t\right ) & \Leftrightarrow \frac{1}{s}\\ U\left ( t-\tau \right ) & \Leftrightarrow \frac{1}{s}e^{-\tau s}\\ e^{-\alpha t}U\left ( t\right ) & \Leftrightarrow \frac{1}{s+\alpha }\\ \sin \left ( \omega t\right ) & \Leftrightarrow \frac{\omega }{s^{2}+\omega ^{2}}\\ \cos \left ( \omega t\right ) & \Leftrightarrow \frac{s}{s^{2}+\omega ^{2}} \end{align*}
Assuming \(\mathcal{L}\left [ y\left ( t\right ) \right ] =Y\left ( s\right ) \), and using the above relations of Laplace transform we find\begin{align*} \mathcal{L}\left [ e^{t}U\left ( t\right ) \right ] & =\frac{1}{s-1}\\\mathcal{L}\left [ e^{t}U\left ( t-2\right ) \right ] & =\frac{e^{-2\left ( s-1\right ) }}{s-1} \end{align*}
Now, taking the Laplace transform of the ODE results in\[ \left ( s^{2}Y\left ( s\right ) -sy\left ( 0\right ) -y^{\prime }\left ( 0\right ) \right ) -5\left ( sY\left ( s\right ) -y\left ( 0\right ) \right ) +6Y\left ( s\right ) =4\left ( \frac{1}{s-1}-\frac{e^{-2\left ( s-1\right ) }}{s-1}\right ) \] Using \(y\left ( 0\right ) =1,y^{\prime }\left ( 0\right ) =-2\) the above simplifies to\begin{align*} \left ( s^{2}Y\left ( s\right ) -s+2\right ) -5\left ( sY\left ( s\right ) -1\right ) +6Y\left ( s\right ) & =\frac{4}{s-1}-\frac{4e^{-2\left ( s-1\right ) }}{s-1}\\ s^{2}Y\left ( s\right ) -s+2-5sY\left ( s\right ) +5+6Y\left ( s\right ) & =\frac{4}{s-1}-\frac{4e^{-2\left ( s-1\right ) }}{s-1}\\ Y(s)\left ( s^{2}-5s+6\right ) -s+7 & =\frac{4}{s-1}-\frac{4e^{-2\left ( s-1\right ) }}{s-1}\\ Y\left ( s\right ) \left ( s^{2}-5s+6\right ) & =\frac{4}{s-1}-\frac{4e^{-2\left ( s-1\right ) }}{s-1}+\left ( s-7\right ) \end{align*}
But \(\left ( s^{2}-5s+6\right ) =\left ( s-3\right ) \left ( s-2\right ) \) and the above becomes\begin{equation} Y\left ( s\right ) =\frac{4}{\left ( s-1\right ) \left ( s-3\right ) \left ( s-2\right ) }-\frac{4e^{2}e^{-2s}}{\left ( s-1\right ) \left ( s-3\right ) \left ( s-2\right ) }+\frac{\left ( s-7\right ) }{\left ( s-3\right ) \left ( s-2\right ) }\tag{1} \end{equation} These are now simplified by partial fractions. The final result is only shown for brevity, since the process of performing partial fraction is a standard one. \[ \frac{1}{\left ( s-1\right ) \left ( s-3\right ) \left ( s-2\right ) }=\frac{1}{2\left ( s-3\right ) }-\frac{1}{s-2}+\frac{1}{2\left ( s-1\right ) }\] And\[ \frac{s-7}{\left ( s-3\right ) \left ( s-2\right ) }=\frac{-4}{s-3}+\frac{5}{s-2}\] Using the above result back in (1) results in\begin{align} Y\left ( s\right ) & =\frac{2}{s-3}-\frac{4}{s-2}+\frac{2}{s-1}-e^{2}e^{-2s}\left ( \frac{2}{\left ( s-3\right ) }-\frac{4}{s-2}+\frac{2}{\left ( s-1\right ) }\right ) -\frac{4}{s-3}+\frac{5}{s-2}\nonumber \\ & =\frac{-2}{s-3}+\frac{1}{s-2}+\frac{2}{s-1}-e^{2}\left ( \frac{2e^{-2s}}{\left ( s-3\right ) }-\frac{4e^{-2s}}{s-2}+\frac{2e^{-2s}}{\left ( s-1\right ) }\right ) \tag{2} \end{align}
Now we apply the inverse Laplace transform. lookup table is also used for this purpose to obtain\begin{align*} -2\mathcal{L}^{-1}\left ( \frac{1}{s-3}\right ) & =-2e^{3t}\\\mathcal{L}^{-1}\left ( \frac{1}{s-2}\right ) & =e^{2t}\\ 2\mathcal{L}^{-1}\left ( \frac{1}{s-1}\right ) & =2e^{t} \end{align*}
And\begin{align*} 2\mathcal{L}^{-1}\left ( \frac{e^{-2s}}{s-3}\right ) & =2e^{3\left ( t-2\right ) }U\left ( t-2\right ) \\ 4\mathcal{L}^{-1}\left ( \frac{e^{-2s}}{s-2}\right ) & =4e^{2\left ( t-2\right ) }U\left ( t-2\right ) \\ 2\mathcal{L}^{-1}\left ( \frac{e^{-2s}}{s-1}\right ) & =2e^{\left ( t-2\right ) }U\left ( t-2\right ) \end{align*}
Putting all these results back into (2) gives the response in time domain as\[ y\left ( t\right ) =-2e^{3t}+e^{2t}+2e^{t}-e^{2}\left ( 2e^{3\left ( t-2\right ) }-4e^{2\left ( t-2\right ) }+2e^{\left ( t-2\right ) }\right ) U\left ( t-2\right ) \] The above can also be written as\[ y\left ( t\right ) =\left \{ \begin{array} [c]{cc}-2e^{3t}+e^{2t}+2e^{t} & 0<t<2\\ -2e^{3t}+e^{2t}+2e^{t}-e^{2}\left ( 2e^{3t}-4e^{2t}+2e^{t}\right ) & t\geq 2 \end{array} \right . \] Since the original ODE is not stable (due to damping term \(-5\) negative in the given ODE, the solution will blow up with time). This is seen by the solution above, where the exponential are all positive, hence growing with time. The following is a plot of the above solution for up to \(\,t=2.2\)
Solution
Let \(Y_{1}\left ( s\right ) =\mathcal{L}\left [ y_{1}\left ( t\right ) \right ] \) and let \(Y_{2}\left ( s\right ) =\mathcal{L}\left [ y_{2}\left ( t\right ) \right ] \). Taking Laplace transform of the two ODE’s gives\begin{align*} s^{2}Y_{1}\left ( s\right ) -sy_{1}\left ( 0\right ) -y_{1}^{\prime }\left ( 0\right ) +5Y_{1}\left ( s\right ) +Y_{2}\left ( s\right ) & =0\\ s^{2}Y_{2}\left ( s\right ) -sy_{2}\left ( 0\right ) -y_{2}^{\prime }\left ( 0\right ) -2Y_{1}\left ( s\right ) +2Y_{2}\left ( s\right ) & =0 \end{align*}
Substituting the given initial conditions results in\begin{align} s^{2}Y_{1}-3s+5Y_{1}+Y_{2} & =0\tag{1}\\ s^{2}Y_{2}-s-2Y_{1}+2Y_{2} & =0\tag{2} \end{align}
The above two ODE’s are now solved for \(Y_{1}\left ( s\right ) ,Y_{2}\left ( s\right ) \)\begin{align*} Y_{1}\left ( s^{2}+5\right ) +Y_{2} & =3s\\ Y_{2}\left ( s^{2}+2\right ) -2Y_{1} & =s \end{align*}
or\[\begin{pmatrix} s^{2}+5 & 1\\ -2 & s^{2}+2 \end{pmatrix}\begin{pmatrix} Y_{1}\\ Y_{2}\end{pmatrix} =\begin{pmatrix} 3s\\ s \end{pmatrix} \] Using Gaussian elimination: Adding \(\left ( \frac{2}{s^{2}+5}\right ) \) times first row to second row gives\begin{align*} \begin{pmatrix} s^{2}+5 & 1\\ 0 & s^{2}+2+\frac{2}{s^{2}+5}\end{pmatrix}\begin{pmatrix} Y_{1}\\ Y_{2}\end{pmatrix} & =\begin{pmatrix} 3s\\ s+\frac{6s}{s^{2}+5}\end{pmatrix} \\\begin{pmatrix} s^{2}+5 & 1\\ 0 & \frac{1}{s^{2}+5}\left ( s^{4}+7s^{2}+12\right ) \end{pmatrix}\begin{pmatrix} Y_{1}\\ Y_{2}\end{pmatrix} & =\begin{pmatrix} 3s\\ \frac{s}{s^{2}+5}\left ( s^{2}+11\right ) \end{pmatrix} \end{align*}
Back substitution: From last row \begin{align} Y_{2}\left ( s\right ) & =\frac{\frac{s}{s^{2}+5}\left ( s^{2}+11\right ) }{\frac{1}{s^{2}+5}\left ( s^{4}+7s^{2}+12\right ) }\nonumber \\ & =\frac{s\left ( s^{2}+11\right ) }{s^{4}+7s^{2}+12}\nonumber \\ & =\frac{s\left ( s^{2}+11\right ) }{\left ( s^{2}+4\right ) \left ( s^{2}+3\right ) }\tag{3} \end{align}
First row gives\[ \left ( s^{2}+5\right ) Y_{1}+Y_{2}=3s \] Using \(Y_{2}\left ( s\right ) \) found from (3), the above becomes\begin{align} \left ( s^{2}+5\right ) Y_{1} & =3s-\frac{s\left ( s^{2}+11\right ) }{\left ( s^{2}+4\right ) \left ( s^{2}+3\right ) }\nonumber \\ Y_{1}\left ( s\right ) & =\frac{3s}{\left ( s^{2}+5\right ) }-\frac{s\left ( s^{2}+11\right ) }{\left ( s^{2}+5\right ) \left ( s^{2}+4\right ) \left ( s^{2}+3\right ) }\tag{4} \end{align}
To obtain the time domain solution we need to inverse Laplace transform (3,4). Starting with (3), and applying partial fractions gives\begin{equation} Y_{2}\left ( s\right ) =\frac{s\left ( s^{2}+11\right ) }{\left ( s^{2}+4\right ) \left ( s^{2}+3\right ) }=\frac{8s}{3+s^{2}}-\frac{7s}{4+s^{2}}\tag{5} \end{equation} From tables we see that \begin{align*} 8\mathcal{L}^{-1}\left [ \frac{s}{3+s^{2}}\right ] & =8\cos \left ( \sqrt{3}t\right ) \\ 7\mathcal{L}^{-1}\left [ \frac{s}{4+s^{2}}\right ] & =7\cos \left ( 2t\right ) \end{align*}
Hence (5) becomes in time domain as\[ y_{2}\left ( t\right ) =8\cos \left ( \sqrt{3}t\right ) -7\cos \left ( 2t\right ) \] Similarly for \(Y_{1}\left ( s\right ) \), from (4) and applying partial fractions\begin{align} Y_{1}\left ( s\right ) & =\frac{3s}{\left ( s^{2}+5\right ) }-\frac{s\left ( s^{2}+11\right ) }{\left ( s^{2}+5\right ) \left ( s^{2}+4\right ) \left ( s^{2}+3\right ) }\nonumber \\ & =\frac{3s}{\left ( s^{2}+5\right ) }-\left ( \frac{4s}{s^{2}+3}-\frac{7s}{s^{2}+4}+\frac{3s}{5+s^{2}}\right ) \tag{6} \end{align}
From inverse Laplace transform table\begin{align*} 3\mathcal{L}^{-1}\left [ \frac{s}{\left ( s^{2}+5\right ) }\right ] & =3\cos \left ( \sqrt{5}t\right ) \\ 4\mathcal{L}^{-1}\left [ \frac{s}{s^{2}+3}\right ] & =4\cos \left ( \sqrt{3}t\right ) \\ 7\mathcal{L}^{-1}\left [ \frac{s}{s^{2}+4}\right ] & =7\cos \left ( 2t\right ) \\ 3\mathcal{L}^{-1}\left [ \frac{s}{5+s^{2}}\right ] & =3\cos \left ( \sqrt{5}t\right ) \end{align*}
Using these in (6), the solution \(y_{1}\left ( t\right ) \) becomes\begin{align*} y_{1}\left ( t\right ) & =3\cos \left ( \sqrt{5}t\right ) -\left ( 4\cos \left ( \sqrt{3}t\right ) -7\cos \left ( 2t\right ) +3\cos \left ( \sqrt{5}t\right ) \right ) \\ & =-4\cos \left ( \sqrt{3}t\right ) +7\cos \left ( 2t\right ) \end{align*}
In summary
\begin{align*} y_{1}\left ( t\right ) & =-4\cos \left ( \sqrt{3}t\right ) +7\cos \left ( 2t\right ) \\ y_{2}\left ( t\right ) & =8\cos \left ( \sqrt{3}t\right ) -7\cos \left ( 2t\right ) \end{align*}
The following is a plot of the solutions for 10 seconds.
Solution
Note: In all the following it is assumed that \(x\) is a vector and that \(x\neq 0\)
Let\begin{equation} A=A_{sy}+A_{sk}\tag{1} \end{equation} Where \(A_{sy}\) is a symmetric matrix, which means \(A_{sy}^{T}=A_{sy}\) and \(A_{sk}\) is skew symmetric matrix which means \(A_{sk}^{T}=-A_{sk}\). Taking the transpose of (1) gives\begin{align} A^{T} & =\left ( A_{sy}+A_{sk}\right ) ^{T}\nonumber \\ & =A_{sy}^{T}+A_{sk}^{T}\nonumber \\ & =A_{sy}-A_{sk}\tag{2} \end{align}
Adding (1)+(2) gives\begin{align} A+A^{T} & =2A_{sy}\nonumber \\ A_{sy} & =\frac{A+A^{T}}{2}\tag{3} \end{align}
Subtracting (2)-(1) gives\begin{align} A^{T}-A & =-2A_{sk}\nonumber \\ A_{sk} & =\frac{A-A^{T}}{2}\tag{4} \end{align}
Therefore for any \(A\), \begin{align} A_{sy} & =\frac{1}{2}\left ( A+A^{T}\right ) \tag{4A}\\ A_{sk} & =\frac{1}{2}\left ( A-A^{T}\right ) \tag{4B} \end{align}
To show that \(A_{sy}\) is indeed symmetric, this is done by construction :\begin{align*} A_{sy}^{T} & =\frac{1}{2}\left ( A+A^{T}\right ) ^{T}\\ & =\frac{1}{2}\left ( A^{T}+\left ( A^{T}\right ) ^{T}\right ) \end{align*}
But \(\left ( A^{T}\right ) ^{T}=A\), and the above becomes\begin{align*} A_{sy}^{T} & =\frac{1}{2}\left ( A^{T}+A\right ) \\ & =A_{sy} \end{align*}
Therefore \(A_{sy}\) is indeed symmetric.
To show that \(A_{sk}\) is skew symmetric matrix :\begin{align*} A_{sk}^{T} & =\frac{1}{2}\left ( A-A^{T}\right ) ^{T}\\ & =\frac{1}{2}\left ( A^{T}-\left ( A^{T}\right ) ^{T}\right ) \\ & =\frac{1}{2}\left ( A^{T}-A\right ) \\ & =-\frac{1}{2}\left ( A-A^{T}\right ) \\ & =-A_{sk} \end{align*}
Hence \(A_{sk}\) is indeed skew symmetric.
Therefore any \(A\) matrix can be written as \(A=A_{sy}+A_{sk}\) where \(A_{sy},A_{sk}\) are given by (4A,4B).
Now we need to show that this is a unique was to write \(A\). Proof is by contradictions. Let there be \(\tilde{A}_{sy}\) matrix which is symmetric and \(\tilde{A}_{sy}\neq A_{sy}\) and let there be \(\tilde{A}_{sy}\) matrix which is skew matrix and \(\tilde{A}_{sk}\neq A_{sk}\). And also let \(A=\tilde{A}_{sy}+\tilde{A}_{sy}\) in addition to \(A=A_{sy}+A_{sk}\). Then\begin{align*} A^{T} & =\left ( \tilde{A}_{sy}+\tilde{A}_{sk}\right ) ^{T}\\ & =\tilde{A}_{sy}^{T}+\tilde{A}_{sk}^{T} \end{align*}
Since \(\tilde{A}_{sy}\) is assumed to be symmetric, then \(\tilde{A}_{sy}^{T}=\tilde{A}_{sy}\) and since \(\tilde{A}_{sk}\) is assumed to be skew symmetric, then \(\tilde{A}_{sk}^{T}=-\tilde{A}_{sk}\) and the above becomes\[ A^{T}=\tilde{A}_{sy}-\tilde{A}_{sk}\] Therefore \begin{align*} \frac{1}{2}\left ( A+A^{T}\right ) & =\frac{1}{2}\left ( \tilde{A}_{sy}+\tilde{A}_{sk}+\tilde{A}_{sy}-\tilde{A}_{sk}\right ) \\ & =\tilde{A}_{sy} \end{align*}
But from (4A) above, we showed that \(\frac{1}{2}\left ( A+A^{T}\right ) =A_{sy}\). Hence \[ A_{sy}=\tilde{A}_{sy}\] Which is a contradiction to our assumption that \(\tilde{A}_{sy}\neq A_{sy}\). Therefore \(A_{sy}\) is unique. The same is done for \(\tilde{A}_{sk}\). From\begin{align*} \frac{1}{2}\left ( A-A^{T}\right ) & =\frac{1}{2}\left ( \tilde{A}_{sy}+\tilde{A}_{sk}-\left ( \tilde{A}_{sy}-\tilde{A}_{sk}\right ) \right ) \\ & =\tilde{A}_{sk} \end{align*}
But from (4) above, we showed that \(\frac{1}{2}\left ( A-A^{T}\right ) =A_{sk}\). Hence \(A_{sk}=\tilde{A}_{sk}\) which is a contradiction. Therefore there is only way to write \(A\) as sum of symmetric and skew symmetric way, which is \[ A=\overset{A_{sy}}{\overbrace{\frac{A+A^{T}}{2}}}+\overset{A_{sk}}{\overbrace{\frac{1}{2}\left ( A-A^{T}\right ) }}\] QED.
Starting with the forward direction. We need to show that given \(A\) is positive definite (p.d.) then this implies \(A_{sy}\) is also p.d.
From part (a) we found that \(A\) can be written as \(A=A_{sy}+A_{sk}\). Since \(A\) is now assumed to be p.d. then this implies\begin{align} x^{T}Ax & >0\nonumber \\ x^{T}\left ( A_{sy}+A_{sk}\right ) x & >0\nonumber \\ x^{T}A_{sy}x+x^{T}A_{sk}x & >0\tag{1} \end{align}
Now we will show that \(x^{T}A_{sk}x=0\) to finish the above proof. First we observe that \begin{align*} \left ( x^{T}A_{sk}x\right ) ^{T} & =\left ( A_{sk}x\right ) ^{T}x\\ & =x^{T}A_{sk}^{T}x \end{align*}
But \(A_{sk}=-A_{sk}^{T}\) by definition of skew symmetric matrix. Therefore the above becomes\[ \left ( x^{T}A_{sk}x\right ) ^{T}=-\left ( x^{T}A_{sk}x\right ) \] But \(x^{T}A_{sk}x\) is a single number, say \(q\). (To be precise, \(q\) is \(1\times 1\) matrix. but since it is \(1\times 1\) we can treat it as a number, since it is one element). But the transpose of a number (or \(1\times 1\) matrix) is itself. Hence the above relation says that \[ q^{T}=-q \] For a number, this is the same as saying \(q=-q\) and this only possible if \(q=0\) or in other words\begin{equation} x^{T}A_{sk}x=0\tag{2} \end{equation} Using (2) in (1) shows immediately that\[ x^{T}A_{sy}x>0 \] Therefore \(A_{sy}\) is positive definite.
Now we need to show the reverse direction. That is, we need to show that if \(A_{sy}\) is p.d. then this implies \(A\) is also p.d.
Since \(A_{sy}\) is now assumed to be p.d. then we can write\[ x^{T}A_{sy}x>0 \] But \(A=A_{sy}+A_{sk}\) therefore \(A_{sy}=A-A_{sk}\) and the above becomes\begin{align*} x^{T}\left ( A-A_{sk}\right ) x & >0\\ x^{T}Ax-x^{T}A_{sk}x & >0 \end{align*}
But we showed in (2) that \(x^{T}A_{sk}x=0\). Therefore the above becomes\[ x^{T}Ax>0 \] Which implies that \(A\) is positive definite, which is what we are asked to show. QED
Solution
Two vectors \(\vec{v},\vec{u}\) are orthogonal if their dot product is zero. This is because \(\vec{v}\cdot \vec{u}=\left \Vert \vec{v}\right \Vert \left \Vert \vec{u}\right \Vert \cos \theta \) where \(\theta \) is the inner angle between the two vectors. Since the vectors are orthogonal, then \(\cos 90^{0}=0\) and therefore \(\vec{v}\cdot \vec{u}=0\). To find which pairs are orthogonal to each others, we compute the inner product between all possible pairs :\begin{align*} \vec{u}_{1}\cdot \vec{u}_{2} & =-2+0+0+2=0\\ \vec{u}_{1}\cdot \vec{u}_{3} & =4+6+0+4=14\\ \vec{u}_{1}\cdot \vec{u}_{4} & =-12+12+0+0=0\\ \vec{u}_{2}\cdot \vec{u}_{3} & =-2+0+3+8=9\\ \vec{u}_{2}\cdot \vec{u}_{4} & =6+0-6+0=0\\ \vec{u}_{3}\cdot \vec{u}_{4} & =-12+8-2+0=-2 \end{align*}
We see from the above that \(\vec{u}_{1}\cdot \vec{u}_{2}=0,\vec{u}_{1}\cdot \vec{u}_{4}=0,\vec{u}_{2}\cdot \vec{u}_{4}=0\). Therefore\begin{align*} S & =\left \{ \vec{u}_{1},\vec{u}_{2},\vec{u}_{4}\right \} \\ & \equiv \left \{ \vec{v}_{1},\vec{v}_{2},\vec{v}_{3}\right \} \end{align*}
Or\begin{align} \vec{v}_{1} & =\left ( 2,3,0,1\right ) \tag{1B}\\ \vec{v}_{2} & =\left ( -1,0,3,2\right ) \nonumber \\ \vec{v}_{3} & =\left ( -6,4,-2,0\right ) \nonumber \end{align}
Form an orthogonal set in \(\mathbb{R} ^{4}\).
Now we need to find the best approximation of \(\vec{w}=\vec{u}_{3}=\left ( 2,2,1,4\right ) \) using the above orthogonal vectors \(\vec{v}_{1},\vec{v}_{2},\vec{v}_{3}\). Using Gram’s theorem, this approximation is \begin{equation} \vec{w}=c_{1}\vec{v}_{1}+c_{2}\vec{v}_{2}+c_{3}\vec{v}_{3}\tag{1A} \end{equation} Where the constants \(c_{i}\) are found from solving the system\[\begin{pmatrix} \vec{v}_{1}\cdot \vec{v}_{1} & \vec{v}_{1}\cdot \vec{v}_{2} & \vec{v}_{1}\cdot \vec{v}_{3}\\ \vec{v}_{2}\cdot \vec{v}_{1} & \vec{v}_{2}\cdot \vec{v}_{2} & \vec{v}_{2}\cdot \vec{v}_{3}\\ \vec{v}_{3}\cdot \vec{v}_{1} & \vec{v}_{3}\cdot \vec{v}_{2} & \vec{v}_{3}\cdot \vec{v}_{3}\end{pmatrix}\begin{pmatrix} c_{1}\\ c_{2}\\ c_{3}\end{pmatrix} =\begin{pmatrix} \vec{v}_{1}\cdot \vec{w}\\ \vec{v}_{2}\cdot \vec{w}\\ \vec{v}_{3}\cdot \vec{w}\end{pmatrix} \] But since \(\vec{v}_{i}\) are all orthogonal to each others then \(\vec{v}_{i}\cdot \vec{v}_{j}=0\) for \(i\neq j\), and \(\vec{v}_{i}\cdot \vec{v}_{i}=\left \Vert v_{i}\right \Vert ^{2}\) and the above becomes\begin{equation} \begin{pmatrix} \left \Vert \vec{v}_{1}\right \Vert ^{2} & 0 & 0\\ 0 & \left \Vert \vec{v}_{2}\right \Vert ^{2} & 0\\ 0 & 0 & \left \Vert \vec{v}_{3}\right \Vert ^{2}\end{pmatrix}\begin{pmatrix} c_{1}\\ c_{2}\\ c_{3}\end{pmatrix} =\begin{pmatrix} \vec{v}_{1}\cdot \vec{w}\\ \vec{v}_{2}\cdot \vec{w}\\ \vec{v}_{3}\cdot \vec{w}\end{pmatrix} \tag{1} \end{equation} But \begin{align*} \vec{v}_{1}\cdot \vec{w} & =\left ( 2,3,0,1\right ) \cdot \left ( 2,2,1,4\right ) =4+6+4=14\\ \vec{v}_{2}\cdot \vec{w} & =\left ( -1,0,3,2\right ) \cdot \left ( 2,2,1,4\right ) =-2+3+8=9\\ \vec{v}_{3}\cdot \vec{w} & =\left ( -6,4,-2,0\right ) \cdot \left ( 2,2,1,4\right ) =-12+8-2=-6 \end{align*}
Hence (1) becomes\begin{equation} \begin{pmatrix} \left \Vert \vec{v}_{1}\right \Vert ^{2} & 0 & 0\\ 0 & \left \Vert \vec{v}_{2}\right \Vert ^{2} & 0\\ 0 & 0 & \left \Vert \vec{v}_{3}\right \Vert ^{2}\end{pmatrix}\begin{pmatrix} c_{1}\\ c_{2}\\ c_{3}\end{pmatrix} =\begin{pmatrix} 14\\ 9\\ -6 \end{pmatrix} \tag{2} \end{equation} Since \begin{align*} \left \Vert \vec{v}_{1}\right \Vert ^{2} & =\left \Vert \left ( 2,3,0,1\right ) \right \Vert ^{2}=4+9+1=14\\ \left \Vert \vec{v}_{2}\right \Vert ^{2} & =\left \Vert \left ( -1,0,3,2\right ) \right \Vert ^{2}=1+9+4=14\\ \left \Vert \vec{v}_{3}\right \Vert ^{2} & =\left \Vert \left ( -6,4,-2,0\right ) \right \Vert ^{2}=36+16+4=56 \end{align*}
Then (2) becomes\begin{equation} \begin{pmatrix} 14 & 0 & 0\\ 0 & 14 & 0\\ 0 & 0 & 56 \end{pmatrix}\begin{pmatrix} c_{1}\\ c_{2}\\ c_{3}\end{pmatrix} =\begin{pmatrix} 14\\ 9\\ -6 \end{pmatrix} \tag{3} \end{equation} From the above we see that \begin{align*} c_{1} & =1\\ c_{2} & =\frac{9}{14}\\ c_{3} & =\frac{-3}{28} \end{align*}
Hence the best approximation using (1A) becomes\begin{align*} \vec{w} & =c_{1}\vec{v}_{1}+c_{2}\vec{v}_{2}+c_{3}\vec{v}_{3}\\ & =\left ( 2,3,0,1\right ) +\frac{9}{14}\left ( -1,0,3,2\right ) -\frac{3}{28}\left ( -6,4,-2,0\right ) \\ & =\left ( 2,\frac{18}{7},\frac{15}{7},\frac{16}{7}\right ) \end{align*}
Therefore \[ \vec{w}=\frac{1}{7}\left ( 14,18,15,16\right ) \] Now we need to find basis \(\vec{v}_{1},\vec{v}_{2},\vec{v}_{3},\vec{v}_{4}\) of orthogonal vectors in \(\mathbb{R} ^{4}\). We already found that from (1B) that \(\vec{v}_{1},\vec{v}_{2},\vec{v}_{3}\) are three such vectors. So we just need to find another \(\vec{v}_{4}=\left [ a_{1},a_{2},a_{3},a_{4}\right ] \) such that it is orthogonal to the other three, in other words we need to solve \begin{align*} \vec{v}_{1}\cdot \vec{v}_{4} & =0\\ \vec{v}_{2}\cdot \vec{v}_{4} & =0\\ \vec{v}_{3}\cdot \vec{v}_{4} & =0 \end{align*}
This implies\begin{align*} \left [ 2,3,0,1\right ] \cdot \left [ a_{1},a_{2},a_{3},a_{4}\right ] & =0\\ \left [ -1,0,3,2\right ] \cdot \left [ a_{1},a_{2},a_{3},a_{4}\right ] & =0\\ \left [ -6,4,-2,0\right ] \cdot \left [ a_{1},a_{2},a_{3},a_{4}\right ] & =0 \end{align*}
Or\begin{align*} 2a_{1}+3a_{2}+a_{4} & =0\\ -a_{1}+3a_{3}+2a_{4} & =0\\ -6a_{1}+4a_{2}-2a_{3} & =0 \end{align*}
Or\[\begin{pmatrix} 2 & 3 & 0 & 1\\ -1 & 0 & 3 & 2\\ -6 & 4 & -2 & 0 \end{pmatrix}\begin{pmatrix} a_{1}\\ a_{2}\\ a_{3}\\ a_{4}\end{pmatrix} =\begin{pmatrix} 0\\ 0\\ 0 \end{pmatrix} \] This system has three equations and 4 unknowns. Therefore it will have one free parameter giving an infinite number of solutions. Using Gaussian elimination:\begin{align*} & \begin{pmatrix} 2 & 3 & 0 & 1\\ -1 & 0 & 3 & 2\\ -6 & 4 & -2 & 0 \end{pmatrix} \overset{R_{2}=R_{2}+\frac{1}{2}R_{1}}{\rightarrow }\begin{pmatrix} 2 & 3 & 0 & 1\\ 0 & \frac{3}{2} & 3 & \frac{5}{2}\\ -6 & 4 & -2 & 0 \end{pmatrix} \overset{R_{3}=R_{3}+3R_{1}}{\rightarrow }\\ & \begin{pmatrix} 2 & 3 & 0 & 1\\ 0 & \frac{3}{2} & 3 & \frac{5}{2}\\ 0 & 13 & -2 & 3 \end{pmatrix} \overset{R_{3}=R_{3}-\frac{26}{3}R_{2}}{\rightarrow }\begin{pmatrix} 2 & 3 & 0 & 1\\ 0 & \frac{3}{2} & 3 & \frac{5}{2}\\ 0 & 0 & -28 & -\frac{56}{3}\end{pmatrix} \end{align*}
We stop the elimination here since no more elimination is possible. We have now this system\[\begin{pmatrix} 2 & 3 & 0 & 1\\ 0 & \frac{3}{2} & 3 & \frac{5}{2}\\ 0 & 0 & -28 & -\frac{56}{3}\end{pmatrix}\begin{pmatrix} a_{1}\\ a_{2}\\ a_{3}\\ a_{4}\end{pmatrix} =\begin{pmatrix} 0\\ 0\\ 0 \end{pmatrix} \] Back substitution: From last row we obtain the equation\begin{align*} -28a_{3}-\frac{56}{3}a_{4} & =0\\ a_{4} & =-\frac{3}{2}a_{3} \end{align*}
The second row gives\begin{align*} \frac{3}{2}a_{2}+3a_{3}+\frac{5}{2}a_{4} & =0\\ a_{2} & =\frac{2}{3}\left ( -3a_{3}-\frac{5}{2}a_{4}\right ) \\ & =-2a_{3}-\frac{5}{3}a_{4} \end{align*}
Since \(a_{4}=-\frac{3}{2}a_{3}\) the above becomes\begin{align*} a_{2} & =-2a_{3}-\frac{5}{3}\left ( -\frac{3}{2}a_{3}\right ) \\ & =\frac{1}{2}a_{3} \end{align*}
First row gives\begin{align*} 2a_{1}+3a_{2}+a_{4} & =0\\ a_{1} & =\frac{1}{2}\left ( -3a_{2}-a_{4}\right ) \end{align*}
Since \(a_{2}=\frac{1}{2}a_{3}\) and \(a_{4}=-\frac{3}{2}a_{3}\) the above becomes\begin{align*} a_{1} & =\frac{1}{2}\left ( -3\left ( \frac{1}{2}a_{3}\right ) -\left ( -\frac{3}{2}a_{3}\right ) \right ) \\ & =0 \end{align*}
Therefore the solution is\[\begin{pmatrix} a_{1}\\ a_{2}\\ a_{3}\\ a_{4}\end{pmatrix} =\begin{pmatrix} 0\\ \frac{1}{2}\\ 1\\ -\frac{3}{2}\end{pmatrix} a_{3}\] The above means that for any arbitrary \(a_{3}\) value there is a solution. \(a_{3}\) is just a scalar which only stretches or shrinks the vector but does not change its direction (orientation). Therefore the vector remains orthogonal to all others for any \(a_{3}\). Let us pick \(a_{3}=1\). Using this \(\vec{v}_{4}\) becomes\[ \vec{v}_{4}=\left [ 0,\frac{1}{2},1,-\frac{3}{2}\right ] \] To verify the result found, we will check that \(\vec{v}_{4}\) is indeed orthogonal with the other three vectors :\begin{align*} \vec{v}_{1}\cdot \vec{v}_{4} & =\left [ 2,3,0,1\right ] \cdot \left [ 0,\frac{1}{2},1,-\frac{3}{2}\right ] =0\\ \vec{v}_{2}\cdot \vec{v}_{4} & =\left [ -1,0,3,2\right ] \cdot \left [ 0,\frac{1}{2},1,-\frac{3}{2}\right ] =0\\ \vec{v}_{3}\cdot \vec{v}_{4} & =\left [ -6,4,-2,0\right ] \cdot \left [ 0,\frac{1}{2},1,-\frac{3}{2}\right ] =0 \end{align*}
QED.
Solution
Let \[ A_{n\times n}=\begin{pmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n}\\ a_{21} & a_{22} & a_{23} & \cdots & a_{2n}\\ a_{31} & a_{32} & a_{33} & \cdots & a_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & a_{n3} & \cdots & a_{nn}\end{pmatrix} ,b=\begin{pmatrix} b_{1}\\ b_{2}\\ b_{3}\\ \vdots \\ b_{n}\end{pmatrix} \]
The first step in Gaussian elimination is to reduce the above matrix to row echelon form :\[ A_{n\times n}=\begin{pmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n}\\ 0 & a_{22} & a_{23} & \cdots & a_{2n}\\ 0 & 0 & a_{33} & \cdots & a_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & a_{nn}\end{pmatrix}\begin{pmatrix} b_{1}\\ \tilde{b}_{2}\\ \tilde{b}_{3}\\ \vdots \\ \tilde{b}_{n}\end{pmatrix} \] Row echelon form has zeros in its the lower left triangle. After this, back substitution starts by solving for \(x_{n}\) from the last row, then solving for \(x_{n-1}\) from the row above the last row and so on until we reach the first row.
Counting operations for forward pass
The first step is to zero out all entries in first column below \(a_{11}\) using \(a_{11}\) as pivot. Next is to zero out all entries in second column below the (updated) \(a_{22}\) value and so on.
To zero out an entry, for example \(a_{21}\), we first need to do one division \(\frac{a_{21}}{a_{11}}=\Delta \) and store this in memory, then do \(a_{2i}=a_{2i}-\Delta a_{1i}\) for all entries in that row, which means for \(i=1\cdots n\). (no need to count \(a_{21}\) since we know it will be zero). We have to remember that this is being applied to the \(b\) vector as well and not just for \(A\) matrix rows.
Hence we need one division to find \(\Delta \), and then \(2n\) multiplication and addition/subtraction operations per row. The division is only needed once per row to find the pivot scaling \(\Delta \).
Since there are \(n-1\) rows then there are \(\left ( n-1\right ) \) divisions and \(\left ( 2n\right ) \left ( n-1\right ) \) multiplications/addition to zero out the first column. After this we have the following system reached\[ A_{n\times n}=\begin{pmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n}\\ 0 & a_{22} & a_{23} & \cdots & a_{2n}\\ 0 & a_{32} & a_{33} & \cdots & a_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & a_{n2} & a_{n3} & \cdots & a_{nn}\end{pmatrix}\begin{pmatrix} b_{1}\\ \tilde{b}_{2}\\ \tilde{b}_{3}\\ \vdots \\ \tilde{b}_{n}\end{pmatrix} \] The total cost now is therefore \(\left ( n-1\right ) +\left ( 2n\right ) \left ( n-1\right ) \).
We now switch to the second row and use the new value of \(a_{22}\) as pivot and repeat the same as above. The only difference now is that there are \(n-2\) rows to process and \(\left ( n-2\right ) \) divisions and therefore \(2\left ( n-1\right ) \left ( n-2\right ) \) multiplications/addition to zero out the second column entries below the second row. After this we reach the following system\[ A_{n\times n}=\begin{pmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n}\\ 0 & a_{22} & a_{23} & \cdots & a_{2n}\\ 0 & 0 & a_{33} & \cdots & a_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & a_{n3} & \cdots & a_{nn}\end{pmatrix}\begin{pmatrix} b_{1}\\ \tilde{b}_{2}\\ \tilde{b}_{3}\\ \vdots \\ \tilde{b}_{n}\end{pmatrix} \] The total cost of the above is therefore \(\left ( n-2\right ) +2\left ( n-1\right ) \left ( n-2\right ) \).
We now switch to the third row and use the new value of \(a_{33}\) as pivot. Now there are now \(\left ( n-3\right ) \) divisions and \(2\left ( n-2\right ) \left ( n-3\right ) \) multiplications/additions to obtain the following system\[ A_{n\times n}=\begin{pmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n}\\ 0 & a_{22} & a_{23} & \cdots & a_{2n}\\ 0 & 0 & a_{33} & \cdots & a_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & a_{nn}\end{pmatrix}\begin{pmatrix} b_{1}\\ \tilde{b}_{2}\\ \tilde{b}_{3}\\ \vdots \\ \tilde{b}_{n}\end{pmatrix} \] The total cost of the above is therefore \(\left ( n-3\right ) +2\left ( n-2\right ) \left ( n-3\right ) .\)
And so on until we reach the row before the last row, where there is only one row below it to process. The cost then is just one division and \(2\) additions and \(2\) multiplications. Therefore the number of total number of multiplication and additions operations for the forward pass is the sum of all the above operations, which can be written as\[ \overset{\text{row 2}}{\overbrace{\left [ \left ( n-1\right ) +\left ( 2n\right ) \left ( n-1\right ) \right ] }}+\overset{\text{row 3}}{\overbrace{\left [ \left ( n-2\right ) +2\left ( n-1\right ) \left ( n-2\right ) \right ] }}+\cdots +\overset{\text{last row}}{\overbrace{\left [ 1+4\right ] }}\] Writing the above as \({\displaystyle \sum \limits _{k=1}^{n-1}} \left ( n-k\right ) +2\left ( n-k+1\right ) \left ( n-k\right ) \) then we need to calculate this sum using known formulas for summations. Let this sum be \(\Delta \), hence\begin{align} \Delta & ={\displaystyle \sum \limits _{k=1}^{n-1}} -3k+2k^{2}+3n-4kn+2n^{2}\nonumber \\ & =-3{\displaystyle \sum \limits _{k=1}^{n-1}} k+2{\displaystyle \sum \limits _{k=1}^{n-1}} k^{2}+3{\displaystyle \sum \limits _{k=1}^{n-1}} n-4{\displaystyle \sum \limits _{k=1}^{n-1}} kn+2{\displaystyle \sum \limits _{k=1}^{n-1}} n^{2}\nonumber \\ & =-3\left ( \frac{n\left ( n-1\right ) }{2}\right ) +2\left ( \frac{n^{3}}{3}-\frac{n^{2}}{2}+\frac{n}{6}\right ) +3\left ( n^{2}-n\right ) -4\left ( \frac{n^{2}\left ( n-1\right ) }{2}\right ) +2\left ( n^{2}\left ( n-1\right ) \right ) \nonumber \\ & =\frac{2}{3}n^{3}+\frac{1}{2}n^{2}-\frac{7}{6}n\tag{1} \end{align}
The above is the number of operations just for for the forward pass (elimination phase).
For example for matrix of size \(3\times 3\) the above gives \(19\) operations, and for matrix of size \(4\times 4\), it gives \(46\) operations and for \(5\times 5\) it gives \(90\) operations and so on.
Counting operations for backward pass In back substitution, we start from the end of the elimination phase above, which will be\[\begin{pmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n}\\ 0 & a_{22} & a_{23} & \cdots & a_{2n}\\ 0 & 0 & a_{33} & \cdots & a_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & a_{nn}\end{pmatrix}\begin{pmatrix} x_{1}\\ x_{2}\\ x_{3}\\ \vdots \\ x_{n}\end{pmatrix} =\begin{pmatrix} b_{1}\\ b_{2}\\ b_{3}\\ \vdots \\ b_{n}\end{pmatrix} \] First step is to solve for \(x_{n}\) by finding \(x_{n}=\frac{b_{n}}{a_{nn}}\). This requires only one division. Next is to solve for \(x_{n-1}\) by finding \(a_{n-1,n-1}x_{n-1}+a_{n-1,n}x_{n}=b_{n-1}\), or\[ x_{n-1}=\frac{b_{n-1}-\left ( a_{n-1,n}\right ) x_{n}}{\left ( a_{n-1,n-1}\right ) }\] We see that this needs one subtraction, one multiplication and one division, or \(3\) operations. The next step is to solve for \(x_{n-2}\) from\[ \left ( a_{n-2,n-2}\right ) x_{n-2}+\left ( a_{n-2,n-1}\right ) x_{n-1}+\left ( a_{n-2,n}\right ) x_{n}=b_{2}\] Hence\[ x_{n-2}=\frac{b_{2}-\left ( a_{n-2,n-1}\right ) x_{n-1}-\left ( a_{n-2,n}\right ) x_{n}}{\left ( a_{n-2,n-2}\right ) }\] Therefore we need \(2\) subtractions, \(2\) multiplication and one division, or \(5\) operations. And so on until we reach the first row to solve for \(x_{1}\). Therefore the total number of operations can be seen as \[ 1+3+5+7+\cdots \] The above can be written as the sum\begin{align} \sum _{k=0}^{n-1}\left ( 2k+1\right ) & =2\sum _{k=0}^{n-1}k+\sum _{k=0}^{n-1}1\nonumber \\ & =2\left ( \frac{n\left ( n-1\right ) }{2}\right ) +n\nonumber \\ & =n\left ( n-1\right ) +n\nonumber \\ & =n^{2}-n+n\nonumber \\ & =n^{2}\tag{2} \end{align}
We see that the cost of the elimination is much greater than the cost of back substitution. One is \(O\left ( n^{3}\right ) \) while the other is \(O\left ( n^{2}\right ) \).
From (1,2), the total number of operations for the complete Gaussian elimination process is\begin{align*} \Delta & =\frac{2}{3}n^{3}+\frac{1}{2}n^{2}-\frac{7}{6}n+n^{2}\\ & =\frac{2}{3}n^{3}+\frac{3}{2}n^{2}-\frac{7}{6}n \end{align*}
For large \(n\) the above is \(O\left ( n^{3}\right ) \).
Given a system of equations \[\begin{pmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n}\\ a_{21} & a_{22} & a_{23} & \cdots & a_{2n}\\ a_{31} & a_{32} & a_{33} & \cdots & a_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & a_{n3} & \cdots & a_{nn}\end{pmatrix}\begin{pmatrix} x_{1}\\ x_{2}\\ x_{3}\\ \vdots \\ x_{n}\end{pmatrix} =\begin{pmatrix} b_{1}\\ b_{2}\\ b_{3}\\ \vdots \\ b_{n}\end{pmatrix} \] Cramer method works as follows :\begin{align*} x_{1} & =\frac{\left \vert A\right \vert }{\left \vert A_{1}\right \vert }\\ x_{2} & =\frac{\left \vert A\right \vert }{\left \vert A_{2}\right \vert }\\ & \vdots \\ x_{n} & =\frac{\left \vert A\right \vert }{\left \vert A_{n}\right \vert } \end{align*}
Where \(\left \vert A\right \vert \) is the determinant of coefficient matrix \(A_{n\times n}\) and \(\left \vert A_{i}\right \vert \) is determinant of coefficient matrix but with the \(i^{th}\) column replaced by the column vector \(b\).
An efficient way to find the determinant is to convert the matrix to row echelon form. In this form, the matrix is upper triangle. Hence the determinant is the product of all elements along the diagonal. This is more efficient than using the matrix cofactor expansion method.
In doing these row operations on the matrix to find \(\left \vert A\right \vert \) the only difference from the elimination steps we did for part(a), is that we have to remember the following rules now during the elimination process
Given the above, let us assume that for each elimination step of a row, we do one multiplication to account for a possible multiplication by \(-1\) or possible multiplication by a scalar. Since we do not know if this will happen every time as this clearly depends on the data in the matrix, then this will be the worst case counting.
This means there is an additional \(\left ( n-1\right ) \) multiplications to add to the cost of doing the elimination step to reach row echelon form at the end.
Another small difference from part(a), is that now we do not have the \(b\) vector added during the forward step.
Therefore, as we did in part(a), the cost to reach this form\[\begin{pmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n}\\ 0 & a_{22} & a_{23} & \cdots & a_{2n}\\ 0 & a_{32} & a_{33} & \cdots & a_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & a_{n2} & a_{n3} & \cdots & a_{nn}\end{pmatrix} \] Is now \(\left ( n-1\right ) +\left ( 2\left ( n-1\right ) \right ) \left ( n-1\right ) \). Recalling from part(a) the cost at this stage was \(\left ( n-1\right ) +\left ( 2n\right ) \left ( n-1\right ) \) here. So we changed \(2n\) to \(2\left ( n-1\right ) \), since there is no \(b\) vector, hence one less element. And as was done in part (a), the cost to reach\[\begin{pmatrix} a_{11} & a_{12} & a_{13} & \cdots & a_{1n}\\ 0 & a_{22} & a_{23} & \cdots & a_{2n}\\ 0 & 0 & a_{33} & \cdots & a_{3n}\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & a_{n3} & \cdots & a_{nn}\end{pmatrix} \] Now becomes \(\left ( n-2\right ) +2\left ( n-2\right ) \left ( n-2\right ) \). Recalling from part(a) the cost at this stage was \(\left ( n-1\right ) +2\left ( n-1\right ) \left ( n-1\right ) \). So we changed \(2\left ( n-1\right ) \) to \(2\left ( n-2\right ) \), since there is no \(b\) vector. This continues to the row before the last as in part (a). Therefore the number of total multiplication and additions operations for just the forward pass is\[ \overset{\text{row 2}}{\overbrace{\left [ \left ( n-1\right ) +\left ( 2\left ( n-1\right ) \right ) \left ( n-1\right ) \right ] }}+\overset{\text{row 3}}{\overbrace{\left [ \left ( n-2\right ) +2\left ( n-2\right ) \left ( n-2\right ) \right ] }}+\cdots +\overset{\text{last row}}{\overbrace{\left [ 1+2\right ] }}\] Hence the cost to put the matrix in row echelon form is \begin{align}{\displaystyle \sum \limits _{k=1}^{n-1}} \left ( n-k\right ) +2\left ( n-k\right ) ^{2} & ={\displaystyle \sum \limits _{k=1}^{n-1}} n-{\displaystyle \sum \limits _{k=1}^{n-1}} k+2{\displaystyle \sum \limits _{k=1}^{n-1}} \left ( n-k\right ) ^{2}\nonumber \\ & =n\left ( n-1\right ) -\frac{n\left ( n-1\right ) }{2}+2\frac{n-3n^{2}+2n^{3}}{6}\nonumber \\ & =\frac{2}{3}n^{3}-\frac{1}{2}n^{2}-\frac{1}{6}n\tag{1} \end{align}
The above cost is very close to part(a) elimination phase as expected which was \(\frac{2}{3}n^{3}+\frac{1}{2}n^{2}-\frac{7}{6}n\). Current cost is smaller because in part(a) we had the \(b\) vector there which added more operations, while here we just operated on \(A\) itself.
Let us now add the \(\left ( n-1\right ) \) multiplication we mentioned earlier to the result above. The cost now becomes\begin{align*} \Delta & =\frac{2}{3}n^{3}-\frac{1}{2}n^{2}-\frac{1}{6}n+\left ( n-1\right ) \\ & =\frac{2}{3}n^{3}-\frac{1}{2}n^{2}+\frac{5}{6}n-1 \end{align*}
We still need to calculate the product of the diagonal elements to find the determinant. For \(n\times n\) matrix, this takes \(n-1\) multiplications. Adding these to the above gives\begin{align*} \Delta & =\frac{2}{3}n^{3}-\frac{1}{2}n^{2}+\frac{5}{6}n-1+\left ( n-1\right ) \\ & =\frac{2}{3}n^{3}-\frac{1}{2}n^{2}+\frac{11}{6}n-2\\ & \approx \frac{2}{3}n^{3}-\frac{1}{2}n^{2}+\frac{11}{6}n \end{align*}
We will use the above as the cost of finding the determinant.
How many times do we need to find determinants? We need to do it one time to find \(\left \vert A\right \vert \) and then \(n\) more time for each \(\left \vert A_{i}\right \vert \). Hence \(\left ( n+1\right ) \) times. This is the main reason why Cramer method becomes much more costly compared to Gaussian elimination.
The number of operations now becomes\begin{align*} \Delta & =\left ( \frac{2}{3}n^{3}-\frac{1}{2}n^{2}+\frac{11}{6}n\right ) \left ( n+1\right ) \\ & =\frac{2}{3}n^{4}+\frac{1}{6}n^{3}+\frac{4}{3}n^{2}+\frac{11}{6}n \end{align*}
We also need to add the cost of the final divisions \(\frac{\left \vert A\right \vert }{\left \vert A_{i}\right \vert }\) to find each \(x_{i}\). So we add \(n\) divisions to the above, giving the final cost as\begin{align*} \Delta & =\frac{2}{3}n^{4}+\frac{1}{6}n^{3}+\frac{4}{3}n^{2}+\frac{11}{6}n+n\\ & =\frac{2}{3}n^{4}+\frac{1}{6}n^{3}+\frac{4}{3}n^{2}+\frac{17}{6}n \end{align*}
We see from above, that Cramer rule for large \(n\) is \(O\left ( n^{4}\right ) \) while Gaussian elimination was \(O\left ( n^{3}\right ) \). Hence Gaussian elimination is much more efficient for large \(n\).
In summary
| \(n\) | cost of Gaussian elimination \(\frac{2}{3}n^{3}+\frac{3}{2}n^{2}-\frac{7}{6}n\) | cost of Cramer \(\frac{2}{3}n^{4}+\frac{1}{6}n^{3}+\frac{4}{3}n^{2}+\frac{17}{6}n\) |
| \(2\) | \(5\) | \(23\) |
| \(3\) | \(19\) | \(79\) |
| \(4\) | \(46\) | \(214\) |
| \(5\) | \(90\) | \(485\) |
| \(6\) | \(155\) | \(965\) |
| \(7\) | \(245\) | \(1743\) |
| \(8\) | \(364\) | \(2924\) |
| \(9\) | \(516\) | \(4629\) |
| \(10\) | \(705\) | \(6995\) |
The following is a graphical illustration of the above
Solution
The eigenvalues of \(A\) are found by solving \(\left \vert A-\lambda I\right \vert =0\) or\begin{align*} \begin{vmatrix} \frac{1}{\sqrt{5}}-\lambda & 0 & \frac{2}{\sqrt{5}}\\ \frac{-2}{\sqrt{5}} & -\lambda & \frac{1}{\sqrt{5}}\\ 0 & -1 & -\lambda \end{vmatrix} & =0\\ \left ( \frac{1}{\sqrt{5}}-\lambda \right ) \begin{vmatrix} -\lambda & \frac{1}{\sqrt{5}}\\ -1 & -\lambda \end{vmatrix} -0+\frac{2}{\sqrt{5}}\begin{vmatrix} \frac{-2}{\sqrt{5}} & -\lambda \\ 0 & -1 \end{vmatrix} & =0\\ \left ( \frac{1}{\sqrt{5}}-\lambda \right ) \left ( \lambda ^{2}+\frac{1}{\sqrt{5}}\right ) +\frac{2}{\sqrt{5}}\left ( \frac{2}{\sqrt{5}}\right ) & =0\\ \frac{1}{5}\left ( \lambda -1\right ) \left ( -5\lambda -5\lambda ^{2}+\sqrt{5}\lambda -5\right ) & =0\\ \frac{1}{5}\left ( \lambda -1\right ) \left ( -5\lambda ^{2}+\lambda \left ( \sqrt{5}-5\right ) -5\right ) & =0\\ \left ( \lambda -1\right ) \left ( -5\lambda ^{2}+\lambda \left ( \sqrt{5}-5\right ) -5\right ) & =0 \end{align*}
Hence \(\lambda =1\). The quadratic formula is used to solve \(-5\lambda ^{2}+\lambda \left ( \sqrt{5}-5\right ) -5=0\). First it is normalized\begin{align*} 5\lambda ^{2}-\lambda \left ( \sqrt{5}-5\right ) +5 & =0\\ \lambda ^{2}-\lambda \left ( \frac{1}{5}\sqrt{5}-1\right ) +1 & =0 \end{align*}
Then \(\lambda =\frac{-b}{2a}\pm \frac{1}{2a}\sqrt{b^{2}-4ac}\) where \(b=-\left ( \frac{1}{5}\sqrt{5}-1\right ) ,c=1,a=1\) and the roots are\begin{align*} \lambda & =\frac{\left ( \frac{1}{5}\sqrt{5}-1\right ) }{2}\pm \frac{1}{2}\sqrt{\left ( -\left ( \frac{1}{5}\sqrt{5}-1\right ) \right ) ^{2}-4}\\ & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) \pm \frac{1}{2}\sqrt{\left ( \frac{1}{5}\sqrt{5}-1\right ) ^{2}-4}\\ & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) \pm \frac{1}{2}\sqrt{1+\frac{5}{25}-\frac{2}{5}\sqrt{5}-4}\\ & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) \pm \frac{1}{2}\sqrt{1+\frac{1}{5}-\frac{2}{5}\sqrt{5}-4}\\ & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) \pm \frac{1}{2}\sqrt{-\frac{14}{5}-\frac{2\sqrt{5}}{5}}\\ & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) \pm \frac{1}{2}\sqrt{\frac{-14-2\sqrt{5}}{5}}\\ & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) \pm \frac{1}{2}i\sqrt{\frac{14+2\sqrt{5}}{5}}\\ & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) \pm \frac{1}{2}i\sqrt{\frac{5\left ( 14+2\sqrt{5}\right ) }{\left ( 5\right ) 5}}\\ & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) \pm \frac{1}{\left ( 2\right ) \left ( 5\right ) }i\sqrt{5\left ( 14+2\sqrt{5}\right ) }\\ & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) \pm \frac{1}{10}i\sqrt{10\sqrt{5}+70} \end{align*}
Therefore the roots are\begin{align*} \lambda _{1} & =1\\ \lambda _{2} & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) +\frac{1}{10}i\sqrt{10\sqrt{5}+70}\\ \lambda _{3} & =\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) -\frac{1}{10}i\sqrt{10\sqrt{5}+70} \end{align*}
Numerically the above becomes\begin{align*} \lambda _{1} & =1\\ \lambda _{2} & =-0.276+0.961i\\ \lambda _{3} & =-0.276\,-0.961i \end{align*}
The following plot shows the locations on the complex plane
To show analytically that the eigenvalues lie on the unit circle means to show that the magnitude of each complex number is \(1\). Clearly \(\lambda _{1}\) already satisfy this condition. We need to check now that \(\left \Vert \lambda _{2}\right \Vert =1\) and that \(\left \Vert \lambda _{3}\right \Vert =1\)\begin{align*} \left \Vert \lambda _{2}\right \Vert & =\sqrt{\operatorname{Re}\left ( \lambda _{2}\right ) ^{2}+\operatorname{Im}\left ( \lambda _{2}\right ) ^{2}}\\ & =\sqrt{\left ( \frac{\sqrt{5}}{10}-\frac{1}{2}\right ) ^{2}+\left ( \frac{1}{10}\sqrt{10\sqrt{5}+70}\right ) ^{2}}\\ & =\sqrt{\left ( \frac{3}{10}-\frac{1}{10}\sqrt{5}\right ) +\left ( \frac{1}{10}\sqrt{5}+\frac{7}{10}\right ) }\\ & =\sqrt{\frac{10}{10}}\\ & =1 \end{align*}
Similarly for \(\lambda _{3}\) since it is the same except for the sign on the complex part (complex conjugate) which does not affect the norm. Therefore all the eigenvalues lie on unit circle in \(\mathbb{C} \). QED.
Let two vectors in the domain of \(A\) be \(\mathbf{x}=\begin{pmatrix} x_{1}\\ x_{2}\\ x_{3}\end{pmatrix} \) and \(\mathbf{y}=\begin{pmatrix} y_{1}\\ y_{2}\\ y_{3}\end{pmatrix} \). And let the two vector after the mapping, which now lie in the range of \(A\) be \(\mathbf{\tilde{x}}=\begin{pmatrix} \tilde{x}_{1}\\ \tilde{x}_{2}\\ \tilde{x}_{3}\end{pmatrix} \) and \(\mathbf{\tilde{y}}=\begin{pmatrix} \tilde{y}_{1}\\ \tilde{y}_{2}\\ \tilde{y}_{3}\end{pmatrix} \). Since \(\mathbf{x\cdot y}=\left \Vert \mathbf{x}\right \Vert \left \Vert \mathbf{y}\right \Vert \cos \theta \) where \(\theta \) is the inner angle between the vectors, and since \(\mathbf{\tilde{x}\cdot \tilde{y}}=\left \Vert \mathbf{\tilde{x}}\right \Vert \left \Vert \mathbf{\tilde{y}}\right \Vert \cos \tilde{\theta }\), then we need to show that \[ \theta =\tilde{\theta }\]
Let \begin{align*} A\mathbf{x} & =\mathbf{\tilde{x}}\\ A\mathbf{y} & =\mathbf{\tilde{y}} \end{align*}
Using the \(A\) given, then\[\begin{pmatrix} \frac{1}{\sqrt{5}} & 0 & \frac{2}{\sqrt{5}}\\ \frac{-2}{\sqrt{5}} & 0 & \frac{1}{\sqrt{5}}\\ 0 & -1 & 0 \end{pmatrix}\begin{pmatrix} x_{1}\\ x_{2}\\ x_{3}\end{pmatrix} =\begin{pmatrix} \frac{1}{\sqrt{5}}x_{1}+\frac{2}{\sqrt{5}}x_{3}\\ \frac{-2}{\sqrt{5}}x_{1}+\frac{1}{\sqrt{5}}x_{3}\\ -x_{2}\end{pmatrix} \] Hence \[ \mathbf{\tilde{x}}=\begin{pmatrix} \frac{1}{\sqrt{5}}x_{1}+\frac{2}{\sqrt{5}}x_{3}\\ \frac{-2}{\sqrt{5}}x_{1}+\frac{1}{\sqrt{5}}x_{3}\\ -x_{2}\end{pmatrix} \] We see from the above that \begin{align*} \tilde{x}_{1} & =\frac{1}{\sqrt{5}}x_{1}+\frac{2}{\sqrt{5}}x_{3}\\ \tilde{x}_{2} & =\frac{-2}{\sqrt{5}}x_{1}+\frac{1}{\sqrt{5}}x_{3}\\ \tilde{x}_{3} & =-x_{2} \end{align*}
Similarly\[ \mathbf{\tilde{y}}=\begin{pmatrix} \frac{1}{\sqrt{5}}y_{1}+\frac{2}{\sqrt{5}}y_{3}\\ \frac{1}{\sqrt{5}}y_{1}-\frac{2}{\sqrt{5}}y_{3}\\ -y_{2}\end{pmatrix} \] We see from the above that \begin{align*} \tilde{y}_{1} & =\frac{1}{\sqrt{5}}y_{1}+\frac{2}{\sqrt{5}}y_{3}\\ \tilde{y}_{2} & =\frac{-2}{\sqrt{5}}y_{1}+\frac{1}{\sqrt{5}}y_{3}\\ \tilde{y}_{3} & =-y_{2} \end{align*}
We now need to determine \(\theta \) and \(\tilde{\theta }\) and show they are the same. From the definition above\[ \theta =\arccos \left ( \frac{\mathbf{x\cdot y}}{\left \Vert \mathbf{x}\right \Vert \left \Vert \mathbf{y}\right \Vert }\right ) \] But \(\mathbf{x\cdot y}=x_{1}y_{1}+x_{2}y_{2}+x_{3}y_{3}\) and \(\left \Vert \mathbf{x}\right \Vert =\sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}\) and \(\left \Vert \mathbf{y}\right \Vert =\sqrt{y_{1}^{2}+y_{2}^{2}+y_{3}^{2}}\), therefore the above becomes\begin{equation} \theta =\arccos \left ( \frac{x_{1}y_{1}+x_{2}y_{2}+x_{3}y_{3}}{\sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}\sqrt{y_{1}^{2}+y_{2}^{2}+y_{3}^{2}}}\right ) \tag{1} \end{equation} Similarly, \(\mathbf{\tilde{x}\cdot \tilde{y}}=\tilde{x}_{1}\tilde{y}_{1}+\tilde{x}_{2}\tilde{y}_{2}+\tilde{x}_{3}\tilde{y}_{3}\). Using the values of \(\tilde{x}_{i},\tilde{y}_{i}\) found above the dot product becomes\begin{align*} \mathbf{\tilde{x}\cdot \tilde{y}} & =\left ( \frac{1}{\sqrt{5}}x_{1}+\frac{2}{\sqrt{5}}x_{3}\right ) \left ( \frac{1}{\sqrt{5}}y_{1}+\frac{2}{\sqrt{5}}y_{3}\right ) +\left ( \frac{-2}{\sqrt{5}}x_{1}+\frac{1}{\sqrt{5}}x_{3}\right ) \left ( \frac{-2}{\sqrt{5}}y_{1}+\frac{1}{\sqrt{5}}y_{3}\right ) +\left ( -x_{2}\right ) \left ( -y_{2}\right ) \\ & =\frac{1}{5}\left ( x_{1}+2x_{3}\right ) \left ( y_{1}+2y_{3}\right ) +\frac{1}{5}\left ( x_{3}-2x_{1}\right ) \left ( y_{3}-2y_{1}\right ) +x_{2}y_{2}\\ & =\frac{1}{5}x_{1}y_{1}+\frac{2}{5}x_{1}y_{3}+\frac{2}{5}x_{3}y_{1}+\frac{4}{5}x_{3}y_{3}+\frac{4}{5}x_{1}y_{1}-\frac{2}{5}x_{1}y_{3}-\frac{2}{5}x_{3}y_{1}+\frac{1}{5}x_{3}y_{3}+x_{2}y_{2} \end{align*}
Which simplifies to \[ \mathbf{\tilde{x}\cdot \tilde{y}}=x_{1}y_{1}+x_{2}y_{2}+x_{3}y_{3}\] And \(\left \Vert \mathbf{\tilde{x}}\right \Vert =\sqrt{\tilde{x}_{1}^{2}+\tilde{x}_{2}^{2}+\tilde{x}_{3}^{2}}\). Using the values of \(\tilde{x}_{i}\) found above, this becomes\begin{align*} \left \Vert \mathbf{\tilde{x}}\right \Vert & =\sqrt{\left ( \frac{1}{\sqrt{5}}x_{1}+\frac{2}{\sqrt{5}}x_{3}\right ) ^{2}+\left ( \frac{-2}{\sqrt{5}}x_{1}+\frac{1}{\sqrt{5}}x_{3}\right ) ^{2}+\left ( -x_{2}\right ) ^{2}}\\ & =\sqrt{\frac{1}{5}x_{1}^{2}+\frac{4}{5}x_{1}x_{3}+\frac{4}{5}x_{3}^{2}+\frac{4}{5}x_{1}^{2}-\frac{4}{5}x_{1}x_{3}+\frac{1}{5}x_{3}^{2}+x_{2}^{2}} \end{align*}
Which simplifies to \[ \left \Vert \mathbf{\tilde{x}}\right \Vert =\sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}\] Similarly, \(\left \Vert \mathbf{\tilde{y}}\right \Vert =\sqrt{\tilde{y}_{1}^{2}+\tilde{y}_{2}^{2}+\tilde{y}_{3}^{2}}\) and using the values of \(\tilde{y}_{i}\) found above, then this becomes\begin{align*} \left \Vert \mathbf{\tilde{y}}\right \Vert & =\sqrt{\left ( \frac{1}{\sqrt{5}}y_{1}+\frac{2}{\sqrt{5}}y_{3}\right ) ^{2}+\left ( \frac{-2}{\sqrt{5}}y_{1}+\frac{1}{\sqrt{5}}y_{3}\right ) ^{2}+\left ( -y_{2}\right ) ^{2}}\\ & =\sqrt{\frac{1}{5}y_{1}^{2}+\frac{4}{5}y_{1}y_{3}+\frac{4}{5}y_{3}^{2}+\frac{4}{5}y_{1}^{2}-\frac{4}{5}y_{1}y_{3}+\frac{1}{5}y_{3}^{2}+y_{2}^{2}} \end{align*}
Which simplifies to\[ \left \Vert \mathbf{\tilde{y}}\right \Vert =\sqrt{y_{1}^{2}+y_{2}^{2}+y_{3}^{2}}\] Therefore\begin{align} \cos \tilde{\theta } & =\frac{\mathbf{\tilde{x}\cdot \tilde{y}}}{\left \Vert \mathbf{\tilde{x}}\right \Vert \left \Vert \mathbf{\tilde{y}}\right \Vert }\nonumber \\ \tilde{\theta } & =\arccos \left ( \frac{\sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}}{\sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}\sqrt{y_{1}^{2}+y_{2}^{2}+y_{3}^{2}}}\right ) \tag{2} \end{align}
Comparing (1) and (2) shows they are the same. Therefore \(\theta =\tilde{\theta }\). QED.
Correction: The set \(S\) shown above should be
\[ S=\left \{ \frac{1}{2\pi },\frac{\cos x}{\pi },\frac{\sin x}{\pi },\frac{\cos 2x}{\pi },\frac{\sin 2x}{\pi },\cdots ,\frac{\cos 5x}{\pi },\frac{\sin x}{\pi }\right \} \] Solution
Two functions \(f,g\) are orthogonal on \(\left [ -\pi ,\pi \right ] \) if \(\int _{-\pi }^{\pi }fgdx=0\). To show this for the set of functions given, we pick \(f=\frac{1}{2\pi }\) and then for \(g\) we pick \(\frac{\cos mx}{\pi }\) and then \(\frac{\sin mx}{\pi }\). i.e. \begin{align*} I_{1} & =\int _{-\pi }^{\pi }\frac{1}{2\pi }\frac{\cos mx}{\pi }dx\\ I_{2} & =\int _{-\pi }^{\pi }\frac{1}{2\pi }\frac{\sin mx}{\pi }dx \end{align*}
For the rest, we have to determine the following 3 cases \begin{align*} I_{3} & =\int _{-\pi }^{\pi }\frac{\cos mx}{\pi }\frac{\cos nx}{\pi }dx\\ I_{4} & =\int _{-\pi }^{\pi }\frac{\cos mx}{\pi }\frac{\sin nx}{\pi }dx\\ I_{5} & =\int _{-\pi }^{\pi }\frac{\sin mx}{\pi }\frac{\sin nx}{\pi }dx \end{align*}
These will take care of all possible combination of any two function in the set \(S\). We could always replace \(m,n\) by a number from \(1\cdots 5\) after evaluating the integrals in order to obtain a specific case. Starting with \(I_{1}\)\[ I_{1}=\frac{1}{2\pi ^{2}}\int _{-\pi }^{\pi }\cos mxdx \] But \(\cos \) function has period \(2\pi \) and therefore the integral above is zero \(I_{1}=0\). This shows that \(f=\frac{1}{2\pi }\) is orthogonal with all \(\frac{\cos mx}{\pi }\) functions in the set. \[ I_{2}=\frac{1}{2\pi ^{2}}\int _{-\pi }^{\pi }\sin mxdx \] As above, \(\sin \) function has period of \(2\pi \) and therefore the integral above is zero \(I_{2}=0\). This shows that \(f=\frac{1}{2\pi }\) is orthogonal with all \(\frac{\sin mx}{\pi }\) functions in the set.\[ I_{3}=\frac{1}{\pi ^{2}}\int _{-\pi }^{\pi }\cos mx\cos nxdx \] From tables, using \(\cos A\cos B=\frac{1}{2}\left ( \cos \left ( A-B\right ) +\cos \left ( A+B\right ) \right ) \), then \[ \cos mx\cos nx=\frac{1}{2}\left ( \cos \left ( \left ( m-n\right ) x\right ) +\cos \left ( \left ( m+n\right ) x\right ) \right ) \] And \(I_{3}\) now becomes\begin{align*} I_{3} & =\frac{1}{2\pi ^{2}}\int _{-\pi }^{\pi }\cos \left ( \left ( m-n\right ) x\right ) +\cos \left ( \left ( m+n\right ) x\right ) dx\\ & =\frac{1}{2\pi ^{2}}\left ( \int _{-\pi }^{\pi }\cos \left ( m-n\right ) xdx+\int _{-\pi }^{\pi }\cos \left ( \left ( m+n\right ) x\right ) dx\right ) \end{align*}
Since the problem is asking us to show orthogonality of different functions in the set, then we assume \(m\neq n\), otherwise the integral will have to be handled as special case when \(m=n\) due to the division. \[ I_{3}=\frac{1}{2\pi ^{2}}\left ( \frac{1}{m-n}\left [ \sin \left ( m-n\right ) x\right ] _{-\pi }^{\pi }+\frac{1}{m+n}\left [ \sin \left ( m+n\right ) x\right ] _{-\pi }^{\pi }\right ) \] But since \(n,m\) are integers, then both terms above are zero since \(\sin \left ( N\pi \right ) =0\) for integer \(N\). Hence \(I_{3}=0\). This shows that\(\ \frac{\cos mx}{\pi }\) is orthogonal with \(\frac{\cos nx}{\pi }\) when \(m\neq n\).\[ I_{4}=\frac{1}{\pi ^{2}}\int _{-\pi }^{\pi }\sin nx\cos mxdx \] Using \(\sin A\cos B=\frac{1}{2}\left ( \sin \left ( A-B\right ) +\sin \left ( A+B\right ) \right ) \), the above becomes\begin{align*} I_{4} & =\frac{1}{2\pi ^{2}}\int _{-\pi }^{\pi }\left ( \sin \left ( n-m\right ) x+\sin \left ( n+m\right ) x\right ) dx\\ & =\frac{1}{2\pi ^{2}}\left ( \int _{-\pi }^{\pi }\sin \left ( n-m\right ) xdx+\int _{-\pi }^{\pi }\sin \left ( n+m\right ) xdx\right ) \end{align*}
Again, since \(n\neq m\), then the above becomes\begin{align*} I_{4} & =\frac{1}{2\pi ^{2}}\left ( \frac{-1}{n-m}\left [ \cos \left ( n-m\right ) x\right ] _{-\pi }^{\pi }+\frac{-1}{n+m}\left [ \cos \left ( m+n\right ) x\right ] _{-\pi }^{\pi }\right ) \\ & =\frac{1}{2\pi ^{2}}\left ( \frac{-1}{n-m}\left [ \cos \left ( \left ( n-m\right ) \pi \right ) -\cos \left ( \left ( n-m\right ) \left ( -\pi \right ) \right ) \right ] +\frac{-1}{n+m}\left [ \cos \left ( \left ( n+m\right ) \pi \right ) -\cos \left ( \left ( n+m\right ) \left ( -\pi \right ) \right ) \right ] \right ) \end{align*}
But \(\cos \left ( -x\right ) =\cos \left ( x\right ) \) and the above becomes\begin{align*} I_{4} & =\frac{1}{2\pi ^{2}}\left ( \frac{-1}{n-m}\left [ \cos \left ( \left ( n-m\right ) \pi \right ) -\cos \left ( \left ( n-m\right ) \pi \right ) \right ] +\frac{-1}{n+m}\left [ \cos \left ( \left ( n+m\right ) \pi \right ) -\cos \left ( \left ( n+m\right ) \pi \right ) \right ] \right ) \\ & =\frac{1}{2\pi ^{2}}\left ( \frac{-1}{n-m}\left [ 0\right ] +\frac{-1}{n+m}\left [ 0\right ] \right ) \\ & =0 \end{align*}
Hence \(I_{4}=0\). This shows that\(\ \frac{\sin mx}{\pi }\) is orthogonal with \(\frac{\sin nx}{\pi }\) when \(m\neq n\).
The final integral is\[ I_{5}=\frac{1}{\pi ^{2}}\int _{-\pi }^{\pi }\sin mx\sin nxdx \] Using \(\sin A\sin B=\frac{1}{2}\left ( \cos \left ( A-B\right ) -\cos \left ( A+B\right ) \right ) \) the above becomes\begin{align*} I_{5} & =\frac{1}{2\pi ^{2}}\int _{-\pi }^{\pi }\cos \left ( \left ( m-n\right ) x\right ) -\cos \left ( \left ( m+n\right ) x\right ) dx\\ & =\frac{1}{2\pi ^{2}}\left ( \int _{-\pi }^{\pi }\cos \left ( \left ( m-n\right ) x\right ) dx-\int _{-\pi }^{\pi }\cos \left ( \left ( m+n\right ) x\right ) dx\right ) \end{align*}
Case \(n=m\)\begin{align*} I_{5} & =\frac{1}{2\pi ^{2}}\left ( \int _{-\pi }^{\pi }dx-\int _{-\pi }^{\pi }dx\right ) \\ & =0 \end{align*}
This shows that \(\frac{\sin mx}{\pi }\) is orthogonal with all \(\frac{\cos nx}{\pi }\) when \(m=n\).
Case \(n\neq m\)\[ I_{5}=\frac{1}{2\pi ^{2}}\left ( \frac{1}{m-n}\left [ \sin \left ( \left ( m-n\right ) x\right ) \right ] _{-\pi }^{\pi }-\frac{1}{m+n}\left [ \sin \left ( \left ( m+n\right ) x\right ) \right ] _{-\pi }^{\pi }\right ) \] But since \(n,m\) are integers, then both terms above are zero since \(\sin \left ( N\pi \right ) =0\) for integer \(N\). Hence \(I_{5}=0\). This shows that \(\frac{\sin mx}{\pi }\)is orthogonal \(\frac{\cos nx}{\pi }\).
The above shows that all the functions in \(S\) are pairwise orthogonal.
To make the set \(S\) orthonormal, we need to find weight \(k\) such that \(\left \Vert kf\left ( x\right ) \right \Vert =1\) or for functions, this is the same as \[ \sqrt{\int _{-\pi }^{\pi }\left ( kf\left ( x\right ) \right ) ^{2}dx}=1 \] For \(f=\frac{1}{2\pi }\), this becomes \begin{align*} \sqrt{\int _{-\pi }^{\pi }\left ( k\frac{1}{2\pi }\right ) ^{2}dx} & =1\\ \frac{k}{2\pi }\sqrt{\int _{-\pi }^{\pi }dx} & =1\\ \frac{k}{2\pi }\sqrt{2\pi } & =1\\ k & =\sqrt{2\pi } \end{align*}
For \(f=\frac{\cos mx}{\pi }\)\begin{align*} \sqrt{\int _{-\pi }^{\pi }\left ( k\frac{\cos mx}{\pi }\right ) ^{2}dx} & =1\\ \frac{k}{\pi }\sqrt{\int _{-\pi }^{\pi }\cos ^{2}mxdx} & =1\\ \frac{k}{\pi }\sqrt{\int _{-\pi }^{\pi }\frac{1}{2}+\frac{1}{2}\cos 2mxdx} & =1\\ \frac{k}{\pi }\sqrt{\left ( \int _{-\pi }^{\pi }\frac{1}{2}dx+\frac{1}{2}\int _{-\pi }^{\pi }\cos 2mxdx\right ) } & =1\\ \frac{k}{\pi }\sqrt{\left ( \pi +\frac{1}{2}\overset{0}{\overbrace{\left [ \frac{\sin \left ( 2mx\right ) }{2m}\right ] _{-\pi }^{\pi }}}\right ) } & =1\\ \frac{k}{\pi }\sqrt{\pi } & =1\\ k & =\sqrt{\pi } \end{align*}
For \(f=\frac{\sin mx}{\pi }\)\begin{align*} \sqrt{\int _{-\pi }^{\pi }\left ( k\frac{\sin mx}{\pi }\right ) ^{2}dx} & =1\\ \frac{k}{\pi }\sqrt{\int _{-\pi }^{\pi }\sin ^{2}mxdx} & =1\\ \frac{k}{\pi }\sqrt{\int _{-\pi }^{\pi }\frac{1}{2}-\frac{1}{2}\cos 2mxdx} & =1\\ \frac{k}{\pi }\sqrt{\left ( \int _{-\pi }^{\pi }\frac{1}{2}dx-\frac{1}{2}\int _{-\pi }^{\pi }\cos 2mxdx\right ) } & =1\\ \frac{k}{\pi }\sqrt{\left ( \pi -\frac{1}{2}\overset{0}{\overbrace{\left [ \frac{\sin \left ( 2mx\right ) }{2m}\right ] _{-\pi }^{\pi }}}\right ) } & =1\\ \frac{k}{\pi }\sqrt{\pi } & =1\\ k & =\sqrt{\pi } \end{align*}
Therefore the orthonormal set now becomes, after using the weights found above as\begin{align*} \tilde{S} & =\left \{ \sqrt{2\pi }\frac{1}{2\pi },\sqrt{\pi }\frac{\cos x}{\pi },\sqrt{\pi }\frac{\sin x}{\pi },\sqrt{\pi }\frac{\cos 2x}{\pi },\sqrt{\pi }\frac{\sin 2x}{\pi },\cdots ,\sqrt{\pi }\frac{\cos 5x}{\pi },\sqrt{\pi }\frac{\sin x}{\pi }\right \} \\ & =\left \{ \frac{1}{\sqrt{2\pi }},\frac{\cos x}{\sqrt{\pi }},\frac{\sin x}{\sqrt{\pi }},\frac{\cos 2x}{\sqrt{\pi }},\frac{\sin 2x}{\sqrt{\pi }},\cdots ,\frac{\cos 5x}{\sqrt{\pi }},\frac{\sin x}{\sqrt{\pi }}\right \} \end{align*}
We now need to approximate \(H\left ( x\right ) =\pi -\left \vert x\right \vert \) using \(\tilde{S}\). The following is a plot of \(H(x)\) over \(\left [ -\pi ,\pi \right ] \)
Counting the number of functions in \(\tilde{S}\), there are \(11\) of them. Using Gram’s theorem, this approximation is \begin{equation} H\left ( x\right ) \approx c_{1}S_{1}+c_{2}S_{2}+c_{3}S_{3}+\cdots \tag{1} \end{equation} Where \(S_{1}=\frac{1}{\sqrt{2\pi }},S_{2}=\frac{\cos x}{\sqrt{\pi }},\cdots ,S_{10}=\frac{\cos 5x}{\sqrt{\pi }},S_{11}=\frac{\sin x}{\sqrt{\pi }}\). Hence\[ H\left ( x\right ) \approx c_{1}\frac{1}{\sqrt{2\pi }}+c_{2}\frac{\cos x}{\sqrt{\pi }}+c_{3}\frac{\sin x}{\sqrt{\pi }}+c_{4}\frac{\cos 2x}{\sqrt{\pi }}+c_{5}\frac{\sin 2x}{\sqrt{\pi }}+\cdots +c_{10}\frac{\cos 5x}{\sqrt{\pi }}+c_{11}\frac{\sin 5x}{\sqrt{\pi }}\] where the constants \(c_{i}\) are found from solving\[\begin{pmatrix} \left \langle S_{1},S_{1}\right \rangle & \left \langle S_{1},S_{2}\right \rangle & \left \langle S_{1},S_{3}\right \rangle & \cdots & \left \langle S_{1},S_{11}\right \rangle \\ \left \langle S_{2},S_{1}\right \rangle & \left \langle S_{2},S_{2}\right \rangle & \left \langle S_{2},S_{3}\right \rangle & \cdots & \left \langle S_{2},S_{11}\right \rangle \\ \left \langle S_{3},S_{1}\right \rangle & \left \langle S_{3},S_{2}\right \rangle & \left \langle S_{3},S_{3}\right \rangle & \cdots & \left \langle S_{3},S_{11}\right \rangle \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \left \langle S_{11},S_{1}\right \rangle & \left \langle S_{11},S_{2}\right \rangle & \left \langle S_{11},S_{3}\right \rangle & \cdots & \left \langle S_{11},S_{11}\right \rangle \end{pmatrix}\begin{pmatrix} c_{1}\\ c_{2}\\ c_{3}\\ \vdots \\ c_{11}\end{pmatrix} =\begin{pmatrix} \left \langle S_{1},H\left ( x\right ) \right \rangle \\ \left \langle S_{2},H\left ( x\right ) \right \rangle \\ \left \langle S_{3},H\left ( x\right ) \right \rangle \\ \vdots \\ \left \langle S_{11},H\left ( x\right ) \right \rangle \end{pmatrix} \] But since \(\left \langle S_{i},S_{j}\right \rangle =0\) for \(i\neq j\), because we showed above they are orthogonal to each others, and since \(S_{i}\) are all normalized now, then \(\left \langle S_{i},S_{i}\right \rangle =\left \Vert S_{i}\right \Vert ^{2}=1\). Hence the above reduces to \begin{equation} \begin{pmatrix} 1 & 0 & 0 & \cdots & 0\\ 0 & 1 & 0 & \cdots & 0\\ 0 & 0 & 1 & \cdots & 0\\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & 1 \end{pmatrix}\begin{pmatrix} c_{1}\\ c_{2}\\ c_{3}\\ \vdots \\ c_{11}\end{pmatrix} =\begin{pmatrix} \left \langle S_{1},H\left ( x\right ) \right \rangle \\ \left \langle S_{2},H\left ( x\right ) \right \rangle \\ \left \langle S_{3},H\left ( x\right ) \right \rangle \\ \vdots \\ \left \langle S_{11},H\left ( x\right ) \right \rangle \end{pmatrix} \tag{2} \end{equation} So we just need to evaluate \(\left \langle S_{i},H\left ( x\right ) \right \rangle \). But we need to do this only for three cases. These are \(\left \langle \frac{1}{\sqrt{2\pi }},H\left ( x\right ) \right \rangle ,\left \langle \frac{\cos mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle ,\left \langle \frac{\sin mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \) and then set \(m=1\cdots 5\).\begin{align*} \left \langle \frac{1}{\sqrt{2\pi }},H\left ( x\right ) \right \rangle & =\int _{-\pi }^{\pi }\frac{1}{\sqrt{2\pi }}H\left ( x\right ) dx\\ & =\frac{1}{\sqrt{2\pi }}\int _{-\pi }^{\pi }H\left ( x\right ) dx\\ & =\frac{1}{\sqrt{2\pi }}\left ( \int _{-\pi }^{0}\left ( \pi +x\right ) dx+\int _{0}^{\pi }\left ( \pi -x\right ) dx\right ) \\ & =\frac{1}{\sqrt{2\pi }}\left ( \left [ \pi x+\frac{x^{2}}{2}\right ] _{-\pi }^{0}+\left [ \pi x-\frac{x^{2}}{2}\right ] _{0}^{\pi }\right ) \\ & =\frac{1}{\sqrt{2\pi }}\left ( \left [ 0-\left ( -\pi ^{2}+\frac{\pi ^{2}}{2}\right ) \right ] +\left [ \pi ^{2}-\frac{\pi ^{2}}{2}\right ] \right ) \\ & =\frac{1}{\sqrt{2\pi }}\left ( \left [ \frac{\pi ^{2}}{2}\right ] +\left [ \pi ^{2}-\frac{\pi ^{2}}{2}\right ] \right ) \\ & =\frac{1}{\sqrt{2\pi }}\pi ^{2}\\ & =\frac{\pi ^{\frac{3}{2}}}{\sqrt{2}} \end{align*}
And\begin{align} \left \langle \frac{\cos mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle & =\int _{-\pi }^{\pi }\frac{\cos mx}{\sqrt{\pi }}H\left ( x\right ) dx\nonumber \\ & =\frac{1}{\sqrt{\pi }}\left ( \int _{-\pi }^{0}\left ( \pi +x\right ) \cos mxdx+\int _{0}^{\pi }\left ( \pi -x\right ) \cos mxdx\right ) \nonumber \\ & =\frac{1}{\sqrt{\pi }}\left ( \int _{-\pi }^{0}\pi \cos mx+\int _{-\pi }^{0}x\cos mxdx+\int _{0}^{\pi }\pi \cos mxdx-\int _{-\pi }^{0}x\cos mxdx\right ) \nonumber \\ & =\frac{1}{\sqrt{\pi }}\left ( \int _{-\pi }^{\pi }\pi \cos mx+\int _{-\pi }^{0}x\cos mxdx-\int _{-\pi }^{0}x\cos mxdx\right ) \tag{3} \end{align}
\(\int x\cos mxdx\) can be evaluated by integration by parts. Let \(u=x,dv=\cos mx\rightarrow du=1,v=\frac{\sin mx}{m}\) hence\begin{align} \int _{-\pi }^{0}x\cos mxdx & =\left [ x\frac{\sin mx}{m}\right ] _{-\pi }^{0}-\int _{-\pi }^{0}\frac{\sin mx}{m}dx\nonumber \\ & =0-\frac{1}{m}\int _{-\pi }^{0}\sin mxdx\nonumber \\ & =-\frac{1}{m}\left ( -\frac{\cos mx}{m}\right ) _{-\pi }^{0}\nonumber \\ & =\frac{1}{m^{2}}\left ( 1-\cos m\pi \right ) \tag{4} \end{align}
And\begin{align} \int _{0}^{\pi }x\cos mxdx & =\left [ x\frac{\sin mx}{m}\right ] _{0}^{\pi }-\int _{0}^{\pi }\frac{\sin mx}{m}dx\nonumber \\ & =0-\frac{1}{m}\int _{0}^{\pi }\sin mxdx\nonumber \\ & =-\frac{1}{m}\left ( -\frac{\cos mx}{m}\right ) _{0}^{\pi }\nonumber \\ & =\frac{1}{m^{2}}\left ( \cos m\pi -1\right ) \tag{5} \end{align}
And \(\int _{-\pi }^{\pi }\pi \cos mx=\pi \int _{-\pi }^{\pi }\pi \cos mx=0\). Using (4,5) in (3), then\begin{align*} \left \langle \frac{\cos mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle & =\frac{1}{\sqrt{\pi }}\left ( \frac{1}{m^{2}}\left ( 1-\cos m\pi \right ) -\frac{1}{m^{2}}\left ( \cos m\pi -1\right ) \right ) \\ & =\frac{1}{m^{2}\sqrt{\pi }}\left ( 1-\cos m\pi -\cos m\pi +1\right ) \\ & =\frac{2\left ( 1-\cos m\pi \right ) }{m^{2}\sqrt{\pi }} \end{align*}
Hence \begin{align*} \left \langle \frac{\cos mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle _{m=1} & =\left \langle \frac{\cos x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle =\frac{2\left ( 1-\cos \pi \right ) }{\sqrt{\pi }}=\frac{2\left ( 1+1\right ) }{\sqrt{\pi }}=\frac{4}{\sqrt{\pi }}\\ \left \langle \frac{\cos mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle _{m=2} & =\left \langle \frac{\cos 2x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle =\frac{2\left ( 1-\cos 2\pi \right ) }{4\sqrt{\pi }}=0\\ \left \langle \frac{\cos mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle _{m=3} & =\left \langle \frac{\cos 3x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle =\frac{2\left ( 1-\cos 3\pi \right ) }{9\sqrt{\pi }}=\frac{2\left ( 1+1\right ) }{9\sqrt{\pi }}=\frac{4}{9\sqrt{\pi }}\\ \left \langle \frac{\cos mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle _{m=4} & =\left \langle \frac{\cos 4x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle =\frac{2\left ( 1-\cos 4\pi \right ) }{16\sqrt{\pi }}=0\\ \left \langle \frac{\cos mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle _{m=5} & =\left \langle \frac{\cos 5x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle =\frac{2\left ( 1-\cos 5\pi \right ) }{25\sqrt{\pi }}=\frac{4}{25\sqrt{\pi }} \end{align*}
Similarly (we expect all the following integrals to be zero, this is because we see from above that \(H\left ( x\right ) \) is an even function and \(\sin \) is odd, hence the product is an odd function and the integral is over the period). This is the same as when in doing Fourier series expansion (which is what we are doing here essentially but using Gram’s theorem instead), all the \(b_{n}\) terms will be zero when the function being approximated is even and all the \(a_{n}\) terms will be zero when the function being approximation is odd.
But we will go ahead and do the integrals to show that this is indeed the case.\begin{align} \left \langle \frac{\sin mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle & =\int _{-\pi }^{\pi }\frac{\sin mx}{\sqrt{\pi }}H\left ( x\right ) dx\nonumber \\ & =\frac{1}{\sqrt{\pi }}\left ( \int _{-\pi }^{0}\left ( \pi +x\right ) \sin mxdx+\int _{0}^{\pi }\left ( \pi -x\right ) \sin mxdx\right ) \nonumber \\ & =\frac{1}{\sqrt{\pi }}\left ( \int _{-\pi }^{0}\pi \sin mx+\int _{-\pi }^{0}x\sin mxdx+\int _{0}^{\pi }\pi \sin mxdx-\int _{-\pi }^{0}x\sin mxdx\right ) \nonumber \\ & =\frac{1}{\sqrt{\pi }}\left ( \int _{-\pi }^{\pi }\pi \sin mx+\int _{-\pi }^{0}x\sin mxdx-\int _{-\pi }^{0}x\sin mxdx\right ) \tag{6} \end{align}
\(\int x\sin mxdx\) is evaluated by integration by parts. Let \(u=x,dv=\sin mx\rightarrow du=1,v=\frac{-\cos mx}{m}\) hence\begin{align} \int _{-\pi }^{0}x\sin mxdx & =-\frac{1}{m}\left [ x\cos mx\right ] _{-\pi }^{0}-\int _{-\pi }^{0}\frac{-\cos mx}{m}dx\nonumber \\ & =-\frac{1}{m}\left [ 0-\left ( -\pi \cos m\pi \right ) \right ] +\frac{1}{m}\int _{-\pi }^{0}\cos mxdx\nonumber \\ & =-\frac{\pi }{m}\left [ \cos m\pi \right ] +\frac{1}{m}\left ( \frac{\sin mx}{m}\right ) _{-\pi }^{0}\nonumber \\ & =-\frac{\pi }{m}\left [ \cos m\pi \right ] \tag{7} \end{align}
And\begin{align} \int _{0}^{\pi }x\sin mxdx & =-\frac{1}{m}\left [ x\cos mx\right ] _{0}^{\pi }-\int _{0}^{\pi }\frac{-\cos mx}{m}dx\nonumber \\ & =-\frac{1}{m}\left [ \pi \cos m\pi \right ] +\frac{1}{m}\int _{0}^{\pi }\cos mxdx\nonumber \\ & =-\frac{\pi }{m}\left [ \cos m\pi \right ] \tag{8} \end{align}
And \(\int _{-\pi }^{\pi }\pi \sin mx=0\). Using (7,8) in (6), then\begin{align*} \left \langle \frac{\sin mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle & =\frac{1}{\sqrt{\pi }}\left ( -\frac{\pi }{m}\left [ \cos m\pi \right ] +\frac{\pi }{m}\left [ \cos m\pi \right ] \right ) \\ & =0 \end{align*}
Hence as expected all the inner products now are zero \[ \left \langle \frac{\sin mx}{\sqrt{\pi }},H\left ( x\right ) \right \rangle _{m}=0\qquad m=1,2,3,4,5 \] Using all the above results in (2) gives\[\begin{pmatrix} 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \end{pmatrix}\begin{pmatrix} c_{1}\\ c_{2}\\ c_{3}\\ c_{4}\\ c_{5}\\ c_{6}\\ c_{7}\\ c_{8}\\ c_{9}\\ c_{10}\\ c_{11}\end{pmatrix} =\begin{pmatrix} \left \langle \frac{1}{\sqrt{2\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\cos x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\sin x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\cos 2x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\sin 2x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\cos 3x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\sin 3x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\cos 4x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\sin 4x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\cos 5x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \\ \left \langle \frac{\sin 5x}{\sqrt{\pi }},H\left ( x\right ) \right \rangle \end{pmatrix} \] Using the results found above, the above becomes\[\begin{pmatrix} 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 0 & 1 \end{pmatrix}\begin{pmatrix} c_{1}\\ c_{2}\\ c_{3}\\ c_{4}\\ c_{5}\\ c_{6}\\ c_{7}\\ c_{8}\\ c_{9}\\ c_{10}\\ c_{11}\end{pmatrix} =\begin{pmatrix} \frac{1}{\sqrt{2}}\pi ^{\frac{3}{2}}\\ \frac{4}{\sqrt{\pi }}\\ 0\\ 0\\ 0\\ \frac{4}{9\sqrt{\pi }}\\ 0\\ 0\\ 0\\ \frac{4}{25\sqrt{\pi }}\\ 0 \end{pmatrix} \] Therefore we see that\[\begin{pmatrix} c_{1}\\ c_{2}\\ c_{3}\\ c_{4}\\ c_{5}\\ c_{6}\\ c_{7}\\ c_{8}\\ c_{9}\\ c_{10}\\ c_{11}\end{pmatrix} =\begin{pmatrix} \frac{1}{\sqrt{2}}\pi ^{\frac{3}{2}}\\ \frac{4}{\sqrt{\pi }}\\ 0\\ 0\\ 0\\ \frac{4}{9\sqrt{\pi }}\\ 0\\ 0\\ 0\\ \frac{4}{25\sqrt{\pi }}\\ 0 \end{pmatrix} \] The above shows that \(c_{1}=\frac{1}{\sqrt{2}}\pi ^{\frac{3}{2}},c_{2}=\frac{4}{\sqrt{\pi }},c_{6}=\frac{4}{9\sqrt{\pi }},c_{10}=\frac{4}{25\sqrt{\pi }}\) and all other \(c^{\prime }s\) are zero. Therefore the best approximation is\begin{align*} H\left ( x\right ) & \approx c_{1}\frac{1}{\sqrt{2\pi }}+c_{2}\frac{\cos x}{\sqrt{\pi }}+c_{3}\frac{\sin x}{\sqrt{\pi }}+c_{4}\frac{\cos 2x}{\sqrt{\pi }}+c_{5}\frac{\sin 2x}{\sqrt{\pi }}+\cdots +c_{10}\frac{\cos 5x}{\sqrt{\pi }}+c_{11}\frac{\sin 5x}{\sqrt{\pi }}\\ & =\frac{1}{\sqrt{2}}\pi ^{\frac{3}{2}}\frac{1}{\sqrt{2\pi }}+\frac{4}{\sqrt{\pi }}\frac{\cos x}{\sqrt{\pi }}+\frac{4}{9\sqrt{\pi }}\frac{\cos 3x}{\sqrt{\pi }}+\frac{4}{25\sqrt{\pi }}\frac{\cos 5x}{\sqrt{\pi }}\\ & =\frac{1}{2}\pi +\frac{4}{\pi }\cos x+\frac{4}{9\pi }\cos 3x+\frac{4}{25\pi }\cos 5x \end{align*}
Or\[ \fbox{$H\left ( x\right ) \approx \frac{1}{2}\pi +\frac{4}{\pi }\cos x+\frac{4}{9\pi }\cos 3x+\frac{4}{25\pi }\cos 5x$}\] To verify the approximation, the above was plotted against the original \(H\left ( x\right ) \), first using one term \(H_{1}\left ( x\right ) \approx \frac{1}{2}\pi \) then using 2 terms \(H_{2}\left ( x\right ) \approx \frac{1}{2}\pi +\frac{4}{\pi }\cos x\) then using 3 terms \(H_{3}\left ( x\right ) \approx \frac{1}{2}\pi +\frac{4}{\pi }\cos x+\frac{4}{9\pi }\cos 3x\) and then using all terms \(H_{4}\left ( x\right ) \approx \frac{1}{2}\pi +\frac{4}{\pi }\cos x+\frac{4}{9\pi }\cos 3x+\frac{4}{25\pi }\cos 5x\). The plot below shows that the approximation improved as more terms added giving the best approximation when all terms are added as expected.
