problem description
solution
Since \(J\left ( \mathbf{u}\right ) \) is a convex function \(J:\Re ^{n}\rightarrow \Re \), then by definition of convex functions we write\[ J\left ( \left ( 1-\lambda \right ) \mathbf{u}^{1}+\lambda \mathbf{u}^{2}\right ) \leq \left ( 1-\lambda \right ) J\left ( \mathbf{u}^{1}\right ) +\lambda J\left ( \mathbf{u}^{2}\right ) \] Where \(\lambda \in \left ( 0,1\right ) \). Rewriting the above as follows\begin{align*} J\left ( \mathbf{u}^{1}-\lambda \mathbf{u}^{1}+\lambda \mathbf{u}^{2}\right ) & \leq J\left ( \mathbf{u}^{1}\right ) -\lambda J\left ( \mathbf{u}^{1}\right ) +\lambda J\left ( \mathbf{u}^{2}\right ) \\ J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) & \leq \lambda \left ( J\left ( \mathbf{u}^{2}\right ) -J\left ( \mathbf{u}^{1}\right ) \right ) \end{align*}
Dividing both sides by \(\lambda \neq 0\) gives\[ \frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }\leq J\left ( \mathbf{u}^{2}\right ) -J\left ( \mathbf{u}^{1}\right ) \] Taking the limit \(\lambda \rightarrow 0\) results in\[ \lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }\leq \lim _{\lambda \rightarrow 0}J\left ( \mathbf{u}^{2}\right ) -J\left ( \mathbf{u}^{1}\right ) \] But \(\lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }=\left . \frac{\partial J\left ( \mathbf{u}\right ) }{\partial \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) }\right \vert _{\mathbf{u}^{1}}=\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \) (appendix below shows how this came about). Therefore the above becomes\begin{align*} \left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) & \leq J\left ( \mathbf{u}^{2}\right ) -J\left ( \mathbf{u}^{1}\right ) \\ J\left ( \mathbf{u}^{2}\right ) & \geq J\left ( \mathbf{u}^{1}\right ) +\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \end{align*}
QED.
More details are given here on why \[ \lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }=\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \] Let \(\mathbf{u}^{2}-\mathbf{u}^{1}=\mathbf{d}\). This is a directional vector, its tail starts at \(\mathbf{u}^{1}\) going to tip of \(\mathbf{u}^{2}\) point. Evaluating \(\lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }\) is the same as saying \begin{align*} \left . \frac{\partial J\left ( \mathbf{u}\right ) }{\partial \mathbf{d}}\right \vert _{\mathbf{u}^{1}} & =\lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }\\ & =\left . \frac{d}{d\lambda }J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right \vert _{\lambda =0} \end{align*}
Using the chain rule gives\begin{align*} \left . \frac{d}{d\lambda }J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right \vert _{\lambda =0} & =\left . \left [ \nabla J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right ] ^{T}\frac{d}{d\lambda }\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right \vert _{\lambda =0}\\ & =\left . \left [ \nabla J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right ] ^{T}\mathbf{d}\right \vert _{\lambda =0}\\ & =\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\mathbf{d} \end{align*}
Replacing \(\mathbf{u}^{2}-\mathbf{u}^{1}=\mathbf{d}\), the above becomes\begin{align*} \lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda } & =\left . \frac{\partial J\left ( \mathbf{u}\right ) }{\partial \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) }\right \vert _{\mathbf{u}^{1}}\\ & =\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \end{align*}
Where \(\nabla J\left ( \mathbf{u}^{1}\right ) \) is the gradient vector of \(J\left ( \mathbf{u}\right ) \) evaluated at \(\mathbf{u}=\mathbf{u}^{1}.\)
solution
Since each \(m_{ij}\left ( q\right ) \) is convex function in \(q\), then\begin{equation} m_{ij}\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) \leq \left ( 1-\alpha \right ) m_{ij}\left ( q^{1}\right ) +\alpha m_{ij}\left ( q^{2}\right ) \tag{1} \end{equation} For \(\alpha \in \left [ 0,1\right ] .\) We also know by Rayleigh quotient theorem which applies for symmetric matrices that largest eigenvalue of a symmetric matrix is given by \[ \lambda _{\max }=\max _{x\in \Re ^{n},\left \Vert x\right \Vert =1}x^{T}Mx \] Therefore, evaluated at point \(q^{\alpha }=\left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\), the above become\begin{equation} \lambda _{\max }\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) =\max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) x_{i}x_{j}\tag{2} \end{equation} Applying (1) in RHS (2) changes \(=\) to \(\leq \) giving\begin{align} \lambda _{\max }\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) & \leq \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}\left ( \left ( 1-\alpha \right ) m_{ij}\left ( q^{1}\right ) +\alpha m_{ij}\left ( q^{2}\right ) \right ) x_{i}x_{j}\nonumber \\ & =\max _{\left \Vert x\right \Vert =1}\left ( \sum _{i,j}^{n}\left ( 1-\alpha \right ) m_{ij}\left ( q^{1}\right ) x_{i}x_{j}+\sum _{i,j}^{n}\alpha m_{ij}\left ( q^{2}\right ) x_{i}x_{j}\right ) \nonumber \\ & =\left ( 1-\alpha \right ) \left ( \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( q^{1}\right ) x_{i}x_{j}\right ) +\alpha \left ( \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( q^{2}\right ) x_{i}x_{j}\right ) \tag{3} \end{align}
Since\[ \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( q^{1}\right ) x_{i}x_{j}=\lambda _{\max }\left ( q^{1}\right ) \] And \[ \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( q^{2}\right ) x_{i}x_{j}=\lambda _{\max }\left ( q^{2}\right ) \] Then (3) becomes\[ \lambda _{\max }\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) \leq \left ( 1-\alpha \right ) \lambda _{\max }\left ( q^{1}\right ) +\alpha \lambda _{\max }\left ( q^{2}\right ) \] This is the definition of convex function, therefore \(\lambda _{\max }\) is a convex function in \(q\).
Note: I tried also to reduce this to a problem where I could argue that the pointwise maximum of convex functions is also a convex function to solve it. I could not get a clear way to do this, so I solved it as above. I hope I did not violate the cardinal rule by using \(\lambda _{\max }=\max _{x\in \Re ^{n},\left \Vert x\right \Vert =1}x^{T}Mx\).
solution
To show \(U\) is bounded, a proof by induction is used. From the definition of constructing \(U\) \[ U=\left \{ x\in \Re ^{n}:x=\sum _{i=1}^{m}\lambda _{i}u^{i}\right \} \] Where \(\sum _{i=1}^{m}\lambda _{i}=1\) and \(\lambda _{i}\geq 0\).
For \(m=1\), \(x=\lambda u^{1}\). So \(U\) contains just one element \(u^{1}\). Since \(\lambda =1\) and \(u^{1}\) is given and bounded, then this is closed and bounded set with one element. Hence compact. Now we assume \(U\) is compact for \(m=k-1\) and we need to show it is compact for \(m=k\). In other words, we assume that each \(x^{\ast }\in U\) generated using\[ x^{\ast }=\sum _{i=1}^{k-1}\lambda _{i}u^{i}\] Is such that \(\left \Vert x^{\ast }\right \Vert <\infty \) and \(x^{\ast }\in U\). Now we need to show that \(U\) is bounded when generator contains \(k\) elements. Now\begin{align*} x & =\sum _{i=1}^{k}\lambda _{i}u^{i}\\ & =\lambda _{1}u^{1}+\lambda _{2}u^{2}+\cdots +\lambda _{k-1}u^{k-1}+\lambda _{k}u^{k} \end{align*}
Multiply and divide by \(\left ( 1-\lambda _{k}\right ) \)\begin{align*} x & =\left ( 1-\lambda _{k}\right ) \left ( \frac{\lambda _{1}u^{1}}{\left ( 1-\lambda _{k}\right ) }+\frac{\lambda _{2}}{\left ( 1-\lambda _{k}\right ) }u^{2}+\cdots +\frac{\lambda _{k-1}u^{k-1}}{\left ( 1-\lambda _{k}\right ) }+\frac{\lambda _{k}}{\left ( 1-\lambda _{k}\right ) }u^{k}\right ) \\ & =\left ( 1-\lambda _{k}\right ) \left ( \sum _{i=1}^{k-1}\frac{\lambda _{i}}{\left ( 1-\lambda _{k}\right ) }u^{i}+\frac{\lambda _{k}}{\left ( 1-\lambda _{k}\right ) }u^{k}\right ) \\ & =\left ( 1-\lambda _{k}\right ) \left ( \sum _{i=1}^{k-1}\frac{\lambda _{i}}{\left ( 1-\lambda _{k}\right ) }u^{i}\right ) +\lambda _{k}u^{k} \end{align*}
But \(\sum _{i=1}^{k-1}\frac{\lambda _{i}}{\left ( 1-\lambda _{k}\right ) }u^{i}=x^{\ast }\) which we assumed in \(U\). Hence the above becomes\[ x=\left ( 1-\lambda _{k}\right ) x^{\ast }+\lambda _{k}u^{k}\] Since \(u^{k}\) is element in the generator set \(G\) and it is in \(U\) by definition, then the above is convex combination of two elements in \(U\). Hence \(x\) in also in \(U\) (it is on a line between \(x^{\ast }\) and \(u^{k}\), both in \(U)\). Therefore \(U\) is closed and bounded for any \(m\) in the generator set. Hence \(U\) is compact.
The extreme points of \(P\) are subset of \(G\). They are the points used to generate \(P\). The set \(P\) is compact (by problem 3) and convex set (by construction, since it is convex combinations of its extreme points). If we can show that \(J^{\ast }\) is at an extreme point of \(P\), then we are done, since an extreme point of \(P\) is in \(G\).
Let \(u^{\ast }\in P\) be the point where \(J\left ( u\right ) \) is maximum. \(u^{\ast }\) is a convex combinations of all extreme points of \(P\), (these are also subset from \(G\) but they can be the whole set \(G\) also if there were no redundant generators), Therefore\[ u^{\ast }={\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}v^{i}\] where \(k\leq N\) and \(v^{i}\in G\). If it happens that all points in \(G\) are extreme points of \(P\), then \(k=N\). Therefore\[ J^{\ast }=J\left ( u^{\ast }\right ) =J\left ({\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}v^{i}\right ) \] Where \(\sum _{i=1}^{k}\lambda _{i}=1\) and \(\lambda _{i}\geq 0\). But \(J\) is convex function (given). Hence by definition of convex function \begin{equation} J^{\ast }=J\left ({\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}v^{i}\right ) \leq{\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}J\left ( v^{i}\right ) \tag{1} \end{equation} The above is generalization of \(J\left ( \left ( 1-\lambda \right ) u^{1}+\lambda u^{2}\right ) \leq \left ( 1-\lambda \right ) J\left ( u^{1}\right ) +\lambda J\left ( u^{2}\right ) \) applied to convex mixtures. Now we look at \(J\left ( v^{i}\right ) \) term in the above. We pick the maximum of \(J\) over all \(v^{i}\). There must be a point in \(G\) where \(J\left ( v\right ) \) is largest. We call this value \(J_{G}^{\ast }\). This is the value of \(J\) where it attains its maximum over generator elements \(v^{i}:i=1\cdots k\). Eq (1) becomes\[ J^{\ast }\leq{\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}J_{G}^{\ast }\] Where we replaced \(J\left ( v^{i}\right ) \) by one value, the maximum \(J_{G}^{\ast }.\) But \(J_{G}^{\ast }\) does not depend on \(i\) now, and can take it outside the sum\[ J^{\ast }\leq J_{G}^{\ast }\left ({\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}\right ) \] But \({\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}=1\) by definition. Therefore the above becomes\[ J^{\ast }\leq J_{G}^{\ast }\] We now see that the maximum of \(J\left ( u\right ) \) over \(P\) is smaller (or equal) than the maximum of \(J\left ( u\right ) \) over the generator set \(G\). Hence a maximum occurs at one of the extreme points \(v^{i}\), since these are by definition taken from \(G\). which is what we are asked to show.
solution
Let us look at the closed loop. Let \(v=0\) and we have, since \(u\left ( t\right ) =kx\left ( t\right ) \)\begin{align*} \dot{x} & =Ax+Bkx\\ & =\left ( A+Bk\right ) x\\ & =A_{c}x \end{align*}
Where \(A_{c}\) is the closed loop system matrix. Since \(J\left ( k\right ) =\int _{0}^{\infty }x^{T}\left ( t\right ) x\left ( t\right ) +\lambda u^{T}\left ( t\right ) u\left ( t\right ) dt\), where \(u\left ( t\right ) =kx\left ( t\right ) \), then\begin{align*} J\left ( k\right ) & =\int _{0}^{\infty }x^{T}x+\lambda \left ( kx\right ) ^{T}\left ( kx\right ) dt\\ & =\int _{0}^{\infty }x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) xdt \end{align*}
Let us find a matrix \(P\), if possible such that \[ d\left ( x^{T}Px\right ) =-\left ( x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x\right ) \] Can we find \(P\)? Since\[ d\left ( x^{T}Px\right ) =x^{T}P\dot{x}+\dot{x}^{T}Px \] Then we need to solve\begin{align*} x^{T}P\dot{x}+\dot{x}^{T}Px & =-\left ( x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x\right ) \\ x^{T}P\left ( A_{c}x\right ) +\left ( A_{c}x\right ) ^{T}Px & =-\left ( x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x\right ) \\ x^{T}P\left ( A_{c}x\right ) +\left ( x^{T}A_{c}^{T}\right ) Px & =-\left ( x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x\right ) \end{align*}
Bring all the \(x\) to LHS then \begin{align*} x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x+x^{T}P\left ( A_{c}x\right ) +\left ( x^{T}A_{c}^{T}\right ) Px & =\mathbf{0}\\ \lambda \left ( k^{T}k\right ) +PA_{c}+A_{c}^{T}P & =-I \end{align*}
Hence the Lyapunov equation to solve for \(P\) is\[ \fbox{$\lambda \left ( k^Tk\right ) +PA_c+A_c^TP=-I$}\] Where \(I\) is the identity matrix. This is the equation to determine matrix \(P\). Without loss of generality, we insist on \(P\) being symmetric matrix. Using this \(P\), now we write\begin{align*} J\left ( k\right ) & =\int _{0}^{\infty }x^{T}x+\lambda \left ( kx\right ) ^{T}\left ( kx\right ) dt\\ & =-\int _{0}^{\infty }d\left ( x^{T}Px\right ) \\ & =\left . x^{T}Px\right \vert _{\infty }^{0}\\ & =x^{T}\left ( 0\right ) Px\left ( 0\right ) -x^{T}\left ( \infty \right ) Px\left ( \infty \right ) \end{align*}
For stable system, \(x\left ( \infty \right ) \rightarrow 0\) (since we set \(v=0\), there is no external input, hence if the system is stable, it must end up in zero state eventually). In part (b) we check for \(k\) range so that the roots are in the left hand side. Therefore\[ J\left ( k\right ) =x^{T}\left ( 0\right ) P\left ( k\right ) x\left ( 0\right ) \] With \(P\left ( k\right ) \) satisfying solution of Lyapunov equation found above.
For \(k=\begin{bmatrix} k_{1} & k_{2}\end{bmatrix} ,x\left ( 0\right ) =\begin{bmatrix} 1\\ 0 \end{bmatrix} \) and system \(y^{\prime \prime }=u\). Hence \(x_{1}^{\prime }=x_{2},x_{2}^{\prime }=u\). Since\[ u=\begin{bmatrix} k_{1} & k_{2}\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \] The system \(\dot{x}=Ax+Bu\) becomes \begin{align*} x^{\prime } & =Ax+Bu\\ & =Ax+Bkx\\ & =\left ( A+Bk\right ) x\\\begin{bmatrix} x_{1}^{\prime }\\ x_{2}^{\prime }\end{bmatrix} & =\left ( \overset{A}{\overbrace{\begin{bmatrix} 0 & 1\\ 0 & 0 \end{bmatrix} }}+\overset{B}{\overbrace{\begin{bmatrix} 0\\ 1 \end{bmatrix} }}\overset{k}{\overbrace{\begin{bmatrix} k_{1} & k_{2}\end{bmatrix} }}\right ) \begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \\ & =\left ( \begin{bmatrix} 0 & 1\\ 0 & 0 \end{bmatrix} +\begin{bmatrix} 0 & 0\\ k_{1} & k_{2}\end{bmatrix} \right ) \begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \\ & =\overset{A_{c}}{\overbrace{\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix} }}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \end{align*}
For stable system, we need \(k_{1},k_{2}<0\) from looking at the roots of the characteristic equation. Now we solve the Lyapunov equation.\begin{align*} \lambda \left ( k^{T}k\right ) +PA_{c}+A_{c}^{T}P & =-I\\ \lambda \begin{bmatrix} k_{1} & k_{2}\end{bmatrix} ^{T}\begin{bmatrix} k_{1} & k_{2}\end{bmatrix} +\begin{bmatrix} p_{11} & p_{12}\\ p_{21} & p_{22}\end{bmatrix}\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix} +\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix} ^{T}\begin{bmatrix} p_{11} & p_{12}\\ p_{21} & p_{22}\end{bmatrix} & =\begin{bmatrix} -1 & 0\\ 0 & -1 \end{bmatrix} \\ \lambda \begin{bmatrix} k_{1}\\ k_{2}\end{bmatrix}\begin{bmatrix} k_{1} & k_{2}\end{bmatrix} +\begin{bmatrix} p_{11} & p_{12}\\ p_{21} & p_{22}\end{bmatrix}\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix} +\begin{bmatrix} 0 & k_{1}\\ 1 & k_{2}\end{bmatrix}\begin{bmatrix} p_{11} & p_{12}\\ p_{21} & p_{22}\end{bmatrix} & =\begin{bmatrix} -1 & 0\\ 0 & -1 \end{bmatrix} \\ \lambda \begin{bmatrix} k_{1}^{2} & k_{1}k_{2}\\ k_{1}k_{2} & k_{2}^{2}\end{bmatrix} +\begin{bmatrix} k_{1}p_{12} & p_{11}+k_{2}p_{12}\\ k_{1}p_{22} & p_{21}+k_{2}p_{22}\end{bmatrix} +\begin{bmatrix} k_{1}p_{21} & k_{1}p_{22}\\ p_{11}+k_{2}p_{21} & p_{12}+k_{2}p_{22}\end{bmatrix} & =\begin{bmatrix} -1 & 0\\ 0 & -1 \end{bmatrix} \\\begin{bmatrix} k_{1}\left ( p_{12}+p_{21}+\lambda k_{1}\right ) & p_{11}+k_{1}p_{22}+k_{2}p_{12}+\lambda k_{1}k_{2}\\ p_{11}+k_{1}p_{22}+k_{2}p_{21}+\lambda k_{1}k_{2} & \lambda k_{2}^{2}+2p_{22}k_{2}+p_{12}+p_{21}\end{bmatrix} & =\begin{bmatrix} -1 & 0\\ 0 & -1 \end{bmatrix} \end{align*}
Hence we have 4 equations to solve for \(p_{11,}p_{12},p_{21,}p_{22}\). (but we know also that \(p_{12}=p_{21}\)). Now let \(\lambda =1\) per the problem, and we obtain the four equations from above as\begin{align*} k_{1}^{2}+k_{1}p_{12}+k_{1}p_{21} & =-1\\ p_{11}+k_{1}k_{2}+k_{1}p_{22}+k_{2}p_{12} & =0\\ p_{11}+k_{1}k_{2}+k_{1}p_{22}+k_{2}p_{21} & =0\\ k_{2}^{2}+2p_{22}k_{2}+p_{12}+p_{21} & =-1 \end{align*}
Solution is (Using Matlab syms).
Gives
\[ P=\begin{bmatrix} -\frac{k_{1}-k_{1}^{2}+k_{1}^{3}-k_{2}^{2}}{2k_{1}k_{2}} & -\frac{k_{1}^{2}+1}{2k_{1}}\\ -\frac{k_{1}^{2}+1}{2k_{1}} & -\frac{k_{1}+k_{1}k_{2}^{2}-k_{1}^{2}-1}{2k_{1}k_{2}}\end{bmatrix} \] Hence\begin{align*} J\left ( k\right ) & =x^{T}\left ( 0\right ) P\left ( k\right ) x\left ( 0\right ) \\ & =\begin{bmatrix} 1 & 0 \end{bmatrix}\begin{bmatrix} -\frac{k_{1}-k_{1}^{2}+k_{1}^{3}-k_{2}^{2}}{2k_{1}k_{2}} & -\frac{k_{1}^{2}+1}{2k_{1}}\\ -\frac{k_{1}^{2}+1}{2k_{1}} & -\frac{k_{1}+k_{1}k_{2}^{2}-k_{1}^{2}-1}{2k_{1}k_{2}}\end{bmatrix}\begin{bmatrix} 1\\ 0 \end{bmatrix} \end{align*}
Therefore\[ \fbox{$J\left ( k\right ) =-\frac{1}{2k_1k_2}\left ( k_1^3-k_1^2+k_1-k_2^2\right ) $ }\] For \(k_{1}=k_{2}=k\), the above becomes\begin{align*} J\left ( k\right ) & =-\frac{\left ( k^{3}-2k^{2}+k\right ) }{2k^{2}}\\ & =-\frac{\left ( k^{2}-2k+1\right ) }{2k} \end{align*}
Or\[ \fbox{$J\left ( k\right ) =-\frac{1}{2k}\left ( k-1\right ) ^2$}\] Now we find the optimal \(J^{\ast }\). Since \[ \frac{dJ\left ( k\right ) }{dk}=\frac{\left ( k-1\right ) ^{2}}{2k^{2}}-\frac{\left ( 2k-2\right ) }{2k}\] Then \(\frac{dJ\left ( k\right ) }{dk}=0\) gives \[ k=1,-1 \] Since \(k\) must be negative for stable system, we pick \[ \fbox{$k^\ast =-1$}\] And\[ \frac{d^{2}J\left ( k\right ) }{dk^{2}}=\frac{\left ( k-1\right ) ^{2}}{k^{3}}-\frac{2\left ( 1-k\right ) }{k^{2}}-\frac{1}{k}\] At \(k^{\ast }=-1\,\)\[ \frac{d^{2}J\left ( k\right ) }{dk^{2}}=1>0 \] Hence this is a minimum. Therefore\[ J^{\ast }=\left . -\frac{1}{2k}\left ( k-1\right ) ^{2}\right \vert _{k=-1}\] Hence\[ \fbox{$J^\ast =2$}\] \(J^{\ast }\) do not get to zero. (same as in the class problem we did without \(\lambda u^{T}u\) term. I thought we will get \(J^{\ast }=0\) now since this I thought it was the reason for using \(\lambda u^{T}u\). I hope I did not make mistake, but do not see where if I did. Below is a plot of \(J\left ( k\right )\).
At \(k=1\) then \(J\left ( 1\right ) =0\), but we can not use \(k=1\) since this will make the system not stable. The system now using \(k^{\ast }=-1\) becomes\begin{align*} \begin{bmatrix} x_{1}^{\prime }\\ x_{2}^{\prime }\end{bmatrix} & =\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \\ & =\begin{bmatrix} 0 & 1\\ -1 & -1 \end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \end{align*}
To verify it is stable. Since\[ \left \vert \left ( \lambda I-A_{c}\right ) \right \vert =\lambda ^{2}+\lambda +1 \] The roots are \[ -\frac{1}{2}\pm \frac{1}{2}i\sqrt{3}\] Hence the system is stable since real part of roots are negative. If we had used \(k=1\), the roots will be \(-0.618,1.618\), and the system would have been unstable.
From last part, we obtained \(P\)\[ P=\begin{bmatrix} -\frac{k_{1}-k_{1}^{2}+k_{1}^{3}-k_{2}^{2}}{2k_{1}k_{2}} & -\frac{k_{1}^{2}+1}{2k_{1}}\\ -\frac{k_{1}^{2}+1}{2k_{1}} & -\frac{k_{1}+k_{1}k_{2}^{2}-k_{1}^{2}-1}{2k_{1}k_{2}}\end{bmatrix} \] When \(k_{1}=k_{2}=k\) the above becomes\[ P=\begin{bmatrix} \frac{-k+2k^{2}-k^{3}}{2k^{2}} & -\frac{k^{2}+1}{2k}\\ -\frac{k^{2}+1}{2k} & \frac{1-k-k^{3}+k^{2}}{2k^{2}}\end{bmatrix} \] Now since \(x\left ( 0\right ) \) is random variable, then\begin{align*} J\left ( k\right ) & =E\left ( x^{T}\left ( 0\right ) Px\left ( 0\right ) \right ) \nonumber \\ & =E\left ( \begin{bmatrix} x_{1}\left ( 0\right ) & x_{2}\left ( 0\right ) \end{bmatrix}\begin{bmatrix} \frac{-k+2k^{2}-k^{3}}{2k^{2}} & -\frac{k^{2}+1}{2k}\\ -\frac{k^{2}+1}{2k} & \frac{1-k-k^{3}+k^{2}}{2k^{2}}\end{bmatrix}\begin{bmatrix} x_{1}\left ( 0\right ) \\ x_{2}\left ( 0\right ) \end{bmatrix} \right ) \nonumber \\ & =E\left ( -\frac{1}{2k^{2}}\left ( k^{3}x_{1}^{2}\left ( 0\right ) +2k^{3}x_{1}\left ( 0\right ) x_{2}\left ( 0\right ) +k^{3}x_{2}^{2}\left ( 0\right ) -2k^{2}x_{1}^{2}\left ( 0\right ) -k^{2}x_{2}^{2}\left ( 0\right ) +kx_{1}^{2}\left ( 0\right ) +2kx_{1}\left ( 0\right ) x_{2}\left ( 0\right ) +kx_{2}^{2}\left ( 0\right ) -x_{2}^{2}\left ( 0\right ) \right ) \right ) \tag{1} \end{align*}
Let \(E\left ( x_{1}\left ( 0\right ) \right ) =\bar{x}_{1}\) and \(E\left ( x_{2}\left ( 0\right ) \right ) =\bar{x}_{2}\) Then \[ J\left ( k\right ) =-\frac{1}{2k^{2}}\left ( k^{3}\bar{x}_{1}^{2}+2k^{3}\bar{x}_{1}\bar{x}_{2}+k^{3}\bar{x}_{2}^{2}-2k^{2}\bar{x}_{1}^{2}-k^{2}\bar{x}_{2}^{2}+k\bar{x}_{1}^{2}+2k\bar{x}_{1}\bar{x}_{2}+k\bar{x}_{2}^{2}-\bar{x}_{2}^{2}\right ) \] But \(E\left ( x_{1}\left ( 0\right ) \right ) =0\), hence \(\bar{x}_{1}=0\) and similarly \(\bar{x}_{2}=0,\) but \(\bar{x}_{1}^{2}=\frac{1}{3}\) since\[ \int _{-1}^{1}x^{2}p\left ( x\right ) dx=\frac{1}{2}\int _{-1}^{1}x^{2}dx=\frac{1}{2}\left ( \frac{x^{3}}{3}\right ) _{-1}^{1}=\frac{1}{3}\] Similarly \(\bar{x}_{2}^{2}=\frac{1}{3}\) and \(\bar{x}_{1}\bar{x}_{2}=0\) (since i.i.d, then \(E\left ( x_{1}\left ( 0\right ) x_{2}\left ( 0\right ) \right ) =E\left ( x_{1}\left ( 0\right ) \right ) E\left ( x_{2}\left ( 0\right ) \right ) =0\). Using these values of expectations, Eq (1) becomes\[ J\left ( k\right ) =-\frac{1}{2k^{2}}\left ( k^{3}\frac{1}{3}+k^{3}\frac{1}{3}-2k^{2}\frac{1}{3}-k^{2}\frac{1}{3}+k\frac{1}{3}+k\frac{1}{3}-\frac{1}{3}\right ) \] Or\begin{equation} \fbox{$J\left ( k\right ) =\frac{-2k^3+3k^2-2k+1}{6k^2}$}\tag{2} \end{equation} To find the optimal: \[ \frac{dJ\left ( k\right ) }{dk}=-\frac{1}{3}-\frac{1}{3k^{3}}+\frac{1}{3k^{2}}\] \(\frac{dJ\left ( k\right ) }{dk}=0\) gives 3 roots. The only one which is real and negative (the other two are complex) is\[ \fbox{$k^\ast =-1.325$}\] At this \(k^{\ast }\), we check \(\frac{d^{2}J\left ( k\right ) }{dk^{2}}\) and find it is \(0.611>0\), hence \(J\) is minimum at \(k^{\ast }\). The value \(J^{\ast }\) at \(k^{\ast }\) is found to be from substituting \(k^{\ast }\) in (2)\[ \fbox{$J^\ast =1.28817$}\]
We now check if the system is stable. (it should be, since \(k^{\ast }<1\)). The system now is\begin{align*} \begin{bmatrix} x_{1}^{\prime }\\ x_{2}^{\prime }\end{bmatrix} & =\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \\ & =\begin{bmatrix} 0 & 1\\ -1.325 & -1.325 \end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \end{align*}
Hence \[ \left \vert \left ( \lambda I-A_{c}\right ) \right \vert =\lambda ^{2}+1.325\lambda +1.325 \] The roots are \[ -0.6625\pm i0.941 \] The system is stable since real part of roots are negative. The following is the step response for system in part(b) and part(c) to compare.
