problem description
solution
Since J\left ( \mathbf{u}\right ) is a convex function J:\Re ^{n}\rightarrow \Re , then by definition of convex functions we write J\left ( \left ( 1-\lambda \right ) \mathbf{u}^{1}+\lambda \mathbf{u}^{2}\right ) \leq \left ( 1-\lambda \right ) J\left ( \mathbf{u}^{1}\right ) +\lambda J\left ( \mathbf{u}^{2}\right ) Where \lambda \in \left ( 0,1\right ) . Rewriting the above as follows\begin{align*} J\left ( \mathbf{u}^{1}-\lambda \mathbf{u}^{1}+\lambda \mathbf{u}^{2}\right ) & \leq J\left ( \mathbf{u}^{1}\right ) -\lambda J\left ( \mathbf{u}^{1}\right ) +\lambda J\left ( \mathbf{u}^{2}\right ) \\ J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) & \leq \lambda \left ( J\left ( \mathbf{u}^{2}\right ) -J\left ( \mathbf{u}^{1}\right ) \right ) \end{align*}
Dividing both sides by \lambda \neq 0 gives \frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }\leq J\left ( \mathbf{u}^{2}\right ) -J\left ( \mathbf{u}^{1}\right ) Taking the limit \lambda \rightarrow 0 results in \lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }\leq \lim _{\lambda \rightarrow 0}J\left ( \mathbf{u}^{2}\right ) -J\left ( \mathbf{u}^{1}\right ) But \lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }=\left . \frac{\partial J\left ( \mathbf{u}\right ) }{\partial \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) }\right \vert _{\mathbf{u}^{1}}=\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) (appendix below shows how this came about). Therefore the above becomes\begin{align*} \left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) & \leq J\left ( \mathbf{u}^{2}\right ) -J\left ( \mathbf{u}^{1}\right ) \\ J\left ( \mathbf{u}^{2}\right ) & \geq J\left ( \mathbf{u}^{1}\right ) +\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \end{align*}
QED.
More details are given here on why \lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }=\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) Let \mathbf{u}^{2}-\mathbf{u}^{1}=\mathbf{d}. This is a directional vector, its tail starts at \mathbf{u}^{1} going to tip of \mathbf{u}^{2} point. Evaluating \lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda } is the same as saying \begin{align*} \left . \frac{\partial J\left ( \mathbf{u}\right ) }{\partial \mathbf{d}}\right \vert _{\mathbf{u}^{1}} & =\lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda }\\ & =\left . \frac{d}{d\lambda }J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right \vert _{\lambda =0} \end{align*}
Using the chain rule gives\begin{align*} \left . \frac{d}{d\lambda }J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right \vert _{\lambda =0} & =\left . \left [ \nabla J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right ] ^{T}\frac{d}{d\lambda }\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right \vert _{\lambda =0}\\ & =\left . \left [ \nabla J\left ( \mathbf{u}^{1}+\lambda \mathbf{d}\right ) \right ] ^{T}\mathbf{d}\right \vert _{\lambda =0}\\ & =\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\mathbf{d} \end{align*}
Replacing \mathbf{u}^{2}-\mathbf{u}^{1}=\mathbf{d}, the above becomes\begin{align*} \lim _{\lambda \rightarrow 0}\frac{J\left ( \mathbf{u}^{1}+\lambda \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \right ) -J\left ( \mathbf{u}^{1}\right ) }{\lambda } & =\left . \frac{\partial J\left ( \mathbf{u}\right ) }{\partial \left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) }\right \vert _{\mathbf{u}^{1}}\\ & =\left [ \nabla J\left ( \mathbf{u}^{1}\right ) \right ] ^{T}\left ( \mathbf{u}^{2}-\mathbf{u}^{1}\right ) \end{align*}
Where \nabla J\left ( \mathbf{u}^{1}\right ) is the gradient vector of J\left ( \mathbf{u}\right ) evaluated at \mathbf{u}=\mathbf{u}^{1}.
solution
Since each m_{ij}\left ( q\right ) is convex function in q, then\begin{equation} m_{ij}\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) \leq \left ( 1-\alpha \right ) m_{ij}\left ( q^{1}\right ) +\alpha m_{ij}\left ( q^{2}\right ) \tag{1} \end{equation} For \alpha \in \left [ 0,1\right ] . We also know by Rayleigh quotient theorem which applies for symmetric matrices that largest eigenvalue of a symmetric matrix is given by \lambda _{\max }=\max _{x\in \Re ^{n},\left \Vert x\right \Vert =1}x^{T}Mx Therefore, evaluated at point q^{\alpha }=\left ( 1-\alpha \right ) q^{1}+\alpha q^{2}, the above become\begin{equation} \lambda _{\max }\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) =\max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) x_{i}x_{j}\tag{2} \end{equation} Applying (1) in RHS (2) changes = to \leq giving\begin{align} \lambda _{\max }\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) & \leq \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}\left ( \left ( 1-\alpha \right ) m_{ij}\left ( q^{1}\right ) +\alpha m_{ij}\left ( q^{2}\right ) \right ) x_{i}x_{j}\nonumber \\ & =\max _{\left \Vert x\right \Vert =1}\left ( \sum _{i,j}^{n}\left ( 1-\alpha \right ) m_{ij}\left ( q^{1}\right ) x_{i}x_{j}+\sum _{i,j}^{n}\alpha m_{ij}\left ( q^{2}\right ) x_{i}x_{j}\right ) \nonumber \\ & =\left ( 1-\alpha \right ) \left ( \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( q^{1}\right ) x_{i}x_{j}\right ) +\alpha \left ( \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( q^{2}\right ) x_{i}x_{j}\right ) \tag{3} \end{align}
Since \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( q^{1}\right ) x_{i}x_{j}=\lambda _{\max }\left ( q^{1}\right ) And \max _{\left \Vert x\right \Vert =1}\sum _{i,j}^{n}m_{ij}\left ( q^{2}\right ) x_{i}x_{j}=\lambda _{\max }\left ( q^{2}\right ) Then (3) becomes \lambda _{\max }\left ( \left ( 1-\alpha \right ) q^{1}+\alpha q^{2}\right ) \leq \left ( 1-\alpha \right ) \lambda _{\max }\left ( q^{1}\right ) +\alpha \lambda _{\max }\left ( q^{2}\right ) This is the definition of convex function, therefore \lambda _{\max } is a convex function in q.
Note: I tried also to reduce this to a problem where I could argue that the pointwise maximum of convex functions is also a convex function to solve it. I could not get a clear way to do this, so I solved it as above. I hope I did not violate the cardinal rule by using \lambda _{\max }=\max _{x\in \Re ^{n},\left \Vert x\right \Vert =1}x^{T}Mx.
solution
To show U is bounded, a proof by induction is used. From the definition of constructing U U=\left \{ x\in \Re ^{n}:x=\sum _{i=1}^{m}\lambda _{i}u^{i}\right \} Where \sum _{i=1}^{m}\lambda _{i}=1 and \lambda _{i}\geq 0.
For m=1, x=\lambda u^{1}. So U contains just one element u^{1}. Since \lambda =1 and u^{1} is given and bounded, then this is closed and bounded set with one element. Hence compact. Now we assume U is compact for m=k-1 and we need to show it is compact for m=k. In other words, we assume that each x^{\ast }\in U generated using x^{\ast }=\sum _{i=1}^{k-1}\lambda _{i}u^{i} Is such that \left \Vert x^{\ast }\right \Vert <\infty and x^{\ast }\in U. Now we need to show that U is bounded when generator contains k elements. Now\begin{align*} x & =\sum _{i=1}^{k}\lambda _{i}u^{i}\\ & =\lambda _{1}u^{1}+\lambda _{2}u^{2}+\cdots +\lambda _{k-1}u^{k-1}+\lambda _{k}u^{k} \end{align*}
Multiply and divide by \left ( 1-\lambda _{k}\right ) \begin{align*} x & =\left ( 1-\lambda _{k}\right ) \left ( \frac{\lambda _{1}u^{1}}{\left ( 1-\lambda _{k}\right ) }+\frac{\lambda _{2}}{\left ( 1-\lambda _{k}\right ) }u^{2}+\cdots +\frac{\lambda _{k-1}u^{k-1}}{\left ( 1-\lambda _{k}\right ) }+\frac{\lambda _{k}}{\left ( 1-\lambda _{k}\right ) }u^{k}\right ) \\ & =\left ( 1-\lambda _{k}\right ) \left ( \sum _{i=1}^{k-1}\frac{\lambda _{i}}{\left ( 1-\lambda _{k}\right ) }u^{i}+\frac{\lambda _{k}}{\left ( 1-\lambda _{k}\right ) }u^{k}\right ) \\ & =\left ( 1-\lambda _{k}\right ) \left ( \sum _{i=1}^{k-1}\frac{\lambda _{i}}{\left ( 1-\lambda _{k}\right ) }u^{i}\right ) +\lambda _{k}u^{k} \end{align*}
But \sum _{i=1}^{k-1}\frac{\lambda _{i}}{\left ( 1-\lambda _{k}\right ) }u^{i}=x^{\ast } which we assumed in U. Hence the above becomes x=\left ( 1-\lambda _{k}\right ) x^{\ast }+\lambda _{k}u^{k} Since u^{k} is element in the generator set G and it is in U by definition, then the above is convex combination of two elements in U. Hence x in also in U (it is on a line between x^{\ast } and u^{k}, both in U). Therefore U is closed and bounded for any m in the generator set. Hence U is compact.
The extreme points of P are subset of G. They are the points used to generate P. The set P is compact (by problem 3) and convex set (by construction, since it is convex combinations of its extreme points). If we can show that J^{\ast } is at an extreme point of P, then we are done, since an extreme point of P is in G.
Let u^{\ast }\in P be the point where J\left ( u\right ) is maximum. u^{\ast } is a convex combinations of all extreme points of P, (these are also subset from G but they can be the whole set G also if there were no redundant generators), Therefore u^{\ast }={\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}v^{i} where k\leq N and v^{i}\in G. If it happens that all points in G are extreme points of P, then k=N. Therefore J^{\ast }=J\left ( u^{\ast }\right ) =J\left ({\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}v^{i}\right ) Where \sum _{i=1}^{k}\lambda _{i}=1 and \lambda _{i}\geq 0. But J is convex function (given). Hence by definition of convex function \begin{equation} J^{\ast }=J\left ({\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}v^{i}\right ) \leq{\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}J\left ( v^{i}\right ) \tag{1} \end{equation} The above is generalization of J\left ( \left ( 1-\lambda \right ) u^{1}+\lambda u^{2}\right ) \leq \left ( 1-\lambda \right ) J\left ( u^{1}\right ) +\lambda J\left ( u^{2}\right ) applied to convex mixtures. Now we look at J\left ( v^{i}\right ) term in the above. We pick the maximum of J over all v^{i}. There must be a point in G where J\left ( v\right ) is largest. We call this value J_{G}^{\ast }. This is the value of J where it attains its maximum over generator elements v^{i}:i=1\cdots k. Eq (1) becomes J^{\ast }\leq{\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}J_{G}^{\ast } Where we replaced J\left ( v^{i}\right ) by one value, the maximum J_{G}^{\ast }. But J_{G}^{\ast } does not depend on i now, and can take it outside the sum J^{\ast }\leq J_{G}^{\ast }\left ({\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}\right ) But {\displaystyle \sum \limits _{i=1}^{k}} \lambda _{i}=1 by definition. Therefore the above becomes J^{\ast }\leq J_{G}^{\ast } We now see that the maximum of J\left ( u\right ) over P is smaller (or equal) than the maximum of J\left ( u\right ) over the generator set G. Hence a maximum occurs at one of the extreme points v^{i}, since these are by definition taken from G. which is what we are asked to show.
solution
Let us look at the closed loop. Let v=0 and we have, since u\left ( t\right ) =kx\left ( t\right ) \begin{align*} \dot{x} & =Ax+Bkx\\ & =\left ( A+Bk\right ) x\\ & =A_{c}x \end{align*}
Where A_{c} is the closed loop system matrix. Since J\left ( k\right ) =\int _{0}^{\infty }x^{T}\left ( t\right ) x\left ( t\right ) +\lambda u^{T}\left ( t\right ) u\left ( t\right ) dt, where u\left ( t\right ) =kx\left ( t\right ) , then\begin{align*} J\left ( k\right ) & =\int _{0}^{\infty }x^{T}x+\lambda \left ( kx\right ) ^{T}\left ( kx\right ) dt\\ & =\int _{0}^{\infty }x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) xdt \end{align*}
Let us find a matrix P, if possible such that d\left ( x^{T}Px\right ) =-\left ( x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x\right ) Can we find P? Since d\left ( x^{T}Px\right ) =x^{T}P\dot{x}+\dot{x}^{T}Px Then we need to solve\begin{align*} x^{T}P\dot{x}+\dot{x}^{T}Px & =-\left ( x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x\right ) \\ x^{T}P\left ( A_{c}x\right ) +\left ( A_{c}x\right ) ^{T}Px & =-\left ( x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x\right ) \\ x^{T}P\left ( A_{c}x\right ) +\left ( x^{T}A_{c}^{T}\right ) Px & =-\left ( x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x\right ) \end{align*}
Bring all the x to LHS then \begin{align*} x^{T}x+\lambda x^{T}\left ( k^{T}k\right ) x+x^{T}P\left ( A_{c}x\right ) +\left ( x^{T}A_{c}^{T}\right ) Px & =\mathbf{0}\\ \lambda \left ( k^{T}k\right ) +PA_{c}+A_{c}^{T}P & =-I \end{align*}
Hence the Lyapunov equation to solve for P is \fbox{$\lambda \left ( k^Tk\right ) +PA_c+A_c^TP=-I$} Where I is the identity matrix. This is the equation to determine matrix P. Without loss of generality, we insist on P being symmetric matrix. Using this P, now we write\begin{align*} J\left ( k\right ) & =\int _{0}^{\infty }x^{T}x+\lambda \left ( kx\right ) ^{T}\left ( kx\right ) dt\\ & =-\int _{0}^{\infty }d\left ( x^{T}Px\right ) \\ & =\left . x^{T}Px\right \vert _{\infty }^{0}\\ & =x^{T}\left ( 0\right ) Px\left ( 0\right ) -x^{T}\left ( \infty \right ) Px\left ( \infty \right ) \end{align*}
For stable system, x\left ( \infty \right ) \rightarrow 0 (since we set v=0, there is no external input, hence if the system is stable, it must end up in zero state eventually). In part (b) we check for k range so that the roots are in the left hand side. Therefore J\left ( k\right ) =x^{T}\left ( 0\right ) P\left ( k\right ) x\left ( 0\right ) With P\left ( k\right ) satisfying solution of Lyapunov equation found above.
For k=\begin{bmatrix} k_{1} & k_{2}\end{bmatrix} ,x\left ( 0\right ) =\begin{bmatrix} 1\\ 0 \end{bmatrix} and system y^{\prime \prime }=u. Hence x_{1}^{\prime }=x_{2},x_{2}^{\prime }=u. Since u=\begin{bmatrix} k_{1} & k_{2}\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} The system \dot{x}=Ax+Bu becomes \begin{align*} x^{\prime } & =Ax+Bu\\ & =Ax+Bkx\\ & =\left ( A+Bk\right ) x\\\begin{bmatrix} x_{1}^{\prime }\\ x_{2}^{\prime }\end{bmatrix} & =\left ( \overset{A}{\overbrace{\begin{bmatrix} 0 & 1\\ 0 & 0 \end{bmatrix} }}+\overset{B}{\overbrace{\begin{bmatrix} 0\\ 1 \end{bmatrix} }}\overset{k}{\overbrace{\begin{bmatrix} k_{1} & k_{2}\end{bmatrix} }}\right ) \begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \\ & =\left ( \begin{bmatrix} 0 & 1\\ 0 & 0 \end{bmatrix} +\begin{bmatrix} 0 & 0\\ k_{1} & k_{2}\end{bmatrix} \right ) \begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \\ & =\overset{A_{c}}{\overbrace{\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix} }}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \end{align*}
For stable system, we need k_{1},k_{2}<0 from looking at the roots of the characteristic equation. Now we solve the Lyapunov equation.\begin{align*} \lambda \left ( k^{T}k\right ) +PA_{c}+A_{c}^{T}P & =-I\\ \lambda \begin{bmatrix} k_{1} & k_{2}\end{bmatrix} ^{T}\begin{bmatrix} k_{1} & k_{2}\end{bmatrix} +\begin{bmatrix} p_{11} & p_{12}\\ p_{21} & p_{22}\end{bmatrix}\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix} +\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix} ^{T}\begin{bmatrix} p_{11} & p_{12}\\ p_{21} & p_{22}\end{bmatrix} & =\begin{bmatrix} -1 & 0\\ 0 & -1 \end{bmatrix} \\ \lambda \begin{bmatrix} k_{1}\\ k_{2}\end{bmatrix}\begin{bmatrix} k_{1} & k_{2}\end{bmatrix} +\begin{bmatrix} p_{11} & p_{12}\\ p_{21} & p_{22}\end{bmatrix}\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix} +\begin{bmatrix} 0 & k_{1}\\ 1 & k_{2}\end{bmatrix}\begin{bmatrix} p_{11} & p_{12}\\ p_{21} & p_{22}\end{bmatrix} & =\begin{bmatrix} -1 & 0\\ 0 & -1 \end{bmatrix} \\ \lambda \begin{bmatrix} k_{1}^{2} & k_{1}k_{2}\\ k_{1}k_{2} & k_{2}^{2}\end{bmatrix} +\begin{bmatrix} k_{1}p_{12} & p_{11}+k_{2}p_{12}\\ k_{1}p_{22} & p_{21}+k_{2}p_{22}\end{bmatrix} +\begin{bmatrix} k_{1}p_{21} & k_{1}p_{22}\\ p_{11}+k_{2}p_{21} & p_{12}+k_{2}p_{22}\end{bmatrix} & =\begin{bmatrix} -1 & 0\\ 0 & -1 \end{bmatrix} \\\begin{bmatrix} k_{1}\left ( p_{12}+p_{21}+\lambda k_{1}\right ) & p_{11}+k_{1}p_{22}+k_{2}p_{12}+\lambda k_{1}k_{2}\\ p_{11}+k_{1}p_{22}+k_{2}p_{21}+\lambda k_{1}k_{2} & \lambda k_{2}^{2}+2p_{22}k_{2}+p_{12}+p_{21}\end{bmatrix} & =\begin{bmatrix} -1 & 0\\ 0 & -1 \end{bmatrix} \end{align*}
Hence we have 4 equations to solve for p_{11,}p_{12},p_{21,}p_{22}. (but we know also that p_{12}=p_{21}). Now let \lambda =1 per the problem, and we obtain the four equations from above as\begin{align*} k_{1}^{2}+k_{1}p_{12}+k_{1}p_{21} & =-1\\ p_{11}+k_{1}k_{2}+k_{1}p_{22}+k_{2}p_{12} & =0\\ p_{11}+k_{1}k_{2}+k_{1}p_{22}+k_{2}p_{21} & =0\\ k_{2}^{2}+2p_{22}k_{2}+p_{12}+p_{21} & =-1 \end{align*}
Solution is (Using Matlab syms).
Gives
P=\begin{bmatrix} -\frac{k_{1}-k_{1}^{2}+k_{1}^{3}-k_{2}^{2}}{2k_{1}k_{2}} & -\frac{k_{1}^{2}+1}{2k_{1}}\\ -\frac{k_{1}^{2}+1}{2k_{1}} & -\frac{k_{1}+k_{1}k_{2}^{2}-k_{1}^{2}-1}{2k_{1}k_{2}}\end{bmatrix} Hence\begin{align*} J\left ( k\right ) & =x^{T}\left ( 0\right ) P\left ( k\right ) x\left ( 0\right ) \\ & =\begin{bmatrix} 1 & 0 \end{bmatrix}\begin{bmatrix} -\frac{k_{1}-k_{1}^{2}+k_{1}^{3}-k_{2}^{2}}{2k_{1}k_{2}} & -\frac{k_{1}^{2}+1}{2k_{1}}\\ -\frac{k_{1}^{2}+1}{2k_{1}} & -\frac{k_{1}+k_{1}k_{2}^{2}-k_{1}^{2}-1}{2k_{1}k_{2}}\end{bmatrix}\begin{bmatrix} 1\\ 0 \end{bmatrix} \end{align*}
Therefore \fbox{$J\left ( k\right ) =-\frac{1}{2k_1k_2}\left ( k_1^3-k_1^2+k_1-k_2^2\right ) $ } For k_{1}=k_{2}=k, the above becomes\begin{align*} J\left ( k\right ) & =-\frac{\left ( k^{3}-2k^{2}+k\right ) }{2k^{2}}\\ & =-\frac{\left ( k^{2}-2k+1\right ) }{2k} \end{align*}
Or \fbox{$J\left ( k\right ) =-\frac{1}{2k}\left ( k-1\right ) ^2$} Now we find the optimal J^{\ast }. Since \frac{dJ\left ( k\right ) }{dk}=\frac{\left ( k-1\right ) ^{2}}{2k^{2}}-\frac{\left ( 2k-2\right ) }{2k} Then \frac{dJ\left ( k\right ) }{dk}=0 gives k=1,-1 Since k must be negative for stable system, we pick \fbox{$k^\ast =-1$} And \frac{d^{2}J\left ( k\right ) }{dk^{2}}=\frac{\left ( k-1\right ) ^{2}}{k^{3}}-\frac{2\left ( 1-k\right ) }{k^{2}}-\frac{1}{k} At k^{\ast }=-1\, \frac{d^{2}J\left ( k\right ) }{dk^{2}}=1>0 Hence this is a minimum. Therefore J^{\ast }=\left . -\frac{1}{2k}\left ( k-1\right ) ^{2}\right \vert _{k=-1} Hence \fbox{$J^\ast =2$} J^{\ast } do not get to zero. (same as in the class problem we did without \lambda u^{T}u term. I thought we will get J^{\ast }=0 now since this I thought it was the reason for using \lambda u^{T}u. I hope I did not make mistake, but do not see where if I did. Below is a plot of J\left ( k\right ).
At k=1 then J\left ( 1\right ) =0, but we can not use k=1 since this will make the system not stable. The system now using k^{\ast }=-1 becomes\begin{align*} \begin{bmatrix} x_{1}^{\prime }\\ x_{2}^{\prime }\end{bmatrix} & =\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \\ & =\begin{bmatrix} 0 & 1\\ -1 & -1 \end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \end{align*}
To verify it is stable. Since \left \vert \left ( \lambda I-A_{c}\right ) \right \vert =\lambda ^{2}+\lambda +1 The roots are -\frac{1}{2}\pm \frac{1}{2}i\sqrt{3} Hence the system is stable since real part of roots are negative. If we had used k=1, the roots will be -0.618,1.618, and the system would have been unstable.
From last part, we obtained P P=\begin{bmatrix} -\frac{k_{1}-k_{1}^{2}+k_{1}^{3}-k_{2}^{2}}{2k_{1}k_{2}} & -\frac{k_{1}^{2}+1}{2k_{1}}\\ -\frac{k_{1}^{2}+1}{2k_{1}} & -\frac{k_{1}+k_{1}k_{2}^{2}-k_{1}^{2}-1}{2k_{1}k_{2}}\end{bmatrix} When k_{1}=k_{2}=k the above becomes P=\begin{bmatrix} \frac{-k+2k^{2}-k^{3}}{2k^{2}} & -\frac{k^{2}+1}{2k}\\ -\frac{k^{2}+1}{2k} & \frac{1-k-k^{3}+k^{2}}{2k^{2}}\end{bmatrix} Now since x\left ( 0\right ) is random variable, then\begin{align*} J\left ( k\right ) & =E\left ( x^{T}\left ( 0\right ) Px\left ( 0\right ) \right ) \nonumber \\ & =E\left ( \begin{bmatrix} x_{1}\left ( 0\right ) & x_{2}\left ( 0\right ) \end{bmatrix}\begin{bmatrix} \frac{-k+2k^{2}-k^{3}}{2k^{2}} & -\frac{k^{2}+1}{2k}\\ -\frac{k^{2}+1}{2k} & \frac{1-k-k^{3}+k^{2}}{2k^{2}}\end{bmatrix}\begin{bmatrix} x_{1}\left ( 0\right ) \\ x_{2}\left ( 0\right ) \end{bmatrix} \right ) \nonumber \\ & =E\left ( -\frac{1}{2k^{2}}\left ( k^{3}x_{1}^{2}\left ( 0\right ) +2k^{3}x_{1}\left ( 0\right ) x_{2}\left ( 0\right ) +k^{3}x_{2}^{2}\left ( 0\right ) -2k^{2}x_{1}^{2}\left ( 0\right ) -k^{2}x_{2}^{2}\left ( 0\right ) +kx_{1}^{2}\left ( 0\right ) +2kx_{1}\left ( 0\right ) x_{2}\left ( 0\right ) +kx_{2}^{2}\left ( 0\right ) -x_{2}^{2}\left ( 0\right ) \right ) \right ) \tag{1} \end{align*}
Let E\left ( x_{1}\left ( 0\right ) \right ) =\bar{x}_{1} and E\left ( x_{2}\left ( 0\right ) \right ) =\bar{x}_{2} Then J\left ( k\right ) =-\frac{1}{2k^{2}}\left ( k^{3}\bar{x}_{1}^{2}+2k^{3}\bar{x}_{1}\bar{x}_{2}+k^{3}\bar{x}_{2}^{2}-2k^{2}\bar{x}_{1}^{2}-k^{2}\bar{x}_{2}^{2}+k\bar{x}_{1}^{2}+2k\bar{x}_{1}\bar{x}_{2}+k\bar{x}_{2}^{2}-\bar{x}_{2}^{2}\right ) But E\left ( x_{1}\left ( 0\right ) \right ) =0, hence \bar{x}_{1}=0 and similarly \bar{x}_{2}=0, but \bar{x}_{1}^{2}=\frac{1}{3} since \int _{-1}^{1}x^{2}p\left ( x\right ) dx=\frac{1}{2}\int _{-1}^{1}x^{2}dx=\frac{1}{2}\left ( \frac{x^{3}}{3}\right ) _{-1}^{1}=\frac{1}{3} Similarly \bar{x}_{2}^{2}=\frac{1}{3} and \bar{x}_{1}\bar{x}_{2}=0 (since i.i.d, then E\left ( x_{1}\left ( 0\right ) x_{2}\left ( 0\right ) \right ) =E\left ( x_{1}\left ( 0\right ) \right ) E\left ( x_{2}\left ( 0\right ) \right ) =0. Using these values of expectations, Eq (1) becomes J\left ( k\right ) =-\frac{1}{2k^{2}}\left ( k^{3}\frac{1}{3}+k^{3}\frac{1}{3}-2k^{2}\frac{1}{3}-k^{2}\frac{1}{3}+k\frac{1}{3}+k\frac{1}{3}-\frac{1}{3}\right ) Or\begin{equation} \fbox{$J\left ( k\right ) =\frac{-2k^3+3k^2-2k+1}{6k^2}$}\tag{2} \end{equation} To find the optimal: \frac{dJ\left ( k\right ) }{dk}=-\frac{1}{3}-\frac{1}{3k^{3}}+\frac{1}{3k^{2}} \frac{dJ\left ( k\right ) }{dk}=0 gives 3 roots. The only one which is real and negative (the other two are complex) is \fbox{$k^\ast =-1.325$} At this k^{\ast }, we check \frac{d^{2}J\left ( k\right ) }{dk^{2}} and find it is 0.611>0, hence J is minimum at k^{\ast }. The value J^{\ast } at k^{\ast } is found to be from substituting k^{\ast } in (2) \fbox{$J^\ast =1.28817$}
We now check if the system is stable. (it should be, since k^{\ast }<1). The system now is\begin{align*} \begin{bmatrix} x_{1}^{\prime }\\ x_{2}^{\prime }\end{bmatrix} & =\begin{bmatrix} 0 & 1\\ k_{1} & k_{2}\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \\ & =\begin{bmatrix} 0 & 1\\ -1.325 & -1.325 \end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} \end{align*}
Hence \left \vert \left ( \lambda I-A_{c}\right ) \right \vert =\lambda ^{2}+1.325\lambda +1.325 The roots are -0.6625\pm i0.941 The system is stable since real part of roots are negative. The following is the step response for system in part(b) and part(c) to compare.
