4.6.2 craps game

Summary of numerical results

The state probability transition matrix was entered and then raised to higher powers. This is the numerical result

To answer part (b) below, we need to run the system from different initial state vector (i.e. different \(\pi ^{\left ( 0\right ) }\)) and observe if the system probability state vector after a long time (i.e. \(\pi ^{\left ( \infty \right ) }\)) will depend on the initial state vector or not. Here is the result for 3 different initial state vectors. In diagram below we show the \(\pi ^{\left ( 0\right ) }\) and to its right \(\pi ^{\left ( \infty \right ) }\).

Analysis of numerical results

part (a)

\(\,\)Yes. The powers of \(P^{n}\) converges as \(n\rightarrow \infty \). This is seen by looking at the above sequence of the \(P\) matrix where we see that the matrix \(P\) converges to the following limiting matrix at around \(n=81\)

We can say the following about the limiting matrix: As \(n\rightarrow \infty \) the matrix \(P\) converges to a fixed value shown above. The entries \(P_{ij}^{n}\) where \(j\) is a transient state goes to zero as \(n\) gets large.

part (b)

From the above numerical result, we see that depending on the initial system probability state vector \(\pi ^{\left ( 0\right ) }\) we obtain a different system probability state vector \(\pi ^{\left ( n\right ) }\) as \(n\) gets very large. This is because some states are transient (states \(\left \{ 4,5,6,8,9,10\right \} \)). In the inventory problem below, we see that we obtained a different result for this part since the inventory problem has no transient states.

part (c) Let \(I\) be the set of all the possible states the system can be in. Hence from definition, we write

\[ \pi _{j}^{\left ( n\right ) }={\displaystyle \sum \limits _{i\in I}} \pi _{i}^{\left ( 0\right ) }P_{ij}^{n}\]

Where \(\pi _{j}^{\left ( n\right ) }\) means the probability that the system will be in state \(j\) after \(n\) steps and \(P_{ij}^{n}\) is the \(n\) steps transition probability. Now take the limit of the above as \(n\rightarrow \infty \) we have

\begin{align*} \lim _{n\rightarrow \infty }\pi _{j}^{\left ( n\right ) } & =\lim _{n\rightarrow \infty }{\displaystyle \sum \limits _{i\in I}} \pi _{i}^{\left ( 0\right ) }P_{ij}^{n}\\ & ={\displaystyle \sum \limits _{i\in I}} \lim _{n\rightarrow \infty }\left ( \pi _{i}^{\left ( 0\right ) }P_{ij}^{n}\right ) \end{align*}

Assume there are \(k\) states, we can expand

\[ \lim _{n\rightarrow \infty }\left ( \pi _{1}^{\left ( 0\right ) }P_{1j}^{n}+\pi _{2}^{\left ( 0\right ) }P_{2j}^{n}+\cdots +\pi _{k}^{\left ( 0\right ) }P_{kj}^{n}\right ) \]

But from part(a) we observed that in the limit, entries of each columns are not equal. Hence \(P_{1j}^{n}\neq P_{2j}^{n}\neq \cdots \neq P_{kj}^{n}\) this means the above sum will produce a different value depending on the initial state probability vector \(\pi ^{\left ( 0\right ) }\). (Compare this to the inventory problem below, where each entry in a column is the same, and we could factor it out of the sum and we reached a different conclusion than here).

Hence we showed depending on the initial \(\pi ^{\left ( 0\right ) }\) then \(\lim _{n\rightarrow \infty }\pi _{j}^{\left ( n\right ) }\) goes to different value as confirmed by the numerical result shown above in part(a). Hence part(a) results could be used to deduce part(b) conclusion.