4.6.3 Inventory problem

Summary of numerical results

An inventory program was written in Mathematica (please see appendix for full source code) which generated the \(P\) matrix for an increasing values of \(n\). The specification of the inventory model is described in the question shown above. The value \(s=3\) and \(S=5\) was used.

The following are few results of the \(P\) matrix for an increasing values of \(n\) and the histogram of the demand distribution used.

To answer part (b) below, we need to run the system from different initial state vector (i.e. different \(\pi ^{\left ( 0\right ) }\)) and observe if the system probability state after a long time (i.e. \(\pi ^{\left ( \infty \right ) }\)) will depend on the initial state vector or not. Since we know that

\begin{align*} \pi ^{\left ( 1\right ) } & =\pi ^{\left ( 0\right ) }P\\ \pi ^{\left ( 2\right ) } & =\pi ^{\left ( 1\right ) }P=\pi ^{\left ( 0\right ) }P^{\left ( 2\right ) }\\ \pi ^{\left ( 3\right ) } & =\pi ^{\left ( 2\right ) }P=\pi ^{\left ( 0\right ) }P^{\left ( 3\right ) }\\ & \cdots \\ \pi ^{\left ( n\right ) } & =\pi ^{\left ( 0\right ) }P^{\left ( n\right ) }\end{align*}

And since \(P^{\left ( n\right ) }=P^{n}\), then all what we have do is pick few \(\pi ^{\left ( 0\right ) }\) vectors, and post multiply them by \(P^{\left ( n\right ) }\) for large \(n\) and see if we obtain the same \(\pi ^{\left ( n\right ) }\). Below is the numerical result for this part showing the initial \(\pi ^{\left ( 0\right ) }\) and the final \(\pi ^{\left ( n\right ) }\). I used \(n=30\) in all cases as this showed it is large enough from the above numerical results. Here are the results. Below we show result of 6 tests. In each one, \(\pi ^{\left ( 0\right ) }\) is shown and to its right \(\pi ^{\left ( n\right ) }\).

Analysis of numerical results

Part (a)

\(\,\)Yes. The powers of \(P^{n}\) converges as \(n\rightarrow \infty \). This is seen by looking at the above sequence of the \(P\) matrix where we see that the matrix \(P\) converges to the following limiting matrix at around \(n=20\)

We can say the following about the limiting matrix: As \(n\rightarrow \infty \) the matrix \(P\) converges to a fixed value shown above. Each column has the same entries in its rows. In addition, all entries are non-zero. This implies that the chain contains no transient states. And since all the values on the converged \(P\) matrix are positive, then we have only one closed set in the chain, which contains all the states.

part (b)

\(\,\)Yes. There is a limiting state probability distribution in all cases. This is show by looking at the numerical result above that shows for different initial probability state vector \(\pi ^{\left ( 0\right ) }\) we obtain the same probability state vector \(\pi ^{\left ( n\right ) }\) when \(n\) is large. So the final \(\pi ^{\left ( \infty \right ) }\) does not depend on which state the system starts from.

part (c)

In this part, we need to show given that \(P^{\infty }\) converges to limiting fixed value, then the \(\pi _{k}^{\left ( \infty \right ) }\) is the same for all states \(k\).

Let \(I\) be the set of all the possible states the system can be in. Hence from definition, we write

\[ \pi _{j}^{\left ( n\right ) }={\displaystyle \sum \limits _{i\in I}} \pi _{i}^{\left ( 0\right ) }P_{ij}^{n}\]

Where \(\pi _{j}^{\left ( n\right ) }\) means the probability that the system will be in state \(j\) after \(n\) steps and \(P_{ij}^{n}\) is the \(n\) steps transition probability. Now take the limit of the above as \(n\rightarrow \infty \) we have

\begin{align*} \lim _{n\rightarrow \infty }\pi _{j}^{\left ( n\right ) } & =\lim _{n\rightarrow \infty }{\displaystyle \sum \limits _{i\in I}} \pi _{i}^{\left ( 0\right ) }P_{ij}^{n}\\ & ={\displaystyle \sum \limits _{i\in I}} \lim _{n\rightarrow \infty }\left ( \pi _{i}^{\left ( 0\right ) }P_{ij}^{n}\right ) \end{align*}

Assume there are \(k\) states, we can expand

\[ \lim _{n\rightarrow \infty }\left ( \pi _{1}^{\left ( 0\right ) }P_{1j}^{n}+\pi _{2}^{\left ( 0\right ) }P_{2j}^{n}+\cdots +\pi _{k}^{\left ( 0\right ) }P_{kj}^{n}\right ) \]

But from part(a) we observed that \(\lim _{n\rightarrow \infty }P_{ij}^{n}\) is a fixed value, which is the limit the transition matrix converged to.  In other words, \(P_{1j}^{n}=P_{2j}^{n}=\cdots =P_{kj}^{n}\) since all entries in the \(j\) column are the same.  Call this entry in \(j^{th}\) column as \(k\) say. So \(k\) is a single number which represents the one step transition probability from state \(i\) to state \(j\) when the system has run for a long time. So we write the above as

\[ \lim _{n\rightarrow \infty }\pi _{j}^{\left ( n\right ) }=k\left ( \pi _{1}^{\left ( 0\right ) }+\pi _{2}^{\left ( 0\right ) }+\cdots +\pi _{k}^{\left ( 0\right ) }\right ) \]

now, \({\displaystyle \sum \limits _{i\in I}} \pi _{i}^{\left ( 0\right ) }\) is the sum of the probabilities of the system being in all its states at time zero, which must be \(1\) hence

\[ \lim _{n\rightarrow \infty }\pi _{j}^{\left ( n\right ) }=k \]

Hence we showed that regardless of the initial \(\pi ^{\left ( 0\right ) }\) then \(\lim _{n\rightarrow \infty }\pi _{j}^{\left ( n\right ) }\) goes to some fixed values. This shows that for any state \(j\) the probability that the system will be in that state after a long time converges to a fixed value regardless of the initial state if the system transition matrix converges in the limit. Hence part(a) results could be used to deduce part(b) conclusion.