A system has three possible states, 0, 1 and 2. Every hour it makes a transition to a different state, which is determined by a coin flip. For example, from state 0, it makes a transition to state 1 or state 2 with probabilities 0.5 and 0.5. a. Find the transition probability matrix.
b. Find the three-step transition probability matrix.
c. Find the steady-state distribution of the Markov chain.

Step-by-step explanation:

From the given information;

the transition probability matrix (TPM) for state 0, 1, 2 is:

The three-step TPM  is done by a simple matrix multiplication which is computed as follows:

The steady-state distribution of the Markov Chain is  determined by first solving the system πp = π together with the normalizing condition

here;

π₁ + π₂ + π₃ = 1

So;

0.5π₂ + 0.5π₃ = π₁

⇒ π₂ + π₃ = 2π₁

0.5π₁ + 0.5π₃ = π₂

⇒ π₁ + π₃ = 2π₂

0.5π₁ + 0.5π₂ = π₃

⇒ π₁ + π₂ = 2π₃

Thus, π₁ + π₂ + π₃ = 1

This can be re-written as:

1 - π₁ = 2π₁

1 - π₂ = 2π₂

1 - π₃ = 2π₃

∴

1 = 3π₁

π₁ = 1/3

1 = 3π₂

π₂ = 1/3

1 = 3π₃

π₃ = 1/3

Hence, the steady-state probability distribution is π = [1/3, 1/3, 1/3]