In Euclidean spaces, a set C is called a convex set if for all x and y in C, the line segment connecting x and y is included in C. In other words, the affine combination tx p1 tqy belongs to C, for all x, y P C, and t P r0, 1s. Prove that if a Markov Chain has stationary distribution, then all stationary distribution of this Markov Chain is a convex set. (b) Let pXnqn0,1,2 be a Markov chain with state space S t1, 2, , Nu. The sum of each column of the stochastic probability matrix P is 1, i. e. ¸ N i1 Pij 1, for all j Show that if this Markov chain is aperiodic and irreducible, then the stationary distribut