Andy and Barney make up a game to pass time during Covid-19 lockdown. At each iteration, each of them picks a penny of himself and secretly turns his penny to heads or tails. Then they reveal their choices simultaneously. If the pennies match (both tails or both heads), Andy wins Barneys penny. If the pennies do not match (one heads and one tails), Barney wins Andys penny.

Suppose the players have a total of 5 pennies between them. If at any time one player has all of the pennies, to keep the game going, he gives one back to the other player on the next iteration and the game will continue.

a) Clearly define the states of the markov chain and draw the state transition diagram.

b) Is this markov chain ergodic? Explain why/why not.

c) On average, in what percent of the iterations Andy has more pennies than Barney? Show all your calculations.

d) Assume that Andy starts the game with 3 pennies and Barney with 2. What is the probability that the first player who drops down to zero pennies is Andy? Show all your calculations.