$.$

Problem 1: (a=b=5, total =10) Suppose every time a child buys a kid's meal at his/her favorite fast-food restaurant, he/she receives one of four superhero action figures (toys). Naturally, the child wants to collect all four action figures and so he/she regularly eats lunch at this restaurant to complete the collection. This process can be described by a Markov chain. In this case, let X[k] = {0, 1, 2, 3, 4} be the number of different action figures that the child has collected after purchasing k meals. Assuming each meal contains one of the four superheroes with equal probability and that the action figure in any meal is independent of what is contained in any previous or future meals. a) Write the corresponding transition probability matrix of the above described case. b) Plot the state diagram for the related Markov chain.