please help with this problem and the parts

Consider a one-player version of the game twenty-one as a Markov decision process. The objective is to draw cards one at a time from an infinite deck of playing cards and acquire a card sum as large as possible without going over 21. For now we will have ten integer states in {12,,21} representing the card sum (sums smaller than 12 are trivially played). At each turn we can take one of two actions from state s. Stopping yields a reward equal to s and immediately ends the game. Hitting yields zero reward, and we will either transition to a state s with probability 131 where s