$(15$ Points$)$ For an MDP $M=(S,A,R,p,r,d_0,),$ write out expressions using the transition function, $p,$ reward function, $r,$ the initial state distribution $d_0,$ and policy $,$ for the following statements. Show your work.

$1\mathrm{.}1(5$ Points$))=s^{\text{'}}|S_7|=s,A_2=a,S_0=(s^{\text{'}\text{'}}$

Answer:

$1\mathrm{.}2(5$ Points$))=s^{\text{'}}|A_1|=(a$

Answer:

$)=s^{\text{'}}|A_1|=(a$

$1\mathrm{.}3(5$ Points$))=r|S_4|=s^{\text{'}},A_2=a,S_2=(s$

Answer:

$)=r|S_4|=s^{\text{'}},A_2=a,S_2=(s$