The Markov Decision Process for this scenario is provided in Figure $1.($see image$)$

The transition probabilities for decreasing and increasing the temperature are provided in blue and red, respectively. The rewards for the transitions are underlined and green.

Using value iteration on the given MDP$,$ compute V$1($under cooked$).$ Write down the intermediate steps, including, the Q$-$values for the state$-$action pairs.

i$=0$ v$0($undercooked$)=0$

i $=1$ v$1($undercooked$)=?$

$1$ Q$($undercooked$,$ increase$)=?$

$2$ Q$($undercooked$,$ decrease$)=?$

Please explain to me q value and the correct steps for markov decision process.