Please explain how did you came up with the answer for a thumbs up!

These questions are based on the Markov Decision Process, reinforcement learning, and statistics.

Thank you!

Consider the simple n-state MDP shown in Figure 1. Starting from state $1, the agent can move to the right (ao) or left (ai) from any state si. Actions are deterministic and always succeed (e.g. going left from state s2 goes to state si, and going left from state si transitions to itself). Rewards are given upon taking an action from the state. Taking any action from the goal state G earns a reward of r = +1 and the agent stays in state G. Otherwise, each move has zero reward (r = 0). Assume a discount factor y