Question 1.1 [First Step Analysis): Consider a. robot exploring an unknown terrain for new resources. After some time, it has learned that its exploration algorithm takes it through three types of terrains [1: forest. 2: mountains. 3: swamps). Let us model the terrain exploration problem as a Markov chain {thrsa with the transition matrix .4 .5 .1 P= a .3 . . [1} [1 u 1 Note that once the robot is in state {3: swamps], it cannot get out. (a) We assume that the reward function r : {1,1, 3} > RED encodes the amount of resources in cash terrain. Let T := min{n. E [l : X\" = 3} be the hitting time of state [3: swamps). Denote the expected reward before reaching the swamps by n*{:r) := E 2: ernHXu = at] . [2) n=EI Using rst step analysis? write down a linear equation that [Whirl]: should satisfy. =13} {b} Now1 assume that the amount of resources in each terrain is given by 1*[1] = 11\".], 1*[2] = 3 and 1"[3] = I}. For the transition matrix [1), compute the solution to the linear equation given in PM [el- (cj Does the numerical solution attained in part {b} corrapond to u*{a:) given in expression [2)? Explain why