Consider a robot exploring an unknown terrain for new resources. After some time, it

has learned that its exploration algorithm takes it through three types of terrains $(1$: forest,

$2$: mountains, $3$: swamps$).$ Let us model the terrain exploration problem as a Markov chain

$\{$Xn$\}$n$>=0,$ with the transition matrix