True or False? $[2$ points$]$ The difference between planning in a known Markov Decision Process $($MDP$)$ and Reinforcement Learning $($RL$)$ is that in RL the agent doesn$$t know what the current state is $($e$.$g$.,$ doesn$$t know its own position when acting in a gridworld$).$

$1\mathrm{.}2[2$ points$]$ If the only difference between two MDPs is the value of the discount factor then they must have the same optimal policy.

$1\mathrm{.}3[2$ points$]$ When getting to act only for a finite number of steps in an MDP$,$ the optimal policy is stationary. $($A stationary policy is a policy that takes the same action in a given state, independent of at what time the agent is in that state.$)$

$1\mathrm{.}4[2$ points$]$ As the number of particles goes to infinity, particle filtering will represent the same probability distribution you would get with exact inference.

$1\mathrm{.}5[6$ points$]$ Consider two particle filtering implementations:

Implementation $1$:

Initialize particles by sampling from initial state distribution and assigning uniform weights. $1.$ Propagate particles, retaining weights

$2.$ Resample according to weights

$3.$ Weight according to observations

Implementation $2$:

Initialize particles by sampling from initial state distribution. $1.$ Propagate unweighted particles

$2.$ Weight according to observations

$3.$ Resample according to weights

Questions:

i$.$ Implementation $2$ will typically provide a better approximation of the estimated distribution than implementation $1.$

ii$.$ If the transition model is deterministic then both implementations provide equally good estimates of distribution

iii. If the observation model is deterministic then both implementations provide equally good estimates of the distribution.