Now, let's see what happens if we try that same control$\_$sequence on a version of the environment where the wheels are a little bit slippery,

so our simple unicycle model does not describe the true system perfectly.

$[]$ class NoisyUnicycle$($Unicycle$)$:

$"\text{'}"\text{'}$Same as Unicycle but add process noise at each step.""$\text{'}"$

def$\_$init$\_($self$,$

process$\_$noise$\_$limits$=$np$.$array $([0\mathrm{.}1,0\mathrm{.}1,0\mathrm{.}5]),$

:

self.process$\_$noise$\_$limits $=$ process$\_$noise$\_$limits

super $()$ :( $v_{-}$min$=v_{-}$min,$v_{-}$max$=v_{-}$max,$w_{-}$min$=w_{-}$min,$w_{-}$max$=w_{-}$max $)$

def step$($self$,$ current$\_$state, action, dt$=0\mathrm{.}1)$ np$.$ndarray:

$"\text{'}"\text{'}"$Add process noise to the parent kinematics model." $"\text{'}"$

next$\_$state $=$super$().$step current$\_$state, action, $dt=dt)$

perturbed$\_$next$\_$state $=$ next$\_$state $+$ np$.$random. uniform$($low$=-$self.process$\_$noise$\_$limits$,$ high$=$self.process$\_$noise$\_$limits$)$

return perturbed$\_$next$\_$state

This class we wrote for you will make it so you can't perfectly measure the system's state. It adds random noise to the obs that the env

returns at each step:

$[]$ from gymnasium import ObservationWrapper

class NoisyStateFeedbackWrapper$($ObservationWrapper$)$:

def $($self$,$ env, sensor$\_$noise$\_$limits$=$np$.$array$([0\mathrm{.}05,0\mathrm{.}05,0\mathrm{.}2]$ :

self. sensor$\_$noise$\_$limits $=$ sensor$\_$noise$\_$limits

super$()$ :$($env$)$

def observation$($self$,$ observation: dict$)$ np$.$ndarray:

obs $=$ observation $["$state$"]+$ np$.$random.uniform$($low$=-$self.sensor$\_$noise$\_$limits$,$ high$=$self.sensor$\_$noise$\_$limits$)$

return obs