Problem 4 (Partially-Observable Markov Processes). Many applications today (e.g., autonomous vehicles, factory automation) make computations and decisions based on the observation of the state of a remote process. Suppose, in such a scenario, the decision at any time k depends on the state X of a Markov process. It may however sometimes be too difficult or too costly to observe the state directly. Thus, the decision-maker may have to act based on a prediction of the otherwise unknown state. For a Markov process model m states, such an estimate is called the probabilistic state, defined for any k by a PMF over the original state space: px, (i) = P(X = i) fori = 1, 2,..., m. (a) Consider the two-state Markov process model with transition probabilities P11 = 1-0, P12 = 0, P22 = 1-4, and p21 = If we have no further information or observations, we can set the probabilistic state at steady-state using steady-state probabilities. Compute the probabilistic state at time k = 1000. In a partially-observable Markov process, a measurement device is available that generates "noisy" obser- vations of the otherwise unknown state. The dependence of each such observation Z on the true state X is defined by a sensor model, typically specified as a conditional distribution of Z given Xk. The conditional PMF P(X= i|Z) for i=1,2,..., m can be viewed as an estimator g(Z) of the probabilis- tic state based on observation Z; then, for any observed value z of Z, the probabilistic state estimate PX|Z = P(Xk = i|Z = z) summarizes the available state information in a manner consistent with the ealized observation as well as all process model and sensor model parameters. (b) Suppose that just after the process defines in part (a) enters X1000, a single observation Z = z is realized according to the following sensor model: fz|x1000 (zi)=. = N( 0) = 1 exp 2Oi (z ) 20 -