Both relate to the prediction task; say policy is being evaluated on MDP (S, A, T,...

Transcribed Image Text:

Both relate to the prediction task; say policy is being evaluated on MDP (S, A, T, R, 7). Assume the MDP is continuing and ergodic; also assume standard conditions for annealing the learning rate. R1. TD (0) in the tabular setting (that is, with a separate entry for each state) converges to the underlying value function V". R2. Linear TD(X), for A € [0, 1], which computes the estimate V as a dot dimensional feature vector of state and learned weight vector w, converges to w Show that R2 implies R1. MSVE(w) ≤ 1-7A min MSVE(w). 1-7 werd product of a d- satisfying Both relate to the prediction task; say policy is being evaluated on MDP (S, A, T, R, 7). Assume the MDP is continuing and ergodic; also assume standard conditions for annealing the learning rate. R1. TD (0) in the tabular setting (that is, with a separate entry for each state) converges to the underlying value function V". R2. Linear TD(X), for A € [0, 1], which computes the estimate V as a dot dimensional feature vector of state and learned weight vector w, converges to w Show that R2 implies R1. MSVE(w) ≤ 1-7A min MSVE(w). 1-7 werd product of a d- satisfying

Answer rating: 100% (QA)

The detailed ... View the full answer