A Markov Decision Process is a four tuple $,$ where S is the finite set of states, A is the finite set of actions and R is the cost or reward being in state s$.$ T is the transition model which specifies.

A$.$ Probability of executing action a in state s at time

B$.$ Probability of at time t$+1,$ given action a in state s at time t$.$

C$.$ Probability of s at time t$,$ given actions in states upto time t$-1$

D$.$ Probability of executing action a in state at time t$+1$