$$advertisement$$ decision is k $=2.$ Past data indicates that that the transition probabilities for each decision is given below:

Moreover, the annual $$profit$$ matrix if decision k is taken when the state is i is estimated as

The marketing manager wants to find the optimal policy in order to maximize the average profit per year.

a$)(10$ points$)$ What is the average profit per year if the manager chooses not to advertise at all in any state.

b$)(10$ points$)$ Determine the primal linear programming formulation in order to find the optimal policy $($do not solve it$),$

c$)(15$ points$)$ Perform only $1$ iteration of the policy improvement algorithm assuming that the initial policy is not to advertise at all in any state.

d$)(5$ points$)$ Write down a linear programming formulation $($primal or dual$)$ if the objective is to maximize the expected total discounted profit where the annual discount factor is $\backslash$alpha $=0\mathrm{.}9($do not solve it$).$