Question $1$

Consider two machines that work simultaneously and independently.

During any day, machine $i=1,2$ fails with probability $q_i.$ Machines that fail during day

$n,$ becomes available for repair at the beginning of day $n+1.$ There is a single repairman

who fixes failed machines. Repair operation takes a day for both machines, i$.$e$.,$ if the

repair starts at the beginning of day $n,$ the machine becomes in operating condition at

the beginning of day $n+1.$ Machine $1$ has repair priority over machine $2$ ; that is$,$ if both

machines are found in down condition at the beginning of day $n,$ then machine $1$ becomes

in operating condition and machine $2$ is still down at the beginning of day $n+1.$

$(10$ points$)$ Model this system as a Markov chain. That is$,$ provide a clear description

of the system state, the state space and the transition probability matrix

$(3$ points$)$ Suppose on Monday both of the machines are working, what is the

probability that machine $1$ is working, and machine $2$ is not on Wednesday? Provide

your answer in terms of multiplication of rows $/$ columns of transition probability

matrix. Do not carry out the multiplication operation.

$(12$ points$)$ Suppose machine $i=1,2$ brings a daily revenue of $r_i$ as long as they

are in operating condition. The repairman is paid $w_i$ if he is working on machine

$i=1,2.$ Express expected daily profit of the system in the long run as a function

of steady state probabilities and other parameters.