$($Queueing Model $1)$ Consider a queueing system with two servers. A customer arrives

according to a Poisson process with rate lambda$>0.$ Each customer brings service times that are exponential with rate lambda$>0.$ When the servers are both empty a customer is assigned to any of the two servers with equal probability. All customers are served independently and the arrival and service processes are mutually independent. Customers are served in the order in which they arrive and servers are "work conserving" meaning that they always keep working unless there are no customers in the system. When a customer is served, they return to the system $($to the back of the queue$)$ to be served again with probability p in $(0,1)$ and leave the system with probability $1-$ p$.$ The new service time in the system is another exponential with rate independent of everything else. In other words, everytime a returning customer interacts with any of the servers a completely new and independent service time is required for the current server$/$customer interaction.

a$.$ Let Q$($t$)$ be the number of customers in the system at time t$.$ Explain why

$\{$Q$($t$)$ : t$>=0\}$ is a continuous time Markov chain? What is the state space?

b$.$ Provide the transition diagram representation for the evolution of Q$.$

c$.$ Now consider a queueing system identical to the one above, but this time

with three servers $($inter$-$arrival and service time parameters are the same as above$).$

Provide the transition diagram describing the evolution of the Markov chain describing the number of customers in the system.