Problem 1 During each time period, a potential customer arrives at a restaurant with probability 1/2. If there are already two people at the restaurant (including the one being served), the potential customer leaves the restaurant immediately and never returns. However, if there is one person or less, he enters the restaurant and becomes an actual customer. The manager has two types of service configurations available. At the beginning of each period, a decision must be made on which configuration to use. If she uses her slow configuration at a cost of $3 and any customers are present during the period, one customer will be served and leave with probability 3/5. If she uses her fast configuration at a cost of $9 and any customers are present during the period, one customer will be served and leave with probability 4/5. The probability of more than one customer arriving or more than one customer being served in a period is zero. A profit of $50 is earned when a customer is served. The manager wants to formulate the problem of choosing the service configuration period by period as a Markov decision process. (1). Identify the state and action set. (2). Collect all information into the state action table. (3). Draw the corresponding state action diagram. (4). List all possible (stationary deterministic) policies. (5). For each policy, either find the corresponding transition matrix and net profit vector, or explain why this policy can never be optimal so you can ignore it. (6). Find the optimal policy with respect to the long-run expected net profit per unit time criterion. up the linear programming formulation for this MDP. (7). Set Problem 1 During each time period, a potential customer arrives at a restaurant with probability 1/2. If there are already two people at the restaurant (including the one being served), the potential customer leaves the restaurant immediately and never returns. However, if there is one person or less, he enters the restaurant and becomes an actual customer. The manager has two types of service configurations available. At the beginning of each period, a decision must be made on which configuration to use. If she uses her slow configuration at a cost of $3 and any customers are present during the period, one customer will be served and leave with probability 3/5. If she uses her fast configuration at a cost of $9 and any customers are present during the period, one customer will be served and leave with probability 4/5. The probability of more than one customer arriving or more than one customer being served in a period is zero. A profit of $50 is earned when a customer is served. The manager wants to formulate the problem of choosing the service configuration period by period as a Markov decision process. (1). Identify the state and action set. (2). Collect all information into the state action table. (3). Draw the corresponding state action diagram. (4). List all possible (stationary deterministic) policies. (5). For each policy, either find the corresponding transition matrix and net profit vector, or explain why this policy can never be optimal so you can ignore it. (6). Find the optimal policy with respect to the long-run expected net profit per unit time criterion. up the linear programming formulation for this MDP. (7). Set