Suppose that in a certain state, every registered professional engineer (P.E.) is required to complete 15 h of continuing professional development (CPD) every year. CPD hours may be earned by successfully completing continuing education courses offered by a professional or trade organization, or offered in-house by a corporation. All registrants need to keep a yearly log showing the type of courses attended. Each year the State Board of Registration for Professional Engineers will audit randomly selected registrants. If selected for an audit, a registrant must submit a CPD activity log and supporting documentation.

A consulting engineer plans to construct a two-state Markov chain model of the possibility that the Board will audit a registrant’s record of continuing engineering education. The state variable is 

Historical data indicates that an engineer who was not audited last year has a 0.3 probability of being audited this year. An engineer who was audited last year has a 0.2 probability of being audited this year.

The consulting engineer is concerned that a portion of her CPD education consists of in-house courses offered by a corporation which provides engineering products and services that she uses and recommends to her clients. Critics have said that such courses are often little more than company marketing in the guise of education. If she is audited, she must demonstrate that the course provider did not bias the course material in favor of its own products and services. To prepare her CPD log and assemble supporting documentation for a possible audit at the end of the current year, she is considering hiring a prominent engineering educator as an audit consultant. The audit consultant will charge her a fee of \($2\),000. Experience indicates that the Board often imposes a fi ne for defi ciencies in CPD courses taken in prior years when a registrant is audited. Past records show that when a CPD log is prepared by an audit consultant, the average fi ne is \($600\) after a log is audited. However, when a CPD log is prepared by the registrant herself, the average fi ne is \($8\),000. The consulting engineer must decide whether or not to hire an audit consultant to prepare her CPD log for a possible audit at the end of the current year. Hiring an audit consultant appears to reduce by 0.05 the probability that a registrant will be audited in the following year: 

(a) Formulate the consulting engineer’s decision alternatives as a two-state recurrent MDP. For each state and decision, specify the associated transition probabilities and calculate the expected immediate cost.

(b) Execute value iteration to find a policy which minimizes the vector of expected total costs that will be incurred in both states after 3 years. Assume zero terminal costs at the end of year 3. 

(c) Use exhaustive enumeration to find a policy that minimizes the expected average cost, or negative gain, over an infinite planning horizon.

(d) Use PI to find an optimal policy over an infinite horizon. 

Transcribed Image Text:

X,, = 1 if a registrant was audited at the end of year n, X,, = 0 if a registrant was not audited at the end of year n.