Dowload data set WomenWage.csv from the book's website. This data set is a tidied-up version of the women's wages data set from [91]. The first column of the data (hours) is the response variable \(Y\). It shows the hours spent in the labor force by married women in the 1970s. We want to understand what factors determine the participation rate of women in the labor force. The predictor variables are:

We observe that some of the responses are \(Y=0\), that is, some women did not participate in the labor force. For this reason, we model the data using the Tobit regression model, in which the response \(Y\) is given as:
\[ Y_{i}=\left\{\begin{array}{ll} Z_{i}, & \text { if } Z_{i}>0 \\ 0, & \text { if } Z_{i} \leqslant 0 \end{array} \quad \boldsymbol{Z} \sim \mathscr{N}\left(\mathbf{X} \boldsymbol{\beta}, \sigma^{2} \mathbf{I}_{n}\right)\right. \]
With \(\boldsymbol{\theta}=\left(\boldsymbol{\beta}, \sigma^{2}\right)\), the likelihood of the data \(\boldsymbol{y}=\left[y_{1}, \ldots, y_{n}\right]^{\top}\) is:
\[ g(\boldsymbol{y} \mid \boldsymbol{\theta})=\prod_{i: y_{i}>0} \varphi_{\sigma^{2}}\left(y_{i}-\boldsymbol{x}_{i}^{\top} \boldsymbol{\beta}\right) \times \prod_{i: y_{i}=0} \Phi\left(\left(u_{i}-\boldsymbol{x}_{i}^{\top} \boldsymbol{\beta}\right) / \sigma\right) \]
where \(\Phi\) is the standard normal cdf. In Exercise 8, we derived the EM algorithm for maximizing the log-likelihood.

(a) Write down the EM algorithm in pseudo code as it applies to this Tobit regression.

(b) Implement the EM algorithm pseudo code in Python. Comment on which factor you think is important in determining the labor participation rate of women living in the USA in the 1970s.

Transcribed Image Text:

Feature kidslt6 kidsge6 age educ exper nwifeinc expersq Table 5.7: Features for the women's wage data set. Description Number of children younger than 6 years. Number of children older than 6 years. Age of the married woman. Number of years of formal education. Number of years of "work experience". Non-wife income, that is, the income of the husband. The square of exper, to capture any nonlinear relationships.