Task $1$: Data cleansing $(50$pt$)-$ Week $3$

Assuming we want to run analysis using variables: Survived, Pclass, Sex, Age, Fare, Embarked, and want to prepare a dataset for the analysis. Please do the items below:

$[5$pt$]$ Create a subset of data which only contains selected variables. This should be a new Python variable only containing selected columns.

$[10$pt$]$ For each variable, please check whether it has any missing values. If yes, how many missing values are there.

$[10$pt$]$ If you see missing values on variables Survived, Pclass, Sex, Embarked, please removing the missing records.

$[10$pt$]$ For missing values on variables Age, Fare, please replace the missing value using average age and average fare.

$[10$pt$]$ If you identify any categorical variable$($s$),$ please use one$-$hot encoding to convert the variables.

$[5$pt$]$ At the end, please print the final dataset, which only contains selected variable and is well cleaned. Please indicate the number of observations in the dataset.

Task $2$: Data Summary $(15$pt$)-$ Week $4$

Data Summary using the cleaned dataset from task $1.$

$[5$pt$]$ Print the descriptive statistics table using all numerical variables.

$[5$pt$]$ Print the correlation matrix using numerical variables.

Task $3$: Regression Analysis $(35$pt$)-$ Week $4$

Please use the dataset you cleaned in task $1.$

$[25$pt$]$ Please build a linear regression model and run the analysis using Fare as dependent varaible, and others are independent variables. In the result, you should see:

The coefficient for each independent variable and the significant level

The R$-$squared for the model.

Number of observations you used to run the analysis.

$[10$pt$]$ Based on your result, which factor$($s$)$ significantly affect ticket fare and is there a negative or positive relationship? $($You can add a text block writing your answer$)$

Hint: