Part I $-$ Understanding the Dataset and Preprocessing $3$ marks

You will become familiar with the dataset and perform the necessary pre$-$processing steps. This includes loading the dataset, explore its structure, and prepare for subsequent modelling tasks.

Loading the Dataset $(1$ mark$)$

Load the provided dataset and display the first few rows.

Exploratory Data Analysis $(1$ mark$)$

Count the number of sentences.

Identify the unique POS tags and named entity tags.

Visualize the distribution of sentence lengths.

Visualize the frequency of different POS tags.

Data Preprocessing $(1$ mark$)$

Tokenize the sentences and map each word to its corresponding POS and named entity tags.

Convert the words and tags into numerical representations suitable for modelling, such as word indices and tag indices