Sequential Sentence Classification with Transformer Models

Objective:

Your primary objective is to develop a Transformer$-$based model that can accurately classify sentences according to their role in the structure of biomedical research paper abstracts $($e$.$g$.,$ objective, methods, results, conclusions$).$

Dataset:

$$ Source: PubMed $20$k RCT dataset

$$ Description: This dataset includes abstracts from randomized controlled trials, with sentences labeled according to their sequential role in the abstract$.$

$$ Access: The dataset can be downloaded from the 'Files' section in a folder named 'Project Files'.

Requirements:

$1.$ Data Preparation: Load and preprocess the data, ensuring the model can effectively process it$.$ This includes tokenization, handling of special tokens, and batch preparation.

$2.$ Model Implementation:

o Implement a Transformer model for sentence classification. You may use pre$-$existing models like BERT, GPT$,$ or develop your own variant.

o Fine$-$tune the model on the dataset, ensuring it is appropriately adjusted to the task of classifying sentences in biomedical abstracts$.$

$3.$ Evaluation:

o Evaluate the model using appropriate metrics such as accuracy, F$1$ score, and confusion matrix.

o Analyze the model's performance, highlighting its strengths and weaknesses in different classification categories.

$4.$ Discussion:

o Discuss how the Transformer architecture benefits the task of sequential sentence classification.

o Compare its performance against a baseline model, such as a simple RNN or LSTM$.$

$5.$ Report:

o Include a comprehensive report detailing your methods, code, experiments, results, and discussions.

o The report should clearly articulate the reasoning behind your model choice and parameter settings.

Deliverables:

$1.$ Code: Ensure the code is well$-$commented and organized.

$2.$ Report: A detailed written report that describes your methodology, findings, and analysis of the results.

Evaluation Criteria:

$$ Model Accuracy and Robustness: How well does your model perform across different classes and abstracts$?$

$$ Quality of Presentation and Report: Organization, clarity, and professionalism of the written report.