Practical data Science with Python Assignment Assessment : Recommender Systems This assignment focuses on recommender systems, a data science application widely used in the real world. You will need to develop and implement appropriate solutions to complete the corresponding tasks and present the results (virtually). These tasks must be completed individually. Introduction This assignment focuses on recommender systems, a data science application widely used in the real world. You will need to develop and implement appropriate solutions to complete the corresponding tasks, and present the results (virtually). These tasks must be completed individually. Academic Integrity All assignments will be checked with plagiarism-detection software; any student found to have plagiarised will be subject to disciplinary actions. Plagiarism includes, e.g., submitting code that is not your own or submitting text that is not your own. Allowing others to copy your work is also considered as plagiarism. All plagiarisms will be penalised; there are no exceptions and no excuses. The Dataset This assignment deals with movie recommendation. The dataset to be used throughout the assignment is the MovieLens 1M Dataset. Task 1: User-based Collaborative Filtering (6 marks) In this task you need to develop, implement, and evaluate user-based (i.e., user-user) collaborative filtering that uses KNN (k-nearest neighbour), i.e., KNN-based Collaborative Filtering. Randomly choose one user (as the active user) and predict this users ratings on movies. Note that in the given dataset, the user might have only rated some of the movies. Specific requirements include:

• Choose an appropriate similarity metric (and other parameters).
• Implement the approach in Python. Add detailed comments for the code (to explain
• your implementation).
• Study the impact of the parameter K (of KNN), with at least 5 different values.
• Use RMSE (root-mean-square error) as the metric for evaluation.
• Summarise the results/findings concisely in slides (and presentation).

Tips on creating comments in Python: https://www.w3schools.com/python/python comments.asp. Task 2: Item-based Filtering (6 marks) In this task you need to develop, implement, and evaluate item-based (i.e., item-item) (collaborative) filtering that uses KNN. Randomly choose one item (i.e., movie) and predict users ratings on this movie. Note that in the given dataset, it is possible that some users didnt rate this movie. Specific requirements include:

• Choose an appropriate value for the parameter K (of KNN) (and other parameters).
• Implement the approach in Python. Add detailed comments for the code (to explain
• your implementation).
• Compare the performance of at least 2 similarity metrics.
• Use RMSE (root-mean-square error) as the metric for evaluation.
• Summarise the results/findings concisely in slides (and presentation).

Task 3: A Better Recommender System (12 marks) In this task you are required to identify/propose a better solution to improve the recommendation quality and evaluate its performance in comparison with two baseline methods. Task 3.1. Develop the new solution for movie recommendation. This can be done in either of the following ways: Option 1: Search and read related publications on recommender systems. Identify/choose one solution that can be used for movie recommendation and will potentially deliver better recommendation quality. The solution must have been published in a peer-reviewed journal or conference. It can be based on KNN or not. Option 2: Based on an extensive literature review (conducted by yourself), propose a new algorithm by yourself. In this case, the idea is yours. You need to justify its originality and novelty. A strong list of references must be provided. Note that parameter tuning only cannot be considered as a (new) approach.

• Describe the idea (in your own words) clearly and precisely in slides (and presentation). Cite references wherever necessary.
• In slides, explicitly give the source of the solution (if you choose Option 1) or provide

a strong list of references (Option 2).

• Name the solution as Option1RecSys if you choose Option 1 or Option2RecSys for

Option 2.

• Implement the approach in Python. Add detailed comments for the code (to explain

your implementation as well as details of the algorithm). Task 3.2. Randomly choose 5 users (from who have rated more than 100 movies each) and recommend Top-30 movies (to each user). Use AP (Average Precision) and NDCG (Normalized Discounted Cumulative Gain) as evaluation metrics. Compare the performance of three solutions:

• Movie Average: recommends items with the highest average ratings (see also Week

9 lec slides). Name it as MovieAvg.

• KNN-based Collaborative Filtering: the approach developed in the above Task 1.

Choose the optimal parameters (based on results of Task 1). Name it as KNNCF.

• The solution developed in the above Task 3.1, i.e., Option1RecSys or Option2RecSys.

Use one appropriate graph/chart to visualise the results of each metric, respectively (i.e., one graph for AP and another for NDCG). Summarise the results/findings concisely in slides (and presentation). ** IMPORTANT** It is required that the solution developed in Task 3.1 (Option1RecSys or Option2RecSys) must achieve the best (overall) performance; if this is not the case, your whole Task 3 will be considered as FAIL (as per the rubric) Task 4: Presentation (6 marks) In this assignment, you need to design slides and record a presentation (for the above tasks and results). Your slides and presentation could include, e.g., but not limited to:

• a cover page/slide containing e.g., your name and student ID, in addition to Assignment info,
• a concise outline, key results, and findings of Task 1,
• a concise outline, key results, and findings of Task 2,
• clear and complete description of the new solution developed in Task 3.1 (with
• proper citations),
• literature review (with proper citations), if applicable,
• necessary/key details of the algorithm of Option1RecSys or Option2RecSys,
• 4
• key results, visualisation, and findings of Task 3.2,
• a list of references.

The following requirements must be strictly met; otherwise, your submission will be considered invalid.

• The slides should be no more than 10 pages in total (that is, no more than 10 slides)
• Save your clean slides as a PDF file and name it A3Slides.pdf. This version should
• not contain any recordings therein.
• The presentation (recording) should be no more than 5 minutes.
• The recording video file must be in MP4 format. Name it A3Presentation.mp4.
• The recording video file must be less than 30MB in size.

Assignment Rubrics

Assignment 3 (Sem 2, 2023)

Criteria

Ratings

Pts

This criterion is linked to a learning outcomeTask 1 User-based Collaborative Filtering

6to >4.8Pts HIGH DISTINCTION All parameters are properly set. The implementation is completely free from errors. Details of the implementation are clear. The code is well commented to explain the implementation. All comments are necessary and precise. The impact of the key parameter is studied adequately with strong support of well-organised numeric results. The correct metric is used. All calculations are free from errors. The results are well displayed and presented, and the findings are summarised correctly and concisely.

4.8to >4.2Pts DISTINCTION All parameters are properly set. The implementation is free from (substantial) errors. Details of the implementation are clear. The code is well commented to explain the implementation. All comments are appropriate. The impact of the key parameter is studied with strong support of numeric results. The correct metric is used. All calculations are free from errors. The results are displayed and presented appropriately, and the findings are summarised correctly.

4.2to >3.6Pts CREDIT All parameters are correctly set. The implementation is free from substantial errors. Most details of the implementation are clear. The code is appropriately commented to explain the implementation. All comments are correct. The impact of the key parameter is studied with support of numeric results. The correct metric is used. All calculations are free from errors. The results are displayed and presented appropriately, and the findings are summarised correctly.

3.6to >3.0Pts PASS All parameters are correctly set. The implementation is free from substantial errors. The code is commented to explain the implementation. All comments are somewhat correct. The impact of the key parameter is studied with support of numeric results. The correct metric is used. All calculations are free from substantial errors. The results are displayed and presented in some way, and the findings are summarised correctly.

3to >0Pts FAIL There are issues with parameter settings, or errors in the implementation or comments. There are no sufficient comments. The study of the impact of the key parameter is (somewhat) insufficient or inappropriate. Key quantitative results are missing or incorrect. Calculations have errors. The results are displayed or presented inappropriately. Or, the findings are not summarised correctly.

6pts

This criterion is linked to a learning outcomeTask 2 Item-based Filtering

6to >4.8Pts HIGH DISTINCTION All parameters are properly set. The implementation is completely free from errors. Details of the implementation are clear. The code is well commented to explain the implementation. All comments are necessary and precise. The performance resulting from different similarity metrics is compared adequately with strong support of well-organised numeric results. The correct metric is used. All calculations are free from errors. The results are well displayed and presented, and the findings are summarised correctly and concisely.

4.8to >4.2Pts DISTINCTION All parameters are properly set. The implementation is free from (substantial) errors. Details of the implementation are clear. The code is well commented to explain the implementation. All comments are appropriate. The performance resulting from different similarity metrics is compared with strong support of numeric results. The correct metric is used. All calculations are free from errors. The results are displayed and presented appropriately, and the findings are summarised correctly.

4.2to >3.6Pts CREDIT All parameters are correctly set. The implementation is free from substantial errors. Most details of the implementation are clear. The code is appropriately commented to explain the implementation. All comments are correct. The performance resulting from different similarity metrics is compared with support of numeric results. The correct metric is used. All calculations are free from errors. The results are displayed and presented appropriately, and the findings are summarised correctly.

3.6to >3.0Pts PASS All parameters are correctly set. The implementation is free from substantial errors. The code is commented to explain the implementation. All comments are somewhat correct. The performance resulting from different similarity metrics is compared with support of numeric results. The correct metric is used. All calculations are free from substantial errors. The results are displayed and presented in some way, and the findings are summarised correctly.

3to >0Pts FAIL There are issues with parameter settings, or errors in the implementation or comments. There are no sufficient comments. The comparison of similarity metrics is (somewhat) insufficient or inappropriate. Key quantitative results are missing or incorrect. Calculations have errors. The results are displayed or presented inappropriately. Or, the findings are not summarised correctly.

6pts

This criterion is linked to a learning outcomeTask 3 A Better Recommender System

12to >9.6Pts HIGH DISTINCTION The idea (of the "new" solution) is clearly and precisely described. All references are properly cited. The source of the solution is clearly specified, if applicable. Key details of the algorithm are provided. All methods are named as required. All parameters are properly set. The implementation of all methods is completely free from errors. Details of the implementation are clear. The code is well commented. All comments are necessary and precise. All settings for performance evaluation are reasonable and properly justified or explained. The performance of different solutions is compared adequately with strong support of well-organised numeric results. All calculations are free from errors. The results are well displayed, visualised, and presented. The findings are summarised correctly and concisely. The "new" solutions achieves the best results.

9.6to >8.4Pts DISTINCTION The idea (of the "new" solution) is clearly and precisely described. All references are properly cited. The source of the solution is clearly specified, if applicable. Key details of the algorithm are provided. All methods are named as required. All parameters are properly set. The implementation of all methods is free from (substantial) errors. Details of the implementation are clear. The code is well commented. All comments are appropriate. All settings for performance evaluation are reasonable. The performance of different solutions is compared with strong support of numeric results. All calculations are free from errors. The results are displayed, visualised, and presented appropriately. The findings are summarised correctly. The "new" solutions achieves the best results.

8.4to >7.2Pts CREDIT The idea (of the "new" solution) is clearly described. All references are properly cited. The source of the solution is clearly specified, if applicable. Key details of the algorithm are provided. All methods are named as required. All parameters are correctly set. The implementation of all methods is free from substantial errors. Most details of the implementation are clear. The code is commented. All comments are correct. All settings for performance evaluation are reasonable. The performance of different solutions is compared with support of numeric results. All calculations are free from errors. The results are displayed, visualised, and presented appropriately. The findings are summarised correctly. The "new" solutions achieves the best results.

7.2to >6.0Pts PASS The idea (of the "new" solution) is described in some way. All references are (properly) cited. The source of the solution is specified, if applicable. Key details of the algorithm are provided. All methods are named as required. All parameters are correctly set. The implementation of all methods is free from substantial errors. The code is commented in some way. All comments are somewhat correct. All settings for performance evaluation are somewhat reasonable. The performance of different solutions is compared with support of numeric results. All calculations are free from substantial errors. The results are displayed, visualised, and presented in some way. The findings are summarised correctly. The "new" solutions achieves the best results.

6to >0Pts FAIL The idea (of the "new" solution) is (somewhat) unclear. References or citations are missing. The source of the solution is unclear, if applicable. Key details of the algorithm are missing. Methods are not named as required. There are issues with parameter settings, or errors in the implementation or comments. There are no sufficient comments. Performance comparison is (somewhat) insufficient or inappropriate. Key quantitative results are missing or incorrect. Calculations have errors. The results are displayed, visualised, or presented inappropriately. The findings are not summarised correctly. Or, the "new" solutions doesn't achieve the best results.

12pts

This criterion is linked to a learning outcomeTask 4 Presentation

6to >4.8Pts HIGH DISTINCTION All requirements for slides and presentation (specified in the Assignment document) are met. The contents are well designed. Presentation is well structured. Organisation of information and presentation are clear, logical, appropriately sequenced and consistent. No obvious grammar or spelling mistakes.

4.8to >4.2Pts DISTINCTION All requirements for slides and presentation (specified in the Assignment document) are met. The contents are adequate and properly formatted, and all necessary elements are included. Presentation is largely well structured. Organisation of information and presentation are clear, consistent and appropriately sequenced. No obvious grammar or spelling mistakes.

4.2to >3.6Pts CREDIT All requirements for slides and presentation (specified in the Assignment document) are met. All necessary elements are included and properly formatted. Presentation is largely well structured. Organisation of information and presentation are clear, and mostly appropriately-sequenced. Some grammar or spelling mistakes.

3.6to >3.0Pts PASS All requirements for slides and presentation (specified in the Assignment document) are met. Most necessary elements are included and properly formatted. Presentation is structured in some way. Organisation of information and presentation are somewhat appropriate. Some grammar or spelling mistakes.

3to >0Pts FAIL Not all requirements for slides and presentation (specified in the Assignment document) are met. Key contents/elements are missing, or the format is poor. Presentation is hard to follow. Organisation of information or presentation is inappropriate. Or, many grammar or spelling mistakes.

6pts

Total points:30

Attachments:

Assignment-de....docx

assignment3-T....ipynb

ml-1m--1-.zip