Please experiment with the ALS algorithm, you can use the following notebook as starting template: https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/42740061589303/2463566369188266/8105921225255291/latest.html

Try to train and run the model on the 20 Million movie ratings dataset instead of the 1 Million one. Files:

/databricks-datasets/cs110x/ml-20m/data-001/movies.csv

/databricks-datasets/cs110x/ml-20m/data-001/ratings.csv

Test with various values of:

ranks

regularization parameter

number of iterations

Compare the models and find the best model based on the error value (i.e RMSE).