I'm having this error in my code. Please share how to resolve. I tried changing userId to integer but columns cant be changed to int. #Databricks #Python

This was my initial question:

Please experiment with the ALS algorithm, you can use the following notebook as starting template: https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/42740061589303/2161112564136160/8105921225255291/latest.html

Try to train and run the model on the 20 Million movie ratings dataset instead of the 1 Million one. Files:

/databricks-datasets/cs110x/ml-20m/data-001/movies.csv

/databricks-datasets/cs110x/ml-20m/data-001/ratings.csv

Test with various values of:

ranks

regularization parameter

number of iterations

Compare the models and find the best model based on the error value (i.e RMSE).

The documenation of the algorithm can be found at: https://spark.apache.org/docs/3.3.1/ml-collaborative-filtering.html

https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.ml.recommendation.ALS.html#pyspark.ml.recommendation.ALS

Prepare a table with at least 10 of your own ratings for the movies that you select and run the model with this data as input and show the top 20 movies that the model recommends for you to watch.

(2) Spark Jobs IllegalArgumentException: requirement failed: Column userId must be of type numeric but was actually of Command took 2.73 seconds -- by hb124075@student.sgh.waw.pl at 1/19/2023,10:38:17 PM on My Cluster