Dataset: MovieLens $25$M Dataset $($A Subset$)$

Use Jupiter notebooks $$ and submit the notebook for review. This should include all your code and outputs.

You can download the full dataset or a smaller subset from GroupLens: https:$//$grouplens$.$org$/$datasets$/$movielens$/)$

This dataset is a collection of movie ratings and tag applications applied to movies. It's a standard dataset for recommender systems and data analysis tasks.

$$ movies.csv: Contains movie information $($movieId$,$ title, genres$)$

$$ ratings.csv: Contains user ratings $($userId$,$ movieId, rating, timestamp$)$

$$ tags.csv: Contains tags applied to movies $($userId$,$ movieId, tag, timestamp$)$

Introduction & Series

$1.$ Load the 'movies.csv$\text{'}$ file into a Pandas DataFrame and examine its structure.

$2.$ Create a Series containing the unique movie genres from the 'genres' column.

$3.$ Count the occurrences of each genre in the Series and display the top $5.$

Series Methods & Handling

$1.$ Filter the Series to include only genres containing the word 'Comedy'.

$2.$ Create a new Series by mapping the genre names to their lengths.

$3.$ Find the longest genre name in the Series.

Working with DataFrames

$1.$ Load the 'ratings.csv$\text{'}$ file into a DataFrame and display the first $10$ rows.

$2.$ Calculate the mean rating for each movie $($movieId$).$

$3.$ Identify the movies with the highest and lowest average ratings.

DataFrames In Depth

$1.$ Add a new column to the 'movies' DataFrame indicating whether a movie is a 'Comedy' or not.

$2.$ Merge the 'movies' and 'ratings' DataFrames based on the 'movieId'.

$3.$ Filter the merged DataFrame to display only movies with an average rating greater than $4\mathrm{.}0.$

Working with Multiple DataFrames

$1.$ Merge the 'movies', 'ratings', and 'tags' DataFrames to create a comprehensive dataset.

$2.$ Identify users who have rated more than $100$ movies.

$3.$ Find the most commonly used tags for movies with a rating greater than $4\mathrm{.}5.$

Going MultiDimensional $($Optional$)$

$1.($If you're comfortable with multi$-$indexing$)$ Explore creating a multi$-$indexed DataFrame with 'userId' and 'movieId' as indices.

GroupBy and Aggregates

$1.$ Group the 'ratings' DataFrame by 'userId' and calculate the mean rating for each user.

$2.$ Identify users who have given a rating of $5\mathrm{.}0$ to more than $50$ movies.

$3.$ Determine the average rating for each genre.

Reshaping with Pivots

$1.$ Create a pivot table with 'userId' as index, 'movieId' as columns, and 'rating' as values.

$2.$ Analyze the sparsity of the pivot table $($how many missing values are there?$).$

Handling Date and Time

$1.$ Convert the 'timestamp' columns in the 'ratings' and 'tags' DataFrames to datetime objects.

$2.$ Determine the most popular time of day for users to rate movies.

$3.$ Calculate the average time between a movie's release and its first rating.

Regex and Text Manipulation

$1.$ Extract the year of release from the 'title' column in the 'movies' DataFrame.

$2.$ Find movies with titles containing a specific actor's name using regular expressions.

Visualizing Data

$1.$ Create a histogram of movie ratings.

$2.$ Plot the average rating for each genre.

$3.$ Generate a scatter plot showing the relationship between the number of ratings and the average rating for each movie.

Data Formats and IO

$1.$ Save the merged DataFrame to a CSV file.