- input: The review text is converted into numerical features using TF-IDF, which captures the importance of words in the review.
- Output (Target): The score is the numerical rating associated with the review, typically on a scale (e.g., 1 to 5 stars).
The model is trained to understand how the content and characteristics of reviews relate to the numerical ratings users assign to movies.
Show Code
import pandas as pdfrom sklearn.model_selection import train_test_splitfrom sklearn.feature_extraction.text import TfidfVectorizerfrom sklearn.ensemble import RandomForestClassifierfrom sklearn.metrics import classification_report, accuracy_scoredf = pd.read_csv("../../data/processed-data/frozen_processed_data.csv")# Ensure the target variable is of integer typedf['score'] = df['score'].astype(int)# Convert review date to timestampdf['review_time'] = pd.to_datetime(df['date']).view('int64') /10**9# Ensure review text is non-null and of string typedf['review_text'] = df['review_text'].fillna("").astype(str)# Add a feature for the length of each reviewdf['review_length'] = df['review_text'].apply(len)# Extract text features using TF-IDFtfidf = TfidfVectorizer(max_features=500)text_features = tfidf.fit_transform(df['review_text']).toarray()# Combine TF-IDF features with other engineered featuresfeatures = pd.DataFrame(text_features)features['sentiment_score'] = df['sentiment_score']features['review_time'] = df['review_time']features['review_length'] = df['review_length']# Ensure all column names are stringsfeatures.columns = features.columns.astype(str)# Split the dataset into training and testing setsX = featuresy = df['score']X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)# Train a Random Forest Classifierclf = RandomForestClassifier(random_state=42)clf.fit(X_train, y_train)# Predict the test sety_pred = clf.predict(X_test)# Evaluate model performanceprint("Accuracy:", accuracy_score(y_test, y_pred))print("Classification Report:\n", classification_report(y_test, y_pred))
C:\Users\tongrui\AppData\Local\Temp\ipykernel_43356\1799330441.py:12: FutureWarning: Series.view is deprecated and will be removed in a future version. Use ``astype`` as an alternative to change the dtype.
df['review_time'] = pd.to_datetime(df['date']).view('int64') / 10**9
The primary objectives of this part of the project are to predict numerical scores using a combination of textual and numerical data, extract meaningful text features with TF-IDF for machine learning, optimize a Random Forest Regressor’s performance through GridSearchCV for hyperparameter tuning, and evaluate the model’s performance using standard metrics like Mean Squared Error (MSE) and a custom accuracy function.
To start, the text data is converted into numerical representations using TF-IDF (Term Frequency-Inverse Document Frequency), which captures the importance of specific words in the reviews. This process limits the feature space to the most significant 500 terms. Next, several numerical features are incorporated: the review date is converted into a numerical timestamp, the length of each review in characters is added, and pre-existing sentiment values are integrated into the feature set. This combination of textual and numerical features creates a robust input for the model.
The dataset is split into training and testing subsets with an 80/20 split to evaluate the model’s generalizability. A Random Forest Regressor, an ensemble learning method that builds multiple decision trees, is employed to make robust predictions. To further enhance the model, GridSearchCV is used for hyperparameter tuning, testing combinations of parameters such as the number of trees, maximum tree depth, and minimum data requirements for splits. Cross-validation ensures the model generalizes well to unseen data.
The optimized model’s performance is assessed using Mean Squared Error (MSE), which measures the average squared difference between predicted and actual scores, indicating overall regression accuracy. Additionally, a custom accuracy function evaluates how many predictions fall within a predefined range (e.g., ±1) of the actual score, providing an intuitive evaluation of prediction quality beyond standard errors.
Ultimately, the function returns a summary of the model’s performance, including the best hyperparameters, custom accuracy, and Mean Squared Error, thus offering a comprehensive evaluation of its effectiveness.
Show Code
import pandas as pdfrom sklearn.model_selection import train_test_split, GridSearchCVfrom sklearn.feature_extraction.text import TfidfVectorizerfrom sklearn.ensemble import RandomForestRegressorfrom sklearn.metrics import mean_squared_errorimport numpy as npdef train_and_evaluate_model(file_path):""" Trains a Random Forest model on the dataset provided in file_path and evaluates its performance using GridSearchCV. Args: file_path (str): Path to the dataset CSV file. Returns: dict: A dictionary containing the best parameters, custom accuracy, and MSE. """# Load data df = pd.read_csv(file_path) df['score'] = df['score'].astype(float)# Feature engineering df['review_time'] = pd.to_datetime(df['date']).view('int64') /10**9# Convert to timestamp df['review_text'] = df['review_text'].fillna("").astype(str) df['review_length'] = df['review_text'].apply(len)# Extract TF-IDF features tfidf = TfidfVectorizer(max_features=500) text_features = tfidf.fit_transform(df['review_text']).toarray()# Combine features features = pd.DataFrame(text_features) features['sentiment_score'] = df['sentiment_score'] features['review_time'] = df['review_time'] features['review_length'] = df['review_length'] features.columns = features.columns.astype(str)# Split data X = features y = df['score'] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)# Define the parameter grid param_grid = {'n_estimators': [100, 200, 300], # Number of trees in the forest'max_depth': [None, 10, 20], # Maximum depth of each tree'min_samples_split': [2, 5, 10], # Minimum samples required to split a node'min_samples_leaf': [1, 2, 4], # Minimum samples required at a leaf node }# Initialize the RandomForestRegressor rf = RandomForestRegressor(random_state=42)# Perform grid search grid_search = GridSearchCV( estimator=rf, param_grid=param_grid, cv=3, scoring='neg_mean_squared_error', verbose=0, n_jobs=-1 ) grid_search.fit(X_train, y_train)# Get the best model best_rf = grid_search.best_estimator_# Predict with the best model y_pred = best_rf.predict(X_test)# Custom accuracy functiondef custom_accuracy(y_true, y_pred, tolerance=1): correct =sum(abs(y_t - y_p) <= tolerance for y_t, y_p inzip(y_true, y_pred))return correct /len(y_true)# Evaluate the best model accuracy = custom_accuracy(y_test, y_pred) mse = mean_squared_error(y_test, y_pred)# Return the results as a dictionaryreturn {"Best Parameters": grid_search.best_params_,"Custom Accuracy": accuracy,"Mean Squared Error": mse }
Best Parameters: {'max_depth': 10, 'min_samples_leaf': 1, 'min_samples_split': 10, 'n_estimators': 300}
Custom Accuracy : 0.88
Mean Squared Error: 0.73
Cultural and Feminism Sentiment Analysis
For the following part, we define two sets of keywords to search within reviews:
Cultural Keywords: Words related to culture, representation, heritage, and equity (e.g., diverse, multicultural, marginalized).
Feminism Keywords: Words related to gender equality, feminism, and female representation (e.g., empower, gender equality, female protagonist).
These keywords are used to filter reviews, then the following codes automate the analysis of cultural and feminism themes in movie reviews by searching for relevant keywords and summarizing results into an easy-to-read table. It is useful for understanding trends in movie reviews related to cultural representation and gender equality, and the overall sentiment associated with these themes.
Show Code
import pandas as pddef analyze_sentiments_by_keywords_to_table(file_paths):""" Analyzes sentiments tied to cultural and feminism keywords in multiple files and outputs the results as a structured table. Args: file_paths (list): List of file paths to CSV datasets. Returns: pd.DataFrame: A DataFrame containing summarized results for all movies. """# Define extended keyword lists cultural_keywords = ["empowering", "diverse", "stereotype", "inclusive", "representation", "heritage", "tradition","identity", "multicultural", "equity", "integration", "underrepresented", "minority", "cultural values","discrimination", "privilege", "cross-cultural", "marginalized", "social justice", "assimilation" ] feminism_keywords = ["empower", "independent", "equality", "gender", "strong female", "feminism", "women rights","patriarchy", "misogyny", "glass ceiling", "equal pay", "female protagonist", "sexism", "intersectional","women empowerment", "gender equality", "matriarchy", "gender roles", "strong woman", "female lead" ]# Initialize results list results = []# Function to analyze a single filedef analyze_file(file_path): df = pd.read_csv(file_path) df['cleaned_review'] = df['cleaned_review'].fillna("").astype(str)# Function to analyze keywordsdef keyword_analysis(keywords): filtered_reviews = df[df['cleaned_review'].str.contains('|'.join(keywords), case=False, na=False)] total_reviews =len(filtered_reviews) avg_sentiment_score = filtered_reviews['sentiment_score'].mean() if total_reviews >0else0return total_reviews, avg_sentiment_score# Analyze cultural and feminism keywords cultural_total, cultural_avg = keyword_analysis(cultural_keywords) feminism_total, feminism_avg = keyword_analysis(feminism_keywords)# Extract movie name movie_name = file_path.split("/")[-1].replace("_processed_data.csv", "").replace("_", " ").title()# Append results results.append({"Movie": movie_name,"Cultural Reviews": cultural_total,"Avg. Cultural Sentiment": round(cultural_avg, 2),"Feminism Reviews": feminism_total,"Avg. Feminism Sentiment": round(feminism_avg, 2) })# Process all file pathsfor file_path in file_paths: analyze_file(file_path)# Convert results to DataFrame results_df = pd.DataFrame(results)return results_df# List of file pathsfile_paths = ["../../data/processed-data/snow_white_processed_data.csv", # Snow White and the Seven Dwarfs (1937)"../../data/processed-data/cinderella_processed_data.csv", # Cinderella (1950)"../../data/processed-data/sleeping_beauty_processed_data.csv", # Sleeping Beauty (1959)"../../data/processed-data/little_mermaid_processed_data.csv", # The Little Mermaid (1989)"../../data/processed-data/beauty_and_the_beast_processed_data.csv", # Beauty and the Beast (1991)"../../data/processed-data/aladdin_processed_data.csv", # Aladdin (1992)"../../data/processed-data/pocahontas_processed_data.csv", # Pocahontas (1995)"../../data/processed-data/mulan_processed_data.csv", # Mulan (1998)"../../data/processed-data/princess_and_the_frog_processed_data.csv", # The Princess and the Frog (2009)"../../data/processed-data/tangled_processed_data.csv", # Tangled (2010)"../../data/processed-data/brave_processed_data.csv", # Brave (2012)"../../data/processed-data/frozen_processed_data.csv", # Frozen (2013)"../../data/processed-data/moana_processed_data.csv", # Moana (2016)"../../data/processed-data/moana_2_processed_data.csv"]# Generate and display the results tableresults_table = analyze_sentiments_by_keywords_to_table(file_paths)results_table
Movie
Cultural Reviews
Avg. Cultural Sentiment
Feminism Reviews
Avg. Feminism Sentiment
0
Snow White
9
0.15
5
0.13
1
Cinderella
7
0.09
5
0.06
2
Sleeping Beauty
8
0.22
3
0.32
3
Little Mermaid
7
0.22
7
0.24
4
Beauty And The Beast
3
0.28
1
0.41
5
Aladdin
8
0.25
3
0.27
6
Pocahontas
13
0.13
5
0.23
7
Mulan
35
0.24
75
0.31
8
Princess And The Frog
86
0.21
18
0.21
9
Tangled
26
0.20
9
0.27
10
Brave
25
0.16
25
0.21
11
Frozen
19
0.16
20
0.24
12
Moana
30
0.26
27
0.28
13
Moana 2
7
0.37
3
0.29
Show Code
import matplotlib.pyplot as pltplt.figure(figsize=(12, 6))plt.plot(results_table["Movie"], results_table["Avg. Cultural Sentiment"], marker='o', label='Avg. Cultural Sentiment', linestyle='-')plt.plot(results_table["Movie"], results_table["Avg. Feminism Sentiment"], marker='o', label='Avg. Feminism Sentiment', linestyle='--')plt.title('Comparison of Average Cultural and Feminism Sentiments')plt.xlabel('Movies')plt.ylabel('Average Sentiment Score')plt.xticks(rotation=45)plt.legend()plt.grid(True)plt.tight_layout()plt.show()
The table summarizes the frequency and average sentiment scores of reviews containing cultural and feminist-related keywords for 14 Disney princess movies. In terms of cultural keywords, Moana 2 (2024) and Moana (2016) stand out with higher average cultural sentiment scores of 0.37 and 0.26, respectively, indicating a more positive cultural representation. However, older movies like Cinderella (1950) and Snow White and the Seven Dwarfs (1937) have relatively lower cultural sentiment averages (0.09 and 0.15) alongside fewer cultural reviews, which may reflect dated representations of culture. Notably, The Princess and the Frog (2009) received the highest number of cultural-related reviews (86) with a sentiment average of 0.21, highlighting its cultural themes but with mixed reactions.
For feminist keywords, Mulan (1998) leads with the highest number of feminism-related reviews (75) and a relatively strong average sentiment score of 0.31, suggesting that audiences recognize and appreciate its portrayal of female empowerment. Meanwhile, Beauty and the Beast (1991) has the highest average feminist sentiment score of 0.41, despite having only one related review, indicating extremely positive feedback for its feminist themes. Movies like Sleeping Beauty (1959) and Brave (2012) also receive moderate attention to feminist themes, with sentiment averages of 0.32 and 0.21, respectively.
Overall, modern Disney movies such as Moana, Frozen, and Brave exhibit greater engagement and positive sentiment around cultural and feminist representations compared to earlier films like Snow White and Cinderella. This trend reflects Disney’s evolving portrayal of diversity and female empowerment, aligning with contemporary audience expectations.