Supervised Learning


Regression Task: Predicts the score (rating)

- input: The review text is converted into numerical features using TF-IDF, which captures the importance of words in the review.
- Output (Target): The score is the numerical rating associated with the review, typically on a scale (e.g., 1 to 5 stars).

The model is trained to understand how the content and characteristics of reviews relate to the numerical ratings users assign to movies.

Show Code
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, accuracy_score

df = pd.read_csv("../../data/processed-data/frozen_processed_data.csv")
# Ensure the target variable is of integer type
df['score'] = df['score'].astype(int)

# Convert review date to timestamp
df['review_time'] = pd.to_datetime(df['date']).view('int64') / 10**9
# Ensure review text is non-null and of string type
df['review_text'] = df['review_text'].fillna("").astype(str)
# Add a feature for the length of each review
df['review_length'] = df['review_text'].apply(len)

# Extract text features using TF-IDF
tfidf = TfidfVectorizer(max_features=500)
text_features = tfidf.fit_transform(df['review_text']).toarray()

# Combine TF-IDF features with other engineered features
features = pd.DataFrame(text_features)
features['sentiment_score'] = df['sentiment_score']
features['review_time'] = df['review_time']
features['review_length'] = df['review_length']

# Ensure all column names are strings
features.columns = features.columns.astype(str)

# Split the dataset into training and testing sets
X = features
y = df['score']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train a Random Forest Classifier
clf = RandomForestClassifier(random_state=42)
clf.fit(X_train, y_train)

# Predict the test set
y_pred = clf.predict(X_test)

# Evaluate model performance
print("Accuracy:", accuracy_score(y_test, y_pred))
print("Classification Report:\n", classification_report(y_test, y_pred))
C:\Users\tongrui\AppData\Local\Temp\ipykernel_43356\1799330441.py:12: FutureWarning: Series.view is deprecated and will be removed in a future version. Use ``astype`` as an alternative to change the dtype.
  df['review_time'] = pd.to_datetime(df['date']).view('int64') / 10**9
Accuracy: 0.4375
Classification Report:
               precision    recall  f1-score   support

           0       0.46      0.43      0.44        14
           1       0.50      0.14      0.22         7
           2       1.00      0.05      0.09        22
           3       0.53      0.47      0.49        45
           4       0.32      0.23      0.26        53
           5       0.44      0.84      0.58        51

    accuracy                           0.44       192
   macro avg       0.54      0.36      0.35       192
weighted avg       0.49      0.44      0.39       192

The primary objectives of this part of the project are to predict numerical scores using a combination of textual and numerical data, extract meaningful text features with TF-IDF for machine learning, optimize a Random Forest Regressor’s performance through GridSearchCV for hyperparameter tuning, and evaluate the model’s performance using standard metrics like Mean Squared Error (MSE) and a custom accuracy function.

To start, the text data is converted into numerical representations using TF-IDF (Term Frequency-Inverse Document Frequency), which captures the importance of specific words in the reviews. This process limits the feature space to the most significant 500 terms. Next, several numerical features are incorporated: the review date is converted into a numerical timestamp, the length of each review in characters is added, and pre-existing sentiment values are integrated into the feature set. This combination of textual and numerical features creates a robust input for the model.

The dataset is split into training and testing subsets with an 80/20 split to evaluate the model’s generalizability. A Random Forest Regressor, an ensemble learning method that builds multiple decision trees, is employed to make robust predictions. To further enhance the model, GridSearchCV is used for hyperparameter tuning, testing combinations of parameters such as the number of trees, maximum tree depth, and minimum data requirements for splits. Cross-validation ensures the model generalizes well to unseen data.

The optimized model’s performance is assessed using Mean Squared Error (MSE), which measures the average squared difference between predicted and actual scores, indicating overall regression accuracy. Additionally, a custom accuracy function evaluates how many predictions fall within a predefined range (e.g., ±1) of the actual score, providing an intuitive evaluation of prediction quality beyond standard errors.

Ultimately, the function returns a summary of the model’s performance, including the best hyperparameters, custom accuracy, and Mean Squared Error, thus offering a comprehensive evaluation of its effectiveness.

Show Code
import pandas as pd
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_squared_error
import numpy as np

def train_and_evaluate_model(file_path):
    """
    Trains a Random Forest model on the dataset provided in file_path
    and evaluates its performance using GridSearchCV.

    Args:
        file_path (str): Path to the dataset CSV file.

    Returns:
        dict: A dictionary containing the best parameters, custom accuracy, and MSE.
    """
    # Load data
    df = pd.read_csv(file_path)
    df['score'] = df['score'].astype(float)

    # Feature engineering
    df['review_time'] = pd.to_datetime(df['date']).view('int64') / 10**9  # Convert to timestamp
    df['review_text'] = df['review_text'].fillna("").astype(str)
    df['review_length'] = df['review_text'].apply(len)

    # Extract TF-IDF features
    tfidf = TfidfVectorizer(max_features=500)
    text_features = tfidf.fit_transform(df['review_text']).toarray()

    # Combine features
    features = pd.DataFrame(text_features)
    features['sentiment_score'] = df['sentiment_score']
    features['review_time'] = df['review_time']
    features['review_length'] = df['review_length']
    features.columns = features.columns.astype(str)

    # Split data
    X = features
    y = df['score']
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

    # Define the parameter grid
    param_grid = {
        'n_estimators': [100, 200, 300],  # Number of trees in the forest
        'max_depth': [None, 10, 20],      # Maximum depth of each tree
        'min_samples_split': [2, 5, 10], # Minimum samples required to split a node
        'min_samples_leaf': [1, 2, 4],   # Minimum samples required at a leaf node
    }

    # Initialize the RandomForestRegressor
    rf = RandomForestRegressor(random_state=42)

    # Perform grid search
    grid_search = GridSearchCV(
        estimator=rf,
        param_grid=param_grid,
        cv=3,
        scoring='neg_mean_squared_error',
        verbose=0,
        n_jobs=-1
    )
    grid_search.fit(X_train, y_train)

    # Get the best model
    best_rf = grid_search.best_estimator_

    # Predict with the best model
    y_pred = best_rf.predict(X_test)

    # Custom accuracy function
    def custom_accuracy(y_true, y_pred, tolerance=1):
        correct = sum(abs(y_t - y_p) <= tolerance for y_t, y_p in zip(y_true, y_pred))
        return correct / len(y_true)

    # Evaluate the best model
    accuracy = custom_accuracy(y_test, y_pred)
    mse = mean_squared_error(y_test, y_pred)

    # Return the results as a dictionary
    return {
        "Best Parameters": grid_search.best_params_,
        "Custom Accuracy": accuracy,
        "Mean Squared Error": mse
    }

Aladdin

Show Code
results = train_and_evaluate_model("../../data/processed-data/aladdin_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': None, 'min_samples_leaf': 4, 'min_samples_split': 10, 'n_estimators': 100}
Custom Accuracy : 0.88
Mean Squared Error: 0.81

Beauty and the Beast

Show Code
results = train_and_evaluate_model("../../data/processed-data/beauty_and_the_beast_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': None, 'min_samples_leaf': 2, 'min_samples_split': 10, 'n_estimators': 300}
Custom Accuracy : 0.89
Mean Squared Error: 1.04

Brave

Show Code
results = train_and_evaluate_model("../../data/processed-data/brave_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 10, 'min_samples_leaf': 1, 'min_samples_split': 10, 'n_estimators': 300}
Custom Accuracy : 0.80
Mean Squared Error: 0.82

Cinderella

Show Code
results = train_and_evaluate_model("../../data/processed-data/cinderella_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 20, 'min_samples_leaf': 2, 'min_samples_split': 2, 'n_estimators': 200}
Custom Accuracy : 0.81
Mean Squared Error: 0.68

Frozen

Show Code
results = train_and_evaluate_model("../../data/processed-data/frozen_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 10, 'min_samples_leaf': 4, 'min_samples_split': 2, 'n_estimators': 200}
Custom Accuracy : 0.76
Mean Squared Error: 0.94

Little Mermaid

Show Code
results = train_and_evaluate_model("../../data/processed-data/little_mermaid_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 20, 'min_samples_leaf': 4, 'min_samples_split': 2, 'n_estimators': 300}
Custom Accuracy : 0.83
Mean Squared Error: 0.67

Moana

Show Code
results = train_and_evaluate_model("../../data/processed-data/moana_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 10, 'min_samples_leaf': 1, 'min_samples_split': 2, 'n_estimators': 300}
Custom Accuracy : 0.81
Mean Squared Error: 0.73

Mulan

Show Code
results = train_and_evaluate_model("../../data/processed-data/mulan_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 20, 'min_samples_leaf': 1, 'min_samples_split': 10, 'n_estimators': 300}
Custom Accuracy : 0.87
Mean Squared Error: 0.62

Pocahontas

Show Code
results = train_and_evaluate_model("../../data/processed-data/pocahontas_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 10, 'min_samples_leaf': 1, 'min_samples_split': 10, 'n_estimators': 300}
Custom Accuracy : 0.62
Mean Squared Error: 1.16

Princess & the Frog

Show Code
results = train_and_evaluate_model("../../data/processed-data/princess_and_the_frog_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': None, 'min_samples_leaf': 1, 'min_samples_split': 10, 'n_estimators': 100}
Custom Accuracy : 0.78
Mean Squared Error: 0.86

Sleeping Beauty

Show Code
results = train_and_evaluate_model("../../data/processed-data/sleeping_beauty_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 10, 'min_samples_leaf': 1, 'min_samples_split': 2, 'n_estimators': 300}
Custom Accuracy : 0.76
Mean Squared Error: 0.96

Snow White

Show Code
results = train_and_evaluate_model("../../data/processed-data/snow_white_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 20, 'min_samples_leaf': 2, 'min_samples_split': 2, 'n_estimators': 300}
Custom Accuracy : 0.83
Mean Squared Error: 0.92

Tangled

Show Code
results = train_and_evaluate_model("../../data/processed-data/tangled_processed_data.csv")
print("Best Parameters:", results["Best Parameters"])
print(f"Custom Accuracy : {results['Custom Accuracy']:.2f}")
print(f"Mean Squared Error: {results['Mean Squared Error']:.2f}")
Best Parameters: {'max_depth': 10, 'min_samples_leaf': 1, 'min_samples_split': 10, 'n_estimators': 300}
Custom Accuracy : 0.88
Mean Squared Error: 0.73

Cultural and Feminism Sentiment Analysis

For the following part, we define two sets of keywords to search within reviews:

  1. Cultural Keywords: Words related to culture, representation, heritage, and equity (e.g., diverse, multicultural, marginalized).

  2. Feminism Keywords: Words related to gender equality, feminism, and female representation (e.g., empower, gender equality, female protagonist).

These keywords are used to filter reviews, then the following codes automate the analysis of cultural and feminism themes in movie reviews by searching for relevant keywords and summarizing results into an easy-to-read table. It is useful for understanding trends in movie reviews related to cultural representation and gender equality, and the overall sentiment associated with these themes.

Show Code
import pandas as pd

def analyze_sentiments_by_keywords_to_table(file_paths):
    """
    Analyzes sentiments tied to cultural and feminism keywords in multiple files
    and outputs the results as a structured table.

    Args:
        file_paths (list): List of file paths to CSV datasets.

    Returns:
        pd.DataFrame: A DataFrame containing summarized results for all movies.
    """
    # Define extended keyword lists
    cultural_keywords = [
        "empowering", "diverse", "stereotype", "inclusive", "representation", "heritage", "tradition",
        "identity", "multicultural", "equity", "integration", "underrepresented", "minority", "cultural values",
        "discrimination", "privilege", "cross-cultural", "marginalized", "social justice", "assimilation"
    ]

    feminism_keywords = [
        "empower", "independent", "equality", "gender", "strong female", "feminism", "women rights",
        "patriarchy", "misogyny", "glass ceiling", "equal pay", "female protagonist", "sexism", "intersectional",
        "women empowerment", "gender equality", "matriarchy", "gender roles", "strong woman", "female lead"
    ]

    # Initialize results list
    results = []

    # Function to analyze a single file
    def analyze_file(file_path):
        df = pd.read_csv(file_path)
        df['cleaned_review'] = df['cleaned_review'].fillna("").astype(str)

        # Function to analyze keywords
        def keyword_analysis(keywords):
            filtered_reviews = df[df['cleaned_review'].str.contains('|'.join(keywords), case=False, na=False)]
            total_reviews = len(filtered_reviews)
            avg_sentiment_score = filtered_reviews['sentiment_score'].mean() if total_reviews > 0 else 0
            return total_reviews, avg_sentiment_score

        # Analyze cultural and feminism keywords
        cultural_total, cultural_avg = keyword_analysis(cultural_keywords)
        feminism_total, feminism_avg = keyword_analysis(feminism_keywords)

        # Extract movie name
        movie_name = file_path.split("/")[-1].replace("_processed_data.csv", "").replace("_", " ").title()

        # Append results
        results.append({
            "Movie": movie_name,
            "Cultural Reviews": cultural_total,
            "Avg. Cultural Sentiment": round(cultural_avg, 2),
            "Feminism Reviews": feminism_total,
            "Avg. Feminism Sentiment": round(feminism_avg, 2)
        })

    # Process all file paths
    for file_path in file_paths:
        analyze_file(file_path)

    # Convert results to DataFrame
    results_df = pd.DataFrame(results)
    return results_df

# List of file paths
file_paths = [
    "../../data/processed-data/snow_white_processed_data.csv",             # Snow White and the Seven Dwarfs (1937)
    "../../data/processed-data/cinderella_processed_data.csv",            # Cinderella (1950)
    "../../data/processed-data/sleeping_beauty_processed_data.csv",       # Sleeping Beauty (1959)
    "../../data/processed-data/little_mermaid_processed_data.csv",        # The Little Mermaid (1989)
    "../../data/processed-data/beauty_and_the_beast_processed_data.csv",  # Beauty and the Beast (1991)
    "../../data/processed-data/aladdin_processed_data.csv",               # Aladdin (1992)
    "../../data/processed-data/pocahontas_processed_data.csv",            # Pocahontas (1995)
    "../../data/processed-data/mulan_processed_data.csv",                 # Mulan (1998)
    "../../data/processed-data/princess_and_the_frog_processed_data.csv", # The Princess and the Frog (2009)
    "../../data/processed-data/tangled_processed_data.csv",               # Tangled (2010)
    "../../data/processed-data/brave_processed_data.csv",                 # Brave (2012)
    "../../data/processed-data/frozen_processed_data.csv",                # Frozen (2013)
    "../../data/processed-data/moana_processed_data.csv",                 # Moana (2016)
    "../../data/processed-data/moana_2_processed_data.csv"                
]

# Generate and display the results table
results_table = analyze_sentiments_by_keywords_to_table(file_paths)
results_table
Movie Cultural Reviews Avg. Cultural Sentiment Feminism Reviews Avg. Feminism Sentiment
0 Snow White 9 0.15 5 0.13
1 Cinderella 7 0.09 5 0.06
2 Sleeping Beauty 8 0.22 3 0.32
3 Little Mermaid 7 0.22 7 0.24
4 Beauty And The Beast 3 0.28 1 0.41
5 Aladdin 8 0.25 3 0.27
6 Pocahontas 13 0.13 5 0.23
7 Mulan 35 0.24 75 0.31
8 Princess And The Frog 86 0.21 18 0.21
9 Tangled 26 0.20 9 0.27
10 Brave 25 0.16 25 0.21
11 Frozen 19 0.16 20 0.24
12 Moana 30 0.26 27 0.28
13 Moana 2 7 0.37 3 0.29
Show Code
import matplotlib.pyplot as plt
plt.figure(figsize=(12, 6))
plt.plot(results_table["Movie"], results_table["Avg. Cultural Sentiment"], marker='o', label='Avg. Cultural Sentiment', linestyle='-')
plt.plot(results_table["Movie"], results_table["Avg. Feminism Sentiment"], marker='o', label='Avg. Feminism Sentiment', linestyle='--')


plt.title('Comparison of Average Cultural and Feminism Sentiments')
plt.xlabel('Movies')
plt.ylabel('Average Sentiment Score')
plt.xticks(rotation=45)
plt.legend()
plt.grid(True)

plt.tight_layout()
plt.show()

The table summarizes the frequency and average sentiment scores of reviews containing cultural and feminist-related keywords for 14 Disney princess movies. In terms of cultural keywords, Moana 2 (2024) and Moana (2016) stand out with higher average cultural sentiment scores of 0.37 and 0.26, respectively, indicating a more positive cultural representation. However, older movies like Cinderella (1950) and Snow White and the Seven Dwarfs (1937) have relatively lower cultural sentiment averages (0.09 and 0.15) alongside fewer cultural reviews, which may reflect dated representations of culture. Notably, The Princess and the Frog (2009) received the highest number of cultural-related reviews (86) with a sentiment average of 0.21, highlighting its cultural themes but with mixed reactions.

For feminist keywords, Mulan (1998) leads with the highest number of feminism-related reviews (75) and a relatively strong average sentiment score of 0.31, suggesting that audiences recognize and appreciate its portrayal of female empowerment. Meanwhile, Beauty and the Beast (1991) has the highest average feminist sentiment score of 0.41, despite having only one related review, indicating extremely positive feedback for its feminist themes. Movies like Sleeping Beauty (1959) and Brave (2012) also receive moderate attention to feminist themes, with sentiment averages of 0.32 and 0.21, respectively.

Overall, modern Disney movies such as Moana, Frozen, and Brave exhibit greater engagement and positive sentiment around cultural and feminist representations compared to earlier films like Snow White and Cinderella. This trend reflects Disney’s evolving portrayal of diversity and female empowerment, aligning with contemporary audience expectations.