Data Cleaning

Rotton Tomato Review Text Data-Cleaning

When cleaning the Rotten Tomatoes raw data, our primary goal was to process and standardize audience reviews while preserving meaningful sentiment information.

Show Code
import pandas as pd
import nltk
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize
from sklearn.feature_extraction.text import CountVectorizer
import re
import string
import warnings
warnings.filterwarnings("ignore")

# Download nltk data
nltk.download('stopwords')
nltk.download('punkt')
[nltk_data] Downloading package stopwords to
[nltk_data]     d:\my_conda_env\nltk_data...
[nltk_data]   Unzipping corpora\stopwords.zip.
[nltk_data] Downloading package punkt to d:\my_conda_env\nltk_data...
[nltk_data]   Package punkt is already up-to-date!
True

import Raw Data

Show Code
# import raw datasets
little_mermaid = pd.read_csv("../../data/raw-data/the_little_mermaid_1989.csv")
sleeping_beauty = pd.read_csv("../../data/raw-data/1019187-sleeping_beauty.csv")
pocahontas = pd.read_csv("../../data/raw-data/1063452-pocahontas.csv")
princess_and_the_frog = pd.read_csv("../../data/raw-data/1196003-princess_and_the_frog.csv")
beauty_and_the_beast = pd.read_csv("../../data/raw-data/beauty_and_the_beast_1991.csv")
brave = pd.read_csv("../../data/raw-data/brave.csv")
cinderella = pd.read_csv("../../data/raw-data/cinderella.csv")
aladdin = pd.read_csv("../../data/raw-data/disneys_aladdin.csv")
frozen = pd.read_csv("../../data/raw-data/frozen_2013.csv")
moana = pd.read_csv("../../data/raw-data/moana_2016.csv")
mulan = pd.read_csv("../../data/raw-data/mulan.csv")
snow_white = pd.read_csv("../../data/raw-data/snow_white_and_the_seven_dwarfs.csv")
tangled = pd.read_csv("../../data/raw-data/tangled.csv")
moana_2 = pd.read_csv("../../data/raw-data/moana_2.csv")

clean and normalize the rotton tomato reviews by:

  • Handled Non-String Inputs: Any non-string values in the review column were replaced with an empty string.
  • Lowercased all text was converted to lowercase to ensure uniformity.
  • Removed HTML Tags and URLs: We removed any embedded HTML tags and URLs to focus on the textual content.
  • Removed specialized character: Non-alphabetic characters, numbers, and special symbols were removed using regular expressions.
  • Removed Punctuation: All punctuation marks were stripped to simplify the text.

Methods also within preprocess_text function:

  • Tokenization
  • Remove Stopword stopwords.words(‘english’)
  • Remove Custom Stopwords: We included custom stopwords specific to movie reviews, such as:
    • Generic words: “movie”, “film”, “disney”, “like”, “im”, “one”.
    • Movie-specific words: For example, “frozen”, “little”, “mermaid” depending on which movie we were doing.
Show Code
def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","frozen","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

frozen['cleaned_review'] = frozen['review_text'].apply(preprocess_text)
Show Code
frozen.head()
review_text date score cleaned_review
0 The BEST Disney movie 100 I love everything ab... Nov 16, 2024 5.0 best love everything let go fantastic song cou...
1 One of the most remarkable achievements in the... Nov 16, 2024 4.5 remarkable achievements realm walt animation u...
2 Used to like it when I was little Now grown up... Nov 9, 2024 1.5 used little grown annoying everywhere go still...
3 I think it is good but it isn't very good Oct 31, 2024 3.5 think good isnt good
4 It is the worst movie!!!!!!!!!!! (x 400000000) Oct 19, 2024 0.5 worst x

Above was an example of our cleaning procedure with Frozen (2013)

Instead of using the same preprocess_text function for all 13 films, we customized stopwords for each movie. - Why? Because each movie had its own unique context, terminology, and recurring words that needed to be excluded to ensure accurate sentiment analysis. - High Frequency but Low Sentiment Value: Words like the movie’s title (“frozen”, “mermaid”, “aladdin”) or key descriptors (“disney”, “film”) appeared frequently in reviews but provided little to no meaningful sentiment information. - Context-Specific Words: Certain movies had unique themes or characters that reviewers repeatedly mentioned - Frozen: “frozen”, “elsa”, “olaf” - The Little Mermaid: “mermaid”, “underwater” - Aladdin: “aladdin”, “genie”, “magic” These words often served as neutral placeholders and detracted from the focus on the emotional or qualitative aspects of the review. ### Next we will use the same cleaning procedure across all movies(remove html tags, special characters, tokenization, customized stopwords)

Show Code
# data processing for different movies (different customized stopwords)

def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","im","one","little","mermaid"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

little_mermaid['cleaned_review'] = little_mermaid['review_text'].apply(preprocess_text)

def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","sleeping","beauty","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

sleeping_beauty['cleaned_review'] = sleeping_beauty['review_text'].apply(preprocess_text)

def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","pocahontas","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

pocahontas['cleaned_review'] = pocahontas['review_text'].apply(preprocess_text)


def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","princess","frog","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)
princess_and_the_frog['cleaned_review'] = princess_and_the_frog['review_text'].apply(preprocess_text)


def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","beauty","beast","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)
beauty_and_the_beast['cleaned_review'] = beauty_and_the_beast['review_text'].apply(preprocess_text)


def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","brave","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

brave['cleaned_review'] = brave['review_text'].apply(preprocess_text)


def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","cinderella","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

cinderella['cleaned_review'] = cinderella['review_text'].apply(preprocess_text)


def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","aladdin","williams","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

aladdin['cleaned_review'] = aladdin['review_text'].apply(preprocess_text)


def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","moana","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)


moana['cleaned_review'] = moana['review_text'].apply(preprocess_text)


def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","mulan","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

mulan['cleaned_review'] = mulan['review_text'].apply(preprocess_text)


def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","snow","white","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

snow_white['cleaned_review'] = snow_white['review_text'].apply(preprocess_text)


def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","tangled","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)

tangled['cleaned_review'] = tangled['review_text'].apply(preprocess_text)
Show Code
def preprocess_text(text):
    if not isinstance(text, str):  # Check if input is a string
        return ""  # Replace non-string inputs with an empty string
    
    # Lowercase the text
    text = text.lower()
    
    # Remove HTML tags, URLs, and special characters
    text = re.sub(r'<[^>]*>', '', text)  # HTML tags
    text = re.sub(r'http\S+', '', text)  # URLs
    text = re.sub(r'[^a-z\s]', '', text)  # Special characters and digits
    
    # Remove punctuation
    text = text.translate(str.maketrans('', '', string.punctuation))
    
    # Tokenize and remove stopwords
    tokens = word_tokenize(text)
    
    # Load default stopwords
    stop_words = set(stopwords.words('english'))
    
    # Add custom stopwords
    custom_stopwords = {"movie", "film","disney","like","moana","im","one"} 
    stop_words.update(custom_stopwords)
    
    # Remove stopwords from tokens
    tokens = [word for word in tokens if word not in stop_words]
    
    # Join tokens back to string
    return ' '.join(tokens)


moana_2['cleaned_review'] = moana_2['review_text'].apply(preprocess_text)
Show Code
moana_2.head()
review_text date score cleaned_review
0 My daughter really loved the movie Dec 15, 2024 5.0 daughter really loved
1 A wonderful celebration of Indigenous heritage... Dec 15, 2024 5.0 wonderful celebration indigenous heritage beau...
2 Excellent so glad we went ! Dec 15, 2024 5.0 excellent glad went
3 the second half of the movie was very well don... Dec 15, 2024 5.0 second half well done curious see rd years
4 Lots of fun empowering female characters The R... Dec 15, 2024 3.5 lots fun empowering female characters rock sti...
Show Code
snow_white.head()
review_text date score cleaned_review
0 As someone who grew up with princess disney mo... Nov 12, 2024 5.0 someone grew princess movies got favorites ani...
1 Please Get Snow White and the Seven Dwarfs 2 G... Oct 31, 2024 5.0 please get seven dwarfs get sequel
2 I appreciate this classic especially because i... Oct 19, 2024 4.0 appreciate classic especially came films
3 Minha Crtica a Branca de Neve e os Sete Anes A... Oct 9, 2024 5.0 minha crtica branca de neve e os sete anes pri...
4 The animation is stunning and holds up to this... Sep 21, 2024 4.0 animation stunning holds day particular sequen...
Show Code
cinderella.head()
review_text date score cleaned_review
0 A truly sweet fairytale classic Oct 4, 2024 4.5 truly sweet fairytale classic
1 Sweet and simple Cinderella is a classic for a... Sep 17, 2024 3.0 sweet simple classic reason fairytale magic ad...
2 I enjoyed watching this Disney classic for the... Aug 22, 2024 3.5 enjoyed watching classic first time even thoug...
3 A mustwatch family movie Jun 23, 2024 4.5 mustwatch family
4 One of Disney's Classic films I mean come on w... Mar 29, 2024 3.5 disneys classic films mean come expect good pr...
Show Code
sleeping_beauty.head()
review_text date score cleaned_review
0 The animation the background the musical score... Nov 9, 2024 5.0 animation background musical score everything ...
1 It's much more epic than previous Disney movie... Oct 14, 2024 4.0 much epic previous movies haunting soundtrack ...
2 Another Disney classic with a fantastic and ic... Oct 4, 2024 4.5 another classic fantastic iconic villain malif...
3 Sleeping Beauty is an elegant classic With lik... Sep 17, 2024 3.0 elegant classic likable characters legendary v...
4 What you get most out of this film is the brea... Jul 19, 2024 5.0 get breathtaking animation soundtrack enough c...
Show Code
little_mermaid.head()
review_text date score cleaned_review
0 Such a classic for kids Timeless unforgettable... Nov 18, 2024 4.5 classic kids timeless unforgettable songs
1 This film is a great fuel for those who fantas... Nov 17, 2024 5.0 great fuel fantasise breaking free due ariel l...
2 One of those movies I Liked As A Litte kid But... Nov 9, 2024 3.0 movies liked litte kid teenager much kids terr...
3 Um clssico atemporal da Disney em seus glorios... Sep 21, 2024 5.0 um clssico atemporal da em seus gloriosos temp...
4 I thought this movie is beautiful the songs ar... Aug 16, 2024 5.0 thought beautiful songs wonderful story amazin...
Show Code
beauty_and_the_beast.head()
review_text date score cleaned_review
0 Masterpiece Work of art Even as a kid I could ... Nov 18, 2024 5.0 masterpiece work art even kid could notice way...
1 Truly underrated for an animated movie! Nov 17, 2024 4.5 truly underrated animated
2 Truly a masterpiece One of the best animated f... Nov 12, 2024 5.0 truly masterpiece best animated films time
3 One of the best animated features of all time Nov 6, 2024 5.0 best animated features time
4 Timeless legendary imaginative awardwinning an... Oct 14, 2024 5.0 timeless legendary imaginative awardwinning un...
Show Code
aladdin.head()
review_text date score cleaned_review
0 Voice acting is totally outstanding Legendary ... Nov 18, 2024 5.0 voice acting totally outstanding legendary sou...
1 Great animation an excellent story and catchy ... Oct 1, 2024 5.0 great animation excellent story catchy music m...
2 I love this movie especially Robin Williams as... Aug 16, 2024 5.0 love especially robin genie always makes smile
3 Este filme e lindo de mais Jul 19, 2024 5.0 este filme e lindo de mais
4 Im never an Aladdin guy It seems a bit boring ... Jul 17, 2024 3.0 never guy seems bit boring iago negative annoy...
Show Code
pocahontas.head()
review_text date score cleaned_review
0 "Pocahontas" a beloved classic from Walt Disne... Nov 7, 2024 4.0 beloved classic walt released renaissance era ...
1 Great film Our family loved it Aug 26, 2024 5.0 great family loved
2 The second half is defintely stronger somewhat... Aug 24, 2024 3.5 second half defintely stronger somewhat first ...
3 Pocahontas is a great Disney animated film fro... Jul 28, 2024 4.5 great animated rating
4 Beautiful memorable songs Apr 24, 2024 4.0 beautiful memorable songs
Show Code
mulan.head()
review_text date score cleaned_review
0 Great animation great voice acting and a power... Oct 1, 2024 4.5 great animation great voice acting powerful st...
1 This is my favorite Disney movie of all times ... Sep 18, 2024 5.0 favorite times ive seem hundred times still in...
2 Animated masterpiece A childhood favorite Aug 29, 2024 4.0 animated masterpiece childhood favorite
3 Such a great childhood film! Shan Yu is such a... Aug 15, 2024 4.0 great childhood yu terrifying villain mushu fa...
4 Amazing Was put together perfectly Jun 20, 2024 5.0 amazing put together perfectly
Show Code
princess_and_the_frog.head()
review_text date score cleaned_review
0 Icon beautiful masterpeice I can't find a sing... Nov 19, 2024 5.0 icon beautiful masterpeice cant find single fl...
1 Good and deserved its rewatch! Nov 17, 2024 4.0 good deserved rewatch
2 This animation is simply fantastic it teaches ... Nov 16, 2024 4.0 animation simply fantastic teaches need work h...
3 A good movie with a great message The animatio... Nov 11, 2024 3.5 good great message animation beautiful misses ...
4 Its a pretty good film I watched this with my ... Sep 3, 2024 4.0 pretty good watched cousin kids pretty emotion...
Show Code
tangled.head()
review_text date score cleaned_review
0 I love this movie and I love Rapunzel Nov 20, 2024 5.0 love love rapunzel
1 One of the BEST Disney movies Great song great... Nov 16, 2024 5.0 best movies great song great characters fun st...
2 Bye guys my ride is here Nov 13, 2024 5.0 bye guys ride
3 A lot of people were saying that flynn could c... Nov 8, 2024 5.0 lot people saying flynn could cut rapunzels ha...
4 "Tangled" is a truly enchanting film brought t... Nov 7, 2024 5.0 truly enchanting brought life animation follow...
Show Code
brave.head()
review_text date score cleaned_review
0 "Brave" is a remarkable film from Disney Pixar... Nov 14, 2024 5.0 remarkable pixar standing true masterpieces fi...
1 It has a touching story but the plot is boring... Nov 13, 2024 3.5 touching story plot boring characters interest...
2 Love the theme humour drama and set in Scotlan... Nov 2, 2024 5.0 love theme humour drama set scotland billy con...
3 It is a good movie I like the plot the animati... Oct 19, 2024 4.5 good plot animation amazing music
4 010 BOOOOOOOOOOOOO!!!!!!!!!!!!!!!!!!!!!! (sorr... Oct 19, 2024 0.5 booooooooooooo sorry
Show Code
frozen.head()
review_text date score cleaned_review
0 The BEST Disney movie 100 I love everything ab... Nov 16, 2024 5.0 best love everything let go fantastic song cou...
1 One of the most remarkable achievements in the... Nov 16, 2024 4.5 remarkable achievements realm walt animation u...
2 Used to like it when I was little Now grown up... Nov 9, 2024 1.5 used little grown annoying everywhere go still...
3 I think it is good but it isn't very good Oct 31, 2024 3.5 think good isnt good
4 It is the worst movie!!!!!!!!!!! (x 400000000) Oct 19, 2024 0.5 worst x
Show Code
moana.head()
review_text date score cleaned_review
0 It has great songs and some moments of mastery... Nov 22, 2024 3.0 great songs moments mastery weighed unforgivab...
1 Pretty good movie with a lot of singing especi... Nov 14, 2024 3.5 pretty good lot singing especially beginning s...
2 Yo this movie was awesome This might be one of... Nov 14, 2024 5.0 yo awesome might favorite movies time even tho...
3 Magnificent film! I love Moana and Maui and I ... Nov 7, 2024 5.0 magnificent love maui love songs memorable oce...
4 The story is good but the characters are not g... Nov 1, 2024 2.5 story good characters good enough hate chicken

IMDB data Cleaning

- Since the IMDBPY package already offered a pretty comprehensive and clean dataset, we'll just extract the useful features and clean as we go. 
Show Code
# import dataset
imdb_movie = pd.read_csv("../../data/raw-data/imdb_movies.csv")
princess_detail = pd.read_csv("../../data/raw-data/disney_princess_data.csv")
Show Code
imdb_movie.head()
Unnamed: 0 title year genres directors cast rating votes plot box_office countries languages runtimes Opening_Weekend_US Cumulative_Gross Budget ROI
0 0 Snow White and the Seven Dwarfs 1937 ['Animation', 'Adventure', 'Family', 'Fantasy'... ['William Cottrell', 'David Hand', 'Wilfred Ja... ['Roy Atwell', 'Stuart Buchanan', 'Adriana Cas... 7.6 220525 The beautiful and kindhearted princess Snow Wh... {'Budget': '$2,000,000 (estimated)', 'Opening ... ['United States'] ['English'] ['83'] 7514749.0 418200000.0 2000000.0 208.100000
1 1 Cinderella 1950 ['Animation', 'Family', 'Fantasy', 'Musical', ... ['Clyde Geronimi', 'Wilfred Jackson', 'Hamilto... ['Ilene Woods', 'Eleanor Audley', 'Verna Felto... 7.3 177837 Beautiful Cinderella, a young woman with a hea... {'Budget': '$2,900,000 (estimated)', 'Opening ... ['United States'] ['English'] ['74'] 5165881.0 NaN 2900000.0 NaN
2 2 Sleeping Beauty 1959 ['Animation', 'Adventure', 'Family', 'Fantasy'... ['Les Clark', 'Clyde Geronimi', 'Eric Larson',... ['Mary Costa', 'Bill Shirley', 'Eleanor Audley... 7.2 164202 After the beautiful Princess Aurora is born in... {'Budget': '$6,000,000 (estimated)', 'Opening ... ['United States'] ['English'] ['75'] 2548698.0 1420935.0 6000000.0 -0.763177
3 3 The Little Mermaid 1989 ['Animation', 'Adventure', 'Family', 'Fantasy'... ['Ron Clements', 'John Musker'] ['Rene Auberjonois', 'Christopher Daniel Barne... 7.6 296976 In Disney's beguiling animated romp, rebelliou... {'Budget': '$40,000,000 (estimated)', 'Opening... ['United States'] ['English', 'French'] ['83'] 9814520.0 211343479.0 40000000.0 4.283587
4 4 Beauty and the Beast 1991 ['Animation', 'Family', 'Fantasy', 'Musical', ... ['Gary Trousdale', 'Kirk Wise'] ['Robby Benson', 'Jesse Corti', 'Rex Everhart'... 8.0 487006 Having lived a life in selfishness, young Prin... {'Budget': '$25,000,000 (estimated)', 'Opening... ['United States', 'Japan'] ['English', 'French'] ['84'] 17754905.0 424967620.0 25000000.0 15.998705
Show Code
princess_detail.head()
title princess_name country_of_origin iconic_theme_song princess_description
0 Snow White and the Seven Dwarfs Snow White Germany Someday My Prince Will Come Snow White knows that every day is an opportun...
1 Cinderella Cinderella France A Dream Is a Wish Your Heart Makes Cinderella knows that being a positive influen...
2 Sleeping Beauty Aurora France Once Upon a Dream Also known as Briar Rose, Aurora is graceful, ...
3 The Little Mermaid Ariel Denmark Part of Your World Driven by her independent spirit and curiosity...
4 Beauty and the Beast Belle France Beauty and the Beast Belle is imaginative and smart, and comfortabl...

Sentiment analysis, keyword extraction, and visualization of keywords using word clouds

  1. Sentiment Analysis Function:sentiment_analysis(text)
    • determine the emotional tone of each review: positive, negative, or neutral
    • achieved using the TextBlob library, which computes a sentiment polarity score for each review.
    • Returns a sentiment polarity score in the range of -1 (negative) to 1 (positive).
Show Code
from textblob import TextBlob
df = frozen 
# 1. Perform sentiment analysis on the "review_text" column
def sentiment_analysis(text):
    blob = TextBlob(text)
    return blob.sentiment.polarity  # Returns sentiment polarity (-1 to 1)
# Add a column for sentiment scores
df['sentiment_score'] = df['cleaned_review'].apply(sentiment_analysis)
  1. Sentiment Classification: Function: classify_sentiment(score)
    • Maps each review’s sentiment polarity score to one of the five sentiment categories.
      • Very Positive: Polarity ≥ 0.6
      • Positive: Polarity ≥ 0.2 and < 0.6
      • Neutral: Polarity > -0.2 and < 0.2
      • Negative: Polarity > -0.6 and ≤ -0.2
      • Very Negative: Polarity ≤ -0.6
Show Code
def classify_sentiment(score):
    if score >= 0.6:
        return 'Very Positive'
    elif score >= 0.2:
        return 'Positive'
    elif score > -0.2:
        return 'Neutral'
    elif score > -0.6:
        return 'Negative'
    else:
        return 'Very Negative'

df['sentiment_category'] = df['sentiment_score'].apply(classify_sentiment)
  1. Keyword Extraction: Function: get_keywords(text_list, top_n=50)
    • Extracts the most frequently occurring words (keywords) from the reviews to identify common themes or topics.
    • Extracted keywords are stored as:
      • all_keywords: Keywords from all reviews.
      • positive_keywords: Keywords from positive reviews.
      • negative_keywords: Keywords from negative reviews.
Show Code
from collections import Counter
def get_keywords(text_list, top_n=50):
    all_words = ' '.join(text_list).split()  # Combine all text and split into words
    word_counts = Counter(all_words)  # Count word frequencies
    return word_counts.most_common(top_n)  # Get the top N keywords and their weights

# Extract keywords from all reviews
all_keywords = get_keywords(df['cleaned_review'].tolist())

# Extract keywords from reviews with scores of 4 or higher
positive_keywords = get_keywords(df[df['score'] >= 4]['cleaned_review'].tolist())

# Extract keywords from reviews with scores of 2 or lower
negative_keywords = get_keywords(df[df['score'] <= 2]['cleaned_review'].tolist())

# Store keywords and weights in lists
all_keywords_list = [{"keyword": word, "weight": weight} for word, weight in all_keywords]
positive_keywords_list = [{"keyword": word, "weight": weight} for word, weight in positive_keywords]
negative_keywords_list = [{"keyword": word, "weight": weight} for word, weight in negative_keywords]

4. Visualization Using Word Clouds: Function: generate_wordcloud(word_freq, title)

- Uses the WordCloud library to generate a visual representation of keyword frequencies.
Show Code
from wordcloud import WordCloud
import matplotlib.pyplot as plt
import numpy as np
from PIL import Image

# Define the Rapunzel color palette
rapunzel_colors = ["#7DBBB0", "#DAAFD2", "#9C8CBD", "#A06BAF", "#8550A0"]

# Load the background image
background_image = Image.open("../../assets/disney_shade.jpeg")  # Replace with your image path
background_array = np.array(background_image)

# Get the dimensions of the background image
image_width, image_height = background_image.size

# Function to generate the word cloud
def generate_wordcloud(word_freq, title, background_array, image_width, image_height):
    # Configure the WordCloud with the same dimensions as the background image
    wc = WordCloud(
        width=image_width,  # Match the background image width
        height=image_height,  # Match the background image height
        background_color=None,  # Transparent background
        mode="RGBA",  # Ensure transparency
        max_words=500,
        min_font_size=2,
        prefer_horizontal=0.8,
        color_func=lambda *args, **kwargs: np.random.choice(rapunzel_colors)  # Custom color function
    ).generate_from_frequencies(dict(word_freq))

    # Plot the word cloud over the background image
    plt.figure(figsize=(16, 9))  # Adjust figure size for better resolution
    plt.imshow(background_array, interpolation="bilinear")  # Show the background image
    plt.imshow(wc, interpolation="bilinear", alpha=0.8)  # Overlay the word cloud with some transparency
    plt.axis('off')  # Hide axes
    plt.title(title, fontsize=24)  # Larger title for better visibility
    plt.show()

# Generate word clouds
generate_wordcloud(all_keywords, "All Reviews Keywords", background_array, image_width, image_height)
generate_wordcloud(positive_keywords, "Positive Reviews Keywords", background_array, image_width, image_height)
generate_wordcloud(negative_keywords, "Negative Reviews Keywords", background_array, image_width, image_height)

# Print the output
print("All Keywords with Weights:", all_keywords_list)
print("Positive Keywords with Weights:", positive_keywords_list)
print("Negative Keywords with Weights:", negative_keywords_list)

All Keywords with Weights: [{'keyword': 'good', 'weight': 280}, {'keyword': 'songs', 'weight': 271}, {'keyword': 'story', 'weight': 250}, {'keyword': 'great', 'weight': 245}, {'keyword': 'love', 'weight': 243}, {'keyword': 'elsa', 'weight': 234}, {'keyword': 'characters', 'weight': 221}, {'keyword': 'animation', 'weight': 188}, {'keyword': 'anna', 'weight': 172}, {'keyword': 'go', 'weight': 166}, {'keyword': 'best', 'weight': 156}, {'keyword': 'really', 'weight': 153}, {'keyword': 'time', 'weight': 152}, {'keyword': 'let', 'weight': 151}, {'keyword': 'movies', 'weight': 133}, {'keyword': 'animated', 'weight': 129}, {'keyword': 'also', 'weight': 127}, {'keyword': 'plot', 'weight': 117}, {'keyword': 'even', 'weight': 117}, {'keyword': 'song', 'weight': 116}, {'keyword': 'people', 'weight': 112}, {'keyword': 'see', 'weight': 108}, {'keyword': 'much', 'weight': 108}, {'keyword': 'would', 'weight': 107}, {'keyword': 'olaf', 'weight': 105}, {'keyword': 'music', 'weight': 101}, {'keyword': 'dont', 'weight': 98}, {'keyword': 'kids', 'weight': 98}, {'keyword': 'ever', 'weight': 96}, {'keyword': 'watch', 'weight': 94}, {'keyword': 'first', 'weight': 94}, {'keyword': 'get', 'weight': 91}, {'keyword': 'well', 'weight': 90}, {'keyword': 'character', 'weight': 88}, {'keyword': 'hans', 'weight': 87}, {'keyword': 'princess', 'weight': 86}, {'keyword': 'better', 'weight': 85}, {'keyword': 'little', 'weight': 84}, {'keyword': 'think', 'weight': 79}, {'keyword': 'still', 'weight': 78}, {'keyword': 'made', 'weight': 77}, {'keyword': 'films', 'weight': 74}, {'keyword': 'make', 'weight': 74}, {'keyword': 'bad', 'weight': 71}, {'keyword': 'sister', 'weight': 70}, {'keyword': 'funny', 'weight': 69}, {'keyword': 'family', 'weight': 69}, {'keyword': 'could', 'weight': 68}, {'keyword': 'powers', 'weight': 68}, {'keyword': 'true', 'weight': 68}]
Positive Keywords with Weights: [{'keyword': 'great', 'weight': 168}, {'keyword': 'love', 'weight': 166}, {'keyword': 'story', 'weight': 151}, {'keyword': 'elsa', 'weight': 140}, {'keyword': 'songs', 'weight': 137}, {'keyword': 'characters', 'weight': 129}, {'keyword': 'good', 'weight': 124}, {'keyword': 'best', 'weight': 122}, {'keyword': 'animation', 'weight': 116}, {'keyword': 'anna', 'weight': 105}, {'keyword': 'movies', 'weight': 98}, {'keyword': 'animated', 'weight': 90}, {'keyword': 'go', 'weight': 86}, {'keyword': 'really', 'weight': 86}, {'keyword': 'time', 'weight': 83}, {'keyword': 'let', 'weight': 76}, {'keyword': 'also', 'weight': 67}, {'keyword': 'watch', 'weight': 64}, {'keyword': 'see', 'weight': 62}, {'keyword': 'family', 'weight': 62}, {'keyword': 'would', 'weight': 62}, {'keyword': 'even', 'weight': 61}, {'keyword': 'kids', 'weight': 60}, {'keyword': 'music', 'weight': 59}, {'keyword': 'plot', 'weight': 56}, {'keyword': 'much', 'weight': 56}, {'keyword': 'princess', 'weight': 55}, {'keyword': 'well', 'weight': 55}, {'keyword': 'people', 'weight': 55}, {'keyword': 'ever', 'weight': 55}, {'keyword': 'olaf', 'weight': 53}, {'keyword': 'sister', 'weight': 51}, {'keyword': 'first', 'weight': 51}, {'keyword': 'beautiful', 'weight': 51}, {'keyword': 'hans', 'weight': 48}, {'keyword': 'song', 'weight': 47}, {'keyword': 'funny', 'weight': 47}, {'keyword': 'loved', 'weight': 46}, {'keyword': 'amazing', 'weight': 45}, {'keyword': 'favorite', 'weight': 45}, {'keyword': 'little', 'weight': 44}, {'keyword': 'queen', 'weight': 44}, {'keyword': 'sisters', 'weight': 43}, {'keyword': 'films', 'weight': 43}, {'keyword': 'think', 'weight': 42}, {'keyword': 'better', 'weight': 42}, {'keyword': 'still', 'weight': 42}, {'keyword': 'true', 'weight': 41}, {'keyword': 'classic', 'weight': 40}, {'keyword': 'made', 'weight': 40}]
Negative Keywords with Weights: [{'keyword': 'songs', 'weight': 51}, {'keyword': 'characters', 'weight': 37}, {'keyword': 'story', 'weight': 36}, {'keyword': 'let', 'weight': 35}, {'keyword': 'time', 'weight': 34}, {'keyword': 'annoying', 'weight': 33}, {'keyword': 'good', 'weight': 33}, {'keyword': 'go', 'weight': 32}, {'keyword': 'bad', 'weight': 30}, {'keyword': 'worst', 'weight': 29}, {'keyword': 'plot', 'weight': 29}, {'keyword': 'overrated', 'weight': 27}, {'keyword': 'ever', 'weight': 26}, {'keyword': 'love', 'weight': 26}, {'keyword': 'elsa', 'weight': 24}, {'keyword': 'see', 'weight': 23}, {'keyword': 'animation', 'weight': 23}, {'keyword': 'dont', 'weight': 23}, {'keyword': 'really', 'weight': 23}, {'keyword': 'even', 'weight': 20}, {'keyword': 'people', 'weight': 20}, {'keyword': 'character', 'weight': 19}, {'keyword': 'olaf', 'weight': 19}, {'keyword': 'make', 'weight': 19}, {'keyword': 'little', 'weight': 18}, {'keyword': 'boring', 'weight': 18}, {'keyword': 'made', 'weight': 18}, {'keyword': 'get', 'weight': 17}, {'keyword': 'song', 'weight': 17}, {'keyword': 'also', 'weight': 17}, {'keyword': 'terrible', 'weight': 16}, {'keyword': 'thing', 'weight': 16}, {'keyword': 'anna', 'weight': 16}, {'keyword': 'first', 'weight': 16}, {'keyword': 'know', 'weight': 16}, {'keyword': 'nothing', 'weight': 15}, {'keyword': 'never', 'weight': 15}, {'keyword': 'many', 'weight': 15}, {'keyword': 'could', 'weight': 15}, {'keyword': 'kids', 'weight': 15}, {'keyword': 'villain', 'weight': 14}, {'keyword': 'think', 'weight': 14}, {'keyword': 'powers', 'weight': 14}, {'keyword': 'hate', 'weight': 14}, {'keyword': 'want', 'weight': 13}, {'keyword': 'watch', 'weight': 13}, {'keyword': 'better', 'weight': 12}, {'keyword': 'watched', 'weight': 12}, {'keyword': 'much', 'weight': 12}, {'keyword': 'ive', 'weight': 12}]

Now that we understand how each part works, we’ll run sentiment overview wordcloud for all movies.

Show Code
import pandas as pd
from collections import Counter
from wordcloud import WordCloud
import matplotlib.pyplot as plt
import numpy as np
from PIL import Image

# Define the Rapunzel color palette
rapunzel_colors = ["#7DBBB0", "#DAAFD2", "#9C8CBD", "#A06BAF", "#8550A0"]

# Load the background image
background_image = Image.open("../../assets/disney_shade.jpeg")  # Replace with your image path
background_array = np.array(background_image)

# Get the dimensions of the background image
image_width, image_height = background_image.size

def process_movie_data(df, dataset_name):
    """
    Process a movie dataset to perform sentiment analysis, classify sentiments,
    extract keywords, and save the processed data to a new CSV file.

    Parameters:
    - df: pd.DataFrame, the input dataset.
    - dataset_name: str, the name of the dataset (e.g., "frozen").

    Returns:
    - None
    """
    # 1. Perform sentiment analysis on the "cleaned_review" column
    def sentiment_analysis(text):
        blob = TextBlob(text)
        return blob.sentiment.polarity  # Returns sentiment polarity (-1 to 1)

    # Add a column for sentiment scores
    df['sentiment_score'] = df['cleaned_review'].apply(sentiment_analysis)

    # 2. Classify sentiment into five categories
    def classify_sentiment(score):
        if score >= 0.6:
            return 'Very Positive'
        elif score >= 0.2:
            return 'Positive'
        elif score > -0.2:
            return 'Neutral'
        elif score > -0.6:
            return 'Negative'
        else:
            return 'Very Negative'

    df['sentiment_category'] = df['sentiment_score'].apply(classify_sentiment)

    # 3. Extract keywords and calculate their weights
    def get_keywords(text_list, top_n=60):
        all_words = ' '.join(text_list).split()  # Combine all text and split into words
        word_counts = Counter(all_words)  # Count word frequencies
        return word_counts.most_common(top_n)  # Get the top N keywords and their weights

    # Extract keywords from all reviews
    all_keywords = get_keywords(df['cleaned_review'].tolist())

    # Extract keywords from reviews with scores of 4 or higher
    positive_keywords = get_keywords(df[df['score'] >= 4]['cleaned_review'].tolist())

    # Extract keywords from reviews with scores of 2 or lower
    negative_keywords = get_keywords(df[df['score'] <= 2]['cleaned_review'].tolist())

    # 4. Save the processed DataFrame to a new CSV file
    output_file = f"./../../data/processed-data/{dataset_name}_processed_data.csv"  # Create the output file name
    df.to_csv(output_file, index=False, encoding='utf-8')  # Save to CSV

    print(f"Processed data has been saved to {output_file}.")

    # 5. Generate word clouds with a custom background and colors
    def generate_wordcloud(word_freq, title, background_array, image_width, image_height):
        # Configure the WordCloud with the same dimensions as the background image
        wc = WordCloud(
            width=image_width,  # Match the background image width
            height=image_height,  # Match the background image height
            background_color=None,  # Transparent background
            mode="RGBA",  # Ensure transparency
            max_words=500,
            min_font_size=2,
            prefer_horizontal=0.8,
            color_func=lambda *args, **kwargs: np.random.choice(rapunzel_colors)  # Custom color function
        ).generate_from_frequencies(dict(word_freq))

        # Plot the word cloud over the background image
        plt.figure(figsize=(16, 9))  # Adjust figure size for better resolution
        plt.imshow(background_array, interpolation="bilinear")  # Show the background image
        plt.imshow(wc, interpolation="bilinear", alpha=0.8)  # Overlay the word cloud with some transparency
        plt.axis('off')  # Hide axes
        plt.title(title, fontsize=24)  # Larger title for better visibility
        plt.show()

    # Generate word clouds
    generate_wordcloud(all_keywords, "All Reviews Keywords", background_array, image_width, image_height)
    generate_wordcloud(positive_keywords, "Positive Reviews Keywords", background_array, image_width, image_height)
    generate_wordcloud(negative_keywords, "Negative Reviews Keywords", background_array, image_width, image_height)

    # Print the keywords
    # print("All Keywords with Weights:", all_keywords)
    # print("Positive Keywords with Weights:", positive_keywords)
    # print("Negative Keywords with Weights:", negative_keywords)
Show Code
process_movie_data(moana_2, "moana_2")
Processed data has been saved to ./../../data/processed-data/moana_2_processed_data.csv.

Show Code
moana_2 = pd.read_csv("./../../data/processed-data/moana_2_processed_data.csv")
moana_2.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 My daughter really loved the movie Dec 15, 2024 5.0 daughter really loved 0.700000 Very Positive
1 A wonderful celebration of Indigenous heritage... Dec 15, 2024 5.0 wonderful celebration indigenous heritage beau... 0.925000 Very Positive
2 Excellent so glad we went ! Dec 15, 2024 5.0 excellent glad went 0.750000 Very Positive
3 the second half of the movie was very well don... Dec 15, 2024 5.0 second half well done curious see rd years -0.088889 Neutral
4 Lots of fun empowering female characters The R... Dec 15, 2024 3.5 lots fun empowering female characters rock sti... 0.381481 Positive
Show Code
process_movie_data(frozen, "frozen")
Processed data has been saved to ./../../data/processed-data/frozen_processed_data.csv.

Show Code
frozen = pd.read_csv("./../../data/processed-data/frozen_processed_data.csv")
frozen.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 The BEST Disney movie 100 I love everything ab... Nov 16, 2024 5.0 best love everything let go fantastic song cou... 0.633333 Very Positive
1 One of the most remarkable achievements in the... Nov 16, 2024 4.5 remarkable achievements realm walt animation u... 0.183478 Neutral
2 Used to like it when I was little Now grown up... Nov 9, 2024 1.5 used little grown annoying everywhere go still... -0.493750 Negative
3 I think it is good but it isn't very good Oct 31, 2024 3.5 think good isnt good 0.700000 Very Positive
4 It is the worst movie!!!!!!!!!!! (x 400000000) Oct 19, 2024 0.5 worst x -1.000000 Very Negative
Show Code
process_movie_data(little_mermaid, "little_mermaid")
Processed data has been saved to ./../../data/processed-data/little_mermaid_processed_data.csv.

Show Code
little_mermaid = pd.read_csv("./../../data/processed-data/little_mermaid_processed_data.csv")
little_mermaid.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 Such a classic for kids Timeless unforgettable... Nov 18, 2024 4.5 classic kids timeless unforgettable songs 0.483333 Positive
1 This film is a great fuel for those who fantas... Nov 17, 2024 5.0 great fuel fantasise breaking free due ariel l... 0.218750 Positive
2 One of those movies I Liked As A Litte kid But... Nov 9, 2024 3.0 movies liked litte kid teenager much kids terr... 0.075000 Neutral
3 Um clssico atemporal da Disney em seus glorios... Sep 21, 2024 5.0 um clssico atemporal da em seus gloriosos temp... 0.000000 Neutral
4 I thought this movie is beautiful the songs ar... Aug 16, 2024 5.0 thought beautiful songs wonderful story amazin... 0.825000 Very Positive
Show Code
process_movie_data(sleeping_beauty, "sleeping_beauty")
Processed data has been saved to ./../../data/processed-data/sleeping_beauty_processed_data.csv.

Show Code
sleeping_beauty = pd.read_csv("./../../data/processed-data/sleeping_beauty_processed_data.csv")
sleeping_beauty.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 The animation the background the musical score... Nov 9, 2024 5.0 animation background musical score everything ... 0.250000 Positive
1 It's much more epic than previous Disney movie... Oct 14, 2024 4.0 much epic previous movies haunting soundtrack ... 0.190000 Neutral
2 Another Disney classic with a fantastic and ic... Oct 4, 2024 4.5 another classic fantastic iconic villain malif... 0.355556 Positive
3 Sleeping Beauty is an elegant classic With lik... Sep 17, 2024 3.0 elegant classic likable characters legendary v... 0.411111 Positive
4 What you get most out of this film is the brea... Jul 19, 2024 5.0 get breathtaking animation soundtrack enough c... 0.500000 Positive
Show Code
process_movie_data(pocahontas , "pocahontas")
Processed data has been saved to ./../../data/processed-data/pocahontas_processed_data.csv.

Show Code
pocahontas = pd.read_csv("./../../data/processed-data/pocahontas_processed_data.csv")
pocahontas.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 "Pocahontas" a beloved classic from Walt Disne... Nov 7, 2024 4.0 beloved classic walt released renaissance era ... 0.219485 Positive
1 Great film Our family loved it Aug 26, 2024 5.0 great family loved 0.750000 Very Positive
2 The second half is defintely stronger somewhat... Aug 24, 2024 3.5 second half defintely stronger somewhat first ... 0.194676 Neutral
3 Pocahontas is a great Disney animated film fro... Jul 28, 2024 4.5 great animated rating 0.800000 Very Positive
4 Beautiful memorable songs Apr 24, 2024 4.0 beautiful memorable songs 0.675000 Very Positive
Show Code
process_movie_data(princess_and_the_frog, "princess_and_the_frog")
Processed data has been saved to ./../../data/processed-data/princess_and_the_frog_processed_data.csv.

Show Code
princess_and_the_frog = pd.read_csv("./../../data/processed-data/princess_and_the_frog_processed_data.csv")
princess_and_the_frog.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 Icon beautiful masterpeice I can't find a sing... Nov 19, 2024 5.0 icon beautiful masterpeice cant find single fl... 0.375714 Positive
1 Good and deserved its rewatch! Nov 17, 2024 4.0 good deserved rewatch 0.700000 Very Positive
2 This animation is simply fantastic it teaches ... Nov 16, 2024 4.0 animation simply fantastic teaches need work h... 0.054167 Neutral
3 A good movie with a great message The animatio... Nov 11, 2024 3.5 good great message animation beautiful misses ... 0.521528 Positive
4 Its a pretty good film I watched this with my ... Sep 3, 2024 4.0 pretty good watched cousin kids pretty emotion... 0.210000 Positive
Show Code
process_movie_data(beauty_and_the_beast, "beauty_and_the_beast")
Processed data has been saved to ./../../data/processed-data/beauty_and_the_beast_processed_data.csv.

Show Code
beauty_and_the_beast= pd.read_csv("./../../data/processed-data/beauty_and_the_beast_processed_data.csv")
beauty_and_the_beast.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 Masterpiece Work of art Even as a kid I could ... Nov 18, 2024 5.0 masterpiece work art even kid could notice way... 0.500000 Positive
1 Truly underrated for an animated movie! Nov 17, 2024 4.5 truly underrated animated 0.000000 Neutral
2 Truly a masterpiece One of the best animated f... Nov 12, 2024 5.0 truly masterpiece best animated films time 1.000000 Very Positive
3 One of the best animated features of all time Nov 6, 2024 5.0 best animated features time 1.000000 Very Positive
4 Timeless legendary imaginative awardwinning an... Oct 14, 2024 5.0 timeless legendary imaginative awardwinning un... 0.303495 Positive
Show Code
process_movie_data(brave, "brave")
Processed data has been saved to ./../../data/processed-data/brave_processed_data.csv.

Show Code
brave = pd.read_csv("./../../data/processed-data/brave_processed_data.csv")
brave.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 "Brave" is a remarkable film from Disney Pixar... Nov 14, 2024 5.0 remarkable pixar standing true masterpieces fi... 0.268116 Positive
1 It has a touching story but the plot is boring... Nov 13, 2024 3.5 touching story plot boring characters interest... 0.125000 Neutral
2 Love the theme humour drama and set in Scotlan... Nov 2, 2024 5.0 love theme humour drama set scotland billy con... 0.486667 Positive
3 It is a good movie I like the plot the animati... Oct 19, 2024 4.5 good plot animation amazing music 0.650000 Very Positive
4 010 BOOOOOOOOOOOOO!!!!!!!!!!!!!!!!!!!!!! (sorr... Oct 19, 2024 0.5 booooooooooooo sorry -0.500000 Negative
Show Code
process_movie_data(cinderella , "cinderella")
Processed data has been saved to ./../../data/processed-data/cinderella_processed_data.csv.

Show Code
cinderella = pd.read_csv("./../../data/processed-data/cinderella_processed_data.csv")
cinderella.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 A truly sweet fairytale classic Oct 4, 2024 4.5 truly sweet fairytale classic 0.258333 Positive
1 Sweet and simple Cinderella is a classic for a... Sep 17, 2024 3.0 sweet simple classic reason fairytale magic ad... 0.297917 Positive
2 I enjoyed watching this Disney classic for the... Aug 22, 2024 3.5 enjoyed watching classic first time even thoug... 0.285000 Positive
3 A mustwatch family movie Jun 23, 2024 4.5 mustwatch family 0.000000 Neutral
4 One of Disney's Classic films I mean come on w... Mar 29, 2024 3.5 disneys classic films mean come expect good pr... 0.284167 Positive
Show Code
process_movie_data(aladdin, "aladdin")
Processed data has been saved to ./../../data/processed-data/aladdin_processed_data.csv.

Show Code
aladdin = pd.read_csv("./../../data/processed-data/aladdin_processed_data.csv")
aladdin.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 Voice acting is totally outstanding Legendary ... Nov 18, 2024 5.0 voice acting totally outstanding legendary sou... 0.500000 Positive
1 Great animation an excellent story and catchy ... Oct 1, 2024 5.0 great animation excellent story catchy music m... 0.460714 Positive
2 I love this movie especially Robin Williams as... Aug 16, 2024 5.0 love especially robin genie always makes smile 0.266667 Positive
3 Este filme e lindo de mais Jul 19, 2024 5.0 este filme e lindo de mais 0.000000 Neutral
4 Im never an Aladdin guy It seems a bit boring ... Jul 17, 2024 3.0 never guy seems bit boring iago negative annoy... -0.112500 Neutral
Show Code
process_movie_data(moana, "moana")
Processed data has been saved to ./../../data/processed-data/moana_processed_data.csv.

Show Code
moana = pd.read_csv("./../../data/processed-data/moana_processed_data.csv")
moana.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 It has great songs and some moments of mastery... Nov 22, 2024 3.0 great songs moments mastery weighed unforgivab... 2.312500e-01 Positive
1 Pretty good movie with a lot of singing especi... Nov 14, 2024 3.5 pretty good lot singing especially beginning s... 1.916667e-01 Neutral
2 Yo this movie was awesome This might be one of... Nov 14, 2024 5.0 yo awesome might favorite movies time even tho... 5.555556e-01 Positive
3 Magnificent film! I love Moana and Maui and I ... Nov 7, 2024 5.0 magnificent love maui love songs memorable oce... 5.875000e-01 Positive
4 The story is good but the characters are not g... Nov 1, 2024 2.5 story good characters good enough hate chicken -2.220446e-17 Neutral
Show Code
process_movie_data(mulan, "mulan")
Processed data has been saved to ./../../data/processed-data/mulan_processed_data.csv.

Show Code
mulan = pd.read_csv("./../../data/processed-data/mulan_processed_data.csv")
mulan.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 Great animation great voice acting and a power... Oct 1, 2024 4.5 great animation great voice acting powerful st... 0.429464 Positive
1 This is my favorite Disney movie of all times ... Sep 18, 2024 5.0 favorite times ive seem hundred times still in... 0.458333 Positive
2 Animated masterpiece A childhood favorite Aug 29, 2024 4.0 animated masterpiece childhood favorite 0.500000 Positive
3 Such a great childhood film! Shan Yu is such a... Aug 15, 2024 4.0 great childhood yu terrifying villain mushu fa... 0.230000 Positive
4 Amazing Was put together perfectly Jun 20, 2024 5.0 amazing put together perfectly 0.800000 Very Positive
Show Code
process_movie_data(snow_white, "snow_white")
Processed data has been saved to ./../../data/processed-data/snow_white_processed_data.csv.

Show Code
snow_white = pd.read_csv("./../../data/processed-data/snow_white_processed_data.csv")
snow_white.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 As someone who grew up with princess disney mo... Nov 12, 2024 5.0 someone grew princess movies got favorites ani... 0.210590 Positive
1 Please Get Snow White and the Seven Dwarfs 2 G... Oct 31, 2024 5.0 please get seven dwarfs get sequel 0.000000 Neutral
2 I appreciate this classic especially because i... Oct 19, 2024 4.0 appreciate classic especially came films 0.083333 Neutral
3 Minha Crtica a Branca de Neve e os Sete Anes A... Oct 9, 2024 5.0 minha crtica branca de neve e os sete anes pri... 0.000000 Neutral
4 The animation is stunning and holds up to this... Sep 21, 2024 4.0 animation stunning holds day particular sequen... 0.158654 Neutral
Show Code
process_movie_data(tangled, "tangled")
Processed data has been saved to ./../../data/processed-data/tangled_processed_data.csv.

Show Code
tangled = pd.read_csv("./../../data/processed-data/tangled_processed_data.csv")
tangled.head()
review_text date score cleaned_review sentiment_score sentiment_category
0 I love this movie and I love Rapunzel Nov 20, 2024 5.0 love love rapunzel 0.500000 Positive
1 One of the BEST Disney movies Great song great... Nov 16, 2024 5.0 best movies great song great characters fun st... 0.650000 Very Positive
2 Bye guys my ride is here Nov 13, 2024 5.0 bye guys ride 0.000000 Neutral
3 A lot of people were saying that flynn could c... Nov 8, 2024 5.0 lot people saying flynn could cut rapunzels ha... 0.404167 Positive
4 "Tangled" is a truly enchanting film brought t... Nov 7, 2024 5.0 truly enchanting brought life animation follow... 0.263380 Positive