Sentiment Text Generator

We built and trained an NLP model that generates text based on user prompt and selected sentiment.


Try the product here!
typing_image

Product Overview

What the product does

We wanted to create a text generation model that would generate text with the specific parameter of mimicking a human emotion. The user can select an emotion that they want the model to base the text on, and the model will then create text of the user's choosing and centering it on the emotion it is supposed to portray.

How the product works

In order to try the product, you first select an emotion (positive or negative), enter a brief prompt to begin your text and, then click "generate". The text box then generates a paragraph that builds off of the user's prompt and is able to express satisfaction or displeasure.

Try the product

Choose a sentiment, enter a prompt, click "generate"! When the page reloads, scroll to see your text.

Choose a sentiment...

Why this product?

Expressing sentiment through your text

We want our tool to be a helpful and efficient text generator for people to use when drafting emails, speeches, product reviews, or other things that require sentiment and tone. It will also help businesses understand their customer's clear opinions and make precise decisions based on a clear expression of emotion.

Methods

line graph

Datasets

The best place to learn about simple emotions is online reviews, thus we are using the "Amazon Reviews for Sentiment Analysis" dataset from the website Kaggle in order to train the model. Our pre-processing consisted of splitting reviews into positive and negative categories and removing labels.


View

GPT Model

We're also using a transformer-based model with GPT architecture, called GPT-neo, by EleutherAI. GPT is a language model that uses deep learning to generate human-like text, (which can also include code, stories, poems, etc.) in a matter of seconds. GPT uses methods ranging from self-attention to probability distribution to produce one token after another with a user prompt.


View

Jupyter Notebooks

We used Jupyter Notebooks on the Cocalc servers to learn Python basics and practice using libraries like NumPy and NLTK. We also used this platform and the topics we learned (splitting, loops, lists) to clean and preprocess our dataset. Overall, we learned many skills with this resource that we were able to apply to our project.


View

GPT Algorithms

High Level Explanation

  1. GPT 3 uses a model called a transformer. Transformers are deep learning models that use attention mechanisms to weigh the importance of different words.
  2. This allows it to use something called token probability distribution to assign a score based on likelihood of a certain word coming after another word.
  3. GPT 3 selects the word with the highest likelihood score after each word to piece together sentences.

Self-Attention

GPT-3 uses something called self-attention in order to focus on the words that are most important. It does this by taking in the understanding of other associated words that serve as relevant context to the word that will be processed. This allows GPT-3 to focus on only the most important words, making the results more accurate. The word being process has a query representation of the words that will be used against the other words. All the other words have keys that identify them and value vectors that are calculated to find the relevancy to the word being processed.


Our Approach

1

We researched GPT and familiarized ourselves with the concepts, then we decided to utilize the AITextGen tool to build and train a model for our project.

2

We cleaned our Amazon Review Dataset by splitting it into positive and negative reviews and removing labels with Python.

3

We researched the purpose of the model parameters and trained an instance of our model with initial values for the parameters.

4

After testing our model with prompts, we began the process of hypertuning: adjusting the paramaters, and increasing the amount of data in order to increase the accuracy of our model. We adjusted the learning rate, training time, and number of lines of preprocessed data.

Meet the students!

Verina Ko

Data Scientist, Web Developer

Verina will be a junior at Diamond Bar High School this fall. She loves dancing and singing. She also enjoys shopping in her free time.

Isaac Choi

Product Manager, Web Developer

Isaac will be a ninth grader at University Preparatory Academy this fall. He likes discussing history facts.

Claire Andrzejek

Product Manager, Web Developer

Claire will be an eighth grader at Convent of the Sacred Heart SF this fall. She plays the cello.

Alyzee Sosa

Machine Learning Engineer, Web Developer

Alyzee will be a senior at Downtown Magnets High School this fall. She likes coding and hopes to pursue a technological career in college. She also likes hiking and camping with her family.

Raphael Chaumartin

Data Scientist

Raphael will be a junior at West High School this fall. His favorite video game is Minecraft.

Charlotte Choi

Data Scientist, Web Developer

Charlotte will be a sophomore at Dos Pueblos High School this fall. She enjoys English, History, and her school's Mock Trial program. She also plays the cello.

Justin Yi

2021 Summer TA

Justin is the team's amazing TA.