Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy emotion model #179

Open
wants to merge 2 commits into
base: development
Choose a base branch
from
Open

Conversation

crystalbroderick
Copy link
Contributor

@crystalbroderick crystalbroderick commented Mar 24, 2022

closes #157
Train and deploy emotion model using Sagemaker, and built-in algo "blazingtext."

** I suggest not running each step the notebooks since these process a lot of data and training costs..

Updated Emotions2.ipynb steps and added datacleaning:

  1. Get Data: Get feedback data from s3 bucket
  2. Clean dataset: remove stopwords, punctuation, numbers and convert to lower case
  3. Find emotion per sentence
  • We are using the text2emotion library for determining the emotion per sentence by matching certain words with emotion.
  1. Add emotion to each sentence
  2. Export analysis to CSV for training and deploying the model

Created emotions-model.ipynb for training and deployment for endpoint

  1. Get data from csv created in Emotions2.ipynb
  2. Convert emotions to numerical value
  3. Prepare dataset:
  • move emotional value to first column
  • add label_ for the supervised learning
  • tokenize sentences
  1. split data into 80% training 20% validation
  2. Upload training and validation csv files to notebook and s3 bucket.
  3. Train the model:
  • set up estimator with parameters to launch the training job
  1. Train the algorithm
  2. Get train accuracy and validation accuracy
  3. Deploy the model / endpoint - test sentences and check to see if the emotional value is correct.
  4. delete the endpoint to save costs.

TIME: 20 hours?!

TEST

  1. Go to sagemaker console
  2. Click notebook in left navigation bar
  3. Choose notebook instances
  4. find emotion-analysis notebook
  5. find [emotion-analysis] > choose "open jupyter" https://emotion-analysis-nvi9.notebook.us-west-2.sagemaker.aws/tree
  6. open emotions2 file
  7. review steps (i suggest not running everything)
  8. open emotions-deploy
  9. review steps
  10. if you want to test the endpoint start on 18 (under hosting) and add your own sentence. it will take a few minutes to create a new endpoint. Make sure you delete the endpoint afterwards.
  11. All training jobs are in s3 bucket:

s3 bucket

traindeploy

response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Demo and explain the Emotion algorithm
1 participant