Plan

[tmozgach]

• Get data from 2012.1.1 to 2017.12.31;
• Get data about author, karma, data of join;
• Topic Modeling;
• Sentiment analysis;

[tmozgach]

Plan for this week (19-25 Feb):

• Try to train on small data;
• Integrated visualization; https://github.com/shichaoji/easyLDA/blob/master/build/lib/easyLDA/base.py
• Understand parameters; https://markroxor.github.io/gensim/static/notebooks/lda_training_tips.html
• Introduction to Cedar;

[tmozgach]

2.0
Text analysis on the title and the body of the questions in the thread.

1. for the titles

• a) conduct topic modelling trying 30, 40, 50 as topic numbers
• b) check length of the words. (It’s the number of words)
• d) check the sentiment being positive or negative

2. for the bodies

• a) conduct topic modelling trying 30, 40, 50 as topic numbers
• b) check length of the words
• c) check the sentiment being positive or negative

[tmozgach]

• Delete first row (col1)
• Check Topic Modeling parameters;
• Run on Cedar;
• Combine everything together;
• Write a guide;
• Submit everything;

[tmozgach]

• check if it is a question format (e.g., starting with ‘what', ‘how’ or end with question mark), or statement format 4. Find if a sentence is in a questioning form #11