2. Pre-processing text (For topic modeling)

[tmozgach]

• Tokenize, normalize (lowercase)
• Stop word removal (specific words for the domain)
• Stemming: use weak morphological treatments?
  "At least for English, morphological conflation treatments such
  as stemmers and lemmatizers can worsen topic
  model quality on a variety of measures, while LDA
  turns out to be quite good at combining morpho-
  logical variants by itself."
• Bigrams ? https://markroxor.github.io/gensim/static/notebooks/lda_training_tips.html
  https://github.com/shichaoji/easyLDA/blob/master/build/lib/easyLDA/base.py (n_gram)
  https://www.datascience.com/resources/notebooks/data-science-summarize-hotel-reviews

[tmozgach]

Understanding Text Pre-Processing for Latent Dirichlet Allocation
http://www.cs.cornell.edu/~xanda/winlp2017.pdf

[tmozgach]

Steps for effective text data cleaning (with case study using Python)
https://www.analyticsvidhya.com/blog/2015/06/quick-guide-text-data-cleaning-python/