learning-deep-learning

Course program

• Basics of deep learning

  1. Introduction, backpropagation algorithm
  2. Empirical risk minimization, standard loss functions, linear classification, stochastic optimizers

• Computer vision

  1. Convolutional networks (ConvNets), classifying images. Homework
  2. "Deep" computer vision beyond classification: Verification tasks, object detection architectures, semantic segmentation
  3. Generation networks

    -- Article deadline: Journal club --
  

  4. Generation: AE, VAE, GAN. Homework

• Natural language processing

  1. Word embeddings, word2vec and other variants, convolutional networks for natural language
  2. RNN, LSTM. Homework
  3. Sequence2sequence, attention, transformers and other advanced techniques

  -- Article deadline: present current results --
  

• Deep reinfocrement learning. Homework

• Adverserial examples, MobileNet, distillation, dark knowledge

  -- Artice deadline: present final results --
  

Classes

Lecture 1. (6.02.2019)

Lecture Introduction, backpropagation algorithm .pptx, .pdf

Seminar Introduction to pytorch.

Homework: Fill Seminar 1 notebook and send it to AnyTask until 13.02.19 (8:00)

Lecture 2. (13.02.2019)

Lecture Optimization for Deep Learning .pptx, .pdf.

Seminar High level pytorch.

Homework:

• Please read good logloss explanation here: https://dyakonov.org/2018/03/12/логистическая-функция-ошибки/
• Fill Seminar 2 notebook and send it to AnyTask until 20.02.19 (8:00)

Lecture 3. (20.02.2010)

Lecture Convolutional Networks. .pptx, .pdf

Seminar Cifar10 finetuning.

Homework:

• Fill Seminar 3 notebook
• HW1 Cifar10 classification.

Lecture 4. (27.02.2019)

Lecture Convolutional Networks in Computer Vision (segmentation, detection, verification) .pptx, .pdf

Seminar Dense prediction.

Homework:

• Fill Seminar 4 notebook
• Find an article for journal club (13.03).

Lecture 5. (06.03.2019)

Lecture Generative Convolutional Networks .pptx, .pdf

Seminar Neural Style Transfer.

Homework:

• Fill Seminar 5 notebook
• Find an article for journal club (13.03).

Journal club. (13.03.2019)

Lecture 6. (20.03.2019)

Lecture Autoencoders and GANs. .pptx, .pdf

Useful link – VAE: https://neurohive.io/ru/osnovy-data-science/variacionnyj-avtojenkoder-vae/

Seminar Fashion-MNIST GAN

Homework:

• Fill Seminar 6 notebook

Lecture 7. (27.03.2019)

Lecture NLP intro, ConvNets for NLP, Word embeddings.

Seminar w2v

Homework:

• Fill Seminar 7 notebook

Lecture 8. (3.04.2019)

Lecture RNN, LSTM. .pptx, .pdf

Seminar Char RNN

Homework:

• LSTM – http://colah.github.io/posts/2015-08-Understanding-LSTMs/
• То же на русском – https://habr.com/ru/company/wunderfund/blog/331310/
• Attention – https://www.youtube.com/watch?v=k63pDjKV3Ew
• Fill Seminar 8 notebook
• HW3 Image captioning

Lecture 9. (10.04.2019)

Lecture Speech2Text. Seq2seq. Transformer .pptx, .pdf

Seminar Seq2seq

Homework:

• Fill Seminar 9 notebook
• Upload intermediate results of course work

Lecture 10. (17.04.2019)

Lecture Reinforcement learning w/o NN. .pptx, .pdf

Seminar Q-learning

Homework:

• Fill Seminar 10 notebook

Сourse evaluation criteria

• 10 seminar tasks: 4 points each
• 4 homeworks: 10 points each
• 1 article implementation + journal club. 5 (journal club) + 5 (current results) + 10 (final results) points

Submissions missed deadlines are estimated at half points maximum.

Total sum is 100 points. Course grades:

• 80 points -> 8/10
• 50 points -> 5/10
• 30 points -> 3/10