COMP 3132 - Python ML, part 2

Course outline

• Draft

Book

• Deep Learning with Python by François Chollet
  • Online audio version (karaoke style, first 3 chapters are free)
  • The audio version is also available on Audible
  • Book GitHub page

Introductory videos

• Neural networks playlist by Grant Sanderson of 3blue1brown.com
• Hello World Canada: The Rise of AI - a motivating 50 minute documentary by Bloomberg
• The Neuron - how real neurons in our brain work (2 min). Remember that artificial neural networks are only "inspired" by biological neurons, and don't fully mimic how brain works. Moreover, we don't yet fully understand how brain works.
• Overview of DL tools by Siraj Raval

Installing Keras on your laptop

Note: This will take some time and download several hundred MB.

If you already have Anacanoda, open Anaconda prompt and first update conda by running

conda update conda

Then install keras

conda install keras

Test it by running import keras in python, you should see output like Using TensorFlow backend or Using CNTK backend - either is good. If you see any warnings about lack of GPU, that's also ok.

If you have a recent Nvidia GPU you can try following the CNTK instructions to istall the GPU version of CNTK but this is optional.

Week 1

• Lab - exercises/intro_MNIST.md
• Reading for next week:
  • Chapter 1 of the book (available free on book website)
  • First 2 videos of the Neural networks playlist from 3Blue1Brown
• Home assignment 1 - exercises/home_assignment1.md (due Jan 23rd)

Week 2

• Lecture slides
  • Animation of 2D convolution
  • How Blurs & Filters Work - Computerphile (8 min)
• Lab - exercises/convolution.ipynb
• Reading for next week:
  • Chapter 2 of the book (skip the heavy math)
  • The remaining 2 videos of the Neural networks playlist from 3Blue1Brown
• Reminder: home assignment 1 is due Jan 23rd

Week 3

• Lecture slides
• Lab - exercises/imdb_reviews.md
• Reading for next week:
  • Review model evaluation techiques - videos #5 and #7 from Kevin's sklearn videos
  • Chapter 3 intro and sections 3.1, 3.2 and 3.4

Week 4

• Lecture slides
• Lab - exercises/boston_housing.md
• Reading for next week:
  • Book sections 3.5, 3.6, 3.7
  • Walkthrough of the MNIST example with Keras

Week 5

• Lecture slides
• Lab - exercises/kaggle_housing.md

Week 6

• There was no lecture
• Lab. Revising the MPG dataset using this notebook from the previous semester. Replace the decision tree with a neural network.

Week 7 - Midterm

Prep advice

• Solve lab exercises
• Go over the MNIST video from sentdex
• Read chapters 3 of the book (skip section 3.3)
• Solve more lab exercises
• From chapter 4 read till section 4.3
• Read chapters 1 and 2 of the book

Main topics and concepts to be familiar with

• Data manipulation in NumPy and Pandas
• Metrics used for evaluating and comparing ML models of different types
• Overfitting
• Train/validation/test data splitting and why we do this
• Concepts about neural nets - weights, layers, activation functions, loss functions

Home assignment 2 - due March 13

• Instructions
• This home assignment is also an excellent practice exercise for the midterm

Week 8 - Intersession

Week 9

• Lecture slides
• Lab - exercises/convnets_mnist.md
• Reading:
  • Chapter 5, sections 5.1 & 5.2
  • A great article about convnets with historical background and intuitive explanation of how they work.
  • Videos:
    • 3 minute explanatioin of the convolution in convnets - from this Udacity course
    • Animation of 2D convolution
    • How Blurs & Filters Work - Computerphile (8 min)
    • Lecture on convnets by Siraj Raval (46 min)

Week 10

• Lecture slides
• Lab - exercises/pretrained.md

Week 11

• Lecture slides
• Lab - continue wroking on exercises/pretrained.md

Home assignment 3 - due April 10

• Instructions

Week 12

• Lecture slides
• Lab - exercises/rnn.ipynb

Week 13

• Lecture slides
• Lab - exercises/tuning.ipynb

Week 14

• Lecture slides

Exam prep adivce

• Re-read chapter 4
• Experiment with lab exercise from week 13