(Preliminary schedule, subject to change) Date Topic Reading Assignment due Mon, Aug 29 Introduction and Motivation (book chapter) Wed, Aug 31 From Models to Systems (book chapter) Building Intelligent Systems, Ch. 5, 7, 8 Fri, Sep 02 Git & ML APIs Mon, Sep 05 Labor day, no classes Wed, Sep 07 Model Quality (book chapter 1, chapter 2) Building Intelligent Systems, Ch. 19 I1: ML Product Teamwork Primer Fri, Sep 09 Stream processing: Apache Kafka Mon, Sep 12 Model Testing Beyond Accuracy (book chapter) Behavioral Testing of NLP Models with CheckList Wed, Sep 14 Goals and Measurement (book chapter 1, book chapter 2) Building Intelligent Systems, Ch. 2, 4 Fri, Sep 16 Measurement and Teamwork Mon, Sep 19 Gathering and Untangling Requirements (book chapter) The World and the Machine Wed, Sep 21 Planning for Mistakes (book chapter) Building Intelligent Systems, Ch. 6, 7, 24 M1: Modeling and First Deployment Fri, Sep 23 Requirements and Risk Analysis Mon, Sep 26 Toward Architecture and Design (book chapter 1, chapter 2, chapter 3) Building Intelligent Systems, Ch. 18 & Choosing the right ML alg. Wed, Sep 28 Deploying a Model (book chapter) Building Intelligent Systems, Ch. 13 and Machine Learning Design Patterns, Ch. 16 I2: Requirements Fri, Sep 30 Architecture & Midterm Questions Mon, Oct 03 Testing in Production (book chapter) Building Intelligent Systems, Ch. 14, 15 Wed, Oct 05 Midterm Fri, Oct 07 Containers: Docker (Code) Mon, Oct 10 Infrastructure Quality and MLOps (book chapter 1, book chapter 2, book chapter 3, operations chapter) The ML Test Score Wed, Oct 12 Data Quality (book chapter) Data Cascades in High-Stakes AI I3: Architecture Fri, Oct 14 Unit Tests and Continuous Integration (PDF, Code, Video) Mon, Oct 17 Fall break, no classes Wed, Oct 19 Fall break, no classes Fri, Oct 21 Fall break, no classes Mon, Oct 24 Scaling Data Storage and Data Processing (book chapter) Big Data, Ch. 1 Wed, Oct 26 Process & Technical Debt (book chapter 1, chapter 2) Hidden Technical Debt in Machine Learning Systems Fri, Oct 28 Tartan community day, no classes Mon, Oct 31 Responsible ML Engineering (book chapter 1, chapter 2) Algorithmic Accountability: A Primer Wed, Nov 02 Measuring Fairness (book chapter) Improving Fairness in Machine Learning Systems M2: Infrastructure Quality Fri, Nov 04 Monitoring: Prometheus, Grafana Mon, Nov 07 Building Fairer Products (book chapter) A Mulching Proposal Wed, Nov 09 Explainability & Interpretability (book chapter) Black boxes not required or Stop Explaining Black Box ML Models… I4: MLOps Tools: Aequitas, Aim, Amazon ECS, ArangoDB, Artillery, Assertible, AWS Cloudwatch, AWS DocumentDB, AWS Glue, Azure Pipelines to deploy on Azure Kubernetes Service, Brooklin, ClearML, Cronitor (ML Pipelines), d6tflow, Dagster, DataPrep, deepchecks, Elasticsearch, FastAPI, Guild AI , HuggingFace, Katib, Kedro, Kubeflow, LightFM, Lightning AI, Logstash, Loki, Mlflow, MongoDB Compass, MySQL, Neptune AI, Neural Network Intelligence (NNI), OpenDP, optuna, Pachyderm, Ploomber, Postman, Prefect, PyJanitor, Qlik Sense, Quilt, Spacy, Splunk, TorchServe, Using Airflow , ZenML Fri, Nov 11 Fairness Mon, Nov 14 Transparency & Accountability (book chapter) People + AI, Ch. Explainability and Trust Wed, Nov 16 Versioning, Provenance, and Reproducability (book chapter) Building Intelligent Systems, Ch. 21 & Goods: Organizing Google's Datasets Fri, Nov 18 Model Explainability & Interpretability (PDF, Code, Video) Mon, Nov 21 Debugging (Guest lecture by Sherry Tongshuang Wu) - Wed, Nov 23 Thanksgiving break Fri, Nov 25 Thanksgiving break Mon, Nov 28 Security and Privacy (book chapter) Building Intelligent Systems, Ch. 25 & The Top 10 Risks of Machine Learning Security Wed, Nov 30 Safety (book chapter) Practical Solutions for Machine Learning Safety in Autonomous Vehicles M3: Monitoring and CD Fri, Dec 02 Threat modeling Mon, Dec 05 Fostering Interdisciplinary Teams (book chapter) Collaboration Challenges in Building ML-Enabled Systems Wed, Dec 07 Summary and Reflection M4: Fairness, Security and Feedback Loops Sun, Dec 18 (9:30-11:30am) Final Project Presentations Final report
