A Practical Guide to Bayesian Optimization

This repo is a collection of code associated with the Data Science Nashville meetup held on January 24, 2018: A Practical Guide to Bayesian Optimization. The presentation can be viewed as a powerpoint (Data_science_nashville_jkk.pptx) or a pdf (Data_science_nashville_jkk.pdf).

• bayes_opt.yml: Conda environment necessary to run notebooks.
• bayes_opt_example.ipynb: Walkthough of basic Bayesian Optimization technique.
• gp_example.ipynb: Visualization of Gaussian process regression in 1D and 2D.
• gp_kernel.ipynb: Visualization of different kernels used to fit various target functions.
• gp_scale.ipynb: Exploration of scaling issues that can arise when using Gaussian process regression.
• mercari_modified_utility.py: Example code showing efficient initialization of Bayesian Optimization object.
• mercari_prep.ipynb: Example of Bayesian Optimization used in preprocessing steps on Mercari Price Prediction Challenge.
• mercari_train.ipynb: Example of Bayesian Optimization used to train a Catboost Regressor on output from mercari_prep.ipynb.
• simple_classification.ipynb: Example of Bayesian Optimization used on San Francisco Crime Classification dataset.
• simple_regression.ipynb: Example of Bayesian Optimization used on House Sales in King County, USA dataset.
• tuning_strategies.ipynb: Visualization of traditional tuning strategies.
• utility_functions.ipynb: Visualization of different utility functions used in Bayesian Optimization.

References

Bayesian Optimization Package

• Bayesian Optimization

Excellent series of video from Nando de Freitas' course @ UBC.

• Introduction to Gaussian Processes
• Regression with Gaussian Processes
• Bayesian Optimization and Multi-Arm Bandits

A free book on Gaussian Processes.

• Gaussian Processes for Machine Learning

A nice publication from Nando de Freitas' group @ UBC.

• A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning

Scikit-Learn's excellent documentation on Gaussian Processes.

• Gaussian Processes