A Neural Algorithm of Artistic Style (Implementation)

An implementation of the arXiv preprint A Neural Algorithm of Artistic Style [1] & paper Image Style Transfer Using Convolutional Neural Networks [2].

Original Photograph: Tubingen, Germany

---

Claude Monet's Houses of Parliament

Pablo Picasso's Seated Nude

Edvard Munch's The Scream

Vincent van Gogh's The Starry Night

William Turner's The Shipwreck of The Minotaur

Wassily Kandinsky's Composition VII

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

1. python3 - Programming Environment
2. pip3 - Python Dependency Management

Installing

To install dependent modules:

pip3 install -r requirements.txt

Project Components

neural_stylization contains Python modules with utility methods and classes for the project.

VGG19

This project relies on the VGG19 architecture. VGG19-classification.ipynb outlines some basic image classification using the network with weight-set W pre-trained on the ImageNet dataset. The implementation of VGG19 can be found in neural_stylization/vgg19.py. Utility methods for loading manipulating, and normalizing image can be found in neural_stylization/img_util.py.

Content Reconstruction

content-reconstruction.ipynb describes the content reconstruction process from white noise. Performing gradient descent of the content loss on a white noise input x for a given content p yields a representation of the networks activation for a given layer l.

Style Reconstruction

style-reconstruction.ipynb describes the style reconstruction process from white noise. Performing gradient descent of the style loss on a white noise input x for a given artwork a yields a representation of the networks activation for a given set of layers L.

Style Transfer

style-transfer.ipynb describes the style transfer process between a white noise image x, a content image p, and a style representation a. Performing gradient descent of the content loss and style loss with respect to x impressions the content of p into x, bearing local styles, and colors from a.

Photo-Realistic Style Transfer

photo-realistic-style-transfer.ipynb describes the photo-realistic style transfer process. Opposed to transfering style from an artwork, this notebook explores transfering a nighttime theme from a picture of one city to a daytime picture of another city with mixed results.

Content Layer Selection

effect-of-content-layer.ipynb visualizes how the style transfer is affected by using different layers for content loss.

Style Layer Selection

effect-of-style-layers.ipynb visualizes how the style transfer is affected by using different sets of layers for style loss.

Optimizers

optimizers.ipynb employs gradient descent, adam, and L-BFGS to understand the affect of different blackbox optimizers. Gatys et. al use L-BFGS, but Adam appears to produce competetive results too.

Acknowledgments

• keras-team provides Keras, a high level neural network framework. They also provide the pre-trained ImageNet weights and some tutorials that help build this project.

References

[1] L. A. Gatys, A. S. Ecker, and M. Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2015.

[2] L. A. Gatys, A. S. Ecker, and M. Bethge. Image style transfer using convolutional neural networks. In Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on, pages 2414–2423. IEEE, 2016.