DenseNet with TensorFlow

Two types of Densely Connected Convolutional Networks (DenseNets) are available:

• DenseNet - without bottleneck layers
• DenseNet-BC - with bottleneck layers

Each model can be tested on such datasets:

• Cifar10
• Cifar10+ (with data augmentation)
• Cifar100
• Cifar100+ (with data augmentation)
• SVHN

A number of layers, blocks, growth rate, image normalization and other training params may be changed trough shell or inside the source code.

Example run:

python run_dense_net.py --train --test --dataset=C10

List all available options:

python run_dense_net.py --help

There are also many other implementations - they may be useful also.

Citation:

@article{Huang2016Densely,
       author = {Huang, Gao and Liu, Zhuang and Weinberger, Kilian Q.},
       title = {Densely Connected Convolutional Networks},
       journal = {arXiv preprint arXiv:1608.06993},
       year = {2016}
}

Test run

Test results on various datasets. Image normalization per channels was used. Results reported in paper provided in parenthesis. For Cifar+ datasets image normalization was performed before augmentation. This may cause a little bit lower results than reported in paper.

Model type	Depth	C10	C10+	C100	C100+
DenseNet(k = 12)	40	6.67(7.00)	5.44(5.24)	27.44(27.55)	25.87(24.42)
DenseNet-BC(k = 12)	100	5.54(5.92)	4.87(4.51)	24.88(24.15)	22.85(22.27)

Approximate training time for models on GeForce GTX TITAN X GM200 (12 GB memory):

• DenseNet(k = 12, d = 40) - 17 hrs
• DenseNet-BC(k = 12, d = 100) - 1 day 18 hrs

Difference compared to the original implementation

The existing model should use identical hyperparameters to the original code. If you note some errors - please open an issue.

Also it may be useful to check my blog post Notes on the implementation DenseNet in tensorflow.

Dependencies

• Model was tested with Python 3.4.3+ and Python 3.5.2 with and without CUDA.
• Model should work as expected with TensorFlow >= 0.10. Tensorflow 1.0 support was recently included.

Repo supported with requirements file - so the easiest way to install all just run pip install -r requirements.txt.