README.md

DMF-GAN

DMF-GAN: Deep Multimodal Fusion Generative Adversarial Networks for Text-to-Image Synthesis

Official Pytorch implementation for our paper [DMF-GAN: Deep Multimodal Fusion Generative Adversarial Networks for Text-to-Image Synthesis]

---

Requirements

• python 3.8
• Pytorch 1.9
• At least 1x24GB NVIDIA GPU

Installation

Clone this repo.

git clone https://github.com/tobran/DMF-GAN
pip install -r requirements.txt
cd DMF-GAN/code/

Preparation

Datasets

1. Download the preprocessed metadata for birds coco and extract them to data/
2. Download the birds image data. Extract them to data/birds/
3. Download coco2014 dataset and extract the images to data/coco/images/

Training

cd DF-GAN/code/

Train the DF-GAN model

For bird dataset: python main.py --cfg cfg/bird.yml

For coco dataset: python main.py --cfg cfg/coco.yml

The code is released for academic research use only. For commercial use, please contact Xueqin Xiang.

Reference

• StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks [code]
• AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks [code]
• DM-GAN: Realistic Image Synthesis with Stacked Generative Adversarial Networks [code]
• DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis [code]
• Recurrent Affine Transformation for Text-to-image Synthesis [code]