This repository provides official models from the paper Light-Weight RefineNet for Real-Time Semantic Segmentation, available here
Light-Weight RefineNet for Real-Time Semantic Segmentation
Vladimir Nekrasov, Chunhua Shen, Ian Reid
In BMVC 2018
For flawless reproduction of our results, the Ubuntu OS is recommended. The models have been tested using Python 2.7 and Python 3.6.
pip, pip3
torch>=0.4.0
To install required Python packages, please run pip install -r requirements.txt (Python2), or pip3 install -r requirements3.txt (Python3) - use the flag -u for local installation.
The given examples can be run with, or without GPU.
For the ease of reproduction, we have embedded all our examples inside Jupyter notebooks. One can either download them from this repository and proceed working with them on his/her local machine/server, or can resort to online version supported by the Google Colab service.
If all the installation steps have been smoothly executed, you can proceed with running any of the notebooks provided in the examples/notebooks folder.
To start the Jupyter Notebook server, on your local machine run jupyter notebook. This will open a web page inside your browser. If it did not open automatically, find the port number from the command's output and paste it into your browser manually.
After that, navigate to the repository folder and choose any of the examples given.
The number of FLOPs and runtime are measured on 625x468 inputs using a single GTX1080Ti, mean IoU is given on corresponding validation sets with a single scale input.
| Models | PASCAL VOC | Person-Part | PASCAL Context | NYUv2, 40 | Params, M | FLOPs, B | Runtime, ms |
|---|---|---|---|---|---|---|---|
| RF-LW-ResNet-50 | 78.5 | 64.9 | - | 41.7 | 27 | 33 | 19.56±0.29 |
| RF-LW-ResNet-101 | 80.3 | 66.7 | 45.1 | 43.6 | 46 | 52 | 27.16±0.19 |
| RF-LW-ResNet-152 | 82.1 | 67.6 | 45.8 | 44.4 | 62 | 71 | 35.82±0.23 |
| RF-LW-MobileNet-v2 | 76.2 | - | - | - | 3.3 | 9.3 | - |
Inside the notebook, one can try out their own images, write loops to iterate over videos / whole datasets / streams (e.g., from webcam). Feel free to contribute your cool use cases of the notebooks!
If you do not want to be involved in any hassle regarding the setup of the Jupyter Notebook server, you can proceed by using the same examples inside the Google colab environment - with free GPUs available!
Once time permits, more things will be added to this repository:
- NASNet-Mobile
- CityScapes' models
- Full training pipeline example
- Evaluation scripts
Our most recent work on real-time joint semantic segmentation and depth estimation is built on top of Light-Weight RefineNet with MobileNet-v2. Check out the paper here; the models are soon to be released!
This project is licensed under the MIT License - see the LICENSE file for details
- University of Adelaide and Australian Centre for Robotic Vision (ACRV) for making this project happen
- HPC Phoenix cluster at the University of Adelaide for making the training of the models possible
- PyTorch developers
- Google Colab
- Yerba mate tea