Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations, CVPR 2019 (Oral)

Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations

The code of:

Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations, Jiwoon Ahn, Sunghyun Cho, and Suha Kwak, CVPR 2019 [Paper]

This repository contains a framework for learning instance segmentation with image-level class labels as supervision. The key component of our approach is Inter-pixel Relation Network (IRNet) that estimates two types of information: a displacement vector field and a class boundary map, both of which are in turn used to generate pseudo instance masks from CAMs.

Citation

If you find the code useful, please consider citing our paper using the following BibTeX entry.

@InProceedings{Ahn_2019_CVPR,
author = {Ahn, Jiwoon and Cho, Sunghyun and Kwak, Suha},
title = {Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}

Prerequisite

Python 3.7, PyTorch 1.1.0, and more in requirements.txt
PASCAL VOC 2012 devkit
NVIDIA GPU with more than 1024MB of memory

Usage

Install python dependencies

pip install -r requirements.txt

Download PASCAL VOC 2012 devkit

Follow instructions in http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit

Run run_sample.py or make your own script

python run_sample.py

You can either mannually edit the file, or specify commandline arguments.

Train Mask R-CNN or DeepLab with the generated pseudo labels

For the reports, we used Detectron.
- Run step/make_cocoann.py to create COCO-style annotations.
- Note: Do not employ https://storage.googleapis.com/coco-dataset/external/PASCAL_VOC.zip to measure the performance of the Mask R-CNN! It only contains bounding box annotations.
TorchVision now supports Mask R-CNN and DeepLab. I personally recommend to use this.

TO DO

Training code for MS-COCO
Code refactoring
IRNet v2

irn