Skip to content
/ ISTR Public
forked from hujiecpp/ISTR

ISTR: End-to-End Instance Segmentation with Transformers

License

Notifications You must be signed in to change notification settings

luyao-cv/ISTR

 
 

Repository files navigation

This is the project page for the paper:

ISTR: End-to-End Instance Segmentation via Transformers,
Jie Hu, Liujuan Cao, Lu Yao, ShengChuan Zhang, Yan Wang, Ke Li, Feiyue Huang, Rongrong Ji, Ling Shao
arXiv 2105.00637

Highlights:

  • GPU Friendly: Four 1080Ti/2080Ti GPUs can handle the training for R50, R101 backbones with ISTR.
  • High Performance: On COCO test-dev, ISTR-R50-3x gets 46.8/38.6 box/mask AP, and ISTR-R101-3x gets 48.1/39.9 box/mask AP.

Updates

  • (2021.05.03) The project page for ISTR is avaliable.

Models

Method inf. time box AP mask AP download
ISTR-R50-3x 17.8 FPS 46.8 38.6 model | log
ISTR-R101-3x 13.9 FPS 48.1 39.9 model | log
  • The inference time is evaluated with a single 2080Ti GPU.
  • We use the models pre-trained on ImageNet using torchvision. The ImageNet pre-trained ResNet-101 backbone is obtained from SparseR-CNN.

Installation

The codes are built on top of Detectron2, SparseR-CNN, and AdelaiDet.

Requirements

  • Python=3.8
  • PyTorch=1.6.0, torchvision=0.7.0, cudatoolkit=10.1
  • OpenCV for visualization

Steps

  1. Install the repository (we recommend to use Anaconda for installation.)
conda create -n ISTR python=3.8 -y
conda activate ISTR
conda install pytorch==1.6.0 torchvision==0.7.0 cudatoolkit=10.1 -c pytorch
pip install opencv-python
pip install scipy
pip install shapely
git clone https://github.com/hujiecpp/ISTR.git
cd ISTR
python setup.py build develop
  1. Link coco dataset path
ln -s /coco_dataset_path/coco ./datasets
  1. Train ISTR (e.g., with ResNet50 backbone)
python projects/ISTR/train_net.py --num-gpus 4 --config-file projects/ISTR/configs/ISTR-R50-3x.yaml
  1. Evaluate ISTR (e.g., with ResNet50 backbone)
python projects/ISTR/train_net.py --num-gpus 4 --config-file projects/ISTR/configs/ISTR-R50-3x.yaml --eval-only MODEL.WEIGHTS ./output/model_final.pth
  1. Visualize the detection and segmentation results (e.g., with ResNet50 backbone)
python demo/demo.py --config-file projects/ISTR/configs/ISTR-R50-3x.yaml --input input1.jpg --output ./output --confidence-threshold 0.4 --opts MODEL.WEIGHTS ./output/model_final.pth

Citation

If our paper helps your research, please cite it in your publications:

@article{hu2021ISTR,
  title={ISTR: End-to-End Instance Segmentation via Transformers},
  author={Hu, Jie and Cao, Liujuan and Yao, Lu and Zhang, ShengChuan and Li, Ke and Huang, Feiyue and Ji, Rongrong and Shao, Ling},
  journal={arXiv preprint arXiv:2105.00637},
  year={2021}
}

About

ISTR: End-to-End Instance Segmentation with Transformers

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 91.2%
  • Cuda 4.5%
  • C++ 3.6%
  • Other 0.7%