Negative Evidence Matters in Interpretable Histology Image Classification (MIDL2022)

by Soufiane Belharbi¹, Marco Pedersoli¹, Ismail Ben Ayed¹, Luke McCaffrey², Eric Granger¹

¹ LIVIA, Dept. of Systems Engineering, ÉTS, Montreal, Canada
² Goodman Cancer Research Centre, Dept. of Oncology, McGill University, Montreal, Canada

Abstract

Using only global image-class labels, weakly-supervised learning methods, such as class activation mapping, allow training CNNs to jointly classify an image, and locate regions of interest associated with the predicted class. However, without any guidance at the pixel level, such methods may yield inaccurate regions. This problem is known to be more challenging with histology images than with natural ones, since objects are less salient, structures have more variations, and foreground and background regions have stronger similarities. Therefore, computer vision methods for visual interpretation of CNNs may not directly apply. In this paper, a simple yet efficient method based on a composite loss is proposed to learn information from the fully negative samples (i.e., samples without positive regions), and thereby reduce false positives/negatives. Our new loss function contains two complementary terms: the first exploits positive evidence collected from the CNN classifier, while the second leverages the fully negative samples from training data. In particular, a pre-trained CNN is equipped with a decoder that allows refining the regions of interest. The CNN is exploited to collect both positive and negative evidence at the pixel level to train the decoder. Our method called NEGEV benefits from the fully negative samples that naturally occur in the data, without any additional supervision signals beyond image-class labels. Extensive experiments github.com/sbelharbi/negev show that our proposed method can substantial outperform related state-of-art methods on GlaS (public benchmark for colon cancer), and Camelyon16 (patch-based benchmark for breast cancer using three different backbones). Our results highlight the benefits of using both positive and negative evidence, the first obtained from a classifier, and the other naturally available in datasets.

Code: Pytorch 1.10.0. This branch does not support multi-gpu.

Citation:

@inproceedings{negevsbelharbi2022,
  title={Negative Evidence Matters  in Interpretable Histology Image Classification},
  author={Belharbi, S. and  Pedersoli, M and Ben Ayed, I. and McCaffrey, L. and Granger, E.},
  booktitle = {Medical Imaging with Deep Learning (MIDL)},
  year      = {2022}
}

Content:

• Method
• Results
• Requirements
• Datasets
• Run code

Method:

Results:

Requirements:

• Python 3.7.10
• Pytorch 1.10.0
• torchvision 0.11.1

pip install torch==1.10.0 -f https://download.pytorch.org/whl/cu111/torch-1.10.0%2Bcu111-cp37-cp37m-linux_x86_64.whl
pip install torchvision==0.11.1 -f https://download.pytorch.org/whl/cu111/torchvision-0.11.1%2Bcu111-cp37-cp37m-linux_x86_64.whl

• Full dependencies
• Build and install CRF:
  • Install Swig
  • CRF (not used in this work, but it is part of the code.)

cdir=$(pwd)
cd dlib/crf/crfwrapper/bilateralfilter
swig -python -c++ bilateralfilter.i
python setup.py install
cd $cdir
cd dlib/crf/crfwrapper/colorbilateralfilter
swig -python -c++ colorbilateralfilter.i
python setup.py install

Download datasets :

2.1. Links to dataset:

• GlaS
• Camelyon16

2.2. Download datasets:

• GlaS: ./download-glas-dataset.sh.

You find the splits in ./folds.

Run code :

• CAM-method: CAM over GLAS using ResNet50:

cudaid=$1
export CUDA_VISIBLE_DEVICES=$cudaid
export OMP_NUM_THREADS=50

python main_wsol.py \
       --task STD_CL \
       --encoder_name resnet50 \
       --arch STDClassifier \
       --runmode final-mode \
       --opt__name_optimizer sgd \
       --batch_size 32 \
       --eval_checkpoint_type best_localization \
       --opt__step_size 250 \
       --opt__gamma 0.1 \
       --max_epochs 1000 \
       --freeze_cl False \
       --support_background True \
       --method CAM \
       --spatial_pooling WGAP \
       --dataset GLAS \
       --fold 0 \
       --cudaid 0 \
       --debug_subfolder None \
       --amp True \
       --opt__lr 0.003 \
       --exp_id 11_19_2021_09_32_36_109051__423849

• NEGEV-method: over GLAS using ResNet50:

cudaid=$1
export CUDA_VISIBLE_DEVICES=$cudaid
export OMP_NUM_THREADS=50
python main_wsol.py \
       --task NEGEV \
       --world_size 1 \
       --task NEGEV \
       --encoder_name resnet50 \
       --arch UnetNEGEV \
       --runmode final-mode \
       --opt__name_optimizer sgd \
       --dist_backend mpi \
       --batch_size 32 \
       --eval_checkpoint_type best_localization \
       --opt__step_size 250 \
       --opt__gamma 0.1 \
       --max_epochs 1000 \
       --freeze_cl True \
       --support_background True \
       --method CAM \
       --spatial_pooling WGAP \
       --dataset GLAS \
       --fold 0 \
       --cudaid 0 \
       --debug_subfolder None \
       --amp True \
       --opt__lr 0.1 \
       --negev_ptretrained_cl_cp best_classification \
       --negev_ptretrained_loc_cp best_localization \
       --elb_init_t 1.0 \
       --elb_max_t 10.0 \
       --elb_mulcoef 1.01 \
       --sl_ng True \
       --sl_ng_seeder probability_seeder \
       --sl_ng_lambda 1.0 \
       --sl_ng_start_ep 0 \
       --sl_ng_end_ep -1 \
       --sl_ng_min 1 \
       --sl_ng_max 1 \
       --sl_ng_ksz 3 \
       --crf_ng False \
       --jcrf_ng False \
       --neg_samples_ng False \
       --max_sizepos_ng False \
       --exp_id 12_13_2021_00_49_48_796469__3314599

• Train the CAM-method first. Then, copy the best model from the exp folder into the folder ./pretrained. Copy the whole folder with this name format GLAS-0-resnet50-CAM-WGAP-cp_best_localization and GLAS-0-resnet50-CAM-WGAP-cp_best_classification.