Stereo Loop Detector

Code for the article "Addressing the challenges of loop detection in agricultural environments", published in the Journal of Field Robotics in 2024.
A pre-print version can be found in arxiv.org: https://arxiv.org/abs/2408.15761 An explanatory video can be found on YouTube: https://www.youtube.com/watch?v=DujsHfyyhNI

If you use this work in academic work please cite:

@article{soncini2024argiloopchallenges,
    author = {Nicol\'as, Soncini and Javier, Civera and Taih\'u, Pire},
    title = {Addressing the challenges of loop detection in agricultural environments},
    journal = {Journal of Field Robotics},
    volume = {},
    number = {},
    pages = {},
    keywords = {agricultural fields, loop detection, open fields, SLAM, visual place recognition},
    doi = {https://doi.org/10.1002/rob.22414},
    url = {https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.22414},
    eprint = {https://onlinelibrary.wiley.com/doi/pdf/10.1002/rob.22414}
}

and also

@article{GalvezTRO12,
    author={G\'alvez-L\'opez, Dorian and Tard\'os, J. D.},
    journal={IEEE Transactions on Robotics},
    title={Bags of Binary Words for Fast Place Recognition in Image Sequences},
    year={2012},
    month={October},
    volume={28},
    number={5},
    pages={1188--1197},
    doi={10.1109/TRO.2012.2197158},
    ISSN={1552-3098}
}

Make sure to read the LICENSE uploaded to this repo if you intend to use or modify this code.

For any comments or issues with the code please submit an issue/ticket through the platform or send us an e-mail to the address listed in the article.

Description

StereoLoopDetector works by detecting loops in sequences of stereo images collected by a robot or mobile platform and returning an approximate relative pose between them. It implements the algorithm presented in Gálvez-López D. et al. (2012) with additional modifications to support stereo images and relative pose estimation as presented in Soncini N. et al. (2024).

Building & Installing

All of the software was ran and tested with Ubuntu 22.04 and 24.04.

Dependencies

First you will have to install the dependencies, if you don't have them already installed and discoverable by CMake.

• cmake can be installed via aptitude sudo apt install cmake
• opencv + opencv_contrib (tested w/version 4.9.x)
  contrib modules used: sfm, ...
  usually has to be compiled from source for contrib modules
• boost additional modules used: boost::dynamic_bitset (for BRIEF), boost::program_options (for demos) can be installed via aptitude sudo apt install libboost-dev
  usually has to be compiled for these extra modules
  usually has to be compiled from source for additional modules
• yaml-cpp used for parsing stereo parameters, can be removed from code easily installed automatically when cmake + make runs
• DLib + PR patches installed automatically when cmake + make runs
• DBoW2 + PR patches installed automatically when cmake + make runs

Building

To configure the project using cmake in a build folder:

cmake -S . -B build

add -GNinja if you have ninja installed.

To build:

cmake --build build

or use make from the build folder.

Usage

To run the code you need to have 5 things ready:

• the stereo images
• two files with newline-separated full paths to the images (one for the left and one for the right camera)
• a stereo camera calibration file (example provided in resources/fs_stereo_parameters.yaml)
• a file with the poses of the images (or empty file if not displaying anything)
• a vocabulary generated by DBoW2 of whichever feature detector+descriptor used (we recommend checking ORB_SLAM3, S S-PTAM or similar open-sourced libraries for pre-generated ones)

Then calling the already built StereoLoopDetector is as easy as:

cd build/
./demo_stereo -h

and completing the necessary listed parameters.

Reproducing our results

Our article showcases the results of the StereoLoopDetector on the FieldSAFE dataset, for which we had to extract the images, synchronize them with the GPS positions, project the GPS coordinates to a planar projection (for plotting), generate the calibration files, etc.

We cannot provide the dataset as it is not of our authorship, but we can provide a step-by-step to download and test on them. The scripts for evaluation and visualization are not included just yet.

Steps to reproduce:

1. download the FieldSAFE session of your choice from the dataset website (e.g. FieldSAFE Dynamic obstacle session #1)
2. extract the stereo images from the rosbag, we chose to name them with the timestamp and saving the left and right stereo images to separate folders,
3. filter the images to be certain that every left image has a right image (we detected this was not always the case, even though they were supposedly hardware triggered),
4. (optional) create the poses .csv file by extracting the GPS poses, projecting them to a plane and making sure that each image has a corresponding pose (interpolate if needed); optionally just use the empty file in resources/fs_empty_poses.txt, just be aware that the visualziation tools that were carried on from DLoopDetector will not work,
5. find out the camera parameters and create the stereo parameters yaml (or use the one in resources/fs_stereo_parameters.yaml),
6. download a vocabulary file for the ORB features (e.g. ORB_SLAM3 Vocabulary file pairs with our ORB_SLAM3 ORB extractor)
7. run the demo!

Acknowledgements

We thank the original authors of DLoopDetector for making the code open and available to let us stand on the shoulders of giants to more easily contribute our enhancements for agricultural stereo loop detection. We thank the authors of S-PTAM for makign their code open and allowing us to take inspiration on stereo SLAM. We thank the autors of ORB_SLAM for making their ORB feature detector and descriptor open along with their DBoW2 vocabulary so we didn't have to retrain one of our own for initial tests. We thank ourselves, because getting experimental code to be mostly readable and usable by the community takes time.