Skip to content

Commit

Permalink
Cherry pick Kinetic updates to the new stable melodic branch (#123)
Browse files Browse the repository at this point in the history
* Patch persistent voxel decay model returning the wrong duration

* Fix typo and implement acceleration in persistent decay

* implementing abstract class for configurable frustum models (#93)

* implementing abstract class for configurable frustum models

* stoping iterator deletion from screwing up the cache

* deleting costmap pointer only if created

* fixing no clearing observations but allowing for global decay (#97)

* fixing no clearing observations but allowing for global decay

* saving a little bit of allocation

* removing removed space

* removing instructions tripping up people

* deleting remaining comment on raw install

* Vlp16 (#104)

* Add support for 3D scanning lidars (Velodyne RoboSense, Ouster, etc)

* various formatting cleanup (#105)

* Dynamic reconfigure proposal continuing (#98)

* Added primitive dynamic reconfigure configuration

* Added naive callback for dynamic reconfigure

* Using positive values in voxel_decay due to dynamic reconfigure
limitations

* using only positive values in voxel decay

* Using raw poiters to follow project's convenction

* We use c++17 so we have templated lambdas!

* Added publishing option to dynamic reconfigure

* adresing comments

* added enabled param for each observation buffer (wip)

- added a service to enable or disable each observation buffer
- c++14 istead of 17 kinetic compatible

* wip trying to make boostbind work with services

* ready to test, resolved requested changes

* commented reconfigure

* test with no pointers

* added reconfigure again

* fixed compilation errors due to the merge

* added service feedback

* Update README.md

* small changes

* add a fix for not clearing local map with deactivated device

* Added params to reconfigure

- addressed all requested changes
- fix to CMakelist closes #99

* solved bug, decay model was not beig updated in reconfigure

* removed layer reset when changing camera

* new param to disable services if the user doesn't want realtime disabling

* cleaned some more

* changed param name

* Revert "changed param name"

This reverts commit c195872.

* Revert "cleaned some more"

This reverts commit 7fe6292.

* Revert "new param to disable services if the user doesn't want realtime disabling"

This reverts commit 56df2a7.

* adressed comments

- Added clearer service names.
- Added mutex lock unlock in reconfigure

* changed reconfigure mutex from grid to layer

* added layer_grid locks for ensuring safe interaction

* still playing with mutex

-trying to solve blocking errors

* renamed _layer_lock to _voxel_grid_lock

* advertise publish boxels always

* Cleanup 98 (#106)

* various cleanup from 98

* replacing locking mutexes with scoped locks

* use recursive lock

* Allow for 1 stream to be in 1 source vs. 1 stream in 2 sources (#109)

* clearing after reading both marking and clearing

* remove unnecessary comment

* clear all measurements clear

* Vfov padding (#113)

* Update spatio_temporal_voxel_layer.cpp

* Update spatio_temporal_voxel_layer.cpp

* Update three_dimensional_lidar_frustum.hpp

* Update three_dimensional_lidar_frustum.cpp

* Update measurement_buffer.hpp

* Update measurement_buffer.cpp

* Update measurement_reading.h

* Update spatio_temporal_voxel_grid.cpp

* added missing argument

* Update vlp16_config.yaml

* Update three_dimensional_lidar_frustum.hpp

* Update three_dimensional_lidar_frustum.hpp

* Update spatio_temporal_voxel_layer.cpp

* Update README.md

* Update README.md

* Update measurement_buffer.hpp

* Update measurement_reading.h

* Update measurement_buffer.cpp

* adding list of environments tested in (#116)

* increment up

* adding vlp16 gif to readme (#119)

* adding vlp16 gif to readme

* oops, wrong URL

* try on one line

* well that didnt work...

* boldface

* resolve cherry pick from kinetic misses

* increment up for release
  • Loading branch information
SteveMacenski authored May 28, 2019
1 parent 1949041 commit 3eaa79b
Show file tree
Hide file tree
Showing 20 changed files with 845 additions and 192 deletions.
16 changes: 13 additions & 3 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
cmake_minimum_required(VERSION 2.8.3)
project(spatio_temporal_voxel_layer)
add_compile_options(-std=c++17)

add_definitions (-std=c++17)

set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} "${PROJECT_SOURCE_DIR}/cmake/")

Expand All @@ -18,15 +19,22 @@ find_package(catkin REQUIRED COMPONENTS
tf2_ros
visualization_msgs
message_generation
dynamic_reconfigure
)

add_service_files(DIRECTORY msgs
FILES
SaveGrid.srv
)

generate_dynamic_reconfigure_options(
cfg/SpatioTemporalVoxelLayer.cfg
)

generate_messages(
DEPENDENCIES
std_msgs
dynamic_reconfigure
)

find_package(PCL REQUIRED)
Expand Down Expand Up @@ -70,6 +78,7 @@ catkin_package(
costmap_2d
tf2_ros
visualization_msgs
dynamic_reconfigure
DEPENDS
TBB
OpenVDB
Expand All @@ -82,10 +91,11 @@ add_library(${PROJECT_NAME}
src/spatio_temporal_voxel_layer.cpp
src/spatio_temporal_voxel_grid.cpp
src/measurement_buffer.cpp
src/frustum.cpp
src/frustum_models/depth_camera_frustum.cpp
src/frustum_models/three_dimensional_lidar_frustum.cpp
src/vdb2pc.cpp
)
add_dependencies(${PROJECT_NAME} ${PROJECT_NAME}_generate_messages_cpp)
add_dependencies(${PROJECT_NAME} ${PROJECT_NAME}_generate_messages_cpp ${PROJECT_NAME}_gencfg)
target_link_libraries(${PROJECT_NAME}
${Boost_LIBRARIES}
${EIGEN3_LIBRARIES}
Expand Down
55 changes: 28 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,22 @@ This package sits on top of [OpenVDB](http://www.openvdb.org/), an open-source C

Leveraging OpenVDB, we have the ability to efficiently maintain a 3 dimensional voxel-representative world space. We wrap this with ROS tools and interfaces to the [navigation stack](http://wiki.ros.org/navigation) to allow for use of this layer in standard ROS configurations. It is certainly possible to utilize this package without ROS/Navigation and I invite other competing methodologies to develop here and create interfaces.

An simple example video can be seen [here](https://www.youtube.com/watch?v=8YIFPiI1vrg&feature=youtu.be), another more complex example can be seen [here](https://youtu.be/hpWxa6M1AR8) with global decay and frustum decay while moving with dynamic objects. A short clip from this is shown below.
Sample videos are shown below of a robot using **7 depth cameras** with less than 50% of a core, and another robot using a **VLP-16**.

![ezgif com-video-to-gif](https://user-images.githubusercontent.com/14944147/37010885-b18fe1f8-20bb-11e8-8c28-5b31e65f2844.gif)
7 Depth Cameras | VLP-16 LIDAR
:-------------------------:|:-------------------------:
![ezgif com-video-to-gif](https://user-images.githubusercontent.com/14944147/37010885-b18fe1f8-20bb-11e8-8c28-5b31e65f2844.gif) | ![vlp16](https://github.com/nickovaras/gifs/blob/master/follow.gif?raw=true)

We found in experimental trials with **6** 7hz dense stereo RGBD cameras we ran the major move_base process at **20-50%** nominal from **80-110%** on a 5th gen i7 CPU in the global costmap updating using the existing `voxel_layer`.

We've received feedback from users and have robots operating in the following environments with STVL:
- Retail
- Warehouses
- Factories
- Libraries
- Hospitals
- Hospitality

Steve spoke at ROSCon 2018 about STVL and his presentation is [linked here](https://vimeo.com/292699571) (or click on image).

[![IMAGE ALT TEXT](https://user-images.githubusercontent.com/14944147/46768837-987c9280-cc9e-11e8-99ea-788d3d590dd8.png)](https://vimeo.com/292699571)
Expand All @@ -28,7 +38,7 @@ Below is an example a size of map that is **trivial** for the Spatio-Temportal V
## -**Temporal**
The Temporal in this package is the novel concept of `voxel_decay` whereas we have configurable functions that expire voxels and their occupation over time. Infrasture was created to store times in each voxel after which the voxel will disappear from the map. This is combined with checking inclusion of voxels in current measurement frustums to accelerate the decay of those voxels that do not have measurements but should if still in the scene and remain marked. This is done rather than simply clearing them naively or via costly raytracing. The time it takes to clear depends on the configured functions and acceleration factors.

Voxel acceleration uses given FOV to compute traditional 6-planed cubical frustums. Sensors that cannot be modelled by traditional frustums (i.e. 360 lidars or sensors without flat back planes) the frustum acceleration mathematics breakdown, **do not use frustum acceleration for these class of sensors**. PRs to implement curved frustums are welcome and encouraged.
Voxel acceleration uses given FOV to compute the frustum geometry. Depth cameras (e.g. Intel Realsense) are modeled with traditional 6-planed cubical frustums. 3D lidars (e.g. Velodyne VLP 16) are modeled with their hourglass-shaped FOV. Although many 3D lidars have 360 degree horizontal FOV, it is possible to use a narrower angle for the clearing frustum by setting the hFOV parameter accordingly.

Future extensions will also to query a static map and determine which connected components belong to the map, not in the map, or moving. Each of these three classes of blobs will have configurable models to control the time they persist, and if these things are reported to the user.

Expand Down Expand Up @@ -64,32 +74,8 @@ sudo apt-get install ros-kinetic-spatio-temporal-voxel-layer
### Install from source
Required dependencies ROS Kinetic, navigation, OpenVDB, TBB.

If you are familiar with ROS, I have setup rosdep to work with all the necessary libraries, no need to install manually. In your workspace:

`sudo rosdep init && rosdep update && rosdep install --from-paths src --ignore-src -r -y`

If you are not familiar or cannot get rosdep to work, the dependencies and docs are listed below.

#### ROS

[See install instructions here.](http://wiki.ros.org/kinetic/Installation)

#### Navigation

`sudo apt-get install ros-kinetic-navigation`

#### OpenVDB

`sudo apt-get install libopenvdb3.1 libopenvdb-dev libopenvdb-tools`

#### TBB

`sudo apt-get install libtbb-dev libtbb2`

#### OpenEXR

`sudo apt-get install libopenexr-dev`

## Configuration and Running

### costmap_common_params.yaml
Expand Down Expand Up @@ -138,7 +124,9 @@ rgbd_obstacle_layer:
vertical_fov_angle: 0.7 #default 0.7, radians
horizontal_fov_angle: 1.04 #default 1.04, radians
decay_acceleration: 1. #default 0, 1/s^2. If laser scanner MUST be 0
model_type: 0 #default 0 (depth camera). Use 1 for 3D Lidar
```
More configuration samples are included in the example folder, including a 3D lidar one.

### local/global_costmap_params.yaml

Expand All @@ -151,7 +139,20 @@ Add this plugin to your costmap params file.

`roslaunch [navigation_pkg] move_base.launch`

### Enabing/disabling observation_sources real-time

To enable/disable observation sources use a ros service for each source:

~rgbd_obstacle_layer/$(source_name)/toggle_enabled (std_srvs/SetBool)
- request.data = true // Enable observation source
- request.data = false // Disable observation source

Example:
```
rosservice call /move_base/global_costmap/rgbd_obstacle_layer/rgbd_back/toggle_enabled "data: true"
rosservice call /move_base/local_costmap/rgbd_obstacle_layer/rgbd_back/toggle_enabled "data: false"
```

### Debug and Model Fitting

I have made the frustum transformations available for visualization and debugging. You may enable them by the `VISUALIZE_FRUSTUM` macro, though be aware it takes a substantial decrease on performance since we're creating and destroying a ros publisher at a non-trivial rate.
Expand Down
29 changes: 29 additions & 0 deletions cfg/SpatioTemporalVoxelLayer.cfg
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/usr/bin/env python
from dynamic_reconfigure.parameter_generator_catkin import *

gen = ParameterGenerator()

combination_method = []
combination_method.append(gen.const("largest", int_t, 0, "uses the highest of the layers in the stack"))
combination_method.append(gen.const("overwrite", int_t, 1, "overrides lower layers than this with these values"))
combination_method_enum = gen.enum(combination_method, "Combination methods")

decay_model = []
decay_model.append(gen.const("Linear", int_t, 0, "Linear"))
decay_model.append(gen.const("Exponential", int_t, 1, "Exponential"))
decay_model.append(gen.const("Permanent", int_t, 2, "Permanent"))
decay_model_enum = gen.enum(decay_model, "Decay models")

gen.add("enabled", bool_t, 0, "Enabled or not", True)
gen.add("publish_voxel_map", bool_t, 0, "Publishes the voxel map", False)
gen.add("voxel_size", double_t, 0, "Size of the voxel in meters", 0.05, 0.001, 10.0)
gen.add("combination_method", int_t, 0, "Combination methods", 1, edit_method=combination_method_enum)
gen.add("mark_threshold", double_t, 0, "Over this threshold, the measurement will be marked", 0.0, 0.0, 200.0)
gen.add("update_footprint_enabled", bool_t, 0, "Cleans where the robot's footprint is", True)
gen.add("track_unknown_space", bool_t, 0, "If true, marks will be UNKNOWN (255) otherwise, FREE (0)", True)
gen.add("decay_model", int_t, 0, "Decay models", 0, edit_method=decay_model_enum)
gen.add("voxel_decay", double_t, 0, "Seconds if linear, e^n if exponential", 10.0, -1, 200.0)
gen.add("mapping_mode", bool_t, 0, "Mapping mode", False)
gen.add("map_save_duration", double_t, 60, "f mapping, how often to save a map for safety", 60.0, 1.0, 2000.0)

exit(gen.generate("spatio_temporal_voxel_layer", "spatio_temporal_voxel_layer", "SpatioTemporalVoxelLayer"))
1 change: 1 addition & 0 deletions example/constrained_indoor_environment_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,4 @@ rgbd_obstacle_layer:
vertical_fov_angle: 0.8745 # default 0.7, radians
horizontal_fov_angle: 1.048 # default 1.04, radians
decay_acceleration: 15.0 # default 0, 1/s^2. If laser scanner MUST be 0
model_type: 0 # default 0, model type for frustum. 0=depth camera, 1=3d lidar like VLP16 or similar
1 change: 1 addition & 0 deletions example/outdoor_environment_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,4 @@ rgbd_obstacle_layer:
vertical_fov_angle: 0.8745 # default 0.7, radians
horizontal_fov_angle: 1.048 # default 1.04, radians
decay_acceleration: 1.0 # default 0, 1/s^2. If laser scanner MUST be 0
model_type: 0 # default 0, model type for frustum. 0=depth camera, 1=3d lidar like VLP16 or similar
1 change: 1 addition & 0 deletions example/standard_indoor_environment_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,4 @@ rgbd_obstacle_layer:
vertical_fov_angle: 0.8745 # default 0.7, radians
horizontal_fov_angle: 1.048 # default 1.04, radians
decay_acceleration: 5.0 # default 0, 1/s^2. If laser scanner MUST be 0
model_type: 0 # default 0, model type for frustum. 0=depth camera, 1=3d lidar like VLP16 or similar
42 changes: 42 additions & 0 deletions example/vlp16_config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
rgbd_obstacle_layer:
enabled: true
voxel_decay: 15 # seconds if linear, e^n if exponential
decay_model: 0 # 0=linear, 1=exponential, -1=persistent
voxel_size: 0.05 # meters
track_unknown_space: true # default space is known
max_obstacle_height: 2.0 # meters
unknown_threshold: 15 # voxel height
mark_threshold: 0 # voxel height
update_footprint_enabled: true
combination_method: 1 # 1=max, 0=override
obstacle_range: 3.0 # meters
origin_z: 0.0 # meters
publish_voxel_map: false # default off
transform_tolerance: 0.2 # seconds
mapping_mode: false # default off, saves map not for navigation
map_save_duration: 60 # default 60s, how often to autosave
observation_sources: rgbd1_mark rgbd1_clear
rgbd1_mark:
data_type: PointCloud2
topic: /velodyne_points
marking: true
clearing: false
min_obstacle_height: 0.3 # default 0, meters
max_obstacle_height: 2.0 # default 3, meters
expected_update_rate: 0.0 # default 0, if not updating at this rate at least, remove from buffer
observation_persistence: 0.0 # default 0, use all measurements taken during now-value, 0=latest
inf_is_valid: false # default false, for laser scans
voxel_filter: false # default off, apply voxel filter to sensor, recommend on
clear_after_reading: true # default false, clear the buffer after the layer gets readings from it
rgbd1_clear:
data_type: PointCloud2
topic: /velodyne_points
marking: false
clearing: true
max_z: 8.0 # default 0, meters
min_z: 1.0 # default 10, meters
vertical_fov_angle: 0.523 # default 0.7, radians. For 3D lidars it's the symmetric FOV about the planar axis.
vertical_fov_padding: 0.05 # 3D Lidar only. Default 0, in meters
horizontal_fov_angle: 6.29 # 3D lidar scanners like the VLP16 have 360 deg horizontal FOV.
decay_acceleration: 5.0 # default 0, 1/s^2.
model_type: 1 # default 0, model type for frustum. 0=depth camera, 1=3d lidar like VLP16 or similar
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
/*********************************************************************
*
* Software License Agreement
*
* Copyright (c) 2018, Simbe Robotics, Inc.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials provided
* with the distribution.
* * Neither the name of Simbe Robotics, Inc. nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
* FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
* COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
* ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*
* Author: Steve Macenski (steven.macenski@simberobotics.com)
* Purpose: Structure for handling camera FOVs to construct frustums
* and associated methods
*********************************************************************/

#ifndef DEPTH_FRUSTUM_H_
#define DEPTH_FRUSTUM_H_

// STVL
#include <spatio_temporal_voxel_layer/frustum_models/frustum.hpp>

namespace geometry
{

// visualize the frustum should someone other than me care
#define VISUALIZE_FRUSTUM 0

// A class to model a depth sensor frustum in world space
class DepthCameraFrustum : public Frustum
{
public:
DepthCameraFrustum(const double& vFOV, const double& hFOV,
const double& min_dist, const double& max_dist);
virtual ~DepthCameraFrustum(void);

// transform plane normals by depth camera pose
virtual void TransformModel(void);

// determine if a point is inside of the transformed frustum
virtual bool IsInside(const openvdb::Vec3d& pt);

// set pose of depth camera in global space
virtual void SetPosition(const geometry_msgs::Point& origin);
virtual void SetOrientation(const geometry_msgs::Quaternion& quat);

private:
// utils to find useful frustum metadata
void ComputePlaneNormals(void);
double Dot(const VectorWithPt3D&, const openvdb::Vec3d&) const;
double Dot(const VectorWithPt3D&, const Eigen::Vector3d&) const;

double _vFOV, _hFOV, _min_d, _max_d;
std::vector<VectorWithPt3D> _plane_normals;
Eigen::Vector3d _position;
Eigen::Quaterniond _orientation;
bool _valid_frustum;

#if VISUALIZE_FRUSTUM
std::vector<Eigen::Vector3d> _frustum_pts;
ros::Publisher _frustumPub;
#endif
};

} // end namespace

#endif
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,7 @@
* POSSIBILITY OF SUCH DAMAGE.
*
* Author: Steve Macenski (steven.macenski@simberobotics.com)
* Purpose: Structure for handling camera FOVs to construct frustums
* and associated methods
* Purpose: Abstract class for a frustum model implementation
*********************************************************************/

#ifndef FRUSTUM_H_
Expand All @@ -60,9 +59,6 @@
namespace geometry
{

// visualize the frustum should someone other than me care
#define VISUALIZE_FRUSTUM 0

// A structure for maintaining vectors and points in world spaces
struct VectorWithPt3D
{
Expand Down Expand Up @@ -99,36 +95,23 @@ struct VectorWithPt3D
class Frustum
{
public:
Frustum(const double& vFOV, const double& hFOV,
const double& min_dist, const double& max_dist);
~Frustum(void);

// transform plane normals by depth camera pose
void TransformPlaneNormals(void);
Frustum() {};
virtual ~Frustum(void) {};

// determine if a point is inside of the transformed frustum
bool IsInside(const openvdb::Vec3d& pt);
virtual bool IsInside(const openvdb::Vec3d& pt) = 0;

// set pose of depth camera in global space
void SetPosition(const geometry_msgs::Point& origin);
void SetOrientation(const geometry_msgs::Quaternion& quat);
virtual void SetPosition(const geometry_msgs::Point& origin) = 0;
virtual void SetOrientation(const geometry_msgs::Quaternion& quat) = 0;

private:
// utils to find useful frustum metadata
void ComputePlaneNormals(void);
double Dot(const VectorWithPt3D&, const openvdb::Vec3d&) const;
double Dot(const VectorWithPt3D&, const Eigen::Vector3d&) const;
// transform model to the current coordinates
virtual void TransformModel(void) = 0;

double _vFOV, _hFOV, _min_d, _max_d;
std::vector<VectorWithPt3D> _plane_normals;
private:
Eigen::Vector3d _position;
Eigen::Quaterniond _orientation;
bool _valid_frustum;

#if VISUALIZE_FRUSTUM
std::vector<Eigen::Vector3d> _frustum_pts;
ros::Publisher _frustumPub;
#endif
};

} // end namespace
Expand Down
Loading

0 comments on commit 3eaa79b

Please sign in to comment.