Isaac ROS Freespace Segmentation

Isaac ROS Freespace Segmentation:
ROS2 package to produce occupancy grids for navigation
-By processing a freespace segmentation mask with the pose of the robot relative to the ground, Bi3D Freespace produces an occupancy grid for Nav2, which is used to avoid obstacles during navigation.
-This package is GPU accelerated to provide real-time, low latency results in a robotics application. Bi3D Freespace provides an additional occupancy grid source for mobile robots (ground based).
-isaac_ros_bi3d is used in a graph of nodes to provide a freespace segmentation mask as one output from a time-synchronized input left and right stereo image pair
-The freespace mask is used by isaac_ros_bi3d_freespace with TF pose of the camera relative to the ground to compute planar freespace into an occupancy grid as input to Nav2
-There are multiple methods to predict the occupancy grid as an input to navigation
None of these methods are perfect; each has limitations on the accuracy of its estimate from the sensor providing measured observations.
Each sensor has a unique field of view, range to provide its measured view of the world, and corresponding areas it does not measure.
-Bi3D Freespace provides a diverse approach to identifying obstacles from freespace.
Stereo camera input used for this function is diverse relative to lidar, and has a better vertical field of view than most lidar units, allowing for perception of low lying obstacles that lidar can miss.
Bi3D Freespace provides a robust, vision-based complement to lidar occupancy scanning.

Isaac ROS NITROS Acceleration

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

QuickStart

(1)Set up your development environment
Already Done
(2)Clone isaac_ros_common and this repository under ${ISAAC_ROS_WS}/src

Already Done
Clone isaac_ros_depth_segmentation repository (Thanks NVIDA)

(3)Pull down a rosbag of sample data
Error:

Solution:

isaac_ros_depth_segmentation file doesn’t exist. This is the reason for the error

=>Realized I had the wrong file name, so I went back to step2 and cloned an additional repository called isaac_ros_depth_segmentation
Correct version:

(4)Launch the Docker container using the run_dev.sh script
Error:

Solution:
https://github.com/git-lfs/git-lfs/issues/3964
Similar Issues:
https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_map_localization/issues/19 => Not so helping..
https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/issues/104 => Someone with the same error who posted on 2023-11-30, asked whether solved the problem
https://forums.developer.nvidia.com/t/isaac-ros-freespace-segmentation-launch-error-in-docker-container/277741 => MY question posted on the NVIDIA Forum
(5)Install this package’s dependencies

(6)Download model files for Bi3D (refer to the Model Preparation section for more information)

(7)Convert the .onnx model files to TensorRT engine plan files (refer to the Model Preparation section for more information)
If using x86_64:


(8)Run the launch file to spin up a demo of this package

Error:

Solution:

https://forums.developer.nvidia.com/t/isaac-ros-freespace-segmentation-quickstart-demo-launch-file-error/278116
Corrected Version

Install image pipeline


Build Packages

Run the Demo
(9)Open a second terminal inside the Docker container

(10)Visualize the occupancy grid in RViz

Start RViz

In the left pane, change Fixed Frame to base_link

In the left pane, click the Add button, then select By topic followed by Map to add the occupancy grid
Click Add > Select By Topic > Map
Click Add > Select By Topic > rgb_left > Image
Click Add > Select By Topic > rgb_right >Image

Error Result: Nothing Came out
Correct Version:

Reference:
https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_freespace_segmentation/isaac_ros_bi3d_freespace/index.html#quickstart