Isaac ROS Compression

Overview

Isaac ROS Compression

Isaac ROS Compression provides H.264 image encoder and decoder that leverages the specialized hardware in NVIDIA GPUs and the Jetson platform
isaac_ros_h264_encoder package:
Can compress an image into H.264 data using the NVENC
isaac_ros_h264_decoder package
Can decode the H.264 data into original images using the NVDEC
-Image compression reduces the data footprint of images when written to storage or transmitted between computers
A 1080p camera at 30fps produces 177MB/s of data
-Image compression reduces this by approximately 10 times to 17MB/s of data, reducing the throughput needed to send this to another computer or write out to storage
A one minute 1080p camera recording is reduced from ~10GB to ~1GB
This compression is provided by dedicated hardware acceleration (NvEnc) separate from other hardware engines such as the GPU
-Common use case for image compression during the development of robots
(1)Capture camera images to storage
(2)This captured data is processed offline from the robot to produce training datasets for AI models, test datasets for perception functions, and test data for open-loop re-simulation of software in development with real data
(3)The compression parameters are tuned to minimize visual quality reduction from lossy compression for AI model and perception function development
(4)Compression reduces the amount of data written to storage, the time required to offload the recording, and footprint of the data at rest in a data lake
-Compression can be used with event data recorders to capture camera images to storage when an event of interest occurs, often due to failures on the robot
This provides visual information to assist in the debugging of the event or to improve perception and robot functions
-H.264 is an efficient and popular compression algorithm with broad support across many platforms
Output of the isaac_ros_h264_encoder package can then be decoded with hardware acceleration using the isaac_ros_h264_decoder on Jetson and x86_64 systems
Or by third-party H.264 decoder packages on non-NVIDIA platforms
-Package is powered by NVIDIA Isaac Transport for ROS(NITROS)
Which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes

Note

ROS2 relies on image_transport_plugins for CPU based compression
Recommend using isaac_ros_h264_encoder as part of the graph of nodes when capturing to a rosbag for performance benefits of NITROS
ROS2 type adaptation used by NITROS is not supported by image_transport_plugins, resulting in more CPU load, and less encode performance

Packages

isaac_ros_h264_decoder
isaac_ros_h264_encoder

Supported Platforms

Compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU

Docker

To simplify development
Strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps
This will streamline your development environment setup with the correct versions of dependencies on x86_64 platforms
Customize your Dev Environment
Reference this guide

isaac_ros_h264_decoder QuickStart

Shows an example of how to use the isaac_ros_h264_decoder with a pre-recorded rosbag, which contains compressed H.264 images generated from isaac_ros_h264_encoder with two Argus cameras as the input source
You will be able to visualize the decoded images after the last step
(1)Set up your development environment by following the instructions here
=> Already Done
(2)Clone isaac_ros_common and this repository under ${ISAAC_ROS_WS}/src

(3)Pull down a rosbag of sample data

(4)Launch the Docker container using the run_dev.sh script

(5)Install this package’s dependencies

(6)Run the following launch files to run the demo of this isaac_ros_h264_decoder

(7)Open a second terminal and attach to the container

(8)Visualize and validate the output of the package
Left Image View

Right Image View

isaac_ros_h264_encoder QuickStart

Try More Examples
Tutorial with RealSense
Tutorial with NITROS Graph

Reference
https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_compression/index.html
https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_common/index.html#isaac-ros-docker-development-environment
https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_compression/isaac_ros_h264_decoder/index.html#quickstart