Isaac ROS DNN Inference

Overview

Isaac ROS DNN Inference:
ROS 2 packages for performing DNN inference, providing AI-based perception for robotics applications
DNN inference uses a pre-trained DNN model to ingest an input Tensor and output a prediction to an output Tensor

Typical graph of nodes for DNN inference on image data
-The input image is resized to match the input resolution of the DNN
The image resolution may be reduced to improve DNN inference performance ,which typically scales directly with the number of pixels in the image
-DNN inference requires input Tensors, so a DNN encoder node is used to convert from an input image to Tensors, including any data pre-processing that is required for the DNN model
-DNN inference is performed
-The DNN decoder node is used to convert the output Tensors to results that can be used by the application
-TensorRT and Triton are two separate ROS nodes to perform DNN inference
TensorRT node:
Uses TensorRT to provide high-performance deep learning inference
Optimizes the DNN model for inference on the target hardware, including Jetson and discrete GPUs
Supports specific operations that are commonly used by DNN models
Triton node:
For newer or bespoke DNN models, TensorRT may not support inference on the model
Triton node uses the Triton Inference Server, which provides a compatible frontend supporting a combination of different inference backends
(e.g. ONNX Runtime, TensorRT Engine Plan, TensorFlow, PyTorch)
In-house benchmark results measure little difference between using TensorRT directly or configuring Triton to use TensorRT as a backend
-Some DNN models may require custom DNN encoders to convert the input data to the Tensor format needed for the model, and custom DNN decoders to convert from output Tensors into results that can be used in the application
Leverage the DNN encoder and DNN decoder node(s) for image bounding box detection and image segmentation, or your own custom node(s)

Reference:
https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/index.html

QuickStarts

Isaac ROS TensorRT

(1)Follow steps 1-6 of the Quickstart with Triton
(1)-1 Set up your development environment by following the instructions here
=> Already Done
(1)-2 Clone isaac_ros_common, isaac_ros_image_segmentation, and this repository under ${ISAAC_ROS_WS}/src

(1)-3 Launch the Docker container using the run_dev.sh script

(1)-4 Install this package’s dependencies, along with an additional package used for this quickstart

(1)-5 This example uses PeopleSemSegNet ShuffleSeg. Download the ETLT file and the int8 inference mode cache file:


(1)-6 Convert the ETLT file to a TensorRT plan file:
inference mode cache file:


(2)Install this package’s dependencies

(3)Run the following launch files to spin up a demo of this package. (3)-1 Launch TensorRT:

(3)-2 Then open another terminal, and enter the Docker container again:

Isaac ROS Triton

Reference:
https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_tensor_rt/index.html#quickstart
https://nvidia-isaac-ros.github.io/repositories_and_packages/isaac_ros_dnn_inference/isaac_ros_triton/index.html