Introductory

Isaac Sim Interface

Learning Objectives
Introduces the most commonly used user-interface buttons, menus, and controls in Omniverse Isaac Sim.
After this tutorial, you should be more confident in navigating and discovering content in the Isaac Sim interface.
Getting Started
(1)Adding a cube to the scene

Go to Menu bar
Click Create > Shapes > Cube
Cube is in the center of the Viewport. The Cube is Selected (highlighted in orange), and the Move (W) command is enabled by default.
(2)Transforming the Cube Hover over and then click the Move Gizmo to drag and move the Cube.
Press “E”:

Switch to the Rotate Gizmo, and then click and drag the gizmo to rotate the Cube.
Press “R”:

Switch to the Scale. Click and drag the gizmo to rotate the Cube
Press “W”:

Switch back to the Move Gizmo.
Press “W” again:

Toggle Move with Local Coordinates. Click and drag the gizmo to move the Cube on its local axes.
Press “ESCAPE”:

Deselect the Cube
(3)Practice transforming the Cube through its Property panel

Select the Cube to populate the Property panel

In the Transform > Translate field, click and drag the X value left and right to move the Cube along the X-Axis

Double Click the Y value and type “1” to move the Cube to Y=1

Click the blue square all the way to the right of the Z value to reset the Translate value to its default setting (0,0,0)

Reset the Rotate and Scale values to their default setting

Set the Cube Translate to {X: 0, Y: -2, Z: -0.5} and Rotate to {X: 0, Y: 22.5, Z: 0} by double-clicking on the corresponding X, Y, Z values and typing in the values.
Toggling Local & World Coordinates

Press “W” multiple times:
Toggle between Move Global and Move Local

Press “E” multiple times:
Toggle between Rotate Global and Rotate Local

Pressing and holding the Move/Rotate Icons in the Toolbar can also be used to select the coordinate system

Each Toolbar Icon will turn Orange when the Local coordinate system is active
Footnote:
toggle: On/Off Switch

Navigating the Viewport
Bring another object into the scene by adding a Sphere

Go to the top Menu Bar and Click Create > Shapes > Sphere. There should now be a Cube and a Sphere in the Viewport, with the Sphere selected by default

Translate the Sphere to {X: 0, Y: 2, Z: 0}

Press “F” to Center and Zoom the Viewport Camera on the selected Sphere

Press “LMB”(left mouse button)+“ALT” to Orbit around the Sphere
“Scroll Wheel” or Press “RMB”(right mouse button)+“ALT”: Zoom in.
Press the “MMB” (middle mouse button) to Pan
Footnote:
Pan: Specific type of camera movement or interaction commonly used in 3D graphics and computer-aided design (CAD) applications

Press “ESCAPE” to Deselect the Sphere

Press “F” when nothing is Selected to Zoom All

Using the Stage and Properties Panels
Objects can be directly selected in the Viewport or in the Stage—the Panel at the top right of the Workspace.
Stage: Powerful tree-based widget for organizing and structuring all the content in an Omniverse Isaac Sim scene

Go to the top Menu Bar
Click Create > Xform: There should now be an empty Xform prim in the Stage panel, nested under World

Select the Cube line item from the Stage, and drag and drop onto the Xform

Select the Sphere and drag and drop it onto the Xform as well
=>Nothing changes in the Viewport.But a Parent-Child relationship between the Xform, the Sphere, and the Cube has been created.

Select the Xform
Press “W” to enable the Move Gizmo. Grab and move the Xform
-Moving the Xform: Moves both the Sphere and the Cube.
-Child objects inherit the transforms of their parents. Important concept for how Omniverse Isaac Sim constructs articulated objects, such as Robots

Select just the Cube

Move the Xform

Its Transform Properties in the Property panel are the same, no matter how you move the Xform

An object’s Transform Properties are a local transform, relative to its Parent

Unparent the Cube by selecting the Cube from the Stage, then dragging and dropping it onto the World line item

Right-click on the Xform > Click Delete
=>Cube’s Transform Properties now reflect the world transform
=>Sphere is deleted as well because it is a child of the Xform
Property panel > Transform Properties: Panel has a lot of useful UI inputs for interfacing directly with geometric, material, visual, and USD properties

Select the Cube from the Stage
Scroll through its Property panel -> Click the header of each sub-panel will collapse or show it

Select the defaultLight from the Stage
Scroll through its Property panel and note the different information it shows compared to the Cube. Select the World (defaultPrim) from the Stage and note how its Property panel differs.
-Data in each Property panel differs because World(defaultPrim), Cube, and defaultLight are all different data types in Omniverse Isaac Sim.
The World is an Xform, the defaultLight is a DistantLight, and the Cube is a Cube

Raw USD Properties Raw USD Properties sub-panel: Contains every active USD property for the selected object.
-Most important sub-panel in the Property panel
-Very useful reference when developing more advanced applications in Omniverse Isaac Sim

Customizing the Workspace
Workspace in Omniverse Isaac Sim is very customizable
Easy to resize, dock, undock, add, and remove panels in the user interface
Customize the user interface by adding another Viewport to the workspace

Go to the top Menu Bar
Click Window > Viewport > Viewport 2 (Very bottom of the dropdown menu)
Be a new viewport called Viewport 2 floating in the workspace
-Follow the Camera Inspector Extension: Create additional viewoports.
Click and drag the top header of Viewport 2 to the activate the Docking widget. Drag and drop to the right of the original Viewport to place the viewports side-by-side.
Click and drag the left side of Viewport 2 to the left to make the viewports equal sizes.

Button overlay area in Viewport 2
Click the Perspective camera button in the top left and change the camera to a Top View

Window > Viewport > Viewport 2 dropdown: Dynamically hide/reveal Viewport 2
Click Viewport 2 to hide the new viewport, and click again to turn it back on

Look at the other items in the Window dropdown. Items with check marks are already visible in the Workspace
Click a few items off -> See what happens in the workspace (e.g. Console, Stage, Property, Profiler)

Running the Simulation
Simulation must be running to use any simulation features in Omniverse Isaac Sim
Ex. joints, scripting, or collision meshes
(1)Press the Play button in the Toolbar to begin the simulation
(2)The Play button turns into a Pause button, which can be used to stop the simulation temporarily but allow it to continue from its current state
(3)While the simulation is running or paused, the Stop button is also visible below the Play/Pause button.
The Stop button ends the simulation and resets it, allowing it to be played again from its starting configuration.
Timeline
Timeline: Extension that allows developers to see and modify settings for a scrollable and customizable timeline.

Timeline is disabled by default, to enable it, go to Window > Extensions, type in omni.anim.window.timeline in the search bar, and click on the toggle
At the bottom of the screen, the timeline widget will appear

Press the Play button > Timeline marker will begin to move, and will loop over the timeline
The simulation’s start/stop progress can also be viewed on the Timeline at the bottom of the default layout

Summary
Adding Content to the Scene > Navigating the Viewport > Navigating the Stage and Properties Panels > Adding, Docking, and Removing Window Panels > Starting/Stopping the Timeline and Simulation

Next Steps
Environment Setup

Isaac Sim Workflows

Learning Objectives
Summarizes the different workflows for developing in Omniverse Isaac Sim and ways to interface with it with your existing systems
Understand the uses and benefits of each workflow and the best way to include Omniverse Isaac Sim in application

Getting Started
Three main workflows when developing in Omniverse Isaac Sim: GUI, extensions, standalone Python
Summary of the key features and their recommended usages
GUI
Key features: Visual, intuitive, specialized tools for populating and simulating a virtual world
Recommended usage: World building, assemble robots, attach sensors, initialize ROS bridges
Extensions
Key features: Run asynchronously to allow interactions with the stage, hot reloading to reflect changes immediately, adaptive physics step for real-time simulation
Recommended usage: Building interactive GUIs, custom application modules, real-time sensitive applications
Standalone Python
Key features: Control over timing of physics and rendering steps, can be run in headless mode
Recommended usage: Large scale training for reinforcement learning, systematic world generation and modification
Footnote:
asynchronously:
Not existing or happening at the same time
adaptive physics:
Physics simulations within the software can dynamically adjust their update rate or time step based on the computational resources available
Footnote
physics: Mathematical and computational models used to simulate physical interactions and behaviors in a virtual environment
Physics engines are used in simulations to calculate and simulate these behaviors accurately
ex. gravity, collisions, object dynamics, forces, and other physical phenomena
Rendering: Process of generating 2D or 3D images from a 3D scene description
Involves taking the virtual objects and environments created in the simulation and converting them into visual images that can be displayed on a screen
ex.shading, lighting, and texturing to create realistic or stylized visual representations of the virtual world
Headless Mode:
Running an application or software without a graphical user interface(GUI) or display output
Application performs its tasks or computations without rendering graphics or providing a visual interface to the user.
Headless mode is often used in server environments or for batch processing where graphical rendering is unnecessary or unwanted.

GUI

Omniverse Isaac Sim’s GUI interface
-Features are the same ones used in Omniverse USD Composer, an application that is dedicated for world-building
-Create tools make it easy to assemble, illuminate, simulate, and render scenes large and small, therefore making it the ideal place to build your virtual worlds, assemble robots, examine physics
-To learn more about how to leverage the GUI for your robotics application, continue to our GUI tutorial series, assemble a Simple Robot

Extensions

Core building block of Omniverse Kit based applications
Individually built application modules. All the tools used in Omniverse Isaac Sim are built as extensions
They can be used across different Omniverse applications by simply installing it in the Extensions Manager
-Main feature of this workflow is that the application runs asynchronously
Enables the extension applications to interact with the USD stage without blocking rendering and physics stepping
Allows for hot reloading, so you can change the application code while Omniverse Isaac Sim is running and then see the reflected changes in your application after saving the file, without shutting down or restarting Omniverse Isaac Sim
Most of the action in an extension is done via callbacks that are triggered with certain events, such as a physics or rendering step, stage events, ticks in time

Writing A Custom Python Extension
Easiest way to get started using the Extension workflow is to use the Omniverse Isaac Sim Extension Template Generator to populate a simple UI-based extension on your local machine
The available extension templates give a useful starting point for many Omniverse Isaac Sim applications and are structured to educate the user in building a custom UI tool that meets their needs
Process
(1)Create and enable a new extension using the Extension Template Generator
(2)Create a folder on your computer where you want Omniverse Isaac Sim to search for user extensions
(3)Navigate to Isaac Utils -> Generate Extension Templates in the toolbar
(4)Select the type of template that you want to start with, and fill in the required fields
(5)Navigate to Window -> Extensions in the toolbar to open the Extensions Manager
(6)Click the hamburger icon in the Extensions Manager, and then “Settings” in the sub-menu to add the path to the folder you created for your user extensions
(7)Find your extension in the Extensions Manager under “Third Party Extensions” and enable it. It will now appear in the toolbar
(8)An alternative way to enable extensions is to add command-line arguments when running Omniverse Isaac Sim from the terminal: isaac_sim.sh –ext-folder {path_to_user_ext_folder} –enable {ext_directory_name}
(9)Each extension template is meant to be self-explanatory. You can quickly get familiar with the template code by reading the README.md file in the provided Python module
Note: Isaac Sim Extension Templates
Writing A Custom Cpp Extension
Write your own extension containing C++ code
Omniverse Kit provide an extension template as well as examples of its usage
Go to Kit C++ Extension Template for detailed instructions

Standalone Application

Omniverse Isaac Sim is launched via a python script, inside which the rendering and physics are stepped manually, to guarantee that stepping only happens after ascertain the completion of a set of commands
Running Your First Standalone Application
(1)
(2)
(3)

Using Jupyter to Develop a Standalone Application
Omniverse Isaac Sim can be launched as headless in this workflow (i.e. without a visible and updating GUI). It can also be launched via a Jupyter notebook (Linux only). In the clip below, Jupyter notebook live syncs to the same USD so that changes are reflected in another Isaac Sim process launched through the terminal or the launcher.

Reference:
https://docs.omniverse.nvidia.com/isaacsim/latest/introductory_tutorials/tutorial_intro_interface.html
https://docs.omniverse.nvidia.com/isaacsim/latest/introductory_tutorials/tutorial_intro_workflows.html