” data-medium-file=”https://i2.wp.com/www.marktechpost.com/wp-content/uploads/2021/06/Screen-Shot-2021-06-25-at-10.58.13-PM.png?fit=300%2C143&ssl=1″ data-large-file=”https://i2.wp.com/www.marktechpost.com/wp-content/uploads/2021/06/Screen-Shot-2021-06-25-at-10.58.13-PM.png?fit=696%2C333&ssl=1″>
https://www.youtube.com/watch?v=xsnGhObzOf4 ” data-medium-file=”https://i2.wp.com/www.marktechpost.com/wp-content/uploads/2021/06/Screen-Shot-2021-06-25-at-10.58.13-PM.png?fit=300%2C143&ssl=1″ data-large-file=”https://i2.wp.com/www.marktechpost.com/wp-content/uploads/2021/06/Screen-Shot-2021-06-25-at-10.58.13-PM.png?fit=696%2C333&ssl=1″>
The advance of robotics and artificial intelligence (AI) will significantly influence how we live and work in the years ahead. Gerard Andrews, Senior Product Marketing Manager at NVIDIA, works with the robotics developer ecosystem. Gerard gives the Marktechpost team an overview of NVIDIA Isaac Sim, a robotics simulation platform for AI-based robots.
A). Marktechpost: What are some factors to consider in creating AI models in a simulated versus real-world environment?
Gerard: There are many different factors to consider in creating effective AI models in an industrial robotics environment.
- Sim 2 Real Gap: the simulation to real gap is the accuracy of what you see happening in the simulation, relative to the behavior you expected in the real world. It is very important to minimize the difference between the simulator and what happens in the real world. When the roboticist sees there is some gap, then it loses some of the utility. The roboticist then can use the real hardware robot to do whatever task, as opposed to the simulation option. So it’s really important to drive the simulation adoption to close that sim 2 real gap.
- Limited Functionality: The other thing about simulators is that existing simulators are designed to do one specific task, and many times it’s good at that one specific task. However, to generally adopt the simulator, there is a learning curve. You need a simulator that has a broad set of the most general use cases. A simulator needs to build complex scenes, whether in a factory, a warehouse, or a retail establishment. The simulator must have access to lots of other content and to be able to bring assets from various sources to build that real-world simulation environment.
- Prototyping: there is a benefit from a cost savings perspective in having a simulator involved from the first step of the development cycle. If you start building your software on a simulated version of the hardware, you can iterate into prototyping. Prototyping has tremendous value before committing to a specific hardware design and having that design built by a manufacturer.
- Training AI-based Robots: We need data to train robots, and collecting accurate data in the real world is tricky for various reasons. For instance, it is costly and unsafe to have robots on the warehouse floor, maneuvering around live people. How can we simulate what a robot does and how it behaves? We need a compelling, realistic environment where it can operate. NVIDIA’s solution is to create a photorealistic, physics-accurate simulator that can also generate ground-truth training data to give valuable data to robots.
- Domain Randomization: With this technique we vary the parameters that define a simulated scene. In Isaac Sim, this includes numerous parameters such as lighting, color, texture of materials, poses, rotations and more. The main objective of domain randomization is to enhance the robustness of machine learning (ML) models by exposing the neural network to the widest amount of conditions that the robot might ever encounter. This will give the user the confidence that the model will work well in the real world when the model is deployed, as the model will have encountered that condition during training. Creating this same level of diversity in the training dataset using only real-world data can be prohibitively expensive and time-consuming.
B.) Marktechpost: What is NVIDIA Isaac Sim?
Gerard: NVIDIA Isaac Sim is a scalable robotics simulation application and synthetic data generation tool. Built on the Omniverse platform, Isaac Sim allows robots to be trained and tested more efficiently by providing a realistic simulation for the robot beyond the real world. Isaac Sim powers physically accurate virtual environments to develop, test, and manage AI-based robots.
- Synthetic data generation: allows training data for robots to be collected in a synthetic, simulated environment instead of the real world. Synthetic data generation is not as difficult or dangerous as collecting data in the real world.
- The new Isaac simulation engine also streamlines domain randomization to build ground-truth datasets to train robots in a variety of applications.
C.) Marktechpost: What is NVIDIA Omniverse?
Gerard: NVIDIA Omniverse is a robust, multi-GPU simulation and collaboration platform with universal interoperability across different applications and 3D ecosystem vendors.
- Multi-GPU simulation means a massive amount of data and complexity can be realistically simulated
- A collaboration platform, meaning two or more researchers can use the platform at once
- Universal interoperability means that assets and data from many different applications can be utilized and work together
D.) Marktechpost: What are some sample use cases of NVIDIA Isaac Sim?
Gerard: Different hardware (robots) can run on Isaac Sim. In a video from April 2021, there are use cases for mobile-based robots, robot arms, and quadruped robots, as an example.
Video source: https://www.nvidia.com/en-us/on-demand/session/gtcspring21-s31824/
E.) Marktechpost: What are the capabilities and key features of Isaac Sim in its latest release?
Gerard: Some of the key strengths and capabilities of Isaac Sim are features that are not present in many other simulators.
Here is a quick 90 second video on Isaac Sim: https://youtu.be/xsnGhObzOf4
- Realistic simulation
- Industrial strength robotics simulator for warehouses, factories, retail, and many other environments.
- Realism is derived from the PhysX 5 and advanced rendering capabilities of Isaac.
- Modular, breadth of applications
- No one-size-fits-all simulator, but there is a case in the market for a simulator that can do a lot of use cases
- Modular design can be customized and extended to handle many applications and environments that we don’t even anticipate at this point.
- Seamless connectivity and interoperability
- Connectivity to the rich world of tools used to describe robots, environments, and the 3D assets required to build compelling realistic simulations.
- With NVIDIA Omniverse, Isaac Sim can do collaborative building, sharing, and importing environments and robot models in Universal Scene Description (USD).
New Features in Isaac Sim Open Beta Release:
- Multi-Camera Support
- Fisheye Camera with Synthetic Data
- ROS2 Support
- PTC OnShape Importer
- Improved Sensor Support
- Ultrasonic Sensor
- Force Sensor
- Custom Lidar Patterns
- Downloadable from NVIDIA Omniverse Launcher
F.) Marktechpost: Who is the target audience for Isaac Sim and NVIDIA Omniverse?
Gerard: We built Isaac Sim to be a scalable platform to handle industrial-scale simulations. That being said, we are pleased to see hobbyists and researchers leveraging the platform. In addition, through the early adopter program, we had thousands of developers from hundreds of companies that all gave us excellent feedback on the simulator features that they wanted to see.
About Gerard Andrews: Gerard Andrews is a Sr. Product Marketing Manager focused on the robotics developer community. Prior to joining NVIDIA, Gerard was at Cadence where he was Product Marketing Director, responsible for product planning, marketing, and business development for licensable processor IP. He holds an MS in electrical engineering from Georgia Institute of Technology and a BS in electrical engineering from Southern Methodist University.
More Information on Isaac Sim:
- Join the thousands of developers who have worked with Isaac Sim across the robotics community via our early access program. Get started with the next step in robotics simulation by downloading Isaac Sim.
- Learn more about importing your own robot with the following tutorial: https://www.youtube.com/watch?v=pxPFr58gHmQ
- Use Isaac Sim to train your Jetbot by exploring these developer blogs.
- https://developer.nvidia.com/blog/training-your-jetbot-in-isaac-sim/
- https://developer.nvidia.com/blog/training-your-nvidia-jetbot-to-avoid-collisions-using-nvidia-isaac-sim/
The post NVIDIA Isaac Sim: A Scalable Robotics Simulation and Synthetic Data Generation Tool to Develop, Test, and Manage AI-Based Robots appeared first on MarkTechPost.