The Fascinating Future Of AI-Empowered Robotics
Whether you have a Roomba vacuum scouring your home for dirt and debris, or you’re fortunate enough to have experienced some level of autonomous driving, most folks realize that advanced robotics applications rely on intelligent machine vision to navigate terrain and avoid collisions. However, truly intelligent robots of the future will need not only eyes to see and navigate, but intelligence to interact, communicate and make decisions on their own as well. With pre-trained models and even live training, many robots of the future will need to infer and ingest important information from the world around them, interact seamlessly with humans and also make the right decisions and recommendations autonomously. This area of intelligent machines is a huge market opportunity that NVIDIA has been developing a myriad of solutions for with its Jetson portfolio for years now.
From the petite, $59 Jetson Nano developer kit for makers (2019), to its latest full-powered Jetson AGX Orin developer kit that retails for $1999, and is a derivative of the company’s Drive AGX Orin for autonomous vehicles, NVIDIA has been keenly focused on not only developing the silicon engines that can power the next generation of autonomous machines and intelligent edge devices, but also a full software stack and deployment platform for developers to harness.
Kicking The Tires On Next-Gen AI For Robots And The Intelligent Edge
NVIDIA’s Jetson AGX Orin is available in both a developer kit with 32GB of LPDDR5 memory on board, as well as a family of modules for various embedded applications. My team got some hands-on time with the dev kit, and the brains of the operation is NVIDIA’s Jetson AGX Orin processor in its full fat implementation, with a 12-core Arm Cortex-A78AE 64-bit CPU complex that rides shotgun alongside an NVIDIA Ampere GPU that sports 2,048 CUDA cores and 64 Tensor cores. Think GeForce RTX 3050-class performance, but tapped here for machine learning, specifically inferencing workloads but with some training capabilities as well, which I’ll cover in a bit.
The top-end config of Jetson AGX Orin offers 275 TOPS (Trillion Operations Per Second) of computing power, though Orin boards can scale from 15 – 60 Watts, to address power requirements of various low power edge applications. The dev kit brings out tons of extra IO for access and adaptability as well, including the usual JTAG port and 40-pin GPIO (general purpose IO) header, additional headers for sensors, and enough USB ports to connect not only cameras and mics, but peripherals and external storage. If that weren’t enough, there’s 10-Gigabit Ethernet port and a pair of PCI Express slots too. In a nutshell, NVIDIA’s Jetson AGX Orin developer kit is designed to provide unfettered access to as many features and capabilities of the platform as possible.
Developers On Cutting-Edge AI Hardware Need Powerful Software Tools And Infrastructure
NVIDIA bundles Jetson AGX Orin with Ubuntu Linux and its Jetson Jetpack SDK. Jetpack is NVIDIA’s development environment. Jetpack works in conjunction with NVIDIA ROS (real-time operating system), pre-trained models and various other SDKs and libraries in support of a myriad of applications like machine vision, real-time speech and analytics. In short, the Jetson Software tools suite is designed to let engineers and developers in a myriad of disciplines, from robotics to industrial automation, the IoT and more, program and develop in their native languages while optimizing on and targeting the Jetson platform. To give you a sense of what can be done with these tools and Jetson AGX Orin, Ben Funk over at HotHardware was able to quickly harness NVIDIA’s Riva AI speech SDK for a fun real-time demo. Check it out…
As you can see, NVIDIA’s Riva AI speech recognition engine has no problem keeping up with Ben’s cadence, understanding his pauses and minor corrections and ultimately understanding conversational context for better accuracy. Couple Riva Automated Speech Recognition (ASR) with conversational AI and you’ve got a chatbot that you can personify with additional NVIDIA tools and SDKs, like the company itself did in this demo…
NVIDIA’s Orion bot is fully powered by Jetson AGX Orin and demonstrates machine vision and location awareness for its navigation, conversational AI, recommendation AI and speech recognition. NVIDIA Omniverse Avatar then brings all of these technologies together for a cute and useful roving digital building assistant, again, all powered by Jetson AGX Orin.
Beyond just the trained vision models that NVIDIA provides for Jetson AGX Orin, the company also provides its TAO (Train, Adapt, Optimize) toolkit that affords developers easy access to adapt models to their own applications and build additional functionality locally, or deploy models on NVIDIA’s DeepStream SDK in the cloud. DeepStream can run on NVIDIA A100-powered data center servers, on Microsoft Azure Cloud infrastructure, for a major lift in performance and compute resources, though it can also run locally on prem and at the edge on Jetson itself.
NVIDIA Jetson AGX Orin Developer Kit Taps A Full Intelligent AI Edge Platform Solution
As a technology analyst, for me the key take-away here is that NVIDIA’s Jetson AGX Orin dev kit is a powerful tool suite of silicon engines and software for robotics, autonomous machines and other AI edge IoT (Internet of Things) devices. However, the GPU giant has taken the approach of enabling a full platform solution, with trained models, the ability to adapt and continually train them, as well as SDKs for machine vision, speech, conversational AI and more, along with the ability to deploy and refine locally or in the cloud.
I believe this true platform solution approach is the only way to really do AI or any type of adaptable computing solution right. Competitive solutions like FPGA-based platforms from Xilinx (AMD) take similar wholistic approaches, but NVIDIA has a significant jump in this market, having pioneered AI acceleration and development on GPUs for well over a decade now when its CUDA programming language was first introduced way back in 2007.
NVIDIA is currently offering Jetson AGX Orin developer kits for $1999 through its network of distributors. Various configurations of Jetson AGX Orin production modules above will start shipping in July, with lower cost Jetson Orin NX modules shipping in September. However, I’d expect NVIDIA distributor partners are already lighting-up more than a few orders for these early Jetson AGX Orin dev kits.