Featured in Jensen Huang's GTC 2026 keynote. Officially integrated in NVIDIA's open-source teleoperation framework — no custom integration pipeline required.
The most significant infrastructure decision in a robot policy training workflow is your data collection pipeline. Everything downstream — model quality, generalization, task success rate — is bounded by how well you captured the human demonstrations. For dexterous manipulation, this means how well you captured the hands.
NVIDIA has standardized on MANUS gloves as the official hand-tracking input device across its robotics simulation and teleoperation stack. This article is a technical guide to what that integration covers, how it works, and how to use it — from first setup through policy training.
Isaac Lab and Isaac Teleop: What Each Does
NVIDIA's robotics software stack has two relevant components for dexterous manipulation teams. Understanding how they differ matters before looking at where MANUS fits.
GPU-accelerated framework for robot learning — reinforcement learning, imitation learning, and motion planning in simulation. Built on Isaac Sim (NVIDIA Omniverse).
- Run thousands of parallel simulation environments on GPU
- Native teleoperation device support (v2.3+)
- Dexterous retargeting from human hand to robot joints
- Isaac Lab Mimic for demonstration augmentation
- SkillGen for motion-planner-generated data
- Open source — github.com/isaac-sim/IsaacLab
Unified framework for teleoperation and structured demonstration data collection across both simulation and physical systems. Open-sourced at GTC 2026.
- Standardized human input → robot action pipeline
- MANUS Gloves Plugin via MANUS SDK
- Works with simulation and real-world robots
- Consistent data format across environments
- Reduces per-robot custom integration effort
- Open source — GitHub: IsaacLab
In practice: Isaac Lab is where you run simulation and train policies. Isaac Teleop is the standardized layer that translates MANUS glove inputs into robot commands and structured demonstration data — in simulation (via Isaac Lab) and on real hardware. The two are designed to work together as a continuous pipeline from human demonstration to deployed policy.
How MANUS Integrates with Both
In Isaac Lab 2.3
Isaac Lab 2.3 introduced native MANUS support as a teleoperation input device. The integration provides dexterous retargeting — the system reads MANUS glove joint angles in real time and maps them to the corresponding joints of a simulated robot hand, accounting for kinematic differences between human and robot hand geometry.
The data flow inside Isaac Lab is direct:
In Isaac Teleop
Isaac Teleop integrates MANUS via the MANUS Gloves Plugin, which uses the MANUS SDK to stream tracking data into the Teleop framework. This is a plugin-based architecture — rather than building a custom bridge for each robot or environment, you connect the plugin once and MANUS data becomes available as a standard input across any Isaac Teleop workflow.
The advantage of Isaac Teleop's standardization is reusability: the same MANUS configuration that captures demonstrations in simulation can be used to control a physical robot using the same pipeline. Data format, retargeting logic, and tooling are consistent across both environments.
One integration, every robot
Before Isaac Teleop, teams building teleoperation pipelines had to write custom integration code for each combination of input device and robot. A MANUS-to-Fourier-GR1 pipeline required different work than a MANUS-to-Unitree-G1 pipeline. Isaac Teleop abstracts this: MANUS connects once via the plugin, and the retargeting layer handles the per-robot mapping. Switching to a different robot means changing the retargeting configuration, not rewriting the input integration.
Setup: Installation and Configuration
MANUS integration with Isaac Lab has two software installation modes depending on your operating environment.
Option A: Linux native
MANUS Core runs natively on Linux. This is the recommended path for research teams running Isaac Lab on Linux workstations. Install MANUS Core for Linux, connect the gloves, and configure the Isaac Lab MANUS teleoperation device.
Option B: Windows-to-Linux streaming
If MANUS Core is running on a Windows machine (common in lab setups where the Windows machine handles peripheral management), data can be streamed over the local network to the Linux machine running Isaac Lab. MANUS Core's network streaming mode handles this without additional middleware.
Verification before running
MANUS provides two verification tools that should be checked before any data collection session:
Confirms all glove sensors are active, battery levels are adequate, and the EMF transmitter is calibrated. Run this before every session — data collected with a miscalibrated transmitter will have systematic joint angle errors that corrupt demonstrations.
Confirms that MANUS Core is streaming data and that Isaac Lab is receiving it correctly. Check that all 25 DoF per hand are being reported — missing joints indicate a connection or calibration issue that should be resolved before recording.
Isaac Lab's open-source retargeter maps human hand joint angles to robot joint positions. The retargeter requires per-user hand scale calibration — typically a 30-second process where the operator holds specific hand poses. This scales the skeleton to match the operator's actual hand geometry, which improves retargeting accuracy significantly.
--xr flagWhen using XR devices in Isaac Lab, pass the --xr flag to the Isaac Lab script. This loads the necessary XR extensions and enables the AR panel in the UI, which shows teleoperation status and IK controller alerts during data collection.
The Full Data Collection Workflow
Once MANUS is calibrated and connected, the demonstration recording workflow in Isaac Lab follows a consistent structure:
Select the manipulation task environment in Isaac Lab. Isaac Lab 2.3 includes new dexterous environments for Lift and Reorient tasks, plus updated locomanipulation environments for robots with both arm and locomotion DOF (Unitree G1, Fourier GR1T2).
Set MANUS as the teleoperation input in the per-task configuration. Isaac Lab's retargeter framework allows per-task device configuration — different tasks can use different input devices or retargeting parameters.
The operator wears MANUS gloves and performs the task. Hand configurations map directly to robot joint positions in real time — what the human hand does, the simulated robot hand mirrors. Demonstrations are recorded as structured datasets automatically.
Isaac Lab 2.3 added UI elements that alert the operator to IK controller errors — at-limit joints and no-solve states — during collection. A completion popup notifies the operator when a demonstration is successfully saved. These quality signals reduce the proportion of unusable demonstrations in a dataset.
Recorded demonstrations export as structured datasets compatible with imitation learning frameworks. The format is consistent whether the demonstration was captured in simulation or on real hardware via Isaac Teleop.
Isaac Lab Mimic: Scaling Demonstrations
Recording high-quality demonstrations is time-intensive. A single experienced operator can record perhaps 10–30 demonstrations per hour for a complex task. Training a policy that generalizes across task variations typically requires hundreds to thousands of demonstrations.
Isaac Lab Mimic addresses this gap. It takes a small set of human demonstrations recorded with MANUS gloves and automatically generates a much larger augmented dataset by varying object positions, grasp approaches, and environmental conditions within the simulation. The pipeline:
- Record a small set of demonstrations with MANUS gloves (typically 10–50 for a task)
- Pass the demonstrations to Isaac Lab Mimic
- Mimic generates hundreds or thousands of valid task variations using GPU-accelerated simulation
- Train the policy on the augmented dataset
This is the core value proposition of sim-first data collection. The human time investment is in recording a small, high-quality seed set — Mimic handles scale. MANUS gloves, by ensuring that seed set is data-quality correct (no occlusion dropouts, no drift artifacts), directly improve the quality of everything Mimic generates downstream.
Motion-planner-generated demonstrations
Isaac Lab 2.3 also introduced SkillGen: a workflow that combines human-provided subtask segments with GPU-accelerated motion planning to generate adaptive, collision-free manipulation demonstrations. SkillGen is complementary to Mimic — it's most useful for tasks with well-defined subtask structure (approach, contact, retreat phases). MANUS demonstrations feed the human-provided subtask segments that SkillGen uses as seeds.
Sharpa Wave 22-DoF: Real-World Validation
The most widely documented demonstration of MANUS + Isaac Lab is the Sharpa Wave teleoperation setup shown at NVIDIA GTC 2026. The Sharpa Wave is a 22-degree-of-freedom dexterous robotic hand — one of the highest-DoF robot hands currently available. Teleoperating it with MANUS gloves inside Isaac Lab demonstrates the complete pipeline under demanding conditions.
Key details from the Sharpa Wave demonstration:
- Operators wearing MANUS gloves teleoperated the 22-DoF Sharpa Wave in Isaac Lab in real time — hand configurations map directly to robot joint positions
- The demonstrations were fed directly into Isaac Lab Mimic for augmentation, then into imitation learning pipelines — all within simulation
- Dr. Scott Walter — a pioneer in humanoid robotics — tested the setup live at GTC, including catching a thrown ball, a task requiring sub-100ms latency and precise multi-finger coordination
- Analog Devices used MANUS gloves at GTC as the human motion capture layer in their tactile AI simulation pipeline — pairing IPC physics-generated contact data with MANUS motion data to train anti-slip grasping algorithms
"Millimeter-level finger tracking streamed directly into Isaac Lab. Occlusion-free, drift-free data throughout full operation sessions. Seamless pipeline from human demonstration → Isaac Lab Mimic augmentation → policy training."
MANUS Technology Group — GTC 2026 integration documentationResearch Using This Stack
The MANUS + NVIDIA Isaac ecosystem is active in published research. Two notable examples from 2026:
USC Ψ₀ (Psi-Zero) — Open Foundation Model for Humanoid Loco-Manipulation
USC's Physical Superintelligence Lab used MANUS gloves to capture fine-grained finger motions for their open foundation model, developed with NVIDIA and WorldEngine. Trained on approximately 800 hours of human video and 30 hours of real-world robot data, Ψ₀ outperformed baselines trained on over 10× more data by more than 40% in task success rate. The paper states directly: "Fine-grained finger motions are acquired using MANUS gloves, allowing direct control over all degrees of freedom of the dexterous hands."
NVIDIA EgoScale
Recent NVIDIA-led research on EgoScale highlights the importance of large-scale human data for training dexterous robot policies. MANUS enables the capture of high-fidelity human motion — both at small-scale for targeted teleoperation and at larger scale for the emerging human-to-robot learning workflows these approaches depend on.
Hardware to Order
The MANUS gloves used with Isaac Lab and Isaac Teleop are the same Metagloves available through Knoxlabs. Both integrate natively with NVIDIA's stack via the MANUS SDK and Gloves Plugin.
25 DoF, EMF tracking, native Isaac Lab + Teleop support. Best for demonstration recording and AI training pipelines.
View on Knoxlabs
25 DoF + per-finger haptics. Same Isaac Lab integration — adds tactile feedback for live teleoperation workflows.
View on KnoxlabsFor teams also running full-body teleoperation inside Isaac Lab, the Xsens Link full-body system pairs with MANUS for complete human motion input. See Article 2 for how the two systems integrate.
Setting up a MANUS + Isaac Lab pipeline?
Knoxlabs can configure and ship MANUS gloves with your Xsens and headset stack as one order.
The full component catalog is on the Knoxlabs Robotics & Teleoperation hub, including MANUS, Xsens, VR headsets, and compatible robot hands. For the complete deployment process — provisioning, configuration, and support — see From Vision to Reality: White-Glove XR Deployment That Actually Works and Article 6: Knoxlabs Robotics Deployment.
Leave a comment