Full-Stack Solution Engineer - Sensorized Human at NVIDIA

Job description

At NVIDIA, we are closing the "embodiment gap." We don’t just build robots; we build digital and physical nervous systems that allow humans to teach robots. You will lead the development of DexUMI (Dexterous Universal Manipulation Interface), a framework that leverages human-worn hardware and advanced computer vision to transfer complex skills from human hands to robotic actuators. This is a True Full-Stack role in Solutions Architecture Team: you will touch everything from the tactile sensor firmware on a wearable exoskeleton to the cloud-based data pipelines that train our diffusion policies. What you'll be doing: Hardware-Software Co-Design: Maintain and iterate on the DexUMI wearable exoskeleton. You will bridge the kinematics gap between human hands and diverse robot end-effectors (e.g., XHand, Inspire Hand). Sensor Fusion & Integration: Integrate high-fidelity tactile sensors and IMUs into wearable interfaces. Ensure low-latency data streaming. Vision & Perception Pipelines: Implement and optimize the "software adaptation" layer—using tools to segment human operators out of training data and robot embodiments. Data Engineering for AI: Build robust pipelines to collect, clean, and replay dexterous manipulation data for Imitation Learning and Diffusion Policies. Optimization: Solve bi-level optimization problems to parameterize exoskeleton designs that maximize human wearability while preserving robot-equivalent fingertip workspaces. What we need to see: A MS/PhD in Robotics, Machine Learning, Computer Science, Electrical Engineering, Mechanical Engineering, or a related field (or equivalent experience) with at least 1 years of research and engineering experience. The "Body" (Hardware/Embedded): Proficiency in C/C++ for embedded systems and ROS2, with hands-on experience in tactile sensing, force-feedback (haptics), and motor control. Mechatronics & Prototyping: Experience with CAD (SolidWorks/Fusion360) and rapid prototyping, including 3D printing and PCB design. The "Brain" (Software/AI): Expertise in Python and deep learning frameworks (PyTorch), with familiarity in computer vision (CV) models. Advanced AI Techniques (Plus): Understanding of Imitation Learning or Reinforcement Learning (RL) is a strong plus. The "Bridge" (Integration): Experience with Record3D or iPhone-based spatial tracking, enabling integration between perception and physical systems. Systems & Infrastructure: Experience working in Ubuntu/Linux environments with high-performance data serialization tools such as Protobuf and MQTT.