GR00T VLA Teacher–Student Policy for Bin-Picking Assembly
Machine tending and assembly are classic manufacturing jobs for robots, and the hard part is variation: parts arrive piled at random, overlap, and sit under changing light. Our three-person lab team was selected for KETI’s 2026 robot manufacturing-process competition, where a robot picks peg-in-hole machine parts out of a bin and inserts them into an assembly fixture. Instead of a hand-tuned pipeline, we train a vision-language-action (VLA) policy built on NVIDIA’s GR00T, then compress it into a student policy fast enough for real-time control.
The competition task
- Host: Korea Electronics Technology Institute (KETI), organized with Sungkyunkwan University. The goal is robot technology that raises manufacturing productivity.
- 3–5 types of peg-in-hole parts are piled at random in a bin; the robot must pick each one and insert it into the matching slot of an assembly fixture.
- Each run lasts 20 minutes: single-layer parts for the first 10 minutes, stacked parts for the last 10. Teams are ranked by completed assemblies, with penalties for hitting the floor or walls.
- Schedule: algorithm development Oct 12 – Nov 6, the competition on Nov 10 in Seoul, and the award ceremony with winner demonstrations at the IRC Challenge in Daegu (EXCO), Nov 13–14.
Approach: GR00T as teacher, a lightweight student on the robot
A digital twin of the Franka and its two-finger gripper in Isaac Sim/Isaac Lab varies part pose, stacking, lighting, and sensor error to generate many machine-tending scenes. A world foundation model adds data that is hard to collect on a real line, such as failure cases and unusual working conditions.
GR00T, a pretrained VLA foundation model, serves as the teacher. Trained on real demonstrations and simulation data, it learns the whole task from recognizing a part to grasping, carrying, and inserting it, including how to recover after a failed grasp or insertion.
The large teacher is too slow to run on the robot directly, so its successful and recovery trajectories train a lightweight student specialized for this task. The student gives up generality it does not need here in exchange for fast inference. Failures seen on the real robot are reproduced in simulation and fed back into training.
Targets
| Metric | Target |
|---|---|
| Task success, reference setup | ≥ 90% |
| Task success, out of distribution | ≥ 80% |
| Parts assembled | ≥ 11 of 12 |
| Recovery success | ≥ 80% |
| Student inference time vs. teacher | ≥ 50% shorter |
| Real-time control rate | ≥ 10 Hz |
| Time per part (no retries) | ≈ 17–18 s |
| All 12 parts | ≤ 3.5 min nominal, ≤ 5 min with retries |
Development plan
- Week 1: set up the Franka and gripper environment in Isaac Sim/Isaac Lab and collect real and simulated training data.
- Week 2: train the GR00T teacher and implement the recognize–grasp–transport–insert policy.
- Week 3: validate single-layer and stacked scenes in simulation, add failure and recovery data, and tune the policy.
- Week 4: train the lightweight student, transfer from simulation to the real robot, and run final tests on the Franka.
My role
- Proposed the GR00T-based VLA teacher–student architecture for the team.
- Training the teacher and student policies and verifying the training and deployment code.
- Leading data collection: real-robot demonstrations, simulation data, and failure and recovery episodes.
Status
Our team was selected through the document review on October 2, 2026. The figures above are targets from the development plan submitted in September, not results; results will be added after the competition on November 10.
Resources
Technologies: NVIDIA GR00T (VLA) · Teacher–Student Distillation · Isaac Sim · Isaac Lab · World Foundation Model · Franka Emika Panda