Vision-Based Bin-Picking Assembly on a Rainbow RB10
Machine tending and assembly are classic manufacturing jobs for robots, and the hard part is variation: parts arrive at random, overlap, and sit under changing light. Our three-person lab team took part in KETI’s 2026 robot manufacturing-process competition, where a Rainbow Robotics RB10 picks parts out of a bin and inserts them into matching guides. We started from a VLA-based design, changed course when speed became the constraint, and finished 2nd with a modular pipeline built around reliable perception from the RealSense D415.
- Final ranking · Excellence Award
- 2nd
- Real D415 test targets detected, 0 false positives
- 144 / 144
- Max hand-eye calibration validation residual
- 1.35 mm

The competition task
- Host: Korea Electronics Technology Institute (KETI), organized with Sungkyunkwan University. The goal is robot technology that raises manufacturing productivity.
- Three part types are placed at random in a bin: cylinders, rectangular blocks with a cylindrical hole through the center, and cross-shaped prisms. The robot picks each one and inserts it into the guide of the same shape.
- Hardware is fixed: a Rainbow Robotics RB10 arm, a Zimmer GEH6060IL-03-B gripper, and an Intel RealSense D415 camera. Teams may change the gripper fingertips.
- Each run lasts 20 minutes: single-layer parts for the first 10, stacked parts for the last 10. Only parts completed without a collision count.
- Schedule: algorithm development Oct 12 – Nov 6, the competition on Nov 10 in Seoul, and the award ceremony with winner demonstrations at the IRC Challenge in Daegu (EXCO), Nov 13–14.
From a VLA policy to a modular pipeline
I first proposed using GR00T, a pretrained vision-language-action model, as a teacher policy that learns the whole recognize–grasp–transport–insert task, and distilling it into a lightweight student for the robot. In practice, speed became the deciding constraint.
The team split the task into stages that can each be measured and tuned: segment parts and guides, compute each part’s position and angle from calibrated geometry, then plan the grasp and insertion with collision checks and run them through the official SDK.
The final system
- Perception: a YOLO26n instance-segmentation model reads the RGB image from the wrist-mounted D415 and finds both the parts and the empty guides for all three shapes.
- Pose: for single-layer parts, each mask outline is projected onto the measured bin floor raised by the 30 mm part thickness from CAD, giving the part’s position and yaw. A depth-based method was kept for comparison.
- Calibration: an automated board-based hand–eye calibration from 15 captures, checked on 5 more, kept the largest validation residual at 1.351 mm and 0.425°. A lens recalibration for the venue camera kept validation errors under 0.92 mm and 0.29°.
- Motion: grasps use the gripper’s force-control mode; paths are checked against the bin walls and registered obstacles, and a direct route replaces intermediate stops whenever the straight path is clear.
- Operation: a local web console starts the robot and gripper servers, shows live detections, runs the calibrations, and repeats rounds by itself, comparing the guides with an empty reference image and starting the next round once they are cleared.

Making perception hold up on the real D415
The D415’s colors are noticeably off. A model trained only on images from our simulated D415 found every part in real images but missed all 36 guides and produced 12 false detections. I collected real D415 images of every part and guide from 18 viewpoints and added color-corrected copies, then fine-tuned on them mixed with synthetic data.
On held-out viewpoints the fine-tuned model detected all 144 targets with no false positives, and mask mAP50–95 rose from 43.9% to 91.9%. The test shares the training layout and differs only in viewpoint, so this measures robustness to the camera rather than a general success rate.

What we tested along the way
- Compared observation strategies, pick orders, and how close the camera should be for a final check, trading accuracy against cycle time in simulation.
- Designed 3D-printable fingertips. A shallow V-groove steadied cylinders but left more error on the ring part, so the original metal fingertips stayed on.
- Measured the angle error of a pick-and-place-back motion and stored a per-shape yaw correction only when repeated trials agreed.
My role
- The only undergraduate on a team of three, working with two master’s students.
- Proposed the initial GR00T-based VLA teacher–student architecture.
- Trained the YOLO26 model, collected the real D415 data, and verified the code.
- In the final week, ran the robot and gripper SDK in a container to fix library conflicts, and added TCP position control, lens and yaw calibration, force-controlled gripping, shorter routes, and console updates.
Result
At the competition in Seoul on November 10, 2026, our team placed 2nd and received the Excellence Award. Under the official notice, the 2nd-place award is presented as the Commissioner of the Korean Intellectual Property Office Award. The ceremony is at the IRC Challenge in Daegu (EXCO) on November 13–14; the certificate will be added once it arrives.

Resources
Technologies: YOLO26 (instance segmentation) · Intel RealSense D415 · Rainbow Robotics RB10 · Zimmer GEH6060IL Gripper · MuJoCo · Python · Docker · NVIDIA GR00T (initial design)