All projects

Vision-Based Bin-Picking Assembly on a Rainbow RB10

Oct 2026 – Nov 2026 · 2nd place · Excellence Award · Team Member · Perception, Data & Field Integration · Team WIC, Hanyang University

Machine tending and assembly are classic manufacturing jobs for robots, and the hard part is variation: parts arrive at random, overlap, and sit under changing light. Our three-person lab team took part in KETI’s 2026 robot manufacturing-process competition, where a Rainbow Robotics RB10 picks parts out of a bin and inserts them into matching guides. We started from a VLA-based design, changed course when speed became the constraint, and finished 2nd with a modular pipeline built around reliable perception from the RealSense D415.

Final ranking · Excellence Award
2nd
Real D415 test targets detected, 0 false positives
144 / 144
Max hand-eye calibration validation residual
1.35 mm
The three competition parts (a cylinder, a rectangular block with a cylindrical hole through the center, and a cross-shaped prism) and the fixed setup: a RealSense D415 camera feeding a YOLO26 detector for a Rainbow RB10 with a Zimmer gripper
The three part types and the fixed hardware. YOLO26 segments parts and guides from the D415 image; the camera’s color shift was the main perception problem.

The competition task

  • Host: Korea Electronics Technology Institute (KETI), organized with Sungkyunkwan University. The goal is robot technology that raises manufacturing productivity.
  • Three part types are placed at random in a bin: cylinders, rectangular blocks with a cylindrical hole through the center, and cross-shaped prisms. The robot picks each one and inserts it into the guide of the same shape.
  • Hardware is fixed: a Rainbow Robotics RB10 arm, a Zimmer GEH6060IL-03-B gripper, and an Intel RealSense D415 camera. Teams may change the gripper fingertips.
  • Each run lasts 20 minutes: single-layer parts for the first 10, stacked parts for the last 10. Only parts completed without a collision count.
  • Schedule: algorithm development Oct 12 – Nov 6, the competition on Nov 10 in Seoul, and the award ceremony with winner demonstrations at the IRC Challenge in Daegu (EXCO), Nov 13–14.

From a VLA policy to a modular pipeline

I first proposed using GR00T, a pretrained vision-language-action model, as a teacher policy that learns the whole recognize–grasp–transport–insert task, and distilling it into a lightweight student for the robot. In practice, speed became the deciding constraint.

The team split the task into stages that can each be measured and tuned: segment parts and guides, compute each part’s position and angle from calibrated geometry, then plan the grasp and insertion with collision checks and run them through the official SDK.

The final system

  • Perception: a YOLO26n instance-segmentation model reads the RGB image from the wrist-mounted D415 and finds both the parts and the empty guides for all three shapes.
  • Pose: for single-layer parts, each mask outline is projected onto the measured bin floor raised by the 30 mm part thickness from CAD, giving the part’s position and yaw. A depth-based method was kept for comparison.
  • Calibration: an automated board-based hand–eye calibration from 15 captures, checked on 5 more, kept the largest validation residual at 1.351 mm and 0.425°. A lens recalibration for the venue camera kept validation errors under 0.92 mm and 0.29°.
  • Motion: grasps use the gripper’s force-control mode; paths are checked against the bin walls and registered obstacles, and a direct route replaces intermediate stops whenever the straight path is clear.
  • Operation: a local web console starts the robot and gripper servers, shows live detections, runs the calibrations, and repeats rounds by itself, comparing the guides with an empty reference image and starting the next round once they are cleared.
MuJoCo simulation of the competition workcell: an RB10 arm with a gripper above a yellow parts bin and a guide plate inside an aluminum frame
The workcell rebuilt in MuJoCo with adapters for the official SDK, used to test perception, motion, and fingertip designs before touching the real robot.

Making perception hold up on the real D415

The D415’s colors are noticeably off. A model trained only on images from our simulated D415 found every part in real images but missed all 36 guides and produced 12 false detections. I collected real D415 images of every part and guide from 18 viewpoints and added color-corrected copies, then fine-tuned on them mixed with synthetic data.

On held-out viewpoints the fine-tuned model detected all 144 targets with no false positives, and mask mAP50–95 rose from 43.9% to 91.9%. The test shares the training layout and differs only in viewpoint, so this measures robustness to the camera rather than a general success rate.

Side-by-side detections on real D415 images: the earlier model misses the square-ring guide and several parts, while the D415 fine-tuned model labels every guide and part
Real D415 images at confidence 0.8. Left: an earlier fine-tune on phone photos. Right: the model fine-tuned on real D415 images.

What we tested along the way

  • Compared observation strategies, pick orders, and how close the camera should be for a final check, trading accuracy against cycle time in simulation.
  • Designed 3D-printable fingertips. A shallow V-groove steadied cylinders but left more error on the ring part, so the original metal fingertips stayed on.
  • Measured the angle error of a pick-and-place-back motion and stored a per-shape yaw correction only when repeated trials agreed.

My role

  • The only undergraduate on a team of three, working with two master’s students.
  • Proposed the initial GR00T-based VLA teacher–student architecture.
  • Trained the YOLO26 model, collected the real D415 data, and verified the code.
  • In the final week, ran the robot and gripper SDK in a container to fix library conflicts, and added TCP position control, lens and yaw calibration, force-controlled gripping, shorter routes, and console updates.

Result

At the competition in Seoul on November 10, 2026, our team placed 2nd and received the Excellence Award. Under the official notice, the 2nd-place award is presented as the Commissioner of the Korean Intellectual Property Office Award. The ceremony is at the IRC Challenge in Daegu (EXCO) on November 13–14; the certificate will be added once it arrives.

Team WIC holding the award certificate in front of the competition robot cells under the 2026 Robot Manufacturing Process Competition banner
Team WIC with the award at the KETI Collaborative Intelligence Demonstration Center in Seoul, Nov 10, 2026.

Resources

Technologies: YOLO26 (instance segmentation) · Intel RealSense D415 · Rainbow Robotics RB10 · Zimmer GEH6060IL Gripper · MuJoCo · Python · Docker · NVIDIA GR00T (initial design)