Contribution

Sim2Real Transfer for Vision-Based Grasp Verification

---

Abstract

The verification of successful grasps is a crucial aspect of robot manipulation, particularly when handling deformable objects. Traditional methods relying on force and tactile sensors often struggle with deformable and non-rigid objects. In this work, we present a vision-based approach for grasp verification to determine whether the robotic gripper has successfully grasped an object. Our method employs a two-stage architecture; first YOLO-based object detection model to detect and locate the robot’s gripper and then a ResNet-based classifier determines the presence of an object.To address the limitations of real-world data capture, we introduce HSR-GraspSynth, a synthetic dataset designed to simulate diverse grasping scenarios. Furthermore, we explore the use of Visual Question Answering capabilities as a zeroshot baseline to which we compare our model. Experimental results demonstrate that our approach achieves high accuracy in real-world environments, with potential for integration into grasping pipelines. Code and datasets are publicly available at github.com/pauamargant/HSR-GraspSynth

Keywords: Grasp verification, Robot manipulation, Deformable objects, Vision-based grasping, YOLO object detection, ResNet classification, Synthetic dataset, Visual Question Answering

How to Cite:

Amargant, P., Hönig, P. & Vincze, M., (2025) “Sim2Real Transfer for Vision-Based Grasp Verification”, ARW Proceedings 25(1), 49-54. doi: https://doi.org/10.34749/3061-0710.2025.8