Contribution
Authors: Pau Amargant , Peter Hönig , Markus Vincze
The verification of successful grasps is a crucial aspect of robot manipulation, particularly when handling deformable objects. Traditional methods relying on force and tactile sensors often struggle with deformable and non-rigid objects. In this work, we present a vision-based approach for grasp verification to determine whether the robotic gripper has successfully grasped an object. Our method employs a two-stage architecture; first YOLO-based object detection model to detect and locate the robot’s gripper and then a ResNet-based classifier determines the presence of an object.To address the limitations of real-world data capture, we introduce HSR-GraspSynth, a synthetic dataset designed to simulate diverse grasping scenarios. Furthermore, we explore the use of Visual Question Answering capabilities as a zeroshot baseline to which we compare our model. Experimental results demonstrate that our approach achieves high accuracy in real-world environments, with potential for integration into grasping pipelines. Code and datasets are publicly available at github.com/pauamargant/HSR-GraspSynth
Keywords: Grasp verification, Robot manipulation, Deformable objects, Vision-based grasping, YOLO object detection, ResNet classification, Synthetic dataset, Visual Question Answering
How to Cite: Amargant, P. , Hönig, P. & Vincze, M. (2025) “Sim2Real Transfer for Vision-Based Grasp Verification”, ARW Proceedings. 25(1). doi: https://doi.org/10.34749/3061-0710.2025.8