Correcting Persistent Failures in Point-Cloud Grasping Policies using Human Demonstrations in Virtual Reality
Achyuthan Unni Krishnan, Karthik Rengarajan, Bharath Sivaram Narasimhan, Chris von Csefalvay
Correcting Persistent Failures in Point-Cloud Grasping Policies using Human Demonstrations in Virtual Reality: 7 upvotes on Hugging Face Daily Papers, #16 of 42 papers on 2026-11-17. Day-by-day upvote history.
Upvotes over time
It had 5 upvotes by its first day, 7 upvotes by the next day, 7 upvotes by day 3, 7 upvotes by day 7, and has 7 now. It ranked #61 of 146 papers in its week and #283 of 520 in its month.
Robots in homes, warehouses and farms must pick up objects they have never seen, usually without a 3D model or a category label. A grasping policy that works directly on the point cloud from a depth sensor needs only the geometry the robot observes, so it applies to any new object. Offline grasp generators such as Contact-GraspNet work this way, but they rank grasps by learned plausibility rather than by whether the grasp actually lifts the object. Reinforcement learning (RL) can optimize for that directly and make a point-cloud grasping policy much better on average, yet it leaves specific grasps persistently unsolved. We present a human-in-the-loop failure loop that targets those grasps: situations an RL policy keeps failing are mined automatically, sent to a virtual-reality application where a person picks the object up with their own hands, and returned to training as two grasp points. The person contributes only where to grasp; the robot chooses its own approach. In simulation, RL raises success on unseen objects from 55% with Contact-GraspNet to 83%. Adding 58 human demonstrations nearly doubles success on the hard cases RL kept failing, from 24% to 45%, and recovers performance that RL alone loses on similar cases that were never demonstrated, without reducing success on unseen objects.
Paper page on Hugging Face · arXiv
Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.