PoseLess: Depth-Free Vision-to-Joint Control via Direct Image Mapping with VLM

Alan Dao, Dinh Bach Vu, Tuan Le Duc Anh, Bui Quang Huy

PoseLess: Depth-Free Vision-to-Joint Control via Direct Image Mapping with VLM: 3 upvotes on Hugging Face Daily Papers, #36 of 39 papers on 2025-03-14. Day-by-day upvote history.

This paper introduces PoseLess, a novel framework for robot hand control that eliminates the need for explicit pose estimation by directly mapping 2D images to joint angles using projected representations. Our approach leverages synthetic training data generated through randomized joint configurations, enabling zero-shot generalization to real-world scenarios and cross-morphology transfer from robotic to human hands. By projecting visual inputs and employing a transformer-based decoder, PoseLess achieves robust, low-latency control while addressing challenges such as depth ambiguity and data scarcity. Experimental results demonstrate competitive performance in joint angle prediction accuracy without relying on any human-labelled dataset.

Paper page on Hugging Face · arXiv

Data: hysts-bot-data/daily-papers-stats and the Daily Papers API. Open data: tardellirs/paper-pulse-data. Sister project: Model Pulse, the download history of every model on the Hub.