Papers
arxiv:2606.06155

AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding

Published on Oct 18
· Submitted by
Qize Yu
on Jun 5
Authors:
,
,
,
,
,
,
,
,
,
,
,

Abstract

AffordanceVLA uses structured affordance learning as an intermediate representation to bridge vision-language models and robotic control, improving manipulation through progressive training and automated data augmentation.

Vision-Language-Action (VLA) models leverage the rich world knowledge of pretrained vision-language models (VLMs) to enable instruction-following robotic manipulation. However, the structural mismatch between VLM semantic spaces and embodied control policies often hinders the learning of precise perception-action mappings. To address this challenge, we propose AffordanceVLA, a scalable, unified framework that introduces structured affordance learning as a task-oriented and object-centric intermediate representation to establish a more precise and robust perception-action mapping. Specifically, we progressively model manipulation priors through three complementary components - Which2Act, Where2Act, and How2Act - which jointly produce spatially grounded, semantically conditioned, and action-coupled intermediate representations, thereby naturally bridging vision, language, and action. We integrate these modules into a Mixture-of-Transformer (MoT) architecture and adopt a three-stage progressive training curriculum that leverages 8.8M affordance-centric grounding and pointing samples, translating scalable vision-language supervision into improved robotic manipulation. To overcome the scarcity of dense affordance labels in robotic datasets, we also develop a robust automated data augmentation pipeline. Extensive experiments on ID & OOD simulation and real-world demonstrate that AffordanceVLA outperforms strong VLA baselines across diverse manipulation scenarios, validating structured affordance learning as a scalable and robust bridge that integrates multi-source vision-language supervision while connecting pretrained VLM semantics to embodied robot control.

Community

Paper author Paper submitter

AffordanceVLA introduces a structured affordance-forecasting bridge for VLA models, enabling robots to reason about what to manipulate, where to interact, and how to act for more robust instruction-following manipulation.

Excellent work with plenty of insightful takeaways.

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2606.06155
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2606.06155 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2606.06155 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2606.06155 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.