FAST: Efficient Action Tokenization for Vision-Language-Action Models

arxiv.org · 13,893 words · saved by 1 readers

This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. HTML conversions sometimes display errors due to content that did not convert correctly from the source. This paper uses the following packages that are not yet supported by the HTML conversion tool. Feedback on these issues are not necessary; they are known and are being worked on. Authors: achieve the best HTML results from your LaTeX submissions by following these best practices. Autoregressive sequence models, such as Transformer-based vision-language action (VLA) policies, can be tremendously effective for capturing complex and generalizable robotic behaviors. However, such models require us to choose a tokenization of our continuous action signals, which determines how the discrete symbols predicted by the model map to continuous robot actions. We

FAST: Efficient Action Tokenization for Vision-Language-Action Models Karl Pertsch ∗,1,2,3 , Kyle Stachowicz ∗,2 , Brian Ichter 1 , Danny Driess 1 , Suraj Nair 1 , Quan Vuong 1 , Oier Mees 2 , Chelsea Finn 1,3 , Sergey Levine 1,2 1 Physical Intelligence, 2 UC Berkeley, 3 Stanford https://pi.website/research/fast ∗ : Core contributors Correspondence to: research@physicalintelligence.company Abstract Autoregressive sequence models, such as Transformer-based vision-language action (VLA) policies, can be tremendously effective for capturing complex and generalizable robotic behaviors. However, suc

FAST: Efficient Action Tokenization for Vision-Language-Action Models

related reading