Robots are entering their GPT-3 era. For years, researchers have attempted to train robots using the same autoregressive (AR) models that power large language models (LLMs). If a model can predict the next word in a sentence, it should also be able to predict the next movement for a robotic arm. However, a technological wall has blocked this progress: converting continuous robot movements into individual tokens is difficult.
A team of researchers from Harvard University and Stanford University have released a new framework called Ordered Action Tokenization (OAT) To bridge this gap.
The dirty reality of robot actions
Tokenization transforms complex data into a sequence of individual numbers (tokens). To the robot, these actions are constant signals like joint angles. Previous strategies had fatal flaws:
- Binning: Each action changes the dimension to ‘bin’. While simple, this creates massive sequences that slow down training and inference.
- FAST (Frequency-Spaced Action Sequence Tokenization): Uses mathematics to compress motions into frequency coefficients. It is fast but often produces ‘undecodable’ sequences where small errors cause the robot to stop or move unexpectedly.
- Learned latent tokenizers: These use a learned ‘dictionary’ of movements. They are safe but lack a specific order, meaning the model treats early and late tokens as equally important.
Three Golden Rules of OAT
The research team identified 3 essential properties—desiderata—for a functional robot tokenizer:
- High Compression (P.1): The token sequence should be small to keep the model efficient.
- Total Decodability (P.2): The decoder must be an exhaustive function, ensuring every possible token sequence maps to a valid movement.
- Causal order (p.3): Tokens should have a left-to-right structure where early tokens capture global momentum and later tokens refine the details.
Secret Sauce: Nested Dropouts and Registers
OAT uses a transformer encoder register token To summarize the parts of the action. To force the model to learn the ‘important’ things first, the research team used an innovative approach called nested dropout.
breaking the benchmark
The research team tested OAT on 20+ tasks in 4 major simulation benchmarks. OAT consistently outperforms industry-standard Diffusion Policy (DP) And last tokenizer.
performance results
| benchmark | oat success rate | dp success rate | bin token count | oat token count |
| libero | 56.3% | 36.6% | 224 | 8 |
| robomimic | 73.1% | 67.1% | 224 | 8 |
| metaworld | 24.4% | 19.3% | 128 | 8 |
| robocasa | 54.6% | 54.0% | 384 | 8 |
‘Anytime’ estimation: speed vs. precision
The most practical benefit of OAT is Prefix-based detokenization. Since tokens are ordered by importance, you can stop the model early.
- Coarse tasks: Decoding just 1 or 2 tokens gives the robot a general direction immediately, which is useful for low-latency tasks.
- Great work: Generating all 8 tokens provides the high-precision detail required for complex entries.
This allows an intuitive trade-off between computation cost and action fidelity that previous fixed-length tokenizers could not offer.
key takeaways
- Solving the Tokenization Gap: OAT addresses a fundamental limitation in applying autoregressive models in robotics by introducing a learned tokenizer that simultaneously achieves high compression, total decodability, and causal ordering.
- Ordered representation via nested dropouts: By using nested dropout during training, OAT forces the model to prioritize global, coarse motion patterns in early tokens, while reserving later tokens for finer refinement.
- Total decodability and reliability: Unlike prior frequency-domain methods such as FAST, OAT ensures that the detokenizer is an aggregate function, meaning that every possible token sequence produces a valid action chunk, preventing runtime execution failures.
- Flexible ‘anytime’ estimation: The ordered structure enables prefix-based decoding, allowing robots to execute coarse tasks from just one or two tokens to save computation or full eight-token sequences for high-precision tasks.
- Better performance in all benchmarks: Autoregressive policies equipped with OAT consistently outperform diffusion-based baselines and other token schemes, achieving an overall success rate of 52.3% and superior results in real-world ‘pick and place’ and ‘stack cup’ tasks.
check it out Paper, Repo and Project pages. Also, feel free to follow us Twitter And don’t forget to join us 100k+ ml subreddit and subscribe our newsletter. wait! Are you on Telegram? Now you can also connect with us on Telegram.
Michael Sutter is a data science professional and holds a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michael excels in transforming complex datasets into actionable insights.
