“Large behavior model” became a widely used term when Boston Dynamics showed Atlas performing long, whole-body manipulation sequences. This article uses Toyota Research Institute’s LBM paper and the Atlas write-up to pin down what an LBM takes in, what it outputs, what data and evaluation produced it, and how it differs from the vision-language-action (VLA) models that rose to prominence at the same time.
One-sentence definition: a language-conditioned policy pretrained on many tasks
TRI’s LBM paper treats an LBM as a multitask visuomotor policy pretrained on robot demonstrations from many tasks, and states that the models extend the Diffusion Policy paradigm. The policy receives camera images, proprioception (joint positions and similar signals) and a language instruction, and generates a chunk of continuous actions covering the next several steps.
“Large” here means less about parameter count, as with LLMs, and more about breadth: many tasks and many hours of demonstrations, so that new tasks can be learned quickly. The abstract makes three claims. Multitask pretraining yields policies that are more successful and robust than single-task baselines, new tasks are learned from a fraction of the data, and performance rises predictably as pretraining scale and diversity grow.
Inside TRI’s LBM: architecture and pretraining data
Section 4 of the full paper describes the setup, summarized here:
| Component | TRI LBM setting |
|---|---|
| Generative method | DDIM-based diffusion; the noise-prediction network is a Diffusion Transformer (DiT) with 8 blocks and embedding size 768 |
| Vision encoder | CLS token of a pretrained CLIP ViT-B/16, fine-tuned during training |
| Language encoder | Frozen CLIP text encoder plus a trainable projection |
| Observations | Two time steps, up to six RGB images, proprioception, one language instruction |
| Actions | Predicts 16 steps of 20-dimensional actions and executes the first 8, at 10 Hz |
| Robot platform | Tabletop bimanual station built from two Franka FR3 arms |
The pretraining mix is the most concretely documented part of the study. The paper reports about 1,695 hours of demonstrations in total (rounded to roughly 1,700 in the introduction), broken down as follows:
- TRI-Ramen-Real: 468 hours, 362 tasks, 46,063 demonstrations, nine hardware stations
- TRI-Ramen-Sim: 45 hours, 41 tasks, 7,348 demonstrations, two simulation stations
- TRI-Ramen-UMI: 32 hours, 129 tasks, 10,851 demonstrations
- OXE-Ramen: a subset of the Open X-Embodiment datasets, about 1,150 hours
TRI’s own collection accounts for 545 hours and 532 tasks; the rest is curated from public data. How human demonstrations are collected and quality-checked is the same problem covered in real teleoperation data versus synthetic data.
Why the evaluation design matters as much as the result
The phrase “careful examination” in the title refers to the evaluation protocol. Each real-world task was run 50 times per policy per condition, and each simulated task 200 times. Evaluators did not know which policy they were testing, policy order was randomized, and initial conditions were bundled so every policy faced the same set. Success rates are reported as Bayesian posteriors, and multi-policy comparisons use a Bonferroni correction with Compact Letter Display to show which differences are statistically distinguishable.
The data-efficiency figures come out of that protocol. According to the paper, on the real task SetBreakfastTable an LBM fine-tuned on only 15% of the data statistically outperformed a single-task policy trained on all of it, and in simulation fine-tuning an LBM reached similar performance with less than 30% of the data needed to train from scratch. Unlike a demo reel, the paper reports which differences hold up statistically and under which conditions, which is what makes it worth reading closely.
Atlas: the same idea on a whole-body humanoid
The Boston Dynamics write-up of its collaboration with TRI extends the idea to a humanoid that walks while it manipulates. The key settings:
- Model: a 450M-parameter Diffusion Transformer trained with a flow-matching objective
- Inputs: images from head-mounted HDR stereo cameras, proprioception, and a language prompt that specifies the goal
- Rate: the full robot is controlled at 30 Hz; each inference predicts a 48-step chunk (1.6 seconds) and typically executes 24 steps (0.8 seconds)
- Action space: left and right gripper joints, neck yaw, torso pose, left and right hand poses, left and right foot poses
- Data: VR teleoperation built on Boston Dynamics’ MPC controller, co-trained with simulation data
The post describes a long-horizon sequence of folding Spot’s legs, sorting parts and clearing a bin, plus tasks such as handling a 22 lb tire, tying rope and unfurling a tablecloth. The same post notes that the policy can run roughly 1.5x to 2x faster at inference time without a significant drop in performance. Foot poses in the action space are the clearest difference from the tabletop station: walking and balance are delegated to the MPC controller underneath, while the LBM outputs target poses. That split mirrors the layered control described in humanoid bipedal locomotion and whole-body control.
LBMs and VLAs on the same axes
Both families map images and language to actions, but they start from different places. A VLA uses a vision-language model pretrained on internet-scale data as its backbone; an LBM trains comparatively small encoders and a diffusion transformer primarily on robot demonstrations. The internals of VLAs are covered separately in how VLA models turn images and instructions into joint commands.
| Aspect | TRI LBM | Atlas LBM | OpenVLA | π0 |
|---|---|---|---|---|
| Backbone | CLIP ViT-B/16 + CLIP text + DiT | Diffusion Transformer (450M) | Llama 2 + DINOv2/SigLIP (7B) | PaliGemma 3B + 300M action expert (3.3B total) |
| Action generation | Diffusion (DDIM) | Flow matching | Discrete tokens (256 bins per dimension) | Flow matching |
| Control rate | 10 Hz | 30 Hz | About 6 Hz on an RTX 4090 in bf16 | Up to 50 Hz |
| Source of language understanding | CLIP text encoder | Language prompt | LLM pretraining | VLM pretraining |
OpenVLA values come from the OpenVLA paper and π0 values from the π0 paper. As the table shows, the boundary is blurring: a VLA such as π0, with a flow-matching action expert on top of a VLM, generates actions the same way an LBM does, and LBMs also take language. In practice, the deciding questions are how much language and common-sense generalization the task needs, and how many demonstrations of your own robot you can realistically collect.
Three common misreadings
An LBM is not a released model name. It is a research concept used by TRI and Boston Dynamics, and the two examples differ in model size, action space and control rate. “Using an LBM” is best read as “using a diffusion-family policy pretrained on many tasks’ demonstrations”.
The data-efficiency numbers are conditional. The 15% and sub-30% results are comparisons on particular tasks against single-task baselines. It is reasonable to expect larger gains on tasks that resemble the pretraining distribution and smaller ones elsewhere.
Evaluation infrastructure comes before the model. You can only judge the effect of pretraining if every policy change is re-evaluated from the same initial conditions. If you want to train a policy yourself, start the way the LeRobot ACT training guide does: get demonstration collection and repeatable evaluation working on a small robot first.
What to settle before adopting the LBM approach
- Define the action space and control rate. Whether the policy outputs joint angles directly or hand and foot targets for a lower-level controller changes the data format.
- Build teleoperation hardware and a demonstration quality bar that let you record many tasks in the same format.
- Log the ratio of public data (for example Open X-Embodiment) to your own, and keep a baseline policy so you can measure what pretraining actually adds.
- Fix initial-condition sets and repetition counts per task, and adopt a blind-evaluation rule.
Once those four are in place, model size and the generative method (diffusion or flow matching) can be chosen to fit your GPU budget and control rate. For sizing the onboard compute, the procedure in the VLA edge hardware sizing guide applies in the same way.