When a person carries a cup past an obstacle, they may go around it on the left or on the right. Both are correct. Train on such demonstrations by fitting the average, though, and the robot may drive straight into the middle of the obstacle. Diffusion Policy solves this with a generative model. This article explains why the problem arises, how Diffusion Policy fixes it, and which settings you must choose to use it, based on the original paper.
The problem: with several right answers, the average is wrong
The simplest form of imitation learning fits a regression model that maps an observation to one action. Trained to minimize mean squared error, the model outputs something close to the average of the demonstrated actions for that observation.
If there is only one right answer, that works. But if the same scene appears with both left and right detours in the data, the average becomes “go straight ahead” — an action no demonstration contained. A distribution of correct actions with several peaks is called multimodal, and real manipulation is full of it: where to grasp an object, which hand to use first, which way to rotate a part.
The fix: generate the action instead of regressing it
Diffusion Policy (Chi et al., 2023) represents a robot policy as a conditional denoising diffusion process. Diffusion models, popularized for images by Ho et al. (2020), start from noise and remove it step by step until a data sample emerges. Diffusion Policy changes what is generated: not an image, but an action sequence conditioned on the current observation.
According to the project page, the policy learns the gradient of the action distribution’s score function and, at inference time, follows that gradient to iteratively improve the actions. Different noise starting points can converge to different valid answers, so instead of averaging, the policy picks one mode and carries it out consistently.
Conceptually, generation looks like this:
# Action generation in Diffusion Policy (conceptual)
obs_feat = encoder(images, robot_state) # observation conditioning
actions = sample_gaussian(shape=(T_pred, action_dim)) # start from noise
for k in reversed(range(K)): # K denoising iterations
noise_pred = denoiser(actions, k, obs_feat)
actions = scheduler.step(noise_pred, k, actions)
execute(actions[:T_exec]) # run the first part, then re-predict
T_pred is the length of the predicted sequence, T_exec the part actually executed, and K the number of denoising iterations. These three values are the key settings to tune for each task and each piece of hardware.
Three design choices that make it work on robots
The paper credits three elements with making diffusion practical on physical robots:
- Receding-horizon control. Predict an action sequence, execute only the first part, then predict again from new observations. Sequence prediction keeps motion consistent; regular re-prediction keeps it responsive to disturbances.
- Visual conditioning. Images are used only as conditions, not generated. Generating actions alone keeps computation manageable.
- A time-series diffusion transformer. A network designed for the temporal structure of action sequences, suited to high-dimensional action spaces.
With this combination, the paper reports an average 46.9% improvement in success rate over the prior state of the art across 12 tasks from 4 robot manipulation benchmarks. The project page also shows real-robot tasks: Push-T, pushing a T-shaped block into a target region; flipping a randomly placed mug so it lands lip down with the handle pointing left; and scooping, pouring and spirally spreading sauce on dough. On Push-T, the policy stayed robust when a hand blocked the camera or the block was physically disturbed.
Close relatives: ACT and π0
| Method | How actions are produced | Published results |
|---|---|---|
| Regression (MSE) policy | Observation → one action, approximates the mean | Risk of averaged motion on multimodal tasks |
| ACT | Transformer that generates action chunks | 6 real tasks from ~10 minutes of demos; 80–90% success on examples |
| Diffusion Policy | Iteratively denoises an action sequence from noise | Average 46.9% improvement across 12 tasks |
| π0 | VLM backbone + flow-matching action expert | 50-step action chunks; 10 Euler steps at inference |
All three predict sequences rather than single steps. ACT is often cited for learning precise bimanual tasks — opening a translucent condiment cup, slotting a battery — on low-cost hardware with very few demonstrations. π0 shows the same family of ideas, via flow matching, serving as the action head of a large VLA; its architecture is broken down in VLA architectures compared.
Why sequences beat single steps, even without multimodality
Predicting a whole sequence helps even on tasks with a single correct answer. A policy that outputs one action per step must stay consistent with its own previous choices, but nothing in single-step training enforces that; small errors can make it hesitate or jitter between nearby actions. Predicting several steps at once forces the network to produce a coherent trajectory, and executing several of those steps before re-predicting smooths out the noise between decisions. The ACT authors make a similar argument for action chunking: committing to a chunk reduces the effective number of decisions per task and with it the opportunity for errors to compound. The trade-off is reaction time — the longer the executed segment, the longer the robot runs open loop before it can respond to something new.
Settings to decide before deploying it
Because Diffusion Policy runs several denoising iterations per action, it takes longer to infer than a regression policy of the same size. The safe order is to measure generation time on the target hardware first, then choose how long the robot executes each predicted sequence.
- Check the demonstrations for multimodality. If the task effectively has one correct way, a simpler policy may be more efficient. How to collect and multiply demonstrations is covered in Robot training data: teleoperation vs synthetic.
- Choose the observation horizon (how many past frames to look at), the prediction length
T_predand the execution lengthT_exec. - Measure one generation on the target hardware. It has to finish before the executed segment runs out, or the robot will pause between segments.
- If you need to reduce
K, measure how the success rate changes as you do.
A useful sanity check is to plot several samples drawn for the same observation. If they cluster into distinct groups — some going left, some going right — the task really is multimodal and the generative approach is earning its cost. If they all overlap, a simpler deterministic policy may serve just as well at a fraction of the compute.
The method for budgeting onboard compute is covered in Sizing VRAM and TOPS for on-device models.