Why the Sim-to-Real Gap Happens and How to Close It: Domain Randomization, Dynamics Randomization and Adaptation

A robot that walks perfectly in simulation often stumbles on a real floor. That difference is the sim-to-real gap, or reality gap. This article separates its two causes and walks through the main technique for each, with the results the original papers report, so that an engineer designing a simulation-based training setup can decide which technique to apply first.

Two causes: what the robot sees and how it moves

Simulation became a core tool of Physical AI because no one can run millions of trial-and-error episodes on real hardware. But a simulator only approximates reality, and the mismatch shows up in two ways.

Because the causes differ, the fixes must be applied separately. Randomize only the visuals and you get a policy that sees well but moves badly.

Fix 1: domain randomization of rendering

Tobin et al. (2017) showed that randomizing a simulator’s rendering during training can produce a model that works in the real world without any real images. Their object detector was trained only on simulated RGB images with deliberately unrealistic random textures, yet it located real objects to within 1.5 cm and coped with distractors and partial occlusion. The authors then used it for real robot grasping in clutter.

The key idea is not to make simulation look real, but to make it varied enough that reality looks like just one more variation. It suggests that widening variation can pay off before spending heavily on photorealistic rendering.

Fix 2: dynamics randomization of physical parameters

Peng et al. (2017) applied the same idea to dynamics. By randomizing the simulator’s dynamics during training, they obtained policies that adapt to dynamics outside the training distribution. On a task of pushing an object to a target with a robot arm, the policy trained only in simulation kept a similar level of performance on the real robot and tolerated significant calibration error.

In practice, teams usually keep reference values and ranges together in a config file like the one below. The values are format examples; real ranges should be centered on measurements from your own robot.

# Example dynamics randomization config (values are format examples)
randomization:
  link_mass_scale:      [0.8, 1.2]    # multiplier on reference mass
  joint_friction:       [0.5, 1.5]    # multiplier on reference friction
  motor_strength_scale: [0.9, 1.1]
  action_latency_ms:    [0, 40]       # command latency
  ground_friction:      [0.4, 1.2]
  observation_noise_std: 0.01

Keeping separate ranges for items such as link_mass_scale, joint_friction and action_latency_ms makes it easy to trace which range to widen or narrow when the real robot fails.

Fix 3: automatic domain randomization (ADR)

Hand-picked ranges are easy to set too narrow or too wide. OpenAI’s Rubik’s Cube work addressed this with automatic domain randomization. ADR widens the ranges whenever the policy handles the current ones, producing progressively harder randomized environments on its own. Combined with a robot platform built for learning, it let a humanoid robot hand trained only in simulation solve a real Rubik’s Cube, and the authors report signs of emergent meta-learning in memory-augmented policies.

Fix 4: adaptation modules that infer the real environment

Randomization aims for a policy that is acceptable everywhere; adaptation aims for one that quickly recognizes the current environment and adjusts. RMA (Rapid Motor Adaptation) consists of a base policy and an adaptation module, trained entirely in simulation with no reference trajectories or domain knowledge. Deployed on an A1 quadruped without fine-tuning, it adapted to rocky, slippery and deformable ground, sand, stairs and grass in fractions of a second.

Adaptation also helps with changes during operation, such as varying payloads and wear. The broader evolution of locomotion control is covered in Bipedal locomotion and whole-body control.

Choosing a simulator: where Isaac Lab and MuJoCo fit

The Isaac Lab documentation lists reinforcement learning, learning from demonstrations and motion planning as target workflows and supports domain randomization. The MuJoCo documentation names robotics and biomechanics research as primary uses. If camera perception is part of training and the visual gap matters, Isaac Lab is a common starting point; if fast iteration on dynamics is the focus, MuJoCo often is.

Diagnosing which gap you are facing

When a transferred policy fails, the first job is to find out which gap caused it, because the two call for different fixes. Three quick checks usually settle it:

• Replay recorded real camera frames through the perception stack. If objects are missed or mislocated on real images but found in simulation, the visual gap is the problem; widen texture, lighting and camera-noise randomization or add real images for fine-tuning.
• Feed ground-truth object poses to the policy on the real robot. If motion still fails with perfect perception, the dynamics gap is the problem; revisit mass, friction, motor strength and latency ranges.
• Compare commanded and measured joint trajectories. A consistent lag points to unmodeled control latency, while overshoot or oscillation points to mismatched gains or motor models.

Logging these comparisons during the first real-robot tests turns “it does not work on hardware” into a specific randomization item to adjust.

Order of work: start from measurements on the real robot

1. Measure — or take from datasheets — the real robot’s masses, joint friction, motor latency and camera noise to build reference values.
2. Fit the simulation model to those references (system identification).
3. Train with visual and dynamics randomization centered on the references; if choosing ranges is hard, consider widening them progressively, as ADR does.
4. Run short tests on the real robot, measure the success-rate gap, and adjust the relevant randomization items depending on whether failures are perception or motion failures.
5. If the operating environment changes a lot, add an adaptation module.

When simulation data alone is not enough, how to blend in real teleoperation data is covered in Synthetic data for robot training.