Humanoid robots and autonomous mobile manipulators cannot rely on cloud round-trips for closed-loop motor control. Deploying Vision-Language-Action (VLA) foundation models directly on onboard edge modules (such as NVIDIA Jetson AGX Orin or Jetson Thor) requires careful memory and bandwidth budgeting.
1. Anatomy of On-Device VLA Memory Consumption
Total unified memory consumption ($M_{\text{total}}$) during real-time robotic inference consists of four components:
- Model Weights ($M_{\text{weights}}$): Parameter count $\times$ bytes per parameter (
FP16= 2.0 B,INT8= 1.0 B,INT4/AWQ$\approx$ 0.55 B including group scales). - Multi-Camera Vision Tokens & KV Cache ($M_{\text{vision+kv}}$): Dual wrist cameras plus stereo head cameras generate 256–576 tokens per camera frame. Temporal history windows multiply this KV footprint rapidly.
- Action Expert / Diffusion Denoising Buffer ($M_{\text{action}}$): Activation buffers for flow-matching or diffusion action chunk generation (typically 16–50 future timesteps per step).
- System & Middleware Reserve ($M_{\text{sys}}$): ROS 2 DDS buffers, CUDA context, and camera ISP pipelines require 2.5–4.0 GB of headroom on unified-memory SoCs.
2. Mixed-Precision Quantization Best Practices
- Language & Multimodal Backbone:
W4A16(AWQ) reduces weight memory and memory bandwidth pressure by ~65% with negligible degradation in task planning. - Vision Encoder & Continuous Action Head: Keep the continuous action head (
Diffusion/Flow-MatchingMLP/DiT) in FP16/BF16 to prevent quantization jitter in sub-millimeter manipulation tasks.
3. Interactive Hardware Calculator
Use our VLA & Robotics Edge VRAM Calculator to simulate your exact camera count, model parameter scale, and target control frequency (Hz).