A Unified Inertial Encoder for Human and Humanoid Motion

Abstract

Learning humanoid state estimation and language-grounded motion understanding from robot recordings ties training coverage to the robot's motion repertoire. Human motion datasets offer broader coverage, yet transferring this knowledge requires overcoming differences in body geometry, sensing, and physical disturbances. To enable this transfer, we propose a unified inertial encoder whose shared inertial interface uses kinematic IMU synthesis to represent human and humanoid motion at corresponding body sites. Within this common input space, joint training on human and kinematically retargeted humanoid motion combines velocity estimation and motion–text alignment with continuous-time encoding and sensor and motion randomization. The resulting model transfers zero-shot to a physical Unitree G1 without physics simulation, real-robot training trajectories, or fine-tuning, achieving the lowest velocity error among evaluated methods across five datasets, including 0.099 m/s on real G1. Its motion-description retrieval also extends to retargeted G1 motion categories observed only in human training, demonstrating how human motion knowledge can support the generalization of humanoid inertial odometry and language-grounded motion understanding.

A Shared Inertial Interface

Twelve corresponding body sites connect human and humanoid motion: pelvis, head, and bilateral upper arms, forearms, upper legs, lower legs, and feet. On G1, kinematic transport combines the pelvis IMU, measured joint positions and velocities, and link kinematics to synthesize these streams. It requires no world pose or root velocity as input.

Continuous-time Tokenization

Timestamp-aware encoding accommodates different sampling rates, irregular timing, and missing samples without resampling to a fixed grid. Each token covers a fixed 125 ms of motion rather than a fixed number of samples, so the same motion yields the same tokens whatever the sensor rate.

Inertial Odometry

One checkpoint achieves the lowest AVE and ATE_GT on all five datasets, and the lowest ATE_Pred on four; TLIO leads ATE_Pred on Nymeria. AVE is in m/s, ATE in m; ATE_GT uses ground-truth attitude and ATE_Pred uses gyro-integrated attitude. Both start from the ground-truth initial pose, with no filter updates or trajectory alignment. Lower is better; bold marks column minima. ^† trained separately per dataset, ^* sequences longer than 10 s only.

Motion Caption Retrieval

Motion-to-text recall at 1 (R@1) uses 19 random distractors, averaged over eight draws. Text-to-motion mean average precision (mAP) ranks all annotated windows. Both are percentages; higher is better. On retargeted G1 categories observed only in human training, our model reaches 53.8% R@1 and 11.3% mAP, compared with 17.3% and 0.5% for IMU2CLIP.

Collapse assigns every motion window the same mean text embedding, measuring retrieval without motion-specific information. IMU2CLIP leads R@1 on Nymeria (16.2% versus 14.9%); our model leads mAP there and both metrics on BONES-Seed human motion and both G1 splits. ^† Trained separately per dataset.

What Enables Transfer?

Joint human–humanoid training reduces real-G1 ATE_GT from 5.901 m to 3.153 m compared with humanoid-only training. The shared multi-IMU input improves both G1 trajectory metrics over direct joint inputs, and continuous-time encoding improves results across all three ablation datasets.

ATE in meters; lower is better. Bold marks column minima. More diverse training data improves Nymeria and real-G1 ATE_GT, while quarter-data training performs best on retargeted G1.

Learned Representation

The paper visualizes the trunk [CLS] features of human (blue) and humanoid (gold) motion. Corresponding categories overlap across embodiments. Switch to the language-head output to explore the aligned text space; dashed regions enclose 60% of the windows of each labeled motion category. Rest the pointer on a point for a second (or tap it) to see the 4-s motion it encodes.

Compute-Optimal Scaling

Seven model sizes (2.8–274 M parameters), 235 EMA checkpoints. A Chinchilla-style fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ gives $N_{\mathrm{opt}}\propto C^{0.52}$ and $D_{\mathrm{opt}}\propto C^{0.48}$. Here, $N$ counts parameters, $D$ counts processed tokens including repeated windows, and $C=6ND$ is training compute in FLOPs. Increasing model capacity and training exposure together uses additional compute effectively. The fit extrapolates and optimizes validation error, which need not give the best hardware transfer.