A Unified Inertial Encoder for Human and Humanoid Motions

Anonymous Authors

Paper VideoCode (Coming Soon)

Abstract

Wearable motion capture and humanoid robots both measure motion with IMUs, yet the two embodiments differ in morphology, sensor placement, and sampling rate, and kinematically generated humanoid data lack the contact dynamics and sensor imperfections of hardware. We present one inertial representation that bridges both gaps: a shared interface of twelve anatomically corresponding sites, a continuous-time encoder for arbitrary sampling rates, and a Transformer trained jointly on real human motion and IK-retargeted Unitree G1 motion with velocity, trajectory, denoising, and language objectives. Deployed on a physical G1 without dynamics simulation, real-robot training data, or fine-tuning, a single checkpoint achieves the lowest velocity error among the evaluated methods on three human datasets, retargeted G1 motion, and real-G1 recordings, and transfers language-grounded motion semantics from humans to the robot.

Video

01Motivation

0:53

02Method

1:42

03Results

0:22

Zero-shot on a Real Unitree G1

Trained on no real-robot trajectory, the model estimates the pelvis path on hardware from inertial measurements alone. Left: the recording. Right: our estimate, with the robot posed from the measured joints along the predicted trajectory (gold).

Recording · climbing

Ours

Recording · descending

Our estimated trajectory while descending

Ours

Inertial Odometry

One checkpoint, five datasets. AVE in m/s, ATE in m; ATE_GT uses ground-truth attitude, ATE_Pred the gyro-integrated attitude. ^† trained separately per dataset, ^* sequences longer than 10 s only.

Method	Human									Humanoid (G1)
	Nymeria			Natural Motion			BONES-Seed (Human)			BONES-Seed (G1)			Real-Trajectory (G1)
	AVE	ATE_GT	ATE_Pred	AVE	ATE_GT	ATE_Pred	AVE	ATE_GT	ATE_Pred	AVE	ATE_GT	ATE_Pred	AVE	ATE_GT	ATE_Pred
Pelvis IMU only
TLIO^†	0.122	8.149	17.76	0.189	40.72	81.31	0.245	0.717	0.719	0.173	0.761	0.762	0.371	17.86	17.86
AirIMU^†	12.62	–	–	109.3	–	–	0.212	9.472	505.5	0.281	11.83	306.6	6.231	731.1	–
AirIO^†	0.069	7.481	113.9	0.280	158.1	157.9	0.140	0.530	1.918	0.118	0.720	1.690	0.350	16.82	20.48
TartanIMU	0.301	37.74	35.53	0.332	155.8	156.3	0.328^*	2.970^*	2.964^*	0.219^*	2.176^*	2.176^*	0.391	18.94	18.96
Pelvis IMU + joint positions and velocities
AirIO w/ joints^†	0.060	6.912	39.64	0.080	14.11	209.2	0.205	0.907	1.652	0.123	0.620	1.565	0.190	5.976	22.25
Shared 12-site multi-IMU input
Ours	0.026	2.468	18.20	0.049	7.456	50.54	0.031	0.113	0.148	0.024	0.101	0.127	0.099	3.153	11.50

1 / 6Recordings shown in the paper

One held-out recording per dataset, equal 3D axis scales. Predicted pelvis velocities of ours (gold) and AirIO (blue) are integrated with ground-truth attitude and drawn against ground truth (gray); each take plays at its own speed so all five finish together. Use the arrows for five further sets.

IMU–Language Retrieval

Recall@1 against 19 distractors and mean average precision, in %. The G1 categories withheld from retargeted-G1 training are recovered from human supervision.

Method	Human		Humanoid (G1)
Method	Nymeria R@1 / mAP (%)	BONES-Seed R@1 / mAP (%)	G1 · seen categories R@1 / mAP (%)	G1 · unseen categories R@1 / mAP (%)
Collapse	4.4 / 0.1	19.7 / 0.2	21.9 / 0.8	9.8 / 0.1
ImageBind	8.9 / 0.1	7.1 / 0.4	11.4 / 1.6	4.6 / 0.1
IMU2CLIP^†	16.2 / 0.4	51.8 / 5.2	36.1 / 11.2	17.3 / 0.5
Ours	14.9 / 0.6	60.7 / 15.9	63.4 / 32.9	53.8 / 11.3

Learned Representation

t-SNE of human (blue) and humanoid (gold) window embeddings, either the language-head output or the trunk [CLS] feature; dashed regions enclose 60% of the windows of each labeled motion category. Rest the pointer on a point for a second (or tap it) to see the 4-s motion it encodes.

Loading embedding…

Compute scaling

Seven model sizes (2.8–274 M parameters), 235 EMA checkpoints. A Chinchilla-style fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ gives $N_{\mathrm{opt}}\propto C^{0.52}$ and $D_{\mathrm{opt}}\propto C^{0.48}$ .

(a) Measured scaling

Loading scaling data…

(b) Fitted loss surface

Loading scaling data…

(a) Dots are EMA checkpoints; hover one for its run, measured error, and the law's prediction. Solid lines are seed-averaged measurements, dotted lines the fitted law at that model size, the dashed line the fitted compute-optimal frontier. (b) The fitted law over model size and tokens with the same checkpoints; hover anywhere for the predicted error (snapping to a nearby checkpoint shows its measured error too), drag to rotate. Dashed contours hold 6ND fixed; gold marks fitted minima and their path.