# EgoScale：利用多样化以自我为中心的人类数据扩展灵巧操作能力
郑瑞杰$^{1*}$、牛丹彤$^{1,4*}$、谢雨琪$^{1*}$、王静$^{1}$、徐梦达$^{1}$、蒋云帆$^{1}$、费尔南多·卡斯塔涅达$^{1}$、胡丰远$^{1}$、谭有良$^{1}$、付乐天$^{1,2}$、特雷弗·达雷尔$^{2}$、黄芙蓉$^{3}$、朱玉科$^{1\dagger}$、徐丹菲$^{1\dagger}$、范林熙$^{1\dagger}$$^{1}$, NVIDIA$^{2}$, University of California, Berkeley$^{3}$, University of Maryland
$ ^{*} $Equal Contribution $ {^\dagger} $Project Lead
## Abstract
Human behavior is among the most scalable sources of data for learning physical intelligence, yet how to effectively leverage it for dexterous manipulation remains unclear. While prior work demonstrates human-to-robot transfer in constrained settings, it is unclear whether large-scale human data can support fine-grained, high-degree-of-freedom dexterous manipulation. We present EgoScale, a human-to-dexterous-manipulation transfer framework built on large-scale egocentric human data. We train a Vision-Language-Action (VLA) model on over 20,854 hours of action-labeled egocentric human video—more than $ 20\times $ larger than prior efforts—and uncover a log-linear scaling law between human data scale and validation loss. This validation loss strongly correlates with downstream real-robot performance, establishing large-scale human data as a predictable supervision source. Beyond scale, we introduce a simple two-stage transfer recipe: large-scale human pretraining followed by lightweight aligned human-robot mid-training. This enables strong long-horizon dexterous manipulation and one-shot task adaptation with minimal robot supervision. Our final policy improves average success rate by 54% over a no-pretraining baseline using a 22-DoF dexterous robotic hand, and transfers effectively to robots with lower-DoF hands, indicating that large-scale human motion provides a reusable, embodiment-agnostic motor prior.
### 1. Introduction
Human behavior is one of the most scalable sources of data for learning physical intelligence. Humans routinely perform dexterous manipulation across diverse objects, environments, and task variations at a scale that far exceeds what can be collected through robot teleoperation. As robotic hardware continues to improve toward more human-like kinematics and dexterity, a natural question arises: can human data serve as a primary training signal for dexterous robot manipulation?
Recent work shows that transfer from human data to robots is possible by aligning observations or actions across embodiments [12, 42, 25, 24, 30]. However, existing results remain limited in two respects. First, most approaches rely on relatively small human datasets, typically on the order of tens to hundreds of hours. Second, many focus on grippers or low DoF hands, where fine-grained finger articulation is absent. It therefore remains unclear whether human data can meaningfully support complex, dexterous manipulation at scale.
In this work, we show that human-to-robot transfer for dexterous manipulation is fundamentally a scaling phenomenon, and present EgoScale, a scalable human-to-dexterous-manipulation transfer framework built on large-scale egocentric human data. We pretrain on 20,854 hours of egocentric human manipulation data, over $ 20 \times $ larger than the dataset sizes used in prior studies of human-robot policy transfer, and uncover a clear scaling law: human wrist and hand action prediction validation loss follows a log-linear relationship with data volume. This enables us to extrapolate: as human data scales, validation loss continues to decrease, and the learned representations generalize increasingly well. Crucially, the loss strongly correlates with real robot
---
Figure 1: EgoScale: Two-stage human-to-robot learning framework. A flow-based Vision-Language-Action (VLA) policy is first pretrained on 20,854 hours of egocentric human videos using wrist motion and retargeted dexterous hand actions. A lightweight mid-training stage with aligned human robot play data (pairs highlighted with green and gray boundaries) adapts the representation to robot sensing and control. The resulting policy is post-trained on downstream tasks, enabling efficient learning of dexterous manipulation and one-shot generalization to unseen skills.
performance on long-horizon, complex manipulation tasks. Together, these results establish large-scale human data as a scalable and predictable supervision source for learning dexterous manipulation policies.
Beyond scale, we identify a simple yet effective training recipe that enables new generalization capabilities. We supervise the model using human manipulation behaviors represented as relative wrist motion and retargeted high-DoF hand joint actions. This aligned action space encourages the model to extract information that is directly useful for manipulation, rather than learning task-agnostic visual features. After pretraining, we introduce a small amount of aligned human-robot mid-training data through co-training. The mid-training data includes humans and robots performing similar manipulation tasks in matched tabletop scenes with comparable visual viewpoints. This alignment provides supervisions for grounding the pretrained representations in the robot's sensing and control space.
Importantly, this mid-training stage gives rise to emergent one-shot and few-shot generalization. With only one or a few robot demonstrations, the policy can adapt to new dexterous tasks without requiring extensive task-specific data collection. For instance, using a single robot demonstration, the trained policy achieves up to 88% average success on shirt folding, even though the mid-training data contains only folding behaviors. Moreover, although human actions are supervised in a high-DoF dexterous hand space, the learned representations generalize to substantially different robot embodiments. On the Unitree G1 robot with a tri-finger hand, human-pretrained policies also achieve over 30% absolute improvement in success rate across both evaluated tasks compared to baseline without human pretraining.
Taken together, our results show that effective dexterous human-to-robot transfer requires scale, explicit motion supervision, and a small amount of precise human–robot alignment. Rather than replacing robot data, large-scale human demonstrations dramatically amplify its effectiveness, pointing toward a future where humans can be treated as another scalable embodiment in robot learning. We summarize the contributions as follows:
---
- Human Data Pretraining Scaling Laws. At the scale of over 20k hours of human data, we uncover a clear log-linear relationship between data scale and hand-action prediction loss and show that this loss strongly correlates with real-robot dexterous manipulation performance.
• An Effective Human-to-Robot Transfer Recipe. We combine high-DoF human hand action supervision with a small amount of aligned human-robot mid-training, enabling strong post-training performance with minimal robot data.
- Emergent One-shot Transfer and Generalization. Our approach enables one-shot transfer to previously unseen dexterous task with 22-DoF hands. The pretraining policy also generalizes effectively to robots with lower-DoF hands, indicating that rich human motion provides a reusable motor prior.
### 2. Method
We aim to learn representations from large-scale egocentric human video that are directly useful for dexterous robot control. This setting poses two core challenges. First, human demonstrations are noisy and lack paired robot actions. Second, human and robot embodiments differ substantially in kinematics and control interfaces. Our method (Figure 1) addresses these challenges through two design choices. We first pretrain on human data using explicit supervision of wrist motion and hand articulation extracted from egocentric videos, forcing the model to learn physically grounded action representations. We then introduce a small amount of aligned human-robot data for mid-training, which grounds these representations in executable robot control without requiring large-scale paired demonstrations. Together, this two-stage design decouples data scale from embodiment alignment, enabling effective transfer from large human datasets to dexterous robot manipulation.
#### 2.1. Human Action Representation
Raw Sensor Streams. Each human demonstration consists of egocentric RGB observations captured from a head-mounted camera, together with estimated camera motion and human hand pose obtained from off-the-shelf perception pipelines. We convert these raw sensory signals into a unified action representation suitable for large-scale pretraining and downstream robot execution. Let $ \mathcal{F}_w $ denote the world frame and $ \mathcal{F}_c^t $ the camera frame at time $ t $. The estimated camera pose is represented as $ \mathbf{T}_{w \leftarrow c}^t \in \mathbb{SE}(3) $. Human hand pose is modeled by 21 keypoints, each represented as a rigid transform $ \mathbf{H}_{c,i}^t \in \mathbb{SE}(3) $ in the camera frame, where $ i = 1 $ corresponds to the wrist. The wrist pose in the world frame is given by $ \mathbf{W}_w^t = \mathbf{T}_{w \leftarrow c}^t \mathbf{H}_{c,1}^t $.
Wrist-level Arm Motion. To obtain motion commands that are invariant to global camera movement, we represent arm motion using relative wrist motion between consecutive timesteps. Given timestep $ t $ in an action chunk, $ \Delta \mathbf{W}^t = (\mathbf{W}_w^0)^{-1} \mathbf{W}_w^t $. This relative end-effector formulation removes dependence on absolute camera pose and captures local arm motion in a physically meaningful manner. The same representation is shared across human demonstrations and robot executions, serving as the primary arm-level action abstraction for cross-embodiment learning.
Hand Articulation. For finger-level control, we retarget the 21 human hand keypoints into a dexterous robot hand joint space using an optimization-based procedure that enforces joint limits and kinematic constraints. Our default choice is the 22-DoF hand action space of the Sharpa hand [29], which preserves human finger articulation during pretraining while aligning with the control interface of our target robot. Although this representation is defined using a high-DoF hand, we later show that the learned models transfer effectively to robots with lower-DoF hands.
#### 2.2. Human Data Sources and Processing
Stage I: Large-Scale Egocentric Human Pretraining Data. We pretrain our model on a large-scale mixture of egocentric human activity datasets totaling 20,854 hours of video. The majority consists of in-the-wild egocentric recordings spanning diverse real-world environments (e.g., household, industrial, retail, and educational settings), covering 9,869 scenes, 6,015 tasks, and 43,237 objects, and providing broad coverage of long-tailed
---
manipulation behaviors.
(a) Egocentric Human Data Collection.
(b) EgoScale Model Architecture.
Figure 2: Human Data Collection and Model Architecture. (Left) Aligned human-robot mid-training data are collected using the same sensing setup as the robot. Vive trackers and Manus gloves capture arm and hand motion, while one head-mounted camera and two wrist-mounted cameras record egocentric and wrist views, enabling consistent perception—action alignment. (Right) A flow-based VLA policy with a VLM backbone and DiT action expert. Human and robot data are unified through a wrist-level action representation, with lightweight embodiment-specific adapters for proprioception and hand actions.
All recordings are captured using egocentric RGB cameras at 30 FPS. We apply off-the-shelf SLAM and hand-pose estimation pipelines to recover camera motion and human hand trajectories. Although these estimates are noisy due to unconstrained data collection, the scale and diversity of the data provide effective supervision for learning transferable action representations, which continue to improve downstream performance as data volume increases.
To complement this large-scale but noisy supervision, we additionally incorporate 829 hours of EgoDex dataset [8], collected using Apple Vision Pro with accurate wrist and hand tracking. EgoDex covers 194 tabletop manipulation tasks involving everyday objects and provides higher-precision kinematic signals that help anchor pretraining while preserving scalability.
Stage II: Aligned Human-Robot Mid-Training Data. To further bridge the embodiment gap between human demonstrations and robot execution, we introduce a smaller dataset with both human and teleoperated robot data. We later show that this dataset is critical to anchor the pretrained representations to the robot's sensing and action spaces.
This dataset comprises 344 tabletop manipulation tasks, with each task captured in approximately 30 human trajectories and 5 robot trajectories, totaling about 50 hours of human data and only 4 hours of robot data. As shown in Figure 2a, human demonstrations are collected using the same camera configuration as the robot, with matched viewpoints and calibrated intrinsic, ensuring that visual observations are directly comparable across domains. Human hand motion is captured using the same motion-capture stack as in robot teleoperation: Vive trackers provide wrist pose (3D position and orientation), while Manus gloves record full in-hand pose as 25 joint transforms. All motion signals are synchronized with the video stream.
Compared to the large-scale but unconstrained data used in Stage I, this dataset is significantly smaller but explicitly embodiment-aligned. It focuses on tabletop tasks that match the robot's workspace and kinematics, enabling abstract human actions learned during pretraining to be grounded in executable robot control. Together, Stage I and Stage II decouple scale and alignment: Stage I provides diversity and semantic grounding, while Stage II supplies precise human-robot correspondence for downstream deployment.
---
#### 2.3. Model Architecture
As shown in Figure 2b, our model follows a flow-based VLA architecture similar to GR00T N1 [19]. At each timestep $ t $, the model conditions on an observation $ o_t = (I_t, l_t) $ consisting of an image and a language instruction, which is encoded into a vision-language embedding $ \phi_t $. The model then predicts a chunk of future actions using a flow-matching objective.
For robot data, the model conditions on the robot proprioceptive state $ q_t $, while human demonstrations do not provide such signals. In the absence of proprioception, we replace $ q_t $ with a learnable placeholder token, enabling a unified model formulation without architectural changes. To accommodate multiple robot embodiments with different state and hand action spaces, following GR00T N1 [19], we use lightweight embodiment-conditioned MLP adapters at the input and output interfaces. Specifically, these adapters encode embodiment-specific proprioceptive state and decode hand actions, while relative wrist motion prediction, the vision-language backbone, and the DiT action expert are fully shared. In practice, this mechanism is used only for a small number of additional embodiments (e.g., G1 with a tri-finger hand).
#### 2.4. Training Recipe
We use a three-stage training pipeline. In Stage I (human pretraining), we train on 20K hours of egocentric human data for 100K steps with 256 GB200 GPUs using a global batch size of 8,192 and learning rate $ 5 \times 10^{-5} $, fully unfreezing every parameter of the VLA model to absorb large-scale data. Then in Stage II (aligned mid-training), we train on the aligned human-robot play dataset for 50K steps with batch size 2,048 and learning rate $ 3 \times 10^{-5} $, freezing the vision-language backbone while only updating the vision encoder and DiT action expert to anchor representations to robot sensing and control. In Stage III (post-training), we fine-tune on task-specific robot demonstrations for 10K steps with batch size 512 and learning rate $ 3 \times 10^{-5} $. During post-training, the vision encoder is frozen if mid-training is used and unfrozen otherwise, to accommodate new embodiments when needed.
#### 2.5. Robot Systems and Control
The real world experiments are conducted on the Galaxea R1Pro humanoid robot with 22-DoF Sharpa dexterous robot hands. We refer the readers to Appendix B for the system figure.
Dual-arm Wheeled Humanoid System Galaxea R1Pro. We fix the base and torso and focus on bimanual manipulation, controlling both 7-DoF arms in relative end-effector space where actions specify incremental position and orientation changes, matching the wrist-pose representation used in human demonstrations for direct human-robot alignment.
22-DoF Dexterous Hands. We equip the robot with Sharpa Wave hands with 22 degrees of freedom and joint-space control, where actions directly specify target joint angles, enabling precise articulation and preserving the fine-grained structure of retargeted human hand motion.
Perception System. We use three RGB cameras: a head-mounted camera that provides an egocentric first-person view consistent with human videos, and two wrist ones mounted on the inner side of each wrist facing the palm, capturing close-range hand-object interactions and provide detailed visual feedback essential for fine-grained dexterous manipulation.
### 3. Experiment
In this section, we aim to answer the following research questions through our experiments:
RQ1: Does large-scale egocentric human pretraining improve downstream dexterous manipulation performance compared to training from scratch or embodiment-aligned data alone?
RQ2: How does the scale of human pretraining data affect representation quality and real-robot performance?
RQ3: What role does mid-training play in enabling few-shot adaptation and generalization to novel tasks?
---
RQ4: Do human-pretrained representations transfer across robot embodiments with substantially different kinematics and control interfaces?
RQ5: How does the choice of human action representation during pretraining affect downstream dexterous manipulation?
#### 3.1. Experiment Setup
Figure 3: Post-Training Evaluation Tasks. Five dexterous manipulation tasks used to evaluate post-training performance
Tasks. To evaluate policy performance, we design five highly dexterous manipulation tasks shown in Figure 3.
Each task is provided with 100 teleoperated robot demonstrations, except for Shirt Rolling, a deformable manipulation task that requires less precise control, for which we use only 20 demonstrations.
(Task I) (Shirt) Shirt Rolling. The robot coordinates both hands to alternately fold and roll a T-shirt into a cylindrical shape before placing it into a basket.
(Task II) (Card) Card Sorting. The robot uses its fingers to rub and separate a single card from a tightly stacked deck, and then precisely inserts it into the correct holder based on color.
(Task III) (Tong) Dexterous Tool Use: Tongs for Fruit Transfer. The robot first grasps a pair of tongs from a toolbox and then uses them to pick up a fruit and place it at a target location.
(Task IV) (Bottle) Unscrewing a Bottle Cap. The robot grasps and continuously rotates a small cap to remove it from a bottle. We collect demonstrations on four bottles of different sizes, with 25 trajectories per bottle.
---
Figure 4: Main Experimental Results. Comparison of Human Pre-train + Mid-Training, Human Pretraining, and No Pretraining across five dexterous manipulation tasks under two evaluation metrics.
(Task V) (Syringe) Syringe Liquid Transfer. This is the most challenging task, requiring the robot to pick up a syringe, draw liquid from tube A, inject it into tube B, and discard the syringe into a trash can. The task involves long-horizon, multi-step reasoning, precise spatial alignment for fluid extraction and injection, and dexterous manipulation of the syringe plunger.
Evaluation Metric. To evaluate policy performance, we train each method using two random training seeds. Then, for each trained policy checkpoint, we evaluate performance over 10 trials, except for Task III, where we conduct 4 trials per bottle across four bottle instances, resulting in 16 evaluation trials. To ensure consistency across evaluation runs, we employ an image-overlay-based initialization procedure, in which the robot evaluator is provided with a visual overlay of the target initial scene configuration to reduce variability in initial conditions. For each task, we record both the absolute task success rate and fine-grained task completion score.
#### 3.2. Large-Scale Human Pretraining Is Key to Strong Dexterous Manipulation Policy Performance
To evaluate the impact of large-scale human pretraining and aligned mid-training on policy learning efficiency, we compare four checkpoints: (1) a model trained from scratch, (2) a model pre-trained only on the midtrained aligned human-robot play dataset, (3) a model pretrained on large-scale human data, and (4) a human pretrained model further mid-trained on aligned human-robot data. For each checkpoint, we report both the task completion score and the absolute success rate.
Results are summarized in Figure 4. Across all tasks, human pretraining consistently yields substantial performance gains over training from scratch, improving average task completion by over 55%. Notably, large-scale human pretraining, despite being noisy, unconstrained, and not task- or sensor-aligned, already outperforms the mid-training-only baseline across most tasks. This provides the evidence that scale and diversity of human demonstrations provide strong inductive biases for dexterous manipulation, even in the absence of precise embodiment alignment.
Finally, combining human pretraining with a small amount of aligned mid-training yields the best overall performance, suggesting a complementary effect: large-scale human data supplies general manipulation structure, while mid-training anchors these representations to executable robot control.
#### 3.3. Policy Performance Scales with Pretraining Data Size
We study how the scale of egocentric human pretraining data affects downstream real-robot manipulation performance, and analyze how this behavior is reflected in offline human-action prediction metrics. We pretrain models using 1k, 2k, 4k, 10k, and 20k hours of human data. To isolate the effect of stage II midtraining, we directly post-train each checkpoint on the downstream task and evaluate their performance.
---
Figure 5: Scaling behavior of human pretraining. Left: Human validation loss versus training steps for models pretrained with increasing amounts of egocentric human data (1k–20k hours). Larger datasets yield stable, monotonic improvements, while smaller datasets exhibit early overfitting. Center: Optimal validation loss at convergence as a function of human data scale, revealing a near-perfect log-linear scaling law ( $ R^{2} = 0.9983 $). Right: Downstream robot performance after post-training, measured by average task completion score, improves consistently with increased human data scale. Together, these results demonstrate predictable scaling of learned action representations and their direct translation to improved dexterous manipulation performance.
As shown in Figure 5 (right), increasing the amount of human pretraining data leads to consistent and substantial gains in downstream robot performance. Average task completion rises monotonically from 0.30 at 1k hours to 0.71 at 20k hours, with no signs of saturation in the explored regime. These results indicate that large-scale human data provides increasingly strong priors for dexterous manipulation, even though the human demonstrations are noisy, unconstrained, and not task-aligned.
To better understand this trend, we examine how pretraining data scale affects the quality of learned action representations. We evaluate each pretrained model on a held-out human-video validation set consisting of 2,000 egocentric episodes. For evaluation, we randomly sample 20 timesteps per trajectory; at each timestep, we draw 16 samples from the flow-matching policy, average the predicted action chunks, and compute the mean squared error against ground-truth wrist and hand actions.
Figure 5 (left) shows human validation loss as a function of training steps for different data scales. Models trained with smaller datasets (1k–2k hours) initially reduce validation loss but later plateau or degrade, indicating overfitting to limited behavioral diversity. In contrast, models trained with larger datasets (10k–20k hours) exhibit stable, monotonic improvement throughout training without signs of overfitting.
Strikingly, when we plot the optimal validation loss achieved at convergence against data scale (Figure 5, center), we observe a remarkably clean log-linear scaling law:
$$ \mathbf{L}=0.024-0.003\cdot\ln(\mathbf{D}), $$
其中D表示人类预训练数据的时长（小时数）。拟合曲线的$R^2$达到0.9983，表明在对数空间中二者存在近乎完美的线性关系。关键在于，这种离线缩放行为能很好地预测真实机器人的性能。人类验证损失与不同数据规模下的下游任务完成度紧密相关，因此它不仅是纯粹的离线指标，更是实体控制能力的有效表征。
综合来看，这些结果表明：实现灵巧操作领域的人类到机器人迁移本质上是一种缩放现象。在已探索的范围内，增加人类数据会带来可预测的验证损失降低以及相应的机器人性能提升，且并未出现收益递减的迹象。
---
虽然我们并未对测量范围之外的情况进行推断，但这一趋势表明：随着人类数据规模和模型容量的持续扩大，仍有巨大的性能提升空间。
## 3.4. 对齐的人机中期训练可实现一次性迁移
Nearest-Neighbor Motion in Mid-training
One-Shot Robot Generalization with Human Data
一次性中期训练泛化能力
Figure 6: Aligned mid-training enables emergent one-shot transfer. During post-training, the policy is trained on only a single robot demonstration per task, together with aligned human demonstrations (100 trajectories per object).
我们评估了对齐的人机中期训练是否能在极有限的机器人监督下，帮助模型学习此前未见过的技能。我们从经过人类预训练的模型出发，先在对齐的游戏数据集上进行中期训练，随后仅用一次机器人演示并结合对齐的人类演示，对新任务进行后续训练。我们选取了叠衣服和拧开矿泉水瓶这两个任务，它们均未出现在中期训练数据中。
对于叠衣服任务，我们提供一次机器人演示和100次对齐的人类演示；对于拧开矿泉水瓶任务，我们选用三种几何形状各异的瓶子来评估模型的泛化能力，每种瓶子对应一次机器人演示和100次对齐的人类演示。
如图6所示，缺少大规模人类预训练或对齐中期训练的模型在这种一次性设置下均无法完成任务。相比之下，“预训练+中期训练”模型在叠衣服任务上的成功率达到0.88，在拧开矿泉水瓶任务上的成功率为0.55，展现出强大的小样本泛化能力。失败情况通常是局部的：比如衣服折叠不完整，或拧开瓶盖后无法稳定抓握瓶子。这些结果表明：对齐的中期训练能够催生一种仅靠人类预训练或特定于实体的数据无法实现的迁移形式。
这种泛化能力源于中期训练与评估任务之间共享的运动结构。尽管物体和任务实例差异显著，但中期训练数据让模型接触到了常见的运动基元，因此仅需一次目标机器人的演示，就能实现这类行为的迁移。
### 3.5. 人类预训练可实现跨实体迁移
除了使用高自由度灵巧手进行双手桌面操作外，我们还证明了人类预训练能够学习可迁移的动作表征，这些表征能泛化到新的机器人本体上。在人类预训练中，动作被表示为末端执行器的相对$\mathbb{SE}(3)$运动，同时结合由Sharpa手定义的22自由度灵巧手关节空间。尽管这一动作空间是基于特定的机械手实现的，但它也可作为电机先验，用于编码可复用的抓取与操作基元，比如手部张开、闭合以及手指的协调运动，这些基元可迁移到其他机械手上。为了评估跨本体泛化能力，我们引入了一个差异显著的机器人平台：Unitree G1。该平台拥有更短的机械臂，可达工作空间更小，且配备7自由度的三指手，其运动学特性与之截然不同。
我们针对两项任务评估训练后策略的性能。在第一项任务“笔入盒”中，G1机器人用左臂打开垃圾桶，随后用右手捡起笔并将其放入盒中。在
---
Figure 7: Mid-training enables cross-embodiment generalization to G1 Tasks. Integrating a small amount of G1 play data during mid-training enables strong transfer to a low-DoF tri-finger hand, outperforming mid-training or pre-training only.
第二项任务“盘子入架”中，机器人需将桌上的三个盘子从右侧传递到左侧并放入架子。执行过程中，策略会预测上半身的目标指令，而下半身的平衡与移动则由单独训练的Homie [^2] 策略负责，该策略会输出下半身关节指令。与以往实验一致，我们使用两种随机种子训练每种方法。针对每种种子，我们对生成的策略进行10次试验评估，最终报告两种种子下的平均策略得分与成功率。
如图7所示，与仅使用同一G1本体专属数据训练的模型相比，在训练过程中加入包含G1本体操作数据的数据混合集后，G1机器人在这两项操作任务上的性能均有显著提升。关键在于，这种提升不能归因于对G1轨迹的更多接触：未经人类预训练、直接在同一G1数据集上微调的预训练策略，无法达到相近的成功率。从经验上看，我们还发现基于人类数据预训练的策略表现出更平滑的行为特征。
这些结果表明，EgoScale预训练学习到了可跨本体迁移的可复用操作结构，而中期训练则使该结构适配G1的传感与控制接口。
### 3.6. 用于人类预训练的手部动作空间设计
Figure 8: Task completion score across tasks for different human pretraining action representations.
---
我们研究了预训练阶段选择何种人类手部动作表征会对后续的灵巧操作产生影响。默认设置下，我们使用22自由度的灵巧关节动作空间来表示人类手部运动，同时我们也对比了两种替代方案：(i)仅包含腕部信息的表征，该方案去除了所有手指层面的监督；(ii)基于指尖的表征 [^42] ，它先预测腕部和指尖的$\mathbb{SE}(3)$ 轨迹，再通过多层感知机映射到机器人关节指令。
如图8所示，动作表征对任务性能影响显著。仅包含腕部信息的表征在所有任务中的表现均较差，尤其是在需要精确手指 articulation 与接触时序的任务中（如“夹子”“卡片”任务）。在这类任务中，策略常常因抓取工具力度不足或高度不稳定，或过早闭合手部，导致接触失败或接触不牢固。
基于指尖的表示方法能提供更丰富的几何监督信息，并能提升某些任务上的性能，但效果仍不稳定。指尖姿态中的微小误差往往会导致映射后的关节配置不合理，从而造成抓取不稳定，或在Cards和Bottle等对接触敏感的任务中发生接触丢失。相比之下，使用重定向后的关节空间手部动作进行预训练，能在所有任务上取得最稳定的性能。因此，我们将重定向后的关节空间手部动作作为大规模人类数据预训练的一种实用且有效的选择。
#### 4. 相关工作
基于人类数据的机器人学习。人类演示已被广泛用于扩展机器人学习范围，早期研究主要利用人类视频来进行表征学习或意图推断 [^17] [^36] [^15] [^14] [^45]。后续方法则利用人类数据来指导规划或高层控制，同时依赖机器人演示来执行底层操作 [^34] [^43] [^44] [^18] [^38] [^39]。更近期的方法借助中心视觉感知和3D手部追踪技术的进步，将人类视频视为密集的动作监督信号。EgoMimic [^12]、Qiu等人 [^25] 以及DexWild [^30] 通过显式对齐，在人类和机器人演示上共同训练统一的模仿策略；而EgoVLA [^42] 则先对人类手部运动进行预训练，再通过逆运动学和重定向技术将其迁移到机器人上。同期的研究 [^13] 表明，在大规模多样化的跨本体数据上预训练的VLA模型能够解锁从人类到机器人的迁移能力。
与这些工作不同，我们的研究侧重于仅需极少机器人监督的大规模人类数据预训练，目标是直接从多样化的中心视觉人类视频中学习手腕运动和灵巧手部动作。与那些主要为夹爪式平台迁移手腕运动的方法相比，我们的设置保留了对于灵巧操控至关重要的丰富手部运动信息。与近期同样利用手部级监督的方法相比，我们基于规模大得多的人类数据进行预训练，并系统地证明了扩大人类视频数据的规模能够在人类验证指标和下游机器人性能两方面均带来稳定的提升，从而在先进的多自由度机械手上实现出色的灵巧操控性能。
机器人学习中的规模特性。受语言和视觉领域观察到的规模定律启发，近期研究开始探讨类似原理是否适用于机器人学习。实证结果表明，大规模的机器人数据集和基础模型式的策略表明，增加数据的多样性和覆盖范围能够提升跨任务和环境的鲁棒性与泛化能力 [^11] [^46] [^20] [^33] [^31] [^4]。Hu等人 [^9] 指出，策略的泛化能力遵循与环境及物体多样性相关的近似幂律关系，而额外的演示数据很快便会达到饱和，这凸显出多样性比原始数据量更为重要。这一结论与以往强调通过组合多样性来实现高效数据收集的研究结果一致 [^7] [^37]。与以往主要依赖机器人采集数据来扩大规模的研究不同，我们证明扩大多样化的自然场景下的中心视觉人类数据规模能够系统性地提升灵巧操控能力，从而证实人类视频是一种高效且可扩展的监督来源。
学习灵巧操作。灵巧操作的研究已经从基于分析和控制的抓取方法，发展为对力闭合、接触稳定性和手部运动学进行建模的方法 [^21] [^22] [^26] [^27] [^23] [^6]。
---
如今则转向基于学习的方法，这类方法能从数据中获取丰富的接触行为 [^1] [^16] [^40]。后续研究引入了诸如抓取可供性、接触图以及手-物体交互场等结构化表示方式，以便更好地捕捉灵巧操作的几何特征与物理特性 [^3] [^5] [^10] [^41] [^32]。更近期的方法旨在学习具备通用性的多指操作策略，实现感知与控制的一体化 [^28] [^35]。然而，由于动作空间维度极高、机器人数据收集成本高昂，且当前灵巧手硬件的能力有限，灵巧操作的规模化应用仍面临挑战。我们的研究利用带有密集手部追踪的以自我为中心的视频作为替代监督源，结果表明人类手部运动能为提升机器人灵巧操作能力提供可迁移的信号。
#### 5. 致谢
若无以下机器人操作员的奉献与专业技能，本研究便无法完成：Ivy Tam、Ashley Kim、Aly Khater、Rhea Alve、Noah Huang、Ty Seligman、Mahnoor Kareem、Christian Soto、Chaitanya Kothapalli、Kendra Shu、Rex Asato、Eley Barba Preciados和April Zitkovich。他们在大规模数据收集与评估工作中提供了宝贵支持。我们还要感谢Johan Bjorck、Scott Reed、Runyu Ding和Joel Jang在项目期间给予的深刻讨论与反馈。
#### 6. 结论
在本研究中，我们证明了灵巧操作领域有效的人机迁移本质上是一种规模化现象。通过在超过2万小时以自我为中心的人类操作数据上预训练视觉-语言-动作策略，EgoScale揭示了人类动作预测损失与数据规模之间明显的对数线性缩放规律，并证实这种损失能强烈预示下游机器人实际表现。除了数据规模外，我们还找到了一种简单而有效的迁移方案：将大规模人类数据预训练与少量对齐的人机中期训练相结合，就能实现强大的长 horizon 灵巧操作能力、突发式一次性适应能力，以及在不同运动学与手部设计差异显著的机器人实体间的稳健迁移。
展望未来，仍有多个研究方向有待探索。尽管在已探索的范围内我们并未观察到饱和现象，但同步扩大人类数据与模型容量或许能带来更多突破，包括改进长 horizon 规划能力与组合泛化能力。随着以自我为中心的人类数据持续增长，通过自监督目标整合标注较粗或无标注的视频，可能会进一步放大这些优势。最后，随着机器人硬件在运动学与灵巧性方面越来越接近人类，实体差距自然会缩小，从而实现更强的迁移能力，甚至能在新任务上实现零样本执行。总体而言，这些结果预示着这样一个未来：人类可被视作学习通用具身智能的真正可扩展的载体。
---
#### A. 机器人系统详情
Figure 9: Visualization of the robot system setups on the Galaxea R1 Pro and Unitree G1 platforms. The Galaxea R1 Pro is equipped with 22-DoF Sharpa dexterous hands, while the Unitree G1 is mounted with 7-DoF tri-finger hands. Both robots have with two OAK-1-Wide cameras for the wrist view and an OAK-D-Wide head-mounted camera to provide egocentric visual observations.
#### B. 任务描述与评估标准
在本节中，我们将详细介绍第3节中介绍的训练后评估任务，以及相应的评估标准。每个任务都由自然语言指令指定，并通过[^0]和[^1]中的任务完成得分来评估，该得分旨在除了衡量二元结果外，还能反映任务的部分进展。
根据任务的结构，我们采用两种评分策略中的一种。对于那些可以自然分解为一系列独立且定义明确的子技能的任务（例如工具抓取、放置或离散操作步骤），我们采用累加式评分标准，最终得分为各子项得分之和。而对于涉及可变形物体或紧密耦合的操作阶段、难以单独定义中间状态的任务，我们则采用基于进展的评分标准，得分反映的是所达到的最远任务里程碑。这两种设计旨在提供细致且易于理解的反馈，同时忠实于任务本身的结构。除非另有说明，否则我们对每个检查点均进行10次试验评估。
### R1 Pro Sharpa
- 任务一：衬衫卷曲。文本指令：“将T恤卷起来并放入篮子中。”机器人必须协调双手，逐步折叠、卷曲并将可变形的T恤放入篮子中。
评分标准（基于进展）：
- 0.0：无折叠行为。
- 0.3：执行初步折叠。
- 0.5：执行多次折叠或部分卷曲。
- 0.8：完成连续的卷曲动作，形成紧凑的形状。
---
- 1.0：将卷好的衬衫放入篮子中。
评估试验次数：10次。
- 任务二：卡片分类。文本指令：“捡起卡片并将其分类放入正确的卡片盒中。”机器人必须将一张卡片从紧密堆叠的牌堆中分离出来，并插入正确的盒子中。评分标准（基于进展）：
- 0.0：未能成功捡起卡片；整副牌被扰乱或抬起。
- 0.3：抓取不牢固；无意中捡起了多张卡片。
- 0.5：成功捡起单张卡片。
- 0.7：放置卡片时造成明显扰动或插入位置错误。
- 0.9：将正确的卡片正确放置，仅有轻微扰动。
- 1.0：将单张正确的卡片整齐地放入正确的盒子中。
- 任务三：夹子水果转移。文本指令：“使用夹子捡起水果并放入篮子中。”机器人首先需抓住夹子，然后利用该工具捡起目标物体并放入容器中。此任务针对柠檬和李子两种随机水果进行评估，每种水果各进行5次试验。
评分标准（累加式）：
- +0.4：成功抓住夹子。
– +0.2：使用夹子捡起水果。
– +0.2：将水果放入篮子中。
- +0.2：将夹子放回桌上。
- 任务四：拧开瓶盖。文字说明：“将瓶盖从瓶子上拧开。”机器人必须抓住瓶子，多次旋转瓶盖，将其取下并放在桌上。为了测试对不同物体变体的泛化能力，我们评估了4种高度和瓶盖尺寸各不相同的瓶子配置。总共进行12次试验，每种瓶子分别进行4次试验。
评分标准（累加制）：
- +0.1：抓住瓶子。
– +0.5：至少连续旋转三次以拧开瓶盖。
- +0.2：完全取下瓶盖。
- +0.2：将瓶盖放在桌上。
- 任务五：注射器液体转移。文字说明：“拿起注射器，从A管中吸取液体，将其注入B管，然后把注射器扔进垃圾桶。”该任务需要精确的工具操作、空间对齐以及长时序的动作规划。
评分标准（累加制）：
- +0.1：拿起注射器。
- +0.1：准确对准A管。
- +0.2：拉动活塞以吸取液体。
- +0.1：重新对准B管。
– +0.2：推动活塞以注入液体。
– +0.2：移交或释放注射器。
- +0.1：将注射器丢弃进垃圾桶。
为了评估一次性适应能力，我们在R1 Pro Sharpa平台上增加了两项操作任务。对于每项任务，均利用一次机器人演示对策略进行后续训练，并通过[^0] [^1]中的任务特定完成分数来评估性能。
• 任务八：一次性T恤折叠。文字说明：“折叠T恤。”
评分标准（累加制）：
– +0.4：成功折叠至少一个袖子。
- +0.4：成功折叠两个袖子。
– +0.3：执行底部折叠。
评估过程中，如果仅完成部分折叠但折叠效果杂乱，我们会扣除0.1分。
---
- 任务九：一次性拧开瓶盖。文字说明：“将水瓶的瓶盖拧开。”该任务可自然分解为离散且定义明确的操作步骤，采用累加评分方式进行评估。
评分标准（累加制）：
- +0.1：抓住瓶子。
– +0.2：成功旋转一至两次以拧开瓶盖。
– +0.5：至少连续旋转三次以拧开瓶盖。
- +0.2：完全取下瓶盖。
- +0.2：将瓶盖放在桌上。
对于所有任务，我们还报告二元任务成功率，只有当任务按照说明完整执行完毕时，该次试验才被视为成功。
#### G1任务
除了R1 Pro Sharpa平台的评估外，我们还在Unitree G1平台上通过两项操作任务来评估跨本体泛化能力。这两项任务均采用累加评分方式进行评估，这反映了它们可分解为离散且定义明确的子技能。
- 笔放入盒子。文本说明：“标记物容器任务。”机器人必须打开容器，捡起笔，然后将笔放入容器中。
评分标准（累加制）：
– +0.25：捡起或打开容器。
- +0.25：将容器平稳地放下。
– +0.25：捡起笔。
– +0.25：将笔放入容器中。
- 将盘子放入碗架。文本说明：“把盘子放到碗架上。”机器人会看到桌上放着三个盘子，它必须将它们转移（可能需经多只手传递），然后竖直放入碗架中。
评分标准（累加制，每个盘子单独计算）：对于每个盘子，我们设定如下评分：
- 0.11：捡起盘子。
- 0.11：在双手之间传递盘子。
- 0.11：将盘子竖直放入碗架。
最终的任务完成得分是通过将三个盘子的得分相加得出的，每次试验的最高得分为1.0。
#### C. 真实场景下的人类数据整理细节
#### C.1. 数据集统计与分析
本节对用于第一阶段预训练的大规模第一人称视角人类活动数据进行了详细分析。图10从多个互补的角度对数据集进行了总结，包括上排显示的统计分布以及下排展示的定性第一人称示例。具体而言，左上角的图表展示了物体词汇表，中上部分的图表显示了类别分布，右上角的图表呈现了环境分布，第二排最右侧的图表则说明了任务分布。这些统计数据共同揭示了类别、环境、任务和物体方面的长尾分布特征，凸显了该数据集中所涵盖的真实世界操作行为的规模与多样性。
#### C.2. 类别分布
视频总时长的类别级分布（图中仅统计时长超过50小时的类别）显示在图10的中上部分。该数据集以零售和消费品为主
---
Retail and Consumer Goods
Food and Beverage
Figure 10: Statistical distributions and qualitative examples of the egocentric human activity dataset, showing long-tailed coverage across categories, environments, tasks, and objects.
---
（占比20.1%），其次是时尚类（11.8%）、维修服务类（11.5%）以及食品饮料类（11.5%）。其他占比较大的类别包括家庭环境类（9.5%）、建筑与五金类（7.7%）、食品加工类（6.7%）以及印刷与设计类（4.4%）；其余类别则构成了涵盖酒店业、汽车与交通运输、体育娱乐、教育、医疗保健、能源及工业领域的长尾部分。
#### C.3. 环境多样性
录制时长超过50小时的环境分布显示在图10的左下角条形图中。出现频率最高的环境包括家庭、花店、电子产品维修店、家具维修店、木工坊以及服装店。除了这些主要环境外，还有数十种其他环境——如杂货店、面包店、车间、工厂、工作室、图书馆及其他服务型场所——也贡献了虽小但不可忽视的数据量，从而形成了长尾型的环境分布。
#### C.4. 任务覆盖范围
任务级别的统计数据展示在图10的右下角条形图中。高频任务包括叠衣服、清洁鞋子、栽种植物、熨烫、组装盒子、插花、打磨木材以及食物准备。此外，种类繁多的中低频任务涵盖了打包、分类、修理、装饰、切割、装瓶、充气以及拆卸等操作。许多任务涉及多步骤流程且需要持续的物理交互，这反映了现实世界中长时间的操作行为，而非孤立的抓取与放置动作。
### C.5. 物体词汇表
物体词汇表通过图10左上角的词云图进行可视化展示。常见的物体包括纸板箱、塑料袋、碗、托盘、胶带、衣物、面团、电池和电路板，此外还有大量出现频率较低的物体。物体实例的多样性及其与各类任务的关联程度显示出强烈的组合结构，这有助于学习跨物体、跨任务和跨环境的可迁移动作表征。
### D. 手部重定向算法
我们采用基于每帧优化的方法，将21个人体手部关键点（每只手表示为包含3D位置和方向的25个关键点）重定向到Sharpa Hand的22自由度关节空间[^29]。该机械手通过基于URDF的正向运动学模型来描述，该模型可将关节角度映射为20个机器人关键点的位姿（位置和四元数）。
对于每只手和每个时间步，我们针对22个关节角度求解非线性规划问题，仅受URDF规定的关节限制约束，同时最小化不同目标的加权组合。该优化过程在CasADi中实现，并通过IPOPT求解，求解时以上一帧的结果作为热启动条件。得到的关节角度会进一步通过一阶指数滤波器进行平滑处理，以减少时间上的抖动。这种设计在确保关节限制和运动学一致性的同时，保留了手指的活动能力以及捏合/握拳等语义特征，最终形成用于预训练及与目标机器人交互的22自由度动作空间。
### D.1. 向G1机器人进行跨本体迁移的额外细节
本小节提供了关于如何将基于人类数据预训练的策略适配到Unitree G1机器人的额外实现细节，该机器人在运动学和手部驱动方面与主要的Galaxea R1 Pro平台存在显著差异。
### 共享腕部动作
在所有本体中，我们都使用$SE(3)$中的相对末端执行器位姿来表示手臂运动，该位姿由帧与帧之间的腕部变换定义。这种表示方式在人类演示和机器人执行过程中是通用的，
---
并且不受绝对工作空间、摄像头摆放位置以及手臂运动学差异的影响。
### 手部动作适配
人类预训练是在通过人体手部动作重定向得到的22自由度灵巧手关节空间中进行的。为了适配不同形态的机器人，例如配备7自由度三指手的Unitree G1，我们借鉴GR00T-N1和N1.5的设计思路，在DiT动作模块的输入和输出接口处引入了轻量级的形态条件化MLP适配器[^19]。具体而言，形态专属编码器会将机器人的本体感知状态和含噪声的动作输入映射到共享的潜在动作空间，而形态专属解码器则会将DiT的输出映射回对应的关节动作空间。视觉-语言骨干网络与DiT动作专家模块在所有形态中是完全共享的。
## 形态专属的中期训练
为了将人类预训练的表征锚定在G1的控制空间中，我们在中期训练阶段加入了G1机器人的操作数据。在此阶段，仅更新视觉编码器、DiT动作专家以及状态-动作编码器和解码器，视觉-语言骨干网络则保持冻结状态。这种设计能够让模型在保留源自人类的操控结构的同时，适配G1的传感与驱动接口。
## 讨论
重要的是，G1并非从零开始训练。中期训练的作用是将已学习到的、源自人类的操控表征与新形态对齐。如第3节所示，这种方法的性能远高于仅用G1数据直接训练的情况，这表明大规模的人类预训练能提供一个可复用且不受形态限制的运动先验，能够高效适配具有不同运动学和手部设计的机器人。
---
### 参考文献
[^1]: OpenAI: Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al. Learning dexterous in-hand manipulation. The International Journal of Robotics Research, 39(1):3–20, 2020. 12
[^2]: Qingwei Ben, Feiyu Jia, Jia Zeng, Junting Dong, Dahua Lin, and Jiangmiao Pang. Homie: Humanoid locomaniulation with isomorphic exoskeleton cockpit, 2025. URL
[^3]: Samarth Brahmbhatt, Ankur Handa, James Hays, and Dieter Fox. Contactgrasp: Functional multi-finger grasp synthesis from contact. in 2019 ieee. In RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2386–2393, 2019. 12
[^4]: Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06817, 2022. 11
[^5]: Enric Corona, Albert Pumarola, Guillem Alenya, Francesc Moreno-Noguer, and Grégory Rogez. Ganhand: Predicting human grasp affordances in multi-object scenes. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5031–5041, 2020. 12
[^6]: Hongkai Dai, Anirudha Majumdar, and Russ Tedrake. Synthesis and optimization of force closure grasps via sequential semidefinite programming. In Robotics Research: Volume 1, pages 285–305. Springer, 2017. 11
[^7]: Jensen Gao, Annie Xie, Ted Xiao, Chelsea Finn, and Dorsa Sadigh. Efficient data collection for robotic manipulation via compositional generalization. arXiv preprint arXiv:2403.05110, 2024. 11
[^8]: Ryan Hoque, Peide Huang, David J. Yoon, Mouli Sivapurapu, and Jian Zhang. Egodex: Learning dexterous manipulation from large-scale egocentric video, 2025. URL . 4
[^9]: Yingdong Hu, Fanqi Lin, Pingyue Sheng, Chuan Wen, Jiacheng You, and Yang Gao. Data scaling laws in imitation learning for robotic manipulation. arXiv preprint arXiv:2410.18647, 2024. 11
[^10]: Hanwen Jiang, Shaowei Liu, Jiashun Wang, and Xiaolong Wang. Hand-object contact consistency reasoning for human grasps generation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11107–11116, 2021. 12
[^11]: Dmitry Kalashnikov, Alex Irpan, Peter Pastor, Julian Ibarz, Alexander Herzog, Eric Jang, Deirdre Quillen, Ethan Holly, Mrinal Kalakrishnan, Vincent Vanhoucke, et al. Scalable deep reinforcement learning for vision-based robotic manipulation. In Conference on robot learning, pages 651–673. PMLR, 2018. 11
[^12]: Simar Kareer, Dhruv Patel, Ryan Punamiya, Pranay Mathur, Shuo Cheng, Chen Wang, Judy Hoffman, and Danfei Xu. Egomimic: Scaling imitation learning via egocentric video, 2024. URL , 11
[^13]: Simar Kareer, Karl Pertsch, James Darpinian, Judy Hoffman, Danfei Xu, Sergey Levine, Chelsea Finn, and Suraj Nair. Emergence of human to robot transfer in vision-language-action models, 2025. URL
#### [^14]: Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, and Pierre Sermanet. Learning latent plans from play. In Conference on robot learning, pages 1113–1132. Pmlr, 2020. 11
---
[^15]: Arjun Majumdar, Karmesh Yadav, Sergio Arnaud, Jason Ma, Claire Chen, Sneha Silwal, Aryan Jain, Vincent-Pierre Berges, Tingfan Wu, Jay Vakil, et al. Where are we in the search for an artificial visual cortex for embodied intelligence? Advances in Neural Information Processing Systems, 36:655–677, 2023. 11
[^16]: Anusha Nagabandi, Kurt Konolige, Sergey Levine, and Vikash Kumar. Deep dynamics models for learning dexterous manipulation. In Conference on robot learning, pages 1101–1112. PMLR, 2020. 12
[^17]: Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3m: A universal visual representation for robot manipulation. arXiv preprint arXiv:2203.12601, 2022. 11
[^18]: Dantong Niu, Yuvan Sharma, Giscard Biamby, Jerome Quenum, Yutong Bai, Baifeng Shi, Trevor Darrell, and Roei Herzig. Llarva: Vision-action instruction tuning enhances robot learning. arXiv preprint arXiv:2406.11815, 2024. 11
[^19]: NVIDIA, :, Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi "Jim" Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, Joel Jang, Zhenyu Jiang, Jan Kautz, Kaushil Kundalia, Lawrence Lao, Zhiqi Li, Zongyu Lin, Kevin Lin, Guilin Liu, Edith Llontop, Loic Magne, Ajay Mandlekar, Avnish Narayan, Soroush Nasiriany, Scott Reed, You Liang Tan, Guanzhi Wang, Zu Wang, Jing Wang, Qi Wang, Jiannan Xiang, Yuqi Xie, Yinzhen Xu, Zhenjia Xu, Seonghyeon Ye, Zhiding Yu, Ao Zhang, Hao Zhang, Yizhou Zhao, Ruijie Zheng, and Yuke Zhu. Gr00t n1: An open foundation model for generalist humanoid robots, 2025. URL , 18
[^20]: Open X-Embodiment Collaboration et al. Open X-Embodiment: Robotic learning datasets and RT-X models. International Conference on Robotics and Automation, 2024. 11
[^21]: Jean Ponce, Steve Sullivan, J-D Boissonnat, and J-P Merlet. On characterizing and computing three-and four-finger force-closure grasps of polyhedral objects. In [^1993] Proceedings IEEE International Conference on Robotics and Automation, pages 821–827. IEEE, 1993. 11
[^22]: Jean Ponce, Steve Sullivan, Attawith Sudsang, Jean-Daniel Boissonnat, and Jean-Pierre Merlet. On computing four-finger equilibrium and force-closure grasps of polyhedral objects. The International Journal of Robotics Research, 16(1):11–35, 1997. 11
[^23]: Domenico Prattichizzo, Monica Malvezzi, Marco Gabiccini, and Antonio Bicchi. On the manipulability ellipsoids of underactuated robotic hands with compliance. Robotics and Autonomous Systems, 60(3): 337–346, 2012. 11
[^24]: Ryan Punamiya, Dhruv Patel, Patcharapong Aphiwetsa, Pranav Kuppili, Lawrence Y Zhu, Simar Kareer, Judy Hoffman, and Danfei Xu. Egobridge: Domain adaptation for generalizable imitation from egocentric human data. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. 1
[^25]: Ri-Zhao Qiu, Shiqi Yang, Xuxin Cheng, Chaitanya Chawla, Jialong Li, Tairan He, Ge Yan, David J. Yoon, Ryan Hoque, Lars Paulsen, Ge Yang, Jian Zhang, Sha Yi, Guanya Shi, and Xiaolong Wang. Humanoid policy ~ human policy. In 9th Annual Conference on Robot Learning, 2025. URL . 1, 11
[^26]: Alberto Rodriguez, Matthew T Mason, and Steve Ferry. From caging to grasping. The International Journal of Robotics Research, 31(7):886–900, 2012. 11
#### [^27]: Carlos Rosales, Raúl Suárez, Marco Gabiccini, and Antonio Bicchi. On the synthesis of feasible and prehensile robotic grasps. In 2012 IEEE international conference on robotics and automation, pages 550–556. IEEE, 2012. 11
---
[^28]: Lin Shao, Fabio Ferreira, Mikael Jorda, Varun Nambiar, Jianlan Luo, Eugen Solowjow, Juan Aparicio Ojea, Oussama Khatib, and Jeannette Bohg. Unigrasp: Learning a unified model to grasp with multifingered robotic hands. IEEE Robotics and Automation Letters, 5(2):2286–2293, 2020. 12
[^29]: Sharpa Robotics. Sharpa wave. , 2024. Accessed: 2026-01-31. 3, 17
[^30]: Tony Tao, Mohan Kumar Srirama, Jason Jingzhou Liu, Kenneth Shaw, and Deepak Pathak. Dexwild: Dexterous human interactions for in-the-wild robot policies, 2025. URL , 11
[^31]: Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213, 2024. 11
[^32]: Dylan Turpin, Liquan Wang, Eric Heiden, Yun-Chun Chen, Miles Macklin, Stavros Tsogkas, Sven Dickinson, and Animesh Garg. Grasp'd: Differentiable contact-rich grasp synthesis for multi-fingered hands. In European Conference on Computer Vision, pages 201–221. Springer, 2022. 12
[^33]: Homer Walke, Kevin Black, Abraham Lee, Moo Jin Kim, Max Du, Chongyi Zheng, Tony Zhao, Philippe Hansen-Estruch, Quan Vuong, Andre He, Vivek Myers, Kuan Fang, Chelsea Finn, and Sergey Levine. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning (CoRL), 2023. 11
[^34]: Chen Wang, Linxi Fan, Jiankai Sun, Ruohan Zhang, Li Fei-Fei, Danfei Xu, Yuke Zhu, and Anima Anandkumar. Mimicplay: Long-horizon imitation learning by watching human play. arXiv preprint arXiv:2302.12422, 2023. 11
[^35]: Albert Wu, Michelle Guo, and C Karen Liu. Learning diverse and physically feasible dexterous grasps with generative model and bilevel optimization. arXiv preprint arXiv:2207.00195, 2022. 12
[^36]: Tete Xiao, Ilija Radosavovic, Trevor Darrell, and Jitendra Malik. Masked visual pre-training for motor control. arXiv preprint arXiv:2203.06173, 2022. 11
[^37]: Annie Xie, Lisa Lee, Ted Xiao, and Chelsea Finn. Decomposing the generalization gap in imitation learning for visual robotic manipulation. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 3153–3160. IEEE, 2024. 11
[^38]: Mengda Xu, Zhenjia Xu, Cheng Chi, Manuela Veloso, and Shuran Song. Xskill: Cross embodiment skill discovery. In Conference on robot learning, pages 3536–3555. PMLR, 2023. 11
[^39]: Mengda Xu, Zhenjia Xu, Yinghao Xu, Cheng Chi, Gordon Wetzstein, Manuela Veloso, and Shuran Song. Flow as the cross-domain manipulation interface. arXiv preprint arXiv:2407.15208, 2024. 11
[^40]: Mengda Xu, Han Zhang, Yifan Hou, Zhenjia Xu, Linxi Fan, Manuela Veloso, and Shuran Song. Dexumi: Using human hand as the universal manipulation interface for dexterous manipulation, 2025. URL . 12
[^41]: Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, and Cewu Lu. Cpf: Learning a contact potential field to model the hand-object interaction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11097–11106, 2021. 12
#### [^42]: Ruihan Yang, Qinxi Yu, Yecheng Wu, Rui Yan, Borui Li, An-Chieh Cheng, Xueyan Zou, Yunhao Fang, Xuxin Cheng, Ri-Zhao Qiu, et al. Egovia: Learning vision-language-action models from egocentric human videos. arXiv preprint arXiv:2507.12440, 2025. 1, 11
---
[^43]: Sarah Young, Dhiraj Gandhi, Shubham Tulsiani, Abhinav Gupta, Pieter Abbeel, and Lerrel Pinto. Visual imitation made easy. In Conference on Robot learning, pages 1992–2005. PMLR, 2021. 11
[^44]: Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daumé III, Andrey Kolobov, Furong Huang, and Jianwei Yang. Tracevla: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. arXiv preprint arXiv:2412.10345, 2024. 11
[^45]: Ruijie Zheng, Jing Wang, Scott Reed, Johan Bjorck, Yu Fang, Fengyuan Hu, Joel Jang, Kaushil Kundalia, Zongyu Lin, Loic Magne, Avnish Narayan, You Liang Tan, Guanzhi Wang, Qi Wang, Jiannan Xiang, Yinzhen Xu, Seonghyeon Ye, Jan Kautz, Furong Huang, Yuke Zhu, and Linxi Fan. FLARE: Robot learning with implicit world modeling. In 9th Annual Conference on Robot Learning, 2025. URL . 11
[^46]: Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023. 11