# Human-In-the-Loop Data Collection Human-In-the-Loop (HIL) data collection lets you improve a trained policy by deploying it on a real robot while a human operator monitors and intervenes when needed. The intervention data (recovery movements and corrections) is recorded alongside autonomous segments, producing a richer training dataset that teaches the policy how to handle failures. --- ## Why Human-In-the-Loop? Standard behavioral cloning trains policies on successful demonstrations only. During deployment, small errors can compound and push the robot into states never seen during training (distribution shift). HIL data collection addresses this by: - Running the trained policy on the real robot - Having a human intervene when the robot is about to fail - Recording the human's recovery and correction as training data - Fine-tuning the policy on the combined dataset This produces a policy that not only knows how to perform the task, but also how to recover when things go wrong. --- ## How It Works During a HIL session, the human operator follows this loop within each episode: 1. **Watch** the policy run autonomously 2. **Pause** when failure is imminent, the robot holds its position 3. **Take control** and teleoperate the robot back to a good state (recovery), then correct the behavior 4. **Return control to the policy**, the policy resumes autonomous execution 5. Repeat steps 2–4 as many times as needed during the episode 6. **End the episode** when the task is complete, save and move on to the next rollout By default, only the human-controlled segments (recovery and correction) are recorded, and each correction becomes its own episode. With `--strategy.record_autonomous=true`, the autonomous segments are recorded too: the policy and human can alternate control multiple times, and autonomous execution, recovery and correction are captured in one continuous trajectory. In both modes, the robot continues from its current state after each handoff (no reset required just because an intervention happened). After collection, the combined dataset (original demonstrations + HIL data) is used to fine-tune the policy. This process can be repeated iteratively: deploy, collect, fine-tune, repeat. Each round targets the current policy's failure modes. ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ Policy v0 (trained on demos) │ │ ↓ │ │ HIL Collection (target current failure modes) → Fine-tune → Policy v1 │ │ ↓ │ │ HIL Collection (target new failure modes) → Fine-tune → Policy v2 │ │ ↓ │ │ ... (repeat until satisfactory performance) │ └─────────────────────────────────────────────────────────────────────────┘ ``` --- ## Hardware Requirements ### Teleoperator Requirements The `lerobot-rollout --strategy.type=dagger` mode requires **teleoperators with active motors** that can: - Enable/disable torque programmatically - Move to target positions (to mirror the robot state when pausing) When you pause, the leader arm moves to match the robot, so control passes between the policy and the human without jumps. Jumps would end up in the recorded corrections and teach the policy jerky trajectories. **Compatible teleoperators:** - `bi_openarm_mini` - Bimanual OpenArm Mini - `so100_leader` / `so101_leader` - SO100 / SO101 leader arm - `bi_so_leader` - Bimanual SO100 / SO101 leader arms > [!IMPORTANT] > The provided commands default to `bi_openarm_follower` + `bi_openarm_mini`. > SO100 / SO101 setups work the same way, e.g. `--robot.type=so101_follower` with `--teleop.type=so101_leader`. --- ## Script Use `lerobot-rollout` with `--strategy.type=dagger` for HIL data collection. Select the inference backend with `--inference.type=sync|rtc|remote`: | Mode | Flag | Models | | ------------------------ | ------------------------- | ------------------------------------------------------------------------------------------------------ | | Standard (default) | _(no flag needed)_ | ACT, Diffusion Policy | | Real-Time Chunking (RTC) | `--inference.type=rtc` | Pi0, Pi0.5, SmolVLA | | Remote | `--inference.type=remote` | Checkpoint served by `lerobot-policy-server`; see [remote deployment requirements](./remote_inference) | --- ## Step-by-Step Guide ### Step 1: Pre-train a Base Policy First, train a policy on your demonstration dataset. This example trains ACT, which runs with standard inference in step 2: ```bash lerobot-train \ --dataset.repo_id=${HF_USER}/demo-dataset \ --policy.type=act \ --policy.repo_id=${HF_USER}/act_pretrain \ --output_dir=outputs/pretrain \ --batch_size=32 \ --steps=50000 ``` ### Step 2: Collect HIL Data **Standard inference (ACT, Diffusion Policy):** ```bash lerobot-rollout --strategy.type=dagger \ --robot.type=bi_openarm_follower \ --robot.left_arm_config.port=can1 \ --robot.left_arm_config.side=left \ --robot.right_arm_config.port=can0 \ --robot.right_arm_config.side=right \ --robot.cameras='{left_wrist: {type: opencv, index_or_path: "/dev/video0", width: 1280, height: 720, fps: 30}, right_wrist: {type: opencv, index_or_path: "/dev/video4", width: 1280, height: 720, fps: 30}, base: {type: opencv, index_or_path: "/dev/video2", width: 640, height: 480, fps: 30}}' \ --teleop.type=bi_openarm_mini \ --teleop.left_arm_config.port=/dev/ttyACM0 \ --teleop.right_arm_config.port=/dev/ttyACM1 \ --policy.path=outputs/pretrain/checkpoints/last/pretrained_model \ --dataset.repo_id=${HF_USER}/rollout_hil_dataset \ --dataset.single_task="Fold the T-shirt properly" \ --dataset.fps=30 \ --strategy.num_episodes=50 \ --interpolation_multiplier=2 ``` With `--interpolation_multiplier=N`, the robot receives `N` interpolated commands per policy action (control runs at `fps × N` Hz) while policy inference and dataset recording stay at `--fps`, so the recorded dataset matches its declared FPS. **With RTC for large models (Pi0, Pi0.5, SmolVLA):** For models with high inference latency, enable RTC for smooth execution. This needs a VLA base policy such as Pi0, Pi0.5 or SmolVLA instead of the ACT policy from step 1: ```bash lerobot-rollout --strategy.type=dagger \ --inference.type=rtc \ --inference.rtc.execution_horizon=20 \ --inference.rtc.max_guidance_weight=5.0 \ --inference.rtc.prefix_attention_schedule=LINEAR \ --robot.type=bi_openarm_follower \ --robot.left_arm_config.port=can1 \ --robot.left_arm_config.side=left \ --robot.right_arm_config.port=can0 \ --robot.right_arm_config.side=right \ --robot.cameras='{left_wrist: {type: opencv, index_or_path: "/dev/video0", width: 1280, height: 720, fps: 30}, right_wrist: {type: opencv, index_or_path: "/dev/video4", width: 1280, height: 720, fps: 30}, base: {type: opencv, index_or_path: "/dev/video2", width: 640, height: 480, fps: 30}}' \ --teleop.type=bi_openarm_mini \ --teleop.left_arm_config.port=/dev/ttyACM0 \ --teleop.right_arm_config.port=/dev/ttyACM1 \ --policy.path=outputs/pretrain/checkpoints/last/pretrained_model \ --dataset.repo_id=${HF_USER}/rollout_hil_rtc_dataset \ --dataset.single_task="Fold the T-shirt properly" \ --dataset.fps=30 \ --strategy.num_episodes=50 \ --interpolation_multiplier=3 ``` **Controls:** | Key | Action | | ------- | ----------------------------------------------------------------------- | | `Space` | Pause the policy / resume the policy | | `Tab` | Start / stop a correction (only while paused); corrections are recorded | | `Enter` | Push the dataset to the Hub (corrections-only mode) | | `ESC` | Stop the session | A USB foot pedal can be used instead of the keyboard with `--strategy.input_device=pedal`. See [Policy Deployment](./inference) for all DAgger flags. **The HIL Protocol:** 1. Watch the policy run autonomously (teleop is idle/free) 2. When you see imminent failure, press `Space` to **pause** - Policy stops - Teleoperator moves to match robot position (torque enabled) - No frames recorded during pause 3. Press `Tab` to **take control** - Teleoperator torque disabled, free to move - **Recovery**: Teleoperate the robot back to a good state - **Correction**: Correct the behavior - All movements are recorded 4. Press `Tab` again to end the correction, then `Space` to **return control to the policy** - Policy resumes autonomous execution from the current state - You can intervene again at any time (repeat steps 2–4) 5. In the default mode, each correction is saved as its own episode. With `--strategy.record_autonomous=true`, everything is recorded and split into episodes automatically 6. Press `ESC` to stop the session **Foot Pedal Setup (Linux):** If using a USB foot pedal (PCsensor FootSwitch), ensure access: ```bash sudo setfacl -m u:$USER:rw /dev/input/by-id/usb-PCsensor_FootSwitch-event-kbd ``` ### Step 3: Merge the Datasets `lerobot-rollout` adds a timestamp to the dataset name, so the HIL dataset from step 2 is saved as `your-username/rollout_hil_dataset_YYYYMMDD_HHMMSS`. Use that full name below. The HIL dataset has an extra `intervention` column that marks human-controlled frames, and datasets can only be merged when they have the same features. Remove the column first, then merge the HIL data with your demonstrations: ```bash lerobot-edit-dataset \ --repo_id ${HF_USER}/rollout_hil_dataset_YYYYMMDD_HHMMSS \ --new_repo_id ${HF_USER}/hil_dataset_no_intervention \ --operation.type remove_feature \ --operation.feature_names "['intervention']" lerobot-edit-dataset \ --new_repo_id ${HF_USER}/demo_plus_hil \ --operation.type merge \ --operation.repo_ids "['${HF_USER}/demo-dataset', '${HF_USER}/hil_dataset_no_intervention']" ``` Both datasets must also use the same robot, cameras and fps. See [Using Dataset Tools](./using_dataset_tools#merge-datasets) for more merge options. ### Step 4: Fine-tune the Policy Fine-tune the policy from step 1 on the **merged** dataset: ```bash lerobot-train \ --dataset.repo_id=${HF_USER}/demo_plus_hil \ --policy.path=outputs/pretrain/checkpoints/last/pretrained_model \ --policy.repo_id=${HF_USER}/act_hil_finetune \ --output_dir=outputs/hil_finetune \ --steps=20000 ``` Then deploy the fine-tuned policy and repeat from Step 2 to target its remaining failure modes. --- ## Tips for Effective HIL Collection ### When to Intervene Intervene when you see: - Robot about to make an irreversible mistake - Robot hesitating or showing uncertain behavior - Robot deviating from the expected trajectory ### Recovery: Teleoperating Back to a Good State During recovery, teleoperate the robot back to a state where: - The robot is in a familiar, in-distribution configuration - The current subtask can still be completed - The recovery trajectory itself is informative training data ### Quality of Corrections During correction: - Provide **confident, clean** trajectories - Complete the current subtask fully - Don't overcorrect or add unnecessary movements --- ## Related Work This HIL data collection approach builds on ideas from interactive imitation learning: - **DAgger** (Ross et al., 2011) introduced the core idea: instead of only training on expert demonstrations, query the expert for corrections on states the _learner_ visits. This breaks the compounding-error cycle of standard behavioral cloning by iteratively collecting on-policy data. - **HG-DAgger** (Kelly et al., 2019) made this practical for robotics: a human expert monitors the robot and only intervenes when needed, rather than labeling every state. The gating between autonomous and human control is exactly the pause → takeover → return-to-policy loop used in the scripts here. - **RaC** (Hu et al., 2025) scales this loop to long-horizon tasks by explicitly decomposing interventions into **recovery** (teleoperating back to a good state) and **correction** (demonstrating the right behavior from there). This decomposition is the protocol followed by the DAgger strategy in `lerobot-rollout`. - **π0.6/RECAP** (Physical Intelligence, 2025) applies the same iterative collect-and-finetune loop at scale with VLA models, showing that even large pretrained policies benefit substantially from targeted human corrections on their own failure modes. π0.6 is trained using RECAP. ```bibtex @article{ross2011dagger, title={A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning}, author={Ross, Stéphane and Gordon, Geoffrey and Bagnell, Drew}, journal={Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics}, year={2011} } @article{kelly2019hgdagger, title={HG-DAgger: Interactive Imitation Learning with Human Experts}, author={Kelly, Michael and Sidrane, Chelsea and Driggs-Campbell, Katherine and Kochenderfer, Mykel J}, journal={arXiv preprint arXiv:1810.02890}, year={2019} } @article{hu2025rac, title={RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correction}, author={Hu, Zheyuan and Wu, Robyn and Enock, Naveen and Li, Jasmine and Kadakia, Riya and Erickson, Zackory and Kumar, Aviral}, journal={arXiv preprint arXiv:2509.07953}, year={2025} } @article{pi2025recap, title={π0.6: a VLA That Learns From Experience}, author={Physical Intelligence}, year={2025} } ```