Policy Deployment (lerobot-rollout)

lerobot-rollout is the single CLI for deploying trained policies on real robots. It supports multiple execution strategies and inference backends, from quick evaluation to continuous recording and human-in-the-loop data collection.

Quick Start

No extra dependencies are needed beyond your robot and policy extras.

lerobot-rollout \
    --strategy.type=base \
    --policy.path=lerobot/act_koch_real \
    --robot.type=koch_follower \
    --robot.port=/dev/ttyACM0 \
    --task="pick up cube" \
    --duration=30

This runs the policy for 30 seconds with no recording.

Choosing a device

Pass --device=cpu or --device=cuda to override the checkpoint device on a host where that device is available. The resolved device is applied to the policy configuration before loading the policy and constructing its processors. For example, add --device=cpu to the command above to run a CUDA-trained checkpoint on a CPU-only host, if the policy supports CPU execution. Check that CPU inference can meet your required control rate before deploying on a robot.

When --device is omitted, rollout uses the policy configuration device if set, otherwise it selects an available device automatically. The same behavior applies to RolloutConfig(device=...) with build_rollout_context.

Strategies

Select a strategy with --strategy.type=<name>. Each strategy defines a different control loop with its own recording and interaction semantics.

Base ( --strategy.type=base )

Autonomous policy execution with no data recording. Use this for quick evaluation, demos, or when you only need to observe the robot.

lerobot-rollout \
    --strategy.type=base \
    --policy.path=${HF_USER}/my_policy \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
    --task="Put lego brick into the box" \
    --duration=60

Sentry ( --strategy.type=sentry )

Continuous autonomous recording with periodic upload to the Hugging Face Hub. Episode boundaries are auto-computed from camera resolution and FPS so each saved episode produces a complete video file, keeping uploads efficient.

Policy state (hidden state, RTC queue) persists across episode boundaries: the robot does not reset between episodes.

lerobot-rollout \
    --strategy.type=sentry \
    --strategy.upload_every_n_episodes=5 \
    --policy.path=${HF_USER}/my_policy \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
    --dataset.repo_id=${HF_USER}/rollout_eval_data \
    --dataset.single_task="Put lego brick into the box" \
    --duration=3600

Highlight ( --strategy.type=highlight )

Autonomous rollout with on-demand recording via a memory-bounded ring buffer. The buffer captures the last N seconds of telemetry. Press the save key to flush the buffer and start live recording; press it again to save the episode. With an asynchronous backend, each toggle pauses inference, invalidates queued/interpolated motion and requests a supported local hold before the blocking drain or save. A healthy run resumes from a fresh capture afterward.

lerobot-rollout \
    --strategy.type=highlight \
    --strategy.ring_buffer_seconds=30 \
    --strategy.save_key=s \
    --strategy.push_key=h \
    --policy.path=${HF_USER}/my_policy \
    --robot.type=koch_follower \
    --robot.port=/dev/ttyACM0 \
    --dataset.repo_id=${HF_USER}/rollout_highlight_data \
    --dataset.single_task="Pick up the red cube"

Keyboard controls:

DAgger ( --strategy.type=dagger )

Human-in-the-loop data collection. Alternates between autonomous policy execution and human intervention via a teleoperator. Intervention frames are tagged with intervention=True. Requires a teleoperator (--teleop.type).

See the Human-In-the-Loop Data Collection guide for a detailed walkthrough.

Corrections-only mode (default): Only human correction windows are recorded. Each correction becomes one episode.

lerobot-rollout \
    --strategy.type=dagger \
    --strategy.num_episodes=20 \
    --policy.path=outputs/pretrain/checkpoints/last/pretrained_model \
    --robot.type=bi_openarm_follower \
    --teleop.type=bi_openarm_mini \
    --dataset.repo_id=${HF_USER}/rollout_hil_data \
    --dataset.single_task="Fold the T-shirt"

Continuous recording mode (--strategy.record_autonomous=true): Both autonomous and correction frames are recorded with time-based episode rotation (same as Sentry).

lerobot-rollout \
    --strategy.type=dagger \
    --strategy.record_autonomous=true \
    --strategy.num_episodes=50 \
    --policy.path=${HF_USER}/my_policy \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --teleop.type=so101_leader \
    --teleop.port=/dev/ttyACM1 \
    --dataset.repo_id=${HF_USER}/rollout_dagger_data \
    --dataset.single_task="Grasp the block"

Keyboard controls (default input device):

Foot pedal input is also supported via --strategy.input_device=pedal. Configure pedal codes with --strategy.pedal.* flags.

Where the robot supports the rollout position hold, a pause refreshes the last position target actually applied by the driver, including its clipping and the gripper target. Before the first command, a validated measured pose supplies the target. This preserves the commanded position error instead of changing the target to the measured pose; the robot can finish settling toward that target while policy trajectory advancement stops. Validate loaded-arm and gripper behavior on the actual robot; this software hold does not guarantee retained grip or physical immobility.

Episodic ( --strategy.type=episodic )

Episode-oriented recording that mirrors the behavior of lerobot-record. The policy drives the robot for each episode; an optional teleoperator can drive the robot during the reset phase between episodes.

lerobot-rollout \
    --strategy.type=episodic \
    --policy.path=${HF_USER}/my_policy \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --teleop.type=so100_leader \
    --teleop.port=/dev/ttyACM1 \
    --dataset.repo_id=${HF_USER}/my_eval_data \
    --dataset.num_episodes=20 \
    --dataset.episode_time_s=30 \
    --dataset.reset_time_s=10 \
    --dataset.single_task="Pick up the red cube"

Teleop is optional — if omitted the robot holds its position during the reset phase.

Keyboard controls:

Bring your own strategy

--strategy.type accepts any strategy registered in the environment, not only the built-ins above. A strategy is two classes in a pip-installable package: a config registered on RolloutStrategyConfig, and a RolloutStrategy subclass named after it without the Config suffix — the same SomethingConfig/Something convention as custom hardware, resolved from the config’s module or package.

from dataclasses import dataclass
from typing import ClassVar

from lerobot.rollout import RolloutContext, RolloutStrategy, RolloutStrategyConfig


@RolloutStrategyConfig.register_subclass("patrol")
@dataclass
class PatrolStrategyConfig(RolloutStrategyConfig):
    dataset_mode: ClassVar[str] = "required"  # the engine creates the dataset for you
    legs: int = 2  # becomes --strategy.legs


class PatrolStrategy(RolloutStrategy):
    def run(self, ctx: RolloutContext) -> None:
        ...  # the paced control loop: see the contract on RolloutStrategy.run

    def teardown(self, ctx: RolloutContext) -> None:
        ctx.data.dataset.finalize()
        self._teardown_hardware(ctx.hardware, ctx.runtime.cfg.return_to_initial_position)

The config declares what the engine arranges on the strategy’s behalf, so nothing in LeRobot needs to know its type:

Name the distribution lerobot_strategy_<name> and lerobot-rollout imports the package before parsing the CLI — only its __init__.py runs, so import the config there. A package with any other name is loaded, submodules included, with --strategy.discover_packages_path=<package>. The strategy’s dataclass fields become --strategy.* flags:

lerobot-rollout \
    --strategy.type=patrol \
    --strategy.legs=3 \
    --policy.path=${HF_USER}/my_policy \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --dataset.repo_id=${HF_USER}/rollout_patrol \
    --dataset.single_task="patrol the aisle"

SentryStrategy (recording, interactive-safe) and BaseStrategy (no recording) are the reference implementations to copy from; send_next_action, safe_push_to_hub and estimate_max_episode_seconds from lerobot.rollout.strategies are the helpers they build on.

For asynchronous backends, the dispatch contract is mandatory. Publish each captured observation with engine.notify_observation(obs_processed), call send_next_action(...) on every autonomous motor tick, including interpolation and held ticks, then call engine.pump_query(obs_processed). Calling only engine.get_action() bypasses the permission and hold handshake: a planned text query can wait indefinitely, and a terminal fault may never signal shutdown.

send_next_action is built on lerobot.inference.dispatch_tick and its parts (next_interpolated_action, commit_action, dispatch_permitted), which encode that order once: dispatch_allowed() before and after pulling an action and again before sending, interpolation invalidation plus a supported local hold and acknowledge_hold() when denied, and record_dispatch() after sending. A hold that fails is never acknowledged. Use those helpers rather than re-implementing the sequence; send_next_action only adds the dataset frame, the robot-side action processor and cadence accounting. Remote contexts have ctx.policy.policy, preprocessor and postprocessor set to None; use ctx.policy.inference and the robot-side pipelines in ctx.processors.

Wrap known blocking mid-run recording work in self._pause_for_recording(ctx), including ring-buffer drains and episode finalization. This invalidates asynchronous motion before the recording stall and resumes only if the run is still healthy and active. Strategies with operator-controlled phases must pass their current motion permission as resume_allowed. The next action must come from a fresh observation; saved work must not revive a paused, stopped or faulted run.

Inference Backends

Select a backend with --inference.type=<name>. Local synchronous and RTC inference use the policy configured below. The remote backend obtains its policy contract from a server and does not load weights on the robot computer.

Sync (default)

One policy call per control tick. The main loop blocks until the action is computed.

Works with all policies. No extra flags needed.

Real-Time Chunking ( --inference.type=rtc )

A background thread produces action chunks asynchronously. The main control loop polls for the next ready action while the policy computes the next chunk in parallel.

Use RTC with compatible VLA models (Pi0, Pi0.5, SmolVLA) to overlap inference and motion. Continuous execution still requires enough usable action playback to cover inference latency and its variation.

lerobot-rollout \
    --strategy.type=base \
    --inference.type=rtc \
    --inference.rtc.execution_horizon=10 \
    --inference.rtc.max_guidance_weight=10.0 \
    --policy.path=${HF_USER}/pi0_policy \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
    --task="Pick up the cube" \
    --duration=60 \
    --device=cuda

See the Real-Time Chunking guide for details on tuning RTC parameters.

Local RTC runs in a background thread in the rollout process, without transport. A server and client on the same computer use --inference.type=remote instead. See the remote guide for chunk alignment/blending and the waiting behavior shared by asynchronous backends.

Waiting requires a supported robot. Ordinary local RTC remains available without that capability, but planned text/instruction-change pauses and recoverable starvation need it. Position-controlled SO, OMX, Koch, HopeJr and Rebot followers support it; OpenArm, Reachy2 and bimanual wrappers have configuration constraints listed in the robot support table. With support, an empty buffer can retain the last applied target for action_starvation_grace_s (default 1 second), including the time needed to drain old inference and predict from a fresh capture. After motion has started, zero grace or an unsupported robot faults immediately on exhaustion; startup has its own deadline. Unlike the former local RTC path, exhaustion is no longer an indefinitely skipped tick. Grace never extends expired request or freshness limits.

Local RTC exposes these additional budgets through --inference.:

These are starting values, not measured hardware guarantees. RTC must retain enough post-trim playback for another inference and timing margin. Raising queue_threshold cannot help if inference routinely consumes the usable prediction horizon; reduce latency or validate a longer horizon/different action frequency.

A terminal fault never automatically resumes policy motion. With usable observation acquisition and actuator control, shutdown honors --return_to_initial_position (default true) and disconnects. Returning reads a full observation, so a persistent camera failure can prevent it. Failed command application suppresses further shutdown movement for all backends, including sync. This does not change synchronous prediction or introduce camera-specific recovery.

Interactive Sessions

Add --interactive=true to drive the rollout from the terminal instead of starting immediately. Hardware connects and the policy loads as usual, but the robot stays still until you type /start — useful when you want to position the scene first, re-instruct the policy between attempts, or run several takes without paying the load time again.

lerobot-rollout \
    --strategy.type=base \
    --policy.path=${HF_USER}/my_smolvla_policy \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
    --task="pick up the cube" \
    --interactive=true

> /start
Rollout running — task 'pick up the cube'. /subtask <text> to change it, ...
> /subtask put the cube in the box
Task: 'pick up the cube' → 'put the cube in the box' (applies from the next policy inference)
> /vqa is the cube inside the box?
Asked: 'is the cube inside the box?' — answering from the next observation...
Q: is the cube inside the box?
A: No — the cube is on the table to the left of the box.
> /reset
Task restored to 'pick up the cube'
Resetting — returning the robot to its initial position...
Robot reset — holding at initial position. /start to run.
> /stop

Lines starting with > are what you type; the session renders no prompt of its own.

Ctrl-C still shuts down as usual, and closing stdin (Ctrl-D, or the end of a piped script) ends the session — so a piped script must keep stdin open for the intended duration:

(printf '/start\n'; sleep 60; printf '/stop\n') | lerobot-rollout ... --interactive=true

/subtask updates the task for subsequent inference. With --inference.type=sync, precomputed chunk actions are dropped. With local RTC or remote inference, a change during an action request discards the obsolete result. On a robot supporting local hold, the engine invalidates queued/interpolated motion, waits for the control loop to acknowledge the hold, then requests fresh motion under the new task. Expect a stop during this transition; it is not a guaranteed continuous blend. A change between requests can retain already accepted actions for continuity. Local RTC on a robot without hold support still discards the obsolete result, but can exhaust its remaining actions and terminate. Change instructions between runs if that robot cannot hold appropriately.

With local --use_torch_compile=true, an instruction that changes processed tensor shapes can trigger recompilation. Padding may keep some task changes at the same shape, but later compilation is not guaranteed to fit the remaining playback or the instruction-change deadline: it can cause a terminal fault. Leave compilation off when exploring instructions unless those transitions have been validated. Changes during initial compile warmup do not request a motion hold, because policy motion has not started yet.

/vqa and /autosteer require a policy that declares text generation support and implements PreTrainedPolicy.generate_text(), such as an appropriately trained EO1 checkpoint. Capability is checked before accepting a query; checkpoint-specific behavior still needs validation.

/vqa answers from the next observation and prints when ready, so you can keep typing meanwhile; only one question is in flight at a time (autosteer’s own queries share that slot). Asynchronous backends first enter an acknowledged local hold and invalidate queued/interpolated motion, then submit a fresh observation to the text path. A healthy active run resumes from a fresh action chunk after the answer; paused or stopped runs stay inactive. Synchronous text generation runs inline on the control thread and drops the policy’s precomputed actions, so the next tick resumes from fresh inference. Query time counts against --duration. Questions are only accepted while a run is in progress — an idle engine has no current view. If the instruction or execution context invalidates an asynchronous answer, the question receives one cancellation notice instead of the obsolete answer. Segment end also reports unanswered questions, including generation still in progress; reporting cancellation does not interrupt a running model call or free its query slot early.

/autosteer hands the policy a goal and asks it for the next subtask every --autosteer_interval_s seconds (default 10), applying each one through the same path as /subtask and announcing it in the chat. Every query is a full text generation, so the interval is a cost knob: lower values re-plan sooner but spend more of the loop generating text instead of acting. Plan progress lives in the policy, not the rollout, so the plan dies with the segment; the sequencer also stops on /reset, /subtask, /autosteer off, and the first planning failure.

Logs below ERROR are muted while the session runs so routine output doesn’t interleave with what you’re typing; errors and fatal inference failures still show, and normal logging resumes when the session ends. The cadence summaries are printed in the chat instead, and running without --interactive gets you the live log back.

Sessions work over SSH and on headless machines — no display server needed.

Recording while interactive. --strategy.type=sentry also accepts --interactive=true and records continuously while you steer: each /start…/reset segment saves complete episodes plus one final partial episode, and the dataset stays open until shutdown. Frames carry the task provenance of the action actually sent, so actions already queued or interpolated under the old task keep the old label.

Blocking mid-run episode saves pause asynchronous inference, invalidate queued/interpolated actions and request a supported local hold. They resume from a fresh capture only if the run remains active and healthy. The same protection applies to DAgger saves and Highlight ring-buffer drains/saves. Recording pauses and ordinary starvation share the physical waiting behavior, but their deadlines and reasons for resumption remain separate.

lerobot-rollout \
    --strategy.type=sentry \
    --policy.path=${HF_USER}/my_smolvla_policy \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --dataset.repo_id=${HF_USER}/rollout_cube_sessions \
    --task="pick up the cube" \
    --interactive=true

The other recording strategies (episodic, DAgger, highlight) are not supported: they bind their own keyboard controls and finalize the dataset when run() exits, so they cannot be restarted segment by segment.

Programmatic control

RolloutController exposes the same operations as a library API — thread-safe start() / reset() / stop() / set_task() plus a RolloutEvent callback, with no stdin, printing or log muting attached:

from threading import Event, Thread

from lerobot.rollout import (
    LinkedEvent,
    RolloutController,
    RolloutEvent,
    build_rollout_context,
    create_strategy,
)

parent = Event()  # your application's shutdown signal
ctx = build_rollout_context(cfg, LinkedEvent(parent))  # loads policy, connects robot
strategy = create_strategy(cfg.strategy)
strategy.setup(ctx)

controller = RolloutController(strategy, ctx, on_event=print)  # or your own observer
serve_thread = Thread(target=controller.serve)  # serve() blocks; run it where you like
serve_thread.start()

controller.start()                        # robot starts executing the policy
controller.set_task("grab the red cube")  # re-instruct mid-run
controller.reset()                        # stop movement, return home, stay warm
controller.stop()                         # end serve()

serve_thread.join()
strategy.teardown(ctx)  # teardown stays with the caller

RolloutController requires the context’s shutdown event to be a LinkedEvent, built as above: the controller ends run segments through its local flag, while your parent event still forces a full shutdown. Set play_sounds=False unless you want the vocal announcements.

Common Flags

Cadence Reporting

Every strategy reports what its control loop actually achieved, as do lerobot-record, lerobot-teleoperate and lerobot-replay — they all pace through the same timer. Each saved episode gets a one-line digest, and when the loop ends — including on Ctrl-C — a block covering the whole run:

Cadence (episode 1): 29.84 Hz policy vs 30 Hz target · 1200 ticks, 20.1 s measured · 6/598 cycles over the 33.3 ms budget (work mean 18.7 ms, worst 48.1 ms)
Cadence (episode 2): 29.84 Hz policy vs 30 Hz target · 1200 ticks, 20.1 s measured · 6/599 cycles over the 33.3 ms budget (work mean 18.7 ms, worst 48.1 ms)
Cadence summary — whole run, 2 episodes · target 30 Hz × 2 (16.7 ms tick slot, 33.3 ms cycle budget): 2400 ticks, 1197 cycles judged
  effective cadence: 29.84 Hz policy / 59.67 Hz commands over 40.2 s measured
  cycles over the 33.3 ms work budget: 12/1197 (1.0%) — work mean 18.7 ms, worst 48.1 ms
  ticks over their 16.7 ms slot: 40/2400 (costs interpolation smoothness only)
  loop-body steps (share of measured work):
    observe      mean   3.13 ms · worst   9.00 ms ·  31.7% of work · 2400 calls
    process_obs  mean   0.50 ms · worst   1.00 ms ·   5.1% of work · 2400 calls
    infer        mean   5.18 ms · worst  40.00 ms ·  52.4% of work · 2400 calls
    send         mean   0.40 ms · worst   0.40 ms ·   4.1% of work · 2400 calls
    record       mean   0.67 ms · worst 620.00 ms ·   6.8% of work · 2402 calls
  pacing headroom: 7.4 ms slept per tick on average (max 13.0 ms) — near zero means the loop is saturated

Read it top-down: effective cadence is what you got against --fps, cycles over budget is how often the loop could not hold it, the step breakdown says where the time went, and pacing headroom near zero means there is nothing left to absorb a slow tick. A high infer share points at the policy or the device; a high observe share points at the cameras. A step’s worst far above its mean is a one-off — above, record is 0.67 ms per frame and the 620 ms outlier is the blocking save_episode that closed an episode.

A cycle here is one policy action’s worth of ticks (--interpolation_multiplier of them). Only whole cycles over the 1/fps budget can cost you frames; a single tick overrunning its slot only makes the interpolated motion less smooth, which is why the two are counted separately. Without interpolation a cycle is a single tick, so the report drops the distinction and counts ticks throughout.

In interactive sessions each /start segment owns its own timer, so the report lands in the chat once per segment (/reset, /stop, or --duration) rather than once per process.

Programmatic Usage

For extensions, lerobot.inference is the public package for backend configurations, local engines and shared scheduling contracts. lerobot.remote_inference owns the remote engine, protocol, deployment and session implementation; lerobot.rollout owns the robot loop, strategies, interpolation and recording. Import public symbols from their package rather than implementation files. The optional remote dependencies are needed only when the remote backend or server is used.

For custom deployments (e.g. with kinematics processors), use the rollout module API directly:

from lerobot.rollout import BaseStrategyConfig, RolloutConfig, build_rollout_context
from lerobot.inference import SyncInferenceConfig
from lerobot.rollout.strategies import BaseStrategy
from lerobot.utils.process import ProcessSignalHandler

cfg = RolloutConfig(
    robot=my_robot_config,
    policy=my_policy_config,
    strategy=BaseStrategyConfig(),
    inference=SyncInferenceConfig(),
    fps=30,
    duration=60,
    task="my task",
)

signal_handler = ProcessSignalHandler(use_threads=True)
ctx = build_rollout_context(
    cfg,
    signal_handler.shutdown_event,
    robot_action_processor=my_custom_action_processor,       # optional
    robot_observation_processor=my_custom_obs_processor,     # optional
)

strategy = BaseStrategy(cfg.strategy)
try:
    strategy.setup(ctx)
    strategy.run(ctx)
finally:
    strategy.teardown(ctx)

See examples/so100_to_so100_EE/rollout.py and examples/phone_to_so100/rollout.py for full examples with kinematics processors.