Skip to content

Robots

Robots in Lucky Engine are delivered as packs. Each pack is built around an MJCF (.xml) model file (MuJoCo's standard format), with the meshes and textures it uses, and optionally trained policies, a motion graph and scripts. You download packs from the Content Vault or the Robots panel, and place them in a scene from the Robots panel or the viewport.

A placed robot is a MuJoCo model. Its Robot Controller component holds the policies and the motion graph that move it, and scripts drive both from C#.

In short

Open View → Robots, or click Create (+) at the bottom left of the viewport and choose Robots. Click Download on a robot if it is not installed yet, then click the robot or drag it into the viewport. The editor adds the model and wires up what the pack ships: policies as inactive policy slots, a motion graph, and a script. Activate a policy from C#, or from its Active checkbox in the Inspector. To start from scratch, use a robot template on the Welcome screen.

In the Content Vault

The Content Vault panel (View → Content Vault) lists the packs available from the Lucky Robots library. Choose the Robots category to see robot packs, and click a pack's download icon to install it into the open project. Any C# scripts the pack ships with are compiled automatically. A pack that needs a different editor version shows its requirement instead of the download icon. See Content Vault for the panel itself.

The Franka Emika Panda arm ships with the editor: it is part of the bundled Welcome project, whose Welcome scene uses it. The pack has a single-arm model (panda.xml) and a two-arm model (two_pandas.xml), each with a motion graph.

Other robot packs, such as the Unitree G1 humanoid, the Unitree Go2 quadruped, the AgileX Piper arm and the OpenArm bimanual manipulator, are listed in the panel when they are available for your editor version. The Content Vault panel is the authoritative list; new packs land there as they ship.

What's in a pack

When a pack installs, the editor unpacks the following into the project, in a folder for the pack under Assets/ContentVault/Robots/:

Asset What it is
MJCF (.xml) The robot model. A pack can hold more than one, such as the Panda's single-arm and two-arm models
assets/ Meshes and textures referenced by the MJCF
Policies (.honnx) Trained policies, if any. Each refers to an ONNX model
Motion graph (.hpodgraph) A robot motion graph, if the pack includes one. See Procedural Graphs
Scripts (.cs) C# scripts, if any

Adding a robot to a scene

From the Robots panel or the Create menu

Robots are placed from a grid of the robot packs in the Content Vault. It opens in two places:

  • View → Robots opens the Robots panel, which docks on the right of the editor and stays open while you work.
  • In the Scene viewport, click the Create (+) button at the bottom left and choose Robots. The grid opens in an Add robot box above the button. It closes when you add a robot, or when you press Esc.

Each robot has a card with a button below its picture:

Button What it does
Download Installs the pack into the open project. While it installs, the button reads Installing..., and no other pack can be downloaded until it has finished. If the install fails, the button reads Retry download
Add to scene Adds the robot to the scene. Shown once the pack is installed
Requires and a version The pack needs a different editor version, and cannot be downloaded

To add an installed robot, click Add to scene or the robot's picture, or drag the picture into the viewport. The editor shows Spawning robot... while it builds the robot, then selects it. Robots are added only while the scene is in Edit mode.

Type in Search robots to filter the cards by name. The refresh button next to it reloads the list of robots from the Content Vault.

The Content Vault panel also downloads robot packs, and updates and removes them, but robots are placed only from this grid. See The Viewport → The Create menu for the rest of the Create menu.

From a Welcome screen template

The TEMPLATES row of the Welcome screen has two robot cards: Unitree G1 (5 Fingers) and Emika Panda (Franka). Click one, fill in the new-project form and click Create. The editor downloads what it needs, creates and opens the project, and places the robot in its scene. See Creating a project.

From your own MJCF file

  1. In the Content Browser, right-click an empty area and choose Import, then pick the .xml file. The editor copies the model together with the meshes, textures and included files it references.
  2. Drag the .xml file into the viewport. The editor adds the MuJoCo model, with no Robot Controller, policies or motion graph.
  3. To control it, use Add Component → RobotController, then add policies or a motion graph as described in Adding policies and motion graphs.

Choosing which geoms are shown

MJCF models often hold two versions of their shapes: detailed visual meshes and simpler collision shapes. If both show up in the viewport, choose which MuJoCo render groups are drawn. Double-click the .xml in the Content Browser to open the MuJoCo Import Editor, in a tab named after the file, and tick Group 0 to Group 5 for the groups to show. By MuJoCo convention, groups 0 to 2 hold visual geoms and are shown, and groups 3 to 5 hold collision geoms and are hidden. Some models put their collision shapes in group 1: untick it to hide them. This works for the models in robot packs too.

Click Apply & Reload to save the choice and rebuild every instance of the model in the open scene. The choice is stored next to the model, in <model>.xml.mjimport, so the .xml itself is not changed. Reset to Defaults sets the boxes back to the convention; click Apply & Reload to use it. Render groups change only what is drawn, not the physics: hidden collision geoms still collide.

What placement sets up

Placing a robot pack does the following:

Part What happens
Model The pack's model is added to the scene, with a root entity named after the pack. The root carries a MuJoCo Scene component (MujocoSceneComponent in C#). Each rigid body in the MJCF gets a child entity, whose transform follows MuJoCo every physics step and which carries any visual mesh
Robot Controller Added when the pack ships a policy or a motion graph (RobotControllerComponent in C#)
Policies Each policy the pack declares becomes an inactive slot under Policy Slots. The editor writes a policy_descriptor.<name>.json next to the policy, registers it in the project's PolicyRegistry.yaml, and writes a C# enum of the policy's commands
Motion graph Attached only if the pack declares one
Script The pack's own script, if it has one. Otherwise, a pack with policies gets a generated starter script for its first policy, and a pack with neither policies nor a script gets a starter script named after the robot in Assets/Scripts/Source
Sensors Lidar sensors and RGBD cameras the pack declares are added to the scene's recording setup. See Sensors

The generated policy starter script, <policy>_Policy.cs, activates slot 1 when you press Play and sends one public float field per policy command every update. The fields appear on the Script component in the Inspector, as sliders when the policy declares ranges for its commands.

For the script-level view of these components, see Working with components → MuJoCo bodies. For an end-to-end example driving a walking policy, see the Getting Started walker script.

The Robot Controller

The Robot Controller component is shown in the Inspector, below the entity tree in the Scene Hierarchy panel. From top to bottom it holds:

Part What it is
Motion Graph The robot's motion graph, a .hpodgraph file. May be empty
Motion clip slots One per motion-clip input of the graph, if it has any
Graph inputs One row per input of the motion graph, shown once a graph is assigned. See Attaching a motion graph
Policy Slots The trained policies bound to this robot, each in its own collapsible section, and an Add Policy Slot row with a New Slot button

Policy slots start inactive. Tick a slot's Active checkbox, or call SetPolicyActive(slotId, true) from a script, to run it.

Adding policies and motion graphs

A robot can host many policies and a motion graph at the same time:

  • Policy slots each bind a trained policy (an ONNX model with its descriptor) to the robot, with its own slot ID, priority and driven-joint mask.
  • A single motion graph drives the joints that no active policy has claimed, for example an arm moved by inverse kinematics while a policy walks the legs.

Joint ownership is divided up so they do not overwrite each other. A script can also drive individual actuators directly; see Working with components.

Most packs come pre-wired

Robot packs that ship a motion graph or policies get them attached automatically at placement. The workflows below are for adding more policies on top, swapping the motion graph, or wiring up a robot that did not come with either.

Adding a policy

Policies are .honnx assets in the project. To bind one to a robot:

  1. Right-click the robot's root entity, either in the Scene Hierarchy or in the viewport, and open Add Policy. The menu is shown for entities with a MuJoCo Scene component.
  2. The submenu lists every .honnx policy in the project by file name. In the Scene Hierarchy, hover an entry to see its path. With no policies in the project, Add Policy is greyed out in the Scene Hierarchy, and the viewport's list reads No .honnx policies found.
  3. Pick a policy.

Picking a policy:

  1. Writes policy_descriptor.<name>.json next to the policy. It lists the policy's commands in order, which gives them their 1-based command IDs (for example SetVx = 1, SetVy = 2, SetYawRate = 3), its observation layout and its joint metadata.
  2. Adds or updates the policy's entry in the project's PolicyRegistry.yaml.
  3. Writes a <Policy>Commands C# enum of the policy's commands to Assets/Scripts/Source/Generated.
  4. Adds an inactive slot under Policy Slots, named after the policy, with an Id one higher than the highest in use (1 for the first slot) and a Priority of 0. Adding the same policy again does not add a second slot.
  5. If this is the robot's first policy and it has no Script component, generates the <policy>_Policy.cs starter script next to the policy and attaches it.

Most policies need a model_config.json, the file that lists the joint names, default poses, PD gains and action scales the policy was trained with. The editor looks for it next to the ONNX file, then next to the robot's MJCF file, and otherwise asks for it. ONNX files that carry this information themselves do not need one.

To bring a policy into the project first, use one of these:

  • In the viewport's Add Policy menu, Import Policy... asks for an .onnx file, and for its model_config.json if there is none next to it. It saves the policy as a .honnx in Assets/Policies. It does not add the policy to the robot: pick it from the list afterwards.
  • In the Content Browser, right-click an empty area and choose Import ONNX Policy (.honnx). This does the same, saving into the current folder.

To activate the policy at runtime, flip its slot from a script:

GetComponent<RobotControllerComponent>()?.SetPolicyActive(slotId, true);

See Getting started for the WASD walker example that drives a walking policy end to end.

The policy slot model

Each slot under Policy Slots has these fields:

Field Description
Id The slot's ID, used by scripts in SetPolicyActive, SetFloat and GetFloat
Name The slot's display name. On a manual scene save, it also names the slot's member in the generated <Scene>PolicyIds enum
Descriptor Path Path, inside Assets, to the policy's policy_descriptor.*.json
Active Whether the policy runs. Toggle it here or from a script
Priority Order in which active slots are combined. See below
Clamp Obs For Unclaimed Joints When on, the policy does not see the motion of joints it does not drive, such as joints moved by another slot or by the motion graph
Driven Joints (n / m) Checklist of the joints this policy may drive, out of the joints in its descriptor

The Driven Joints checklist has presets: All, Legs, Arms, Waist, Left Arm and Right Arm. A plain click replaces the selection, Shift + click adds the preset to it, and Ctrl + click removes it. An empty list drives every joint in the policy's descriptor. The checklist needs a Descriptor Path to list the joints.

How active slots are combined:

  • Slots run in ascending Priority. The lowest-priority active slot is the primary policy, and higher-priority slots are overlays.
  • Each slot writes only its Driven Joints. Where two active slots claim the same joint, the higher-priority slot wins. Values are not blended, so keep the masks apart.
  • Inactive slots do not run at all.

New Slot adds an empty slot named Slot 1, Slot 2 and so on. Set its Descriptor Path to a policy descriptor to use it. The X button on a slot's header removes the slot; the policy's ONNX file and descriptor stay on disk.

Give slots names that are valid C# identifiers

On every manual save, the editor writes a <Scene>PolicyIds C# enum to Assets/Scripts/Source/Generated, with one member per named slot, using the slot's Name. Use names that start with a letter and contain only letters, digits and underscores, such as walker, so the enum compiles. Rename new slots before saving.

The Policies panel

View → Policies opens a panel that manages the project's registered policies (PolicyRegistry.yaml):

  • Registered Policies lists them, with a search box.
  • + New opens Import New Policy, with Policy Name, ONNX File, Model Config, Robot XML and Destination (a folder inside Assets). Every field is required. Import generates the policy descriptor and registers the policy.
  • Selecting a policy shows the same fields. Pick new files and click Save Changes to regenerate it, or click Delete to remove it from PolicyRegistry.yaml and delete its descriptor. The ONNX file, model_config.json and robot XML are not touched.

To run a policy imported here, add a slot with New Slot and set its Descriptor Path to the generated descriptor.

Attaching a motion graph

The motion graph is a single asset on the Robot Controller, not a slot. To attach one, or swap it for another:

  1. Select the robot's root entity.
  2. Drag a .hpodgraph robot motion graph from the Content Browser onto the Motion Graph field of the Robot Controller.

A motion graph drives a robot only if it is a robot motion graph: a Motion-type procedural graph with a pose output and an input that reads the robot's pose. See Procedural Graphs → Motion graphs for robots.

Once a graph is assigned, its inputs appear below the field. Edit a value to set it for this robot only. Right-click a row for:

  • Use graph default, which removes this robot's value.
  • Send pulse, on checkbox inputs, which sets the input for a single update. Graphs use this for one-off actions, such as Start on the Panda's IK graph.

If the graph takes motion clips, each clip input gets a slot that accepts a motion clip (a Motion .json asset). Clearing the slot restores the graph's default.

The graph runs every step alongside the active policies, and drives the joints that no active policy has claimed. It skips actuators a script controls directly. The motion graph also runs while the scene is not playing, so the robot previews its motion in the viewport. To snap a robot back to its model's default pose, right-click it in the Scene Hierarchy and choose Reset Pose to MJCF Default. This also pauses the preview so the pose stays; save the scene to keep it. The graph runs again when you press Play.

The script-side view of policy commands and motion-graph inputs is in Controlling robots.

Upgrading from 2026.2

Motion graphs are now procedural graphs (.hpodgraph). .hmograph files are no longer supported, packs that contain them are rejected on install, and a 2026.2 scene whose Robot Controller used a motion graph does not load. See Procedural Graphs → Upgrading from 2026.2.

Where to go next