Lidar¶

A lidar sensor casts ranging beams against the MuJoCo scene and reports a distance per beam. Lucky Engine models both spinning units (Velodyne and Ouster style) and non-repetitive units (Livox style), with a calibrated return model that reproduces range noise, grazing dropout, and reflectivity-dependent maximum range.

Materials can be authored for the sensor's wavelength, so glass, water, mirrors and foliage behave as they do for a real 905 nm lidar, including secondary returns behind a window or a mirror.

In short

Turn on Record Lidar on the Recorder page of Edit → Settings, open the Lidar tab, and click Add Lidar Binding. Attach the binding to a MuJoCo body or site and pick a preset. Read scans from C# with MujocoSceneComponent.GetLidarScan, or over gRPC with LidarService. A range of -1 means the beam returned nothing.

Adding a lidar sensor¶

Lidar is not an Inspector component. It is a binding on the scene's recorder settings, as described in Sensors are bindings, not components.

Stop the scene if it is playing, then open Edit → Settings and select the Recorder page. Enable MuJoCo Observer must be on for the page's tabs to appear.
On the General tab, under Lidar Recording, turn on Record Lidar. The Lidar tab's settings stay greyed out until this is on. The switch on the Lidar heading of the Observer panel, or of the Recorder Status popup on the viewport toolbar, does the same.
Open the Lidar tab and click Add Lidar Binding. Each binding appears as a collapsible entry, labelled Lidar #0 until it has an Output Name.
Set Attach Body or Attach Site to an entity in the scene. The attach target supplies the sensor's pose. Picking one also fills an empty Output Name with the entity's name.
Optionally choose a unit from Apply Preset (Select preset...), then adjust from there.

The binding is saved with the scene. Placing a robot pack that ships a lidar in a scene (see Adding a robot to a scene) adds its bindings for you, attached to the pack's sites. It also turns on Record Lidar and Record Data, sets Recorder Mode to MuJoCo Full Scene, and switches the viewport's Lidar View to Live Preview if it was None.

Attach precedence

A binding may carry both an attach body and an attach site. The site wins when both are set.

Mount Offset displaces the sensor from its attach point, in metres, in the attach body's or site's local frame.

Scan configuration¶

Field	Default	Range	What it does
Output Name	empty		File-name suffix for the recorded Parquet: `lidar-<Name>-NNN.parquet`
Mount Offset	`0, 0, 0`		Offset from the attach point, metres, local frame
Scan Pattern	Spinning (uniform)		Beam layout; see Scan patterns
Channels	`16`	1 to 256	Vertical beams, or laser rows. Spinning only
Azimuth Bins	`1800`	1 to 65535	Horizontal samples per 360 degree revolution. Spinning only
Rotation Hz	`10.0`	0 to 1000	Revolutions per second. Pattern Hz for Livox
Vertical FOV Min (deg)	`-15.0`	-90 to 90	Lowest beam elevation
Vertical FOV Max (deg)	`15.0`	-90 to 90	Highest beam elevation
Min Range (m)	`0.5`	0 to 10000	Near cutoff; closer hits are dropped
Max Range (m)	`100.0`	0 to 100000	Far cutoff; farther hits become misses
Geom Group Mask	`255`	0 to 255	Which MuJoCo geom groups the beams hit
Self Ignore	Mount body only		What the sensor's own beams pass through
Record Video Proxy	off		Also write an H.264 visualisation `.mp4` next to the Parquet file. Needs the external FFmpeg program
Secondary Capture	Off		Extra geometry to record per beam; see Secondary returns
Secondary Scope	Flagged materials only		Where the extra rays are cast. Shown when capture is on

There is no horizontal FOV

Both scan patterns sweep a full 360 degrees. Horizontal resolution is implied rather than configured: it is 360 / Azimuth Bins degrees. At the default 1800 bins that is 0.2 degrees per bin. The bin index is stored as a 16-bit integer, which is why the cap is 65535.

Self Ignore decides what the sensor is blind to, which matters when a sensor is mounted on the robot it is scanning from:

Editor label	Behaviour
None (hit self)	Beams hit everything, including the body they are mounted on, as a real sensor sees its own housing
Mount body only	Beams pass through the attach body but still hit the rest of the robot. The default
Whole robot	Beams pass through the robot's whole kinematic tree, so the robot never occludes its own scan

Geom Group Mask is effectively 6-bit

The field accepts values up to 255 and defaults to all bits set, but MuJoCo defines only six geom groups. The top two bits have no effect. SetLidarConfig from script or gRPC keeps only the low six bits.

Scan patterns¶

Pattern	Layout	Fields
Spinning (uniform)	A uniform grid: Channels beams spread evenly across the vertical FOV, sampled at Azimuth Bins steps around a full revolution	Channels, Azimuth Bins, Rotation Hz
Livox (non-repetitive)	A 360 degree pattern that never repeats, so coverage fills in over time rather than tracing fixed rings	Points / cycle, Pattern Hz

For a spinning sensor the elevation step is (FOV Max - FOV Min) / (Channels - 1) and the azimuth step is 360 / Azimuth Bins.

For the Livox pattern the editor replaces Channels and Azimuth Bins with a single Points / cycle field (1 to 65535) and relabels Rotation Hz as Pattern Hz. Points per cycle multiplied by Pattern Hz gives the effective points per second. Editing Points / cycle, or applying a Livox scan preset, sets Channels to 1 and stores the point count in Azimuth Bins. Selecting the pattern alone leaves the existing values in place. In both patterns the beam count is Channels * Azimuth Bins.

Scan presets¶

Apply Preset overwrites eight fields: scan pattern, channels, azimuth bins, rotation rate, both vertical FOV bounds, and both range bounds. It leaves the name, attach target, mount offset, geom group mask, self-ignore, video proxy and secondary capture untouched.

Preset	Pattern	Channels	Bins	Hz	Vertical FOV	Range (m)
`Unitree L1 (Go2 stock)`	Livox	1	2160	10.0	-45 to 45	0.05 to 30
`Livox Mid-360`	Livox	1	20000	10.0	-7 to 52	0.10 to 70
`Velodyne VLP-16 (spinning)`	Spinning	16	1800	10.0	-15 to 15	0.50 to 100
`Ouster OS1-64 (spinning)`	Spinning	64	1024	10.0	-22.5 to 22.5	0.50 to 120

Two different things are called a preset

These scan presets set beam geometry. A second, unrelated preset list sets the return model calibration, applied with SetLidarPreset. Both lists have four entries, but the entries do not line up: scan preset 0 is the Unitree L1, while return preset 0 is Generic. See Return model presets.

Scan timing¶

The lidar recorder runs on the runner chosen in Lidar Recorder TimeRunner on the Recorder page's General tab. On each tick it advances the scan by Rotation Hz multiplied by the tick length and casts only the beams in that slice, up to one full revolution per tick. For a spinning sensor the slice is a range of azimuth bins, all channels included. For a Livox pattern it is the next run of beams in emission order.

A live scan holds the newest value for each beam. Beams in the current slice are fresh, and the rest are up to one revolution old. For example, a 10 Hz sensor on a 50 Hz runner refreshes one fifth of its beams per tick.
In the recording, every ray cast in the same tick shares one timestamp.
A sensor cannot scan faster than one revolution per tick. Keep Rotation Hz at or below the runner's rate.
The scan restarts from the beginning of the pattern at the start of each episode.

Beam layout and indexing¶

Beam directions are generated in the sensor's local MuJoCo frame: +X forward, +Z up, with +Z as the rotation axis. A scan is returned as one flat array. The index of a beam is:

index = bin * Channels + channel

For a spinning sensor that maps cleanly onto a grid: channel is the vertical row, from the lowest beam at 0, and bin is the azimuth step, with bin 0 pointing along +X.

For Livox, bin and channel are not angles

A non-repetitive pattern has no rings and no azimuth grid. The flat index is emission order, and bin and channel are just the high and low parts of that index. A bin is not an azimuth bin and a channel is not a vertical channel. The recording's sidecar metadata says so explicitly. To get real angles for a beam, call GetLidarBeamAngles rather than deriving them from the index.

GetLidarBeamAngles returns per-beam azimuth and elevation in radians: azimuth as atan2(y, x) in [-pi, pi], elevation as asin(z) in [-pi/2, pi/2].

The return model¶

By default a beam reports a clean geometric distance. The return model replaces that with a simulated measurement: range noise that grows with distance, dropout on dark, glossy or grazing surfaces, and a maximum range that depends on target reflectivity.

The return model applies to live scans requested with the material option (material in C# and gRPC, and in the lidar_ranges observation) and needs a material bake. Recordings always store the clean geometric scan, so the same recording can be re-processed later; the Lidar Replay panel applies the model when it exports a variant, using its own copy of the parameters.

The calibration is one set for the whole scene, shared by every lidar. It is not part of a binding and is not saved with the scene: it starts at the Generic preset, and is set from script or gRPC.

The parameters, in the order GetLidarReturnParams returns them:

Parameter	Default	Unit	What it does
`maxRangeAtRef`	`20.0`	m	Detection range for a surface of the reference reflectivity
`refReflectivity`	`0.90`		The reflectivity that `maxRangeAtRef` is quoted at
`rangeExp`	`0.32`		How strongly detection range scales with reflectivity
`noiseSigma0`	`0.02`	m	Base one-sigma range noise
`grazingCosCutoff`	`0.10`		Incidence cosine below which returns become unreliable and start to drop, about 84 degrees
`intensityRefRange`	`5.0`	m	Range at which the inverse-square intensity term equals 1. Scales reported intensity only
`detectFloor`	`0.03`		Return strength below which a beam drops out
`nearRangeR0`	`0.6`	m	Distance over which intensity ramps up close to the sensor
`noiseSlopePerM`	`0.0005`	m/m	Growth of range noise per metre
`specSlopeFloor`	`0.03`	rad	Minimum surface slope assumed for glossy surfaces, so a perfectly flat one still gives a glint of finite width
`specPeakMax`	`900.0`		Cap on glint strength, so a polished surface cannot return unbounded signal
`retroGain`	`800.0`		How much brighter a retroreflector is than a perfect matte surface
`retroCosExp`	`5.0`		How quickly a retroreflector dims away from head-on
`foldMaxSlope`	`0.20`	rad	Surface slope above which a mirror-like surface no longer produces a recognisable phantom
`rangeFloor`	`0.01`	m	Smallest range noise can produce. Not the sensor's minimum range
`minReturnSeparation`	`1.0`	m	Two echoes closer than this merge into one
`maxReturns`	`1`		How many echoes the hardware can report per beam. Single-echo units keep the strongest

Maximum range scales with reflectivity as Rmax(rho) = maxRangeAtRef * (rho / refReflectivity) ^ rangeExp, and range noise grows linearly as sigma(R) = noiseSigma0 + noiseSlopePerM * R.

Tune detectFloor first

If too much or too little is dropping out, detectFloor is the parameter to adjust first. It is a threshold on return strength, so dark, glossy and grazing surfaces fall below it first. Lower it for fewer dropouts. The right value depends on the scene, so a value tuned for a small indoor scene may not suit a large outdoor one.

Return model presets¶

SetLidarPreset selects a calibration by index. Applying a preset resets every return parameter, including the optics parameters, to that preset's values.

Index	Preset	Max range at ref (m)	Ref reflectivity	Range exponent	Max returns
`0`	Generic	20	0.90	0.32	1
`1`	Unitree L1 (Go2)	20	0.90	0.32	1
`2`	Livox Mid-360	70	0.80	0.27	1
`3`	Ouster OS1	170	0.80	0.31	2

Generic is the default calibration. SetLidarReturnParams and SetLidarReturnParamsEx set the parameters explicitly, and GetLidarReturnParams reads them back.

Which fields a setter writes

From C#, SetLidarReturnParams takes the first nine parameters and leaves the optics parameters alone, while SetLidarReturnParamsEx writes all seventeen. Over gRPC, SetLidarReturnParams always writes the nine base fields, so send them all, ideally from a GetLidarReturnParams read. The eight optics fields, from spec_slope_floor to max_returns, keep their current values when the request leaves them out.

Material bakes¶

The return model needs to know how reflective each surface is at the sensor's wavelength. That comes from a material bake: a 3D grid of the scene's near-infrared (NIR) material values, built from the scene's visual meshes. A bake is required for:

Reflectivity-dependent range and dropout. Without a bake the model has no per-surface reflectivity to work from.
Live secondary returns. GetLidarScanSecondary, and want_secondary over gRPC, compute the hit behind a window or a mirror from the bake.

Bakes are saved to MaterialBakes/<name>.lvxb in the project folder, and default to the name scene. From script or gRPC, bake with BakeLidarMaterial or BakeLidarMaterialEx, then use SaveLidarBake and LoadLidarBake to persist and restore it. The bake used by script and gRPC is not saved with the scene, so load it again when a new play session starts.

The Lidar Replay panel builds the same kind of bake from the editor, but keeps its own copy, separate from the one scripts and gRPC use. To use a panel bake from the API, click Save bake to disk in the panel, which writes MaterialBakes/scene.lvxb, then call LoadLidarBake("scene").

Bake settings are saved with the scene and edited on the Lidar Replay panel's Noise Profile tab. BakeLidarMaterialEx and gRPC BakeLidarMaterial fall back to those settings for any field left at zero, so a bake made from the API matches one made in the editor, and a client can override one setting without knowing the others:

Field	0 means	Otherwise
`cell_size`	Scene setting	Cell size in metres
`max_dim`	Scene setting	Cap on cells per axis
`splat_radius`	Scene setting	The radius; pass `-1` to request an explicit `0`
`include_hidden`	Scene setting	`1` = include editor-hidden meshes, `2` = exclude them

Stale bakes¶

A bake is a snapshot. Nothing re-reads a material after it is baked, so editing a material leaves the old values in the grid, and a scan reports the materials as they were when baked, with nothing in the scan itself to say so. A bake also counts as stale when it was written by an older version of the editor.

GetLidarBakeStatus reports what is loaded: whether a bake is loaded at all, its format version, cell size, splat radius, dimensions, filled-cell count, the size of its per-geom material table, a material fingerprint, and a stale flag. Check stale before trusting a material-informed scan. LoadLidarBake loads a stale bake rather than refusing it, because over the API the decision is yours, and its result does not say the bake is stale. The Lidar Replay panel offers Re-bake and refuses to export against a stale bake.

Surface classes¶

A material's continuation class says what the pulse does after the first surface. This is what separates a window from a puddle: both are see-through, and they behave completely differently at 905 nm.

Class	Behaviour	Typical material
Opaque	The pulse stops	Walls, floors, most objects
Slab (glass)	Passes straight through; the two faces of a pane cancel, so the beam exits parallel	Window glass
Medium (absorbing)	Refracts and stays bent, absorbing along the submerged path	Water
Porous (foliage)	The beam footprint splits: part hits, part passes through a gap	Foliage
Specular (mirror)	Folds about the surface normal onto whatever it points at	Mirrors, polished metal

Two consequences worth expecting:

Water is nearly invisible. Water absorbs at 905 nm within about 15 cm, so a pond surface returns essentially nothing except a near-normal glint, and its bottom fades out quickly with depth. This is correct behaviour, not a missing return.
A mirror produces a phantom. The pulse folds and reports the surface it lands on, booked at the total path length, so the phantom appears behind the mirror, along the original beam. Again, correct.

Because a medium refracts, a submerged surface is also reported further away than it geometrically is: time of flight measures the optical path, which in water is about 33% longer than the depth.

Authoring a material¶

A continuation class is a property of a material. Open the material in the Material Editor and expand LiDAR (NIR), which is collapsed by default. These channels are separate from the visible PBR ones: a surface that looks like glass to the eye is still opaque to the sensor until its NIR channels say otherwise. The same channels drive the depth sensor models.

Channel	Range	What it sets
NIR reflectivity	0 to 1	Diffuse reflectivity at 905 nm. Unauthored materials fall back to the brightness of the visible albedo
Retroreflective	0 to 1	Blend toward a retroreflective response, which returns far more than a matte surface near head-on
Interface F0 (905nm)	0 to 1	Specular reflectance at normal incidence. For metals this differs from the visible colour
NIR transmission	0 to 1	Fraction carried onward. For Porous this is the chance a beam finds a gap. `0`, the default, is opaque
Absorption (1/m)	0 to 100	Absorption inside the medium beyond the surface. Used only for Medium
Continuation	class	Which of the five classes above applies

The Glass section does not affect the sensors

The Glass section of a transparent material (IOR, Attenuation, Dispersion, Tint) controls only how the surface looks when rendered. Lidar and depth sensors read only the LiDAR (NIR) channels, so a window needs both.

Material class (Apply preset...) applies a preset in one click and then shows Applied: with its name. Each preset sets every channel the model reads, including roughness and metalness: a glass material left at an opaque default roughness produces no glint and behaves nothing like glass. Hover a preset in the list for a summary.

Preset	Class	What to expect
Diffuse (matte)	Opaque	Ordinary concrete, plaster, painted wall
Retroreflector	Opaque	Sign, plate or hi-vis tape: hundreds of times brighter than a white wall, falling off with entrance angle
Clear float glass	Slab	About 71% passes through. Bright glint within a couple of degrees of normal, near-total dropout elsewhere, plus a secondary off whatever is behind
Tinted / solar glass	Slab	Absorbs most of the NIR, about 28% through, so the secondary behind it is much weaker than clear glass
Low-E / coated glass	Slab	Clear to the eye, a mirror at 905 nm. Around 1% gets through
Mirror	Specular	No pass-through. The pulse folds about the normal and returns off a third surface, as a phantom behind the glass
Polished metal	Specular	Machined aluminium or stainless: strong glint near normal, dropout elsewhere, same folded phantom
Still water	Medium	A void off-normal. Absorbed within about 15 cm, so a bottom return survives only in very shallow water
Wet / specular ground	Opaque	Wet asphalt: the water film throws energy forward, cutting the return compared with dry road
Vegetation canopy	Porous	Bright in NIR. Gaps let a sparse subset of pulses through to the ground behind
Dark absorptive	Opaque	Soot, black rubber, tar paper: weak return with range-limited dropout
Gloss dark paint	Opaque	Black car clearcoat: the classic 905 nm worst case, very dark under a glossy coat

Metal preset (Set from metal...) is the narrower tool: it sets Interface F0 (905nm) for a chosen metal and sets the material's metalness to 1, leaving everything else alone. Metals reflect differently at 905 nm than in visible light, so the values differ from the base colour: Aluminium 0.90, Silver 0.97, Steel 0.60, Copper 0.96, Gold 0.97, Chrome 0.62.

MuJoCo geom materials save to the scene, not to an asset

A material with its own asset file stores these channels in that file. Materials that come from a MuJoCo model own no file, because they are rebuilt from the model on every load, so their NIR authoring is kept in the scene instead, keyed by material name. The section then reads Saved with the scene, as an override for, followed by the material's name. Save the scene after authoring one, or the work is discarded on reload.

Editing any of these channels makes an existing bake stale. The section reminds you with Re-bake the lidar material LUT after editing. Re-bake before scanning, as described under Material bakes.

Secondary returns¶

A beam that passes through glass, water or foliage, or folds off a mirror, can report a second, weaker hit behind the first. Two things have to line up for one to appear.

The recording has to capture the geometry. This is opt-in per sensor, on the binding, because each captured ray adds five columns per beam:

Setting	Values	Meaning
Secondary Capture	Off, Pass-through, Reflected, Both	Which extra ray to record: straight on past the first hit, folded about the surface normal, or both. Off is the default
Secondary Scope	Flagged materials only, All surfaces	Whether to cast only where the hit surface's material could need it, or at every hit. Shown when capture is not Off

Flagged materials only casts a pass-through ray where the hit surface's continuation class is Slab, Medium or Porous, and a reflected ray where it is Specular. It also casts where the surface's MuJoCo material is translucent (pass-through) or has a non-zero reflectance (reflected). All surfaces is much slower. Use it only if you may later re-author an opaque material as glass and want the geometry already recorded.

The sensor has to be able to report one. maxReturns reflects real hardware. The Unitree L1 and Livox Mid-360 are single-echo and keep only the stronger of the two, which may well be the ghost, since a window's reflection of the wall behind it can be brighter than the pane's own off-normal return. The Ouster OS1 reports two.

Set capture in the Lidar tab, or from script and gRPC with SetLidarSecondaryCapture. Changes are ignored while a recording is in progress, because the file's columns are fixed when it opens. The captured geometry is what the Lidar Replay panel uses when it exports secondaries. Live secondaries from GetLidarScanSecondary are computed from the material bake instead, and are all -1 unless maxReturns is at least 2.

Previewing in the viewport¶

The viewport can draw a sensor's returns as coloured points, so a lidar's placement and coverage can be checked before recording. The preview is an editor overlay only. It never appears in recorded video or in gRPC camera frames.

Turn it on with Lidar View, under Debug in the viewport toolbar's Overlay menu (the Overlay options arrow beside the overlays toggle), or as Lidar view on the Lidar Replay panel's Display tab. Both edit the same scene setting, which is saved with the scene. The points are one of the viewport's overlays, so Hide Overlays on the overlays toggle hides them too.

Lidar View	What the viewport shows
None	Nothing. The default, until a robot pack with a lidar is placed in the scene
Live Preview	The sensor's live returns while the scene plays
Replay	Recorded returns, played by the Lidar Replay panel while it is open
Auto (Play=Live, Edit=Replay)	Live preview in Play, replay in Edit

The preview needs the sensor to be firing

Live preview draws the same in-memory scan the recorder produces, so points appear only while the lidar is firing: during a recording, or after SetLidarLive(true), and in both cases only with Record Lidar on.

Colouring the points¶

Colour by, Ramp, Shape and Marker size are on the Lidar Replay panel's Display tab, and apply to both live preview and replay. Colour by selects the value each point is tinted by:

Colour by	Points coloured by
Channel	Laser row (ring) index. A single colour for Livox, which has one channel
Range (distance)	Distance from the sensor. The default
Height	World height of the hit
Azimuth	Horizontal angle around the sensor
Sample order	The beam's emission order
Solid	A single flat green
Incidence	Angle between the beam and the hit surface
Intensity	Return-model intensity. Includes the inverse-square falloff, so it saturates close in
NIR reflectivity	Baked near-infrared reflectivity
Metal reflectance	Baked metal reflectance
Dropout prob	Return-model drop probability
Calibrated reflectivity	Range-compensated return strength on a logarithmic scale, spanning about four decades. The one to use for judging materials

The material modes work only in Replay

Five modes, Intensity, NIR reflectivity, Metal reflectance, Dropout prob and Calibrated reflectivity, read the return model and a material bake. They show real values only in Replay, with Preview noise profile in viewport on in the Lidar Replay panel and a bake loaded there. Otherwise, and always in Live Preview, they draw the flat green of Solid, so an unconfigured material read is unmistakable. Where the bake has no material for a hit, the point is drawn dim grey.

Ramp maps the scalar modes onto a colour gradient: Warm-Green-Blue, Grayscale, Heat, or Warm Red-Yellow (the default).

Shape selects the marker drawn at each hit:

Shape	Marker
Square	Camera-facing square. The default
Circle	Camera-facing circle
Ring	Ring facing the sensor, the beam's footprint
Ray to source	A short line from the hit back toward the sensor
Ray + normal	Ray to source, plus a line along the surface normal
Surface ring	Ring lying flat on the hit surface
Surface ring + ray	Surface ring plus the ray to source
Ring + ray	Beam-facing ring plus the ray to source
Surface square	Filled square on the hit surface

Marker size is the world-space size of each marker in metres, a square or circle radius or a ray length. It defaults to 0.035 and ranges from 0.002 to 2.0.

Reading scans from C#¶

The lidar API lives on MujocoSceneComponent. Sensors are addressed by 0-based index in binding order.

using Hazel;

public class LidarReader : Entity
{
    private MujocoSceneComponent m_Scene;
    private float[] m_Ranges;

    protected override void OnCreate()
    {
        m_Scene = GetComponent<MujocoSceneComponent>();

        // Scans are only produced during recording unless the sensor is set live.
        m_Scene.SetLidarLive(true);

        m_Ranges = new float[m_Scene.GetLidarBeamCount(0)];
    }

    protected override void OnUpdate(float ts)
    {
        int beams = m_Scene.GetLidarScan(0, m_Ranges);

        for (int i = 0; i < beams; i++)
        {
            if (m_Ranges[i] < 0.0f)
                continue;   // this beam returned nothing

            // m_Ranges[i] is metres along the beam.
        }
    }
}

Member	What it does
`GetLidarBeamCount(sensorIndex)`	Beam count for a sensor: `Channels * AzimuthBins`
`GetLidarScan(sensorIndex, outRanges, material = false)`	Fills `outRanges` with metres per beam; returns the count written
`GetLidarScanChannels(sensorIndex, outRanges, outIntensity, outReflectivityCalibrated, material = true)`	As above, plus per-beam return strength. Either channel array may be null; one shorter than `outRanges` makes the call return 0
`GetLidarScanSecondary(sensorIndex, outSecondary)`	Secondary ranges behind the first hit; requires a bake, and all `-1` when `maxReturns` is below 2
`GetLidarBeamAngles(sensorIndex, outAz, outEl)`	Per-beam azimuth and elevation, radians. Both arrays must be the same length
`SetLidarLive(live)`	Fire the lidar every step even when not recording
`SetLidarConfig(...)`	Rewrite a sensor's scan configuration for the running scene
`GetLidarConfig(sensorIndex, outInts, outFloats)`	Read a sensor's scan configuration back
`SetLidarSecondaryCapture(sensorIndex, capture, scope)`	Choose which extra rays the recording captures
`SetLidarPreset(preset)`	Apply a return model calibration by index
`SetLidarReturnParams(...)`	Set the nine base return model parameters
`SetLidarReturnParamsEx(...)`	Set all seventeen return model parameters
`GetLidarReturnParams(outParams)`	Read them back; returns how many were written
`BakeLidarMaterial(cellSize)`	Bake the scene's NIR materials from its visual meshes
`BakeLidarMaterialEx(cellSize, maxDim, splatRadius, includeHidden)`	The same bake with every setting exposed. Each argument falls back to the scene's setting when left at `0`
`GetLidarBakeStatus(outInts, outLongs, out cellSize)`	Read the loaded bake's identity, including whether it is stale
`SaveLidarBake(name)` / `LoadLidarBake(name)`	Persist and restore a bake

The full member list is in the MujocoSceneComponent API reference.

GetLidarConfig unpacks into two caller-supplied arrays:

outInts   = [scanPattern, channels, azimuthBins, geomGroupMask, selfIgnore,
             secondaryCapture, secondaryScope]
outFloats = [vfovMin, vfovMax, minRange, maxRange, rotationHz]

The call is capacity-checked: it writes at most as many values as the arrays hold, so a shorter array receives the leading fields. Size the int array for 7 to read the secondary-capture settings. It returns false when there is no sensor at that index.

GetLidarBakeStatus unpacks the same way, into two arrays and an out parameter:

outInts   = [valid, formatVersion, splatRadius, dimX, dimY, dimZ]       // int[],   at least 6
outLongs  = [filledCells, geomMaterials, materialFingerprint, stale]    // ulong[], at least 4
cellSize                                                                // out float

It returns false if either array is undersized, or when no bake is loaded. stale is 1 when the bake no longer matches the scene's materials, or was written by an older version.

Things to know

SetLidarLive(true) is required to read scans outside recording. Without it a scan is only produced while the recorder runs. It needs Enable MuJoCo Observer and Record Lidar on, and calling it while the recorder is idle arms the recording session, so the Observer panel shows it as armed. The recorder loads the lidar bindings when it arms, so until then GetLidarBeamCount returns 0 and GetLidarConfig returns false. Call SetLidarLive(true) first, as in the example.
SetLidarConfig applies to the running scene only. It is not written back to the scene's Recorder settings, and arming a new recording session reloads the configuration from the scene. It is ignored while recording.
SetLidarConfig can change the beam count. Re-query GetLidarBeamCount and resize the destination array after any configuration change.
SetLidarSecondaryCapture is kept across sessions. Unlike SetLidarConfig, it writes back to the running scene's Recorder settings, so arming a new recording session keeps it. Like other changes made during Play, it is discarded when the scene stops.
The return model and the bake start fresh. Neither is saved with the scene, so set the calibration and call LoadLidarBake when the scene starts playing.

Recorded output¶

The recorder does not write a point cloud. It writes a row per beam, as Parquet, one file per episode:

DataSessions/session_<date>_<time>/data/chunk-NNN/lidar-<Name>-NNN.parquet

DataSessions is in the project folder, unless an Output Directory is set on the Recorder page's General tab. The full session layout is described in Dataset format.

Column	Type	Meaning
`timestamp`	float64	Seconds since the start of the episode. Every ray cast in one tick shares a timestamp
`bin`	uint16	Azimuth bin, or the high part of the emission index for Livox
`channel`	uint16	Vertical channel, or the low part of the emission index for Livox
`range`	float32	Metres along the beam; `-1.0` on a miss
`geom_id`	int32	MuJoCo geom hit; `-1` on a miss
`hit`	uint8	1 when the beam returned inside the configured range
`normal_x`, `normal_y`, `normal_z`	float32	Surface normal in the sensor frame; 0 on a miss

A beam counts as a hit when geom_id >= 0 and the range falls inside [Min Range, Max Range).

When secondary capture is on, the schema gains five columns per captured ray. Pass-through adds:

Column	Type	Meaning
`sec_range`	float32	Metres from the sensor to the next surface past the first hit; `-1` = none
`sec_geom_id`	int32	MuJoCo geom of that surface; `-1` = none
`sec_normal_x/y/z`	float32	That surface's normal, sensor frame

and Reflected adds refl_range (metres from the first hit, along the folded direction), refl_geom_id and refl_normal_x/y/z. A phantom is booked at range + refl_range.

These columns are material-agnostic geometry: the recording never applies a material model, so the same capture serves any later re-tuning. The sidecar records which capture mode produced the file.

Each Parquet file has a JSON sidecar with the same name, recording the sensor configuration that produced it: the beam layout, ranges, FOV, mount offset, self-ignore mode, scan pattern and a description of each column. For Livox sensors the sidecar also spells out that bin and channel are parts of the emission index rather than angles. With Record Video Proxy on, a visualisation .mp4 with the same name sits beside them.

The Lidar Replay panel¶

The Lidar Replay panel loads a recorded session from disk and plays it back in the viewport. It is the tool for reviewing captured scans: scrubbing through an episode, inspecting per-beam metrics, baking scene materials, and exporting material-informed variants of a recording. Open it from View → Lidar Replay.

It is covered in full on its own page, The Lidar Replay panel, including the bake controls, the model self-check, the exported schema, and the augmentation.json provenance manifest.

Over gRPC¶

LidarService exposes the same surface to any language. It is a polled service: unlike camera frames, lidar scans do not ride along on StepResponse. A client calls GetLidarScan when it wants a scan.

lidar = rpc.LidarServiceStub(channel)

lidar.SetLidarLive(pb.SetLidarLiveRequest(live=True))
beams = lidar.GetLidarBeamCount(pb.LidarSensorRequest(sensor=0)).beams

scan = lidar.GetLidarScan(pb.GetLidarScanRequest(sensor=0))
ranges = scan.ranges          # metres per beam; -1.0 means no return

RPC	Purpose
`GetLidarScan`	A scan: `beams`, `ranges[]`, optional `secondary_ranges[]`, `intensity[]`, `reflectivity_calibrated[]`
`GetLidarBeamCount`	Beam count for a sensor
`GetLidarBeamAngles`	Per-beam `azimuth[]` and `elevation[]`, radians
`SetLidarLive`	Fire the lidar every step even when not recording
`SetLidarConfig` / `GetLidarConfig`	Write and read the scan configuration
`SetLidarReturnParams` / `GetLidarReturnParams`	Write and read the return model calibration
`SetLidarSecondaryCapture`	Choose which extra rays the recording captures
`SetLidarPreset`	Apply a return model calibration by index
`BakeLidarMaterial`	Bake NIR materials from the visual meshes
`GetLidarBakeStatus`	Bake state: validity, format version, dimensions, cell count, and whether it is stale
`LoadLidarBake` / `SaveLidarBake`	Persist and restore a bake

GetLidarScanRequest takes sensor (0-based), material, want_secondary, and want_intensity. Each optional array is empty unless its flag was set, and matches the length of ranges when present.

Setting want_intensity adds the two per-beam strength channels the material model computes, the same ones the Lidar Replay export writes per point. It needs material=true to mean anything: without the material model there is no return strength to report, only geometry.

intensity carries the inverse-square falloff and the near-range ramp, so it saturates at close range.
reflectivity_calibrated is range-compensated, in multiples of a perfect matte surface at normal incidence: the surface property, and what lidar vendors call calibrated reflectivity. Prefer it for judging materials.

A -1 in either means the material was not resolvable: no bake, a hole in the bake, or material was false.

A single-echo sensor returns a full array of -1 secondaries

secondary_ranges is sized to ranges whenever want_secondary was set, even when no secondaries were found. With max_returns at 1, the default and the value for the Unitree L1 and Livox Mid-360 presets, every entry comes back -1 however the capture is scoped. A non-empty array is not evidence of a secondary. Read GetLidarReturnParams and check max_returns before concluding that a scene has none.

Enum values over the wire¶

The proto carries these as plain integers, so a client needs the mapping:

Field	Values
`scan_pattern`	`0` spinning uniform, `1` Livox non-repetitive
`self_ignore`	`0` none, `1` mount body, `2` whole robot
`secondary_capture`	`0` off, `1` pass-through, `2` reflected, `3` both
`secondary_scope`	`0` flagged materials only, `1` all surfaces
`include_hidden` (bake)	`0` use the scene setting, `1` include, `2` exclude
`preset` (`SetLidarPreset`)	`0` Generic, `1` Unitree L1, `2` Livox Mid-360, `3` Ouster OS1

Secondary capture is read-only on GetLidarConfig

LidarConfigResponse reports secondary_capture and secondary_scope, but SetLidarConfig cannot set them; use SetLidarSecondaryCapture. SetLidarConfig replaces the whole scan configuration, so keeping capture out of it means a client can only change capture by asking to.

GetLidarBakeStatus reports valid, format_version, cell_size_m, splat_radius, dim_x/y/z, filled_cells, geom_materials (the size of the per-geom material table; 0 means the bake has none), material_fingerprint, and stale. The fingerprint is the identity the export manifest records, so a variant can be traced back to the exact bake that produced it.

Sensors are addressed by 0-based index in binding order. A bad index is rejected rather than silently ignored: SetLidarConfig and SetLidarSecondaryCapture return ok=false ("no lidar sensor at index N"), GetLidarConfig returns found=false, and GetLidarScan and GetLidarBeamCount return zero beams.

The service acts on the registered external agent's entity first, and otherwise on the first entity carrying a MujocoSceneComponent. It reports a missing scene as no MuJoCo scene in the active world.

See Sensors over gRPC for how this fits the wider API. The .proto files ship with the editor.

As a reinforcement learning observation¶

Lidar ranges are available to a task contract as the built-in observation component lidar_ranges:

Parameter	Default	Meaning
`beams`	required	Beam count, `Channels * AzimuthBins`. Emits nothing if absent or not positive
`sensor`	`0`	Which sensor to read
`material`	off	Apply the return model. Turned on by `1` or `true`

The observation width is set by beams, so it must match the sensor's real beam count. Beams beyond the scan are filled with -1. Task contracts are covered on the gRPC API page.

Where to go next¶

RGBD and depth is the other simulated sensor, and shares the NIR materials.
The Lidar Replay panel replays recordings and exports material-informed variants.
gRPC API covers the server, the step loop, and the action gate.
Recording covers the recorder the Parquet output belongs to.