The Lidar Replay panel¶
The Lidar Replay panel loads a recorded session from disk and plays it back in the viewport. It is the tool for reviewing captured scans: scrubbing an episode, inspecting per-beam metrics, baking scene materials, and exporting material-informed variants of a recording.
The panel never modifies a recording. A clean capture is left untouched, and every variant is written alongside it, so the same recording can be re-processed with different materials or sensor calibrations.
In short
Open View → Lidar Replay, pick a session and an episode on the General tab, and
press Play. To produce a noisy, material-informed copy of the recording, bake the
scene's materials on the Noise Profile tab, tune the return model, and click
Export material-informed parquet. Each export folder carries an
augmentation.json that records exactly how it was made.
Opening the panel¶
Open it from View → Lidar Replay. It opens as a tab in the Main zone, can be dragged to any other dock zone, and is not part of the built-in layouts. See Layouts and docking.
The panel is a transport bar above four tabs: General, Display, Noise Profile, and Metrics. Replayed points are drawn in the viewport only while the panel is open and the viewport shows its overlays.
Loading a session¶
The General tab picks what plays. The session dropdown lists the recordings in the
project's DataSessions folder, newest first, and shows No sessions in DataSessions/
when there are none. Sessions recorded to a custom Output Directory on the Recorder
page do not appear in the list.
The episode dropdown picks one episode within the session, and switching episodes reloads just that episode. There is no free-text path field and no separate Load button: selecting an entry loads it. The newest session loads automatically, paused, the first time the panel opens. A Loaded: line confirms what is loaded.
The Sensors list shows every lidar stream in the loaded episode, each with its ray, channel and bin counts and a checkbox to show or hide it. Hiding a sensor also removes it from the Metrics tab.
Where a replayed sensor sits
A sensor follows its recorded pose when the session captured one. When it did not, the panel shows No recorded body pose - anchoring replay to the attached entity's live transform., and the sensor anchors to the attached entity's live transform instead. A replay in a scene whose robot has since moved draws the scan from the robot's current position, not the one it was recorded at.
Transport¶
Play and Pause, a Loop toggle (on by default), a Speed slider from 0.1x to
4.0x, and a time scrubber. Playback runs in real time scaled by Speed, not frame by
frame, so the point density at 0.1x matches what the sensor actually delivered.
Display¶
The Display tab carries the point-drawing controls: Lidar view, Colour by (twelve modes), Ramp, Shape and Marker size, described under Previewing in the viewport. They are scene settings, saved with the scene, and they also apply to the live preview during Play. Lidar view is the same setting as Lidar View in the viewport toolbar's Overlay menu.
Below them are the replay window controls, shown once a session is loaded:
| Control | Default | What it does |
|---|---|---|
| Window back (s) | 0.1 |
How much recorded time behind the playhead is drawn at once |
| Window fwd (s) | 0.0 |
Time ahead of the playhead to draw. 0 shows a trailing window only |
| Max points | 50000 |
Per-sensor draw budget, 1000 to 90000. Excess points are thinned out evenly |
A readout below shows how many of the window's points were drawn. Setting both window values to their maximum draws a whole episode at once, which is useful for seeing accumulated coverage and the quickest way to spot a sensor that stopped returning partway through.
Metrics¶
The Metrics tab renders a range image for the frame at the playhead: azimuth across the full 360 degrees on the X axis, elevation on the Y, coloured by one metric. Colours scale to the data in the current frame, from blue for low to red for high, and black means no return. A histogram above, Range this frame (scrub), shows the distribution of ranges in the same frame.
| Metric | Shows |
|---|---|
| Range (distance) | The recorded distance |
| Incidence angle | The recorded incidence |
| Intensity | Return-model intensity |
| NIR reflectivity | Baked reflectivity |
| Dropout prob | Return-model drop probability |
The last three need a material bake, but not the noise profile preview. When a material variant is selected instead of the clean recording, the tab shows the variant, the clean reference, and their difference side by side, with the difference labelled diff (magenta = dropped). This is the fastest way to see what a model change actually did.
Two separate Colour by controls
The Display tab and the Metrics tab both have a control labelled Colour by, and they are independent. The Display one has twelve modes and tints the 3D points in the viewport. The Metrics one has five and tints the 2D range image. They share several names, so they look like one setting, but changing one does not affect the other.
Noise Profile¶
The Noise Profile tab is the editor's path to the material bake. It bakes the scene's NIR materials, previews the return model live over the replayed hits, and exports a material-informed copy of the recording.
Preview noise profile in viewport runs the return model live over the replayed hits, dropping points, nudging their ranges, and colouring them exactly as an export would. It writes no files. It needs a bake, and it is what enables the material colour modes on the Display tab.
Baking¶
Under Material bake (voxel):
| Control | Default | Range | What it does |
|---|---|---|---|
| Cell size (m) | 0.10 |
0.01 to 5 | Size of each bake cell in metres. Smaller captures finer material detail but uses more memory and takes longer |
| Max grid dim | 512 |
16 to 1024 | Cap on cells per axis. If the scene would exceed it, the cell size grows to fit |
| Splat radius | 1 |
0 to 8 | Spreads each surface sample into neighbouring cells. 0 keeps exact cells |
| Bake hidden meshes | on | Include meshes hidden in the editor, since beams hit them regardless |
These settings are saved with the scene, so they survive an editor restart, and they are
the settings BakeLidarMaterial uses over gRPC for any field left at
zero. A cell size of about 0.1 m is fine for calibration; drop to a few centimetres for a
full data pass.
Leave Splat radius at 1
With a radius of 0, a noticeable share of hits land just outside a filled cell and
find no material. A radius of 1 or 2 closes those small gaps, so a hit just off a
surface still resolves a material.
Bake current scene builds the bake from the scene's visual meshes, using the settings
above. Once a bake exists, Save bake to disk writes it to MaterialBakes/scene.lvxb in
the project folder, and a status line reports the grid dimensions, cell size, and how many
cells came back filled. Load saved bake restores the saved file and is greyed out until
one exists. Show bake voxels (debug) draws the filled cells in the viewport, coloured
by the baked material, so the bake's coverage can be checked by eye.
The panel's bake is its own
The panel keeps its bake separate from the one scripts and gRPC use for live scans. To
use a panel bake from the API, click Save bake to disk, then call
LoadLidarBake("scene").
When a bake goes stale¶
A bake is a snapshot of the materials at the time it was built. The panel treats it as stale, and shows a warning with a Re-bake button, when:
- a material changed after baking: Materials have changed since this bake.
- the bake settings above no longer match the ones it was built with: Bake settings above differ from the loaded bake (built at cell X m, splat N).
- the bake file was written by an older version of the editor.
Exports are refused until you re-bake, with the message The bake no longer matches the scene or the bake settings - re-bake before exporting. An export against a stale bake would write the old materials into the recording without saying so.
The return model¶
LiDAR return / noise model (feeds the export) holds the return model parameters the preview and the export use. Sensor preset loads a calibration for a known unit (Generic, Unitree L1 (Go2), Livox Mid-360, Ouster OS1), and the sliders below adjust it:
| Slider | Range | Parameter on the Lidar page |
|---|---|---|
| Max range @ ref (m) | 1 to 200 | maxRangeAtRef |
| Ref reflectivity | 0.1 to 1 | refReflectivity |
| Range exponent | 0 to 1 | rangeExp |
| Intensity ref range (m) | 0.5 to 20 | intensityRefRange |
| Range noise sigma0 (m) | 0 to 0.2 | noiseSigma0 |
| Grazing cos cutoff | 0 to 0.5 | grazingCosCutoff |
| Detection floor | 0 to 0.2 | detectFloor |
| Near-range R0 (m) | 0 to 3 | nearRangeR0 |
| Noise slope (m per m) | 0 to 0.01 | noiseSlopePerM |
Two plots update as the sliders move: P(return) vs range and P(return) vs incidence angle, each drawn for a set of reference materials, from a white matte surface to gold. angle-plot range (m) sets the distance the second plot is evaluated at.
The remaining optics parameters are not shown here. The export uses their defaults, or the values the chosen Sensor preset sets; Ouster OS1 is the only preset that reports two echoes per beam.
These values are separate from the calibration that scripts and gRPC use for live scans,
and they are not saved with the scene. Each export records the values it used in its
augmentation.json.
Sliders stay live during an export
An export runs in the background while the panel stays interactive. Every row is evaluated with the parameters taken when the export started, and the manifest records that snapshot, not the panel's current values. Moving a slider mid-export does not corrupt the provenance, but it does not affect the running export either.
Model validation¶
Model validation (self-check), collapsed by default, evaluates the model against its own reference points and shows a PASS or FAIL row for each. It answers "is the model still behaving?" after a calibration change, without leaving the editor.
| Check | Asserts |
|---|---|
| Datasheet anchors | Max range at 10% reflectivity lands close to each preset's published figure: Unitree L1 about 10 m, Livox Mid-360 about 40 m, Ouster OS1 about 90 m |
| Dropout with range | Probability of return falls with range and never rises |
| Grazing noise | Range noise grows markedly from normal to grazing incidence |
| Calibrated reflectivity | A matte surface's calibrated reflectivity recovers its authored value |
A FAIL means the current parameters are outside the range the model was calibrated for, usually because a slider was pushed well past a datasheet value, not because the export is broken.
Exporting a material-informed variant¶
Export material-informed parquet writes a new copy of the recording with the model's noisy range, intensity, calibrated reflectivity, materials and dropout applied. The clean recording is never touched.
Until the export can run, the panel lists what is still missing:
- an open scene,
- a recorded session, loaded on the General tab,
- a bake, either freshly built or loaded from disk.
The bake must also not be stale. Then:
- Choose an Export scope: This episode or Whole session.
- Set a Name for the variant. It defaults to
material. - Click Export material-informed parquet. A progress bar shows the episode being written, with a Cancel export button.
Output lands in the session folder:
DataSessions/session_<date>_<time>/materials/<name>/
├── lidar-<Sensor>-<episode>.parquet one per sensor and episode
├── lidar-<Sensor>-<episode>.json copy of the recording's sidecar
└── augmentation.json how the variant was made
If a folder with that name already exists, the export writes to <name>-2, <name>-3 and
so on, so an earlier export is never overwritten. The noise seed is derived from the final
folder name and recorded in augmentation.json. The new variant appears in the Noise
profile selector alongside clean.
Exported schema¶
The export writes its own schema, not the clean recording's:
| Column | Type | Meaning |
|---|---|---|
timestamp |
float64 | Seconds since the start of the episode, as in the clean recording |
bin, channel |
uint16 | Beam address, as in the clean recording |
range |
float32 | Range after the model, with noise and dropout applied |
hit |
uint8 | 1 when the beam survived the model |
geom_id |
int32 | MuJoCo geom hit; -1 when unknown |
normal_x/y/z |
float32 | Surface normal, sensor frame |
range_clean |
float32 | The originally recorded range, kept so the two can be compared |
intensity |
float32 | Return strength, with the inverse-square falloff and the near-range ramp |
reflectivity_calibrated |
float32 | Range-compensated surface property |
nir_reflectivity |
float32 | The baked reflectivity used for this point |
nir_metal |
float32 | The baked metal reflectance used for this point |
return_index |
uint8 | 0 = first surface, 1 = secondary |
The export is long-form; the clean recording is wide-form
A clean recording puts a secondary in extra columns on
the same row (sec_range, refl_range, and so on), one row per beam. The export
instead emits a second row with return_index = 1. The two files therefore do not
have the same row count for the same episode, and code that assumes one row per beam
will double-count secondaries. Filter on return_index == 0 to recover a
first-surface-only cloud.
Very bright retroreflector returns can also add spurious returns in neighbouring beams that
had none of their own, as a saturated real sensor does. These rows have return_index = 1
and geom_id = -1.
A point with -1 materials is a bake hole, not a sensor reading
Where the bake has no material at a hit, the export keeps the point, because the pulse
genuinely returned and the recording measured its range, and marks every material
column -1. Every real value is >= 0, so these points are easy to filter.
augmentation.json¶
Every export folder gets an augmentation.json. Together, seed, model_version,
params and bake reproduce the variant from the unchanged clean recording. The
accounting block explains why any given point is missing.
{
"model_version": 2,
"seed": 1234567,
"source_profile": "clean",
"scope": "session",
"episodes": [0, 1, 2],
"bake": { "format_version": 8, "material_fingerprint": 14418..., "cell_size_m": 0.1000,
"splat_radius": 1, "dims": [412, 96, 388], "filled_cells": 2113402, "geom_materials": 184 },
"secondary_source": "recorded",
"secondary_transmit_min": 0.1500,
"accounting": { "rays": 8640000, "kept": 7982114, "...": "..." },
"params": { "MaxRangeAtRef": 40.0000, "DetectFloor": 0.0300, "...": "..." }
}
| Key | Meaning |
|---|---|
model_version |
Return-model version the export ran. Changes when model behaviour changes |
seed |
Reproduces the exact noise draw |
source_profile |
The noise profile selected when the export ran, usually clean |
scope, episodes |
Whether one episode or the session was exported, and which episodes |
bake |
Identity of the bake used: format version, material fingerprint, cell size, splat radius, dimensions, filled cells, and the size of its per-geom material table |
secondary_source |
How secondaries were resolved; see below |
secondary_transmit_min |
Transmission below which a first surface is treated as opaque for secondaries |
accounting |
What happened to every ray; see The accounting funnel |
params |
The return model parameters the export ran with |
secondary_source decides whether two exports are comparable
It reports recorded, raymarched, or mixed. Secondaries either come from rays the
recording captured with Secondary Capture, or are traced
through the material bake at export time. Two exports with identical seed and
parameters are not reproducible from each other if they resolved their secondaries
by different routes. Check this before treating a pair as a controlled comparison.
bake.splat_radius reports -1 when the bake was made by an older version that did not
record its build settings. The manifest reports the gap rather than guessing a value.
The accounting funnel¶
accounting is a full census of what happened to every ray. It should reconcile, and a
missing point should always be attributable.
| Field | Meaning |
|---|---|
rays |
Rays evaluated |
kept |
Points written |
kept_unknown_material |
Kept, but the bake had no material: the -1 points above |
dropped_max_range |
Beyond the reflectivity-scaled maximum range |
dropped_grazing |
Past the grazing-incidence cutoff |
dropped_detect_floor |
Return strength under the detect floor |
secondaries_recorded / secondaries_marched |
How each secondary was resolved: from the recording, or through the bake |
secondaries_emitted |
Secondaries actually written as rows |
secondary_funnel breaks the secondary path down stage by stage. Most of its fields are
diagnostic. The two to read are merged_dead_zone, secondaries merged with the first
return because the two echoes were closer than minReturnSeparation, and
lost_to_echo_limit, secondaries discarded because a single-echo sensor kept the stronger
first return instead.
Expect most secondaries to disappear at lost_to_echo_limit
On a single-echo sensor, which includes the Generic, Unitree L1 and Livox Mid-360
presets, a secondary that survives the physics never appears as a second row. It is
either discarded, and counted under lost_to_echo_limit, or, when it is the brighter
echo, kept as the beam's only return with return_index = 0. That is correct
behaviour, and the funnel is what distinguishes it from a bake problem. Choose the
Ouster OS1 sensor preset to export two echoes per beam.
secondary_response summarises secondary return strength by path length. Behind an
absorbing medium such as water it should fall as the path gets longer, which makes it a
quick check on an export without re-running anything.
Switching between variants¶
The Noise profile selector, on the General and Metrics tabs, chooses which copy
plays. clean is the original recording; the other entries are the variants exported
here. Switching a variant reloads its scans while keeping the pose from the clean
reference, so the sensor stays put and only the returns change.
Where to go next¶
- Lidar covers bindings, the return model, materials and secondary returns.
- Recording covers how sessions are recorded.
- Dataset format describes the session folders the panel reads.