Deterministic recording with sync points

By default, episodes start at slightly different engine steps from run to run. This is a side effect of a control loop and a recorder ticking at different rates, so their phase offset varies. (The stepping itself stays deterministic; only the boundary phase moves.) For general training data this is desirable: real-world deployments have the same kind of jitter, and learning across that variance produces a more robust policy.

Some workflows need the opposite. Regression tests, determinism proofs, and locked-cadence training all need every episode to start on the same engine step. The Sync Point System provides the beat for that: it emits a sync point on every tick of the time runner it is bound to, and scripts gate BeginEpisode, resets, and end-of-episode on those sync points.

In short

The Sync Point System is enabled in every new scene and fires at 10 Hz on a runner of its own; bind it to another runner for a different drumbeat. On its own the signal changes nothing: a script opts in by reading Hazel.TimeSync and starting episodes, resets and episode ends only on a sync point. Reach for this when episodes must start on the same engine step every run. Skip it to keep the recorder's natural timing variance in the training data.

When to reach for sync points

Sync points are useful when:

• Proving determinism: two runs of the same scene must produce bit-identical episode data.
• A regression test suite requires every episode to start from a known engine state.
• A training target requires episodes to start from a fixed phase relationship between runners.

Sync points are usually wrong when:

• The task is general training data. Real-world deployments have timing jitter between sensors and controllers; letting that variance into the dataset trains a more robust policy.
• The recorder should capture variance in how the control loop behaves across the natural phase offsets of its sensors. Sync points hide that variance by collapsing it to a single fixed phase.

How a sync point works

The Sync Point System fires one sync point on every tick of its runner, the Sync Point TimeRunner, so that runner's frequency is the sync rate. Scripts watch for sync points through the Hazel.TimeSync API and use them as gates on the episode lifecycle:

graph LR
    S1["SYNC"]:::sync -.->|episode N| S2["SYNC"]:::sync -.->|episode N+1| S3["SYNC"]:::sync -.->|episode N+2| S4["SYNC"]:::sync
    classDef sync fill:#f5503d,stroke:#f5503d,color:#ffffff;

Each episode is bounded by sync beats. Because the engine state on every beat is the same fixed phase of the simulation, two runs of the scene reach the same engine state when they reach the same beat. Datasets become comparable across runs.

Picking a drumbeat

The sync rate expresses a particular pattern of alignment between the other runners. The greatest common divisor of the rates being locked together is the natural choice: every sync point lands on a step where all those runners coincide. For a scene running control at 50 Hz and recording at 30 Hz, that is 10 Hz.

Sub-multiples of the GCD give looser patterns that still land on natural alignments. 5 Hz catches every other alignment, 1 Hz catches every tenth, and so on. The right cadence is whichever pattern matches how often episodes should lock.

Avoid frequencies that are not divisors of the relevant rates. A 7 Hz sync rate against 50 Hz and 30 Hz lands on a different phase of each runner at every tick, which defeats the point of using sync to enforce determinism.

Check the rates your scene actually uses

In a new scene, scripts, the data recorder and the video recorder all run on the 50 Hz Robot runner, so any divisor of 50 Hz works, 10 Hz included. If you move the recorders to the 30 Hz Capture runner, or add your own runners, pick a divisor of every rate involved. See The Time Manager for runners.

Scene setup

The Sync Point System is enabled in new scenes. Its settings are on the Subsystems page of the scene settings (Settings → Scene → Subsystems), under Sync Point Subsystem Configuration:

Setting	Default	What it does
Enable Sync Point System	On	Emits a sync point on every tick of the selected runner
Sync Point TimeRunner	See below	The runner whose ticks are sync points. Its frequency is the sync rate
Execution Phase	Acquisition Phase [1st]	The phase in which the sync point fires
Internal Phase Priority	-200	Its order within that phase. Lower runs first

In a new scene, the Sync Point System runs on a 10 Hz runner of its own, SyncPoint, which it adds when Play starts, and it keeps that runner when the scene is saved. The SyncPoint runner is not in the scene's list of runners, so Sync Point TimeRunner shows the scene's first runner, usually Robot [50.00Hz], even though sync points fire at 10 Hz. Choosing a runner in Sync Point TimeRunner binds the system to that runner instead.

Scenes saved by earlier versions can be bound to the 50 Hz Robot runner, and the bundled Welcome scene is. Every 50 Hz step is then a sync point, which is too fine to act as a drumbeat. To set the drumbeat explicitly, and see it in the settings, give the system a runner of its own:

1. Open Settings → Scene → Time Runners and click Add Time Runner. Rename the new runner, for example to Sync, and set its frequency to the drumbeat you picked, such as 10 Hz.
2. Open Settings → Scene → Subsystems. Under Sync Point Subsystem Configuration, make sure Enable Sync Point System is ticked and select the new runner in Sync Point TimeRunner.
3. Leave Execution Phase at Acquisition Phase [1st] and Internal Phase Priority at -200, then save the scene.

With priority -200 in the Acquisition phase, the sync point fires ahead of the script callbacks that read TimeSync later in the same step, so they see a fresh beat. See Phases for the phase order.

To confirm the rate in use, log TimeSync.SyncHz once after Play starts.

The settings are saved in the scene file. An abridged excerpt from a scene set up this way:

SceneSettings:
  TimeRunners:
    - { ID: 0, Name: Robot,   Frequency: 50 }
    - { ID: 1, Name: Capture, Frequency: 30 }
    - { ID: 2, Name: Sync,    Frequency: 10 }  # the drumbeat: a divisor of both runner rates

  UpdateSyncPointSystemSettings:
    Enabled: true
    RunnerID: 2
    RunnerName: Sync
    Frequency: 10
    Priority: -200
    Phase: 0          # Acquisition

The TimeSync API

Three static members on Hazel.TimeSync:

Member	Type	What it returns
IsAtSyncPoint	bool	true only during the engine step on which a sync point fired.
CurrentSyncIndex	ulong	The number of sync points since Play started. It resets to 0 each time Play starts. For "save the index, wait until it changes" patterns.
SyncHz	double	The sync rate. Use it to convert a duration in seconds into an exact integer number of sync ticks. It is 0 when the Sync Point System is disabled.

Pattern: gate BeginEpisode on a sync tick

The most common use. A script that wants every episode to start on the same drumbeat waits for the next sync tick after it started watching, then starts the episode:

using Hazel;
using Hazel.Data;

public class SyncedEpisodeStart : Entity
{
    private bool  m_Begun;
    private ulong m_StartSyncIndex;

    protected override void OnCreate()
    {
        m_Begun          = false;
        m_StartSyncIndex = TimeSync.CurrentSyncIndex;
    }

    protected override void OnUpdate(float ts)
    {
        if (m_Begun)
            return;

        // Begin on the first sync STRICTLY after OnCreate, so episode frame 0
        // lands the same fixed gap after every reset.
        if (!TimeSync.IsAtSyncPoint)
            return;
        if (TimeSync.CurrentSyncIndex <= m_StartSyncIndex)
            return;

        if (!Observer.IsRecording)
            Observer.StartRecording();      // opens the first episode itself
        else if (!Observer.IsEpisodeOpen)
            Observer.BeginEpisode();        // every later episode

        m_Begun = true;
    }
}

Observer.StartRecording() opens the session and its first episode in one call, so the script calls BeginEpisode() only when a session is already recording with no episode open. Calling BeginEpisode() while an episode is open is ignored.

The latched m_StartSyncIndex matters. Without it, if a sync tick happened to fire on the same engine step that ran OnCreate, the episode would begin immediately instead of on the next clean drumbeat. The strictly-greater-than guard keeps it on the next beat.

Pattern: deterministic sync-tick waits

For repeating cycles (run, reset, run again, reset...), ts-accumulating waits drift relative to the sync clock. The bit-exact pattern converts each wait into an integer number of sync ticks and counts on the sync itself:

using System;
using Hazel;

public class SyncCycle : Entity
{
    [Tooltip("How long to hold quiet before resetting.")]
    public float QuietSeconds = 1.0f;

    [Tooltip("How long to settle after a reset before the next run.")]
    public float SettleSeconds = 2.0f;

    private enum Stage { WaitForStart, Quiet, Settle, Done }

    private Stage m_Stage;
    private ulong m_StageStartSyncIndex;
    private ulong m_QuietSyncTicks;
    private ulong m_SettleSyncTicks;

    protected override void OnCreate()
    {
        double syncHz     = Math.Max(1.0, TimeSync.SyncHz);
        m_QuietSyncTicks  = (ulong)Math.Max(1.0, Math.Ceiling(QuietSeconds  * syncHz));
        m_SettleSyncTicks = (ulong)Math.Max(1.0, Math.Ceiling(SettleSeconds * syncHz));
        m_Stage           = Stage.WaitForStart;
    }

    protected override void OnUpdate(float ts)
    {
        if (!TimeSync.IsAtSyncPoint)
            return;

        if (m_Stage == Stage.WaitForStart)
        {
            m_StageStartSyncIndex = TimeSync.CurrentSyncIndex;
            m_Stage = Stage.Quiet;
            return;
        }

        if (m_Stage == Stage.Quiet)
        {
            if (TimeSync.CurrentSyncIndex - m_StageStartSyncIndex < m_QuietSyncTicks)
                return;

            PerformReset();
            m_StageStartSyncIndex = TimeSync.CurrentSyncIndex;
            m_Stage = Stage.Settle;
            return;
        }

        if (m_Stage == Stage.Settle)
        {
            if (TimeSync.CurrentSyncIndex - m_StageStartSyncIndex < m_SettleSyncTicks)
                return;

            BeginNextCycle();
            m_Stage = Stage.Done;
        }
    }

    private void PerformReset()    { /* ... */ }
    private void BeginNextCycle()  { /* ... */ }
}

Each stage transition fires on the exact sync tick that is N drumbeats after the previous stage's start. The gap between reset and the next cycle's first recorded frame is the same every iteration. No ts accumulation, no rounding drift. The Math.Max(1.0, ...) guard keeps the tick counts valid when SyncHz is 0 because the Sync Point System is disabled.

The pattern fits chained reset cycles (quiet wait → MuJoCo reset → settle wait → re-enqueue) where every iteration's frame 0 must land on the same beat.

Pattern: trigger N runner steps after a sync

The sync drumbeat is the coarse rhythm. For finer alignment, count subsequent control-runner updates after each beat and trigger work on a specific one. With OnUpdate on the default 50 Hz Robot runner, "the second update after the sync" lands a fixed 40 ms after the beat, every beat, without a faster sync rate.

using Hazel;

public class PostSyncTrigger : Entity
{
    [Tooltip("How many control steps to wait after each sync beat before firing.")]
    public int StepsAfterSync = 2;

    private ulong m_LastSyncIndex;
    private int   m_StepsSinceSync;
    private bool  m_FiredThisBeat;

    protected override void OnCreate()
    {
        m_LastSyncIndex  = 0;
        m_StepsSinceSync = 0;
        m_FiredThisBeat  = true;   // wait for the first real sync before arming
    }

    protected override void OnUpdate(float ts)
    {
        // A new sync beat resets the per-beat counter and re-arms the trigger.
        if (TimeSync.IsAtSyncPoint && TimeSync.CurrentSyncIndex != m_LastSyncIndex)
        {
            m_LastSyncIndex  = TimeSync.CurrentSyncIndex;
            m_StepsSinceSync = 0;
            m_FiredThisBeat  = false;
            return;
        }

        if (m_FiredThisBeat)
            return;

        m_StepsSinceSync += 1;

        if (m_StepsSinceSync == StepsAfterSync)
        {
            DoTheThing();
            m_FiredThisBeat = true;
        }
    }

    private void DoTheThing() { /* ... */ }
}

The trigger re-arms on every new sync beat and fires exactly once per beat, on the configured step. The exact same engine step fires every iteration, so the determinism guarantee of sync gating carries through to the in-between work.

When to skip sync points entirely

For general training data, start and end episodes whenever the task calls for it and don't gate them on TimeSync. The Sync Point System can stay enabled: on its own, the signal changes nothing in the scene or the recording. The recorder runs at its configured rate, the controller runs at its own, and the phase between them varies slightly across episodes. That variance is what trains a robust policy, so preserve it. See Recording with the Observer for the default path.

Where to go next

• Recording with the Observer covers sessions, episodes and the full Observer API.
• The Time Manager explains runners and phases.
• Dataset Format describes the files a recording session writes.