Reproducibility is a scoped promise, not a wish. NeuronScope defines it precisely so a failure to reproduce is a bug in a specific stage, not a mystery.
The contract
Given the same fingerprinted model, dataset, prompt, and environment, two recordings produce byte-identical artifacts under the same NeuronScope major version. Two artifacts are considered "the same experiment" if and only if their input fingerprints match.
What is guaranteed
- Fingerprints are stable across machines and OSes for a given canonicalizer version.
- The artifact schema is versioned; a reader can decode any artifact from a supported major version without loss.
- Random seeds set through
ns.seed_all(...)propagate to every framework the recorder touches. - The order of observations in an artifact is deterministic — it follows the recording plan, not process scheduling.
What is not
- Non-deterministic ops (e.g.
torch.use_deterministic_algorithms(False)) are not corrected — they will produce different tensor bytes. - Data races in user code are not detected. Fingerprints only prove input identity.
- Framework upgrades across a minor version can change the environment fingerprint. This is intentional; results may legitimately differ.
A reproducible run
- Pin your framework version.
uv.lockor arequirements.txtis enough. - Call
ns.seed_all(0)before touching any random source. - Record with
deterministic=True— this refuses to run if an installed op is known non-deterministic. - Commit the resulting
.nszto a reference location (git-LFS or a cloud bucket). - In CI, re-record and assert
ns.assert_matches(new_artifact, reference).
Common failure modes
- Silent driver upgrade. CUDA 12.1 → 12.4 changes the environment fingerprint. Detected automatically.
- Non-deterministic dataloader. A shuffled dataset with a missing seed will fingerprint differently every run. Use
ns.Dataset.from_iter(..., shuffle_seed=0). - In-place mutation. Mutating a model between fingerprint and record invalidates the fingerprint. Fingerprint immediately before recording.