NeuronScope
Interactive diagramBetasince v0.7.0

Serialization flow

How a recorded experiment becomes an .nsz artifact — manifest, columnar frames, chunked payloads, and the final atomic rename.

Audience: ContributorsRead: 4 minEdit on GitHub

The chunk index is the read-side contract: as long as its layout is preserved, older readers can consume newer artifacts.

The pipeline

Interactive pipelinehover · focus · click
InputRecorded runStageManifestStageColumnar encodeCoreChunk indexStageStaged writeOutput.nsz
Input
Recorded run

The in-memory result of a recorder pass: probe outputs, manifest, and profiling metadata.

Manifest and columnar encoding run concurrently; the index waits for both.

Interpretation notes

  • Tensor columns and metadata columns share the same frame layout so a reader can mmap either.
  • Dictionary compression is applied per-column, not per-file — a schema change never invalidates unrelated columns.
  • The atomic rename is the only durability step; everything before it is safe to abandon.
Related