NeuronScope
ConceptStablesince v0.9.2

Telemetry Serialization & `.nsz` Format

The .nsz columnar telemetry container format — stable schema, self-describing headers, input fingerprints, zero-copy memory mapping, and high-performance streaming I/O.

Audience: AI Engineers & Storage ArchitectsRead: 10 minEdit on GitHub

Format Overview

Telemetry artifacts are the portable currency of NeuronScope. They store distributed execution traces, model activation tensors, prompt variable bindings, vector store retrieval results, evaluation benchmarks, and performance profiling data.

The .nsz (NeuronScope Zip) format is a versioned, columnar binary container engineered for high-throughput AI telemetry I/O. It guarantees byte-level reproducibility, self-describing metadata schemas, zero-copy memory mapping, and streaming access over object stores like S3 or GCS.

Why a Columnar Container Format

Traditional AI logging tools dump unstructured JSON logs or raw Python pickle objects. Pickling poses security vulnerabilities and breaks across library versions, while pure JSON cannot efficiently store multi-gigabyte neural network layer activation tensors.

NeuronScope solves this by combining structured JSON metadata with uncompressed/compressed raw binary tensor columns in an open Zip container:

  • Content-Addressed Provenance: The header embeds BLAKE3 digests for all inputs (fp:model, fp:dataset, fp:prompt, fp:env).
  • Columnar Tensor Slicing: Telemetry streams are stored in contiguous binary columns. Inspecting a single step or layer is an O(1) offset range-read rather than deserializing the whole file.
  • Zero-Copy Memory Mapping: Uncompressed float32/float16 binary streams inside the archive can be directly memory-mapped into NumPy or PyTorch tensors without heap allocations.
  • Open Standard: The container uses standard PKZip layout. Any zip utility can inspect the JSON headers and telemetry manifest without proprietary tools.

On-Disk Container Layout

Every .nsz file is structured as a directory tree inside an optimized Zip archive:

text
run-001.nsz
├── header.json           # Schema version, input fingerprints, tensor offset index
├── plan.json             # Frozen execution & observation plan
├── env.json              # Captured runtime environment snapshot (hardware, seeds, packages)
├── evaluation.json       # Benchmark scores (hallucination, accuracy, latency)
├── telemetry/
│   ├── traces.jsonl       # Distributed execution spans, agent tool calls, and RAG hops
│   ├── layer_0_activations.f32   # Raw binary float32 tensor column
│   └── layer_15_activations.f32  # Raw binary float32 tensor column
└── SIGNATURE             # Cryptographic BLAKE3 checksum over the sealed container

Binary Stream Specification

Tensor streams inside the telemetry/ subdirectory are stored as raw C-contiguous memory blobs matching IEEE 754 precision formats:

  • .f32 — Standard 32-bit single precision floats.
  • .f16 — 16-bit half precision floats.
  • .bf16 — Brain Floating Point 16-bit.
  • .i64 — 64-bit signed integers (typically used for token IDs).

Because byte order is standardized to Little-Endian across modern hardware (x86_64 and ARM64), tensors can be directly cast into native RAM buffers without byte swapping.

Schema Compatibility Contract

Zero-Copy & Memory Mapping

Both the Python SDK and Rust crate leverage memory mapping (mmap) for reading .nsz files. When querying a specific model layer from a 10GB artifact, only the requested byte range is paged into virtual memory by the OS kernel:

Python Lazy Read
python
import neuronscope as ns

# 1. Observe and record run into Scope
with ns.Scope("production_eval") as scope:
    # Model execution, retrieval, and evaluation steps
    output = model.generate(prompt)

# 2. Seal scope to immutable .nsz artifact
artifact_path = scope.save("production_eval.nsz")

# 3. Read back artifact header without uncompressing full file
artifact = ns.load_artifact(artifact_path)
print(f"Artifact Version: {artifact.version}")
print(f"Full Fingerprint: {artifact.fingerprint.full}")
print(f"Layers Recorded:  {len(artifact.layers)}")

# 4. Lazy tensor read — only reads layer 15 activations into memory
layer_15 = artifact.get_layer("model.layers.15").read_tensor()
print(f"Layer 15 Tensor Shape: {layer_15.shape}, Dtype: {layer_15.dtype}")

Streaming I/O & Cloud Storage

NeuronScope supports direct cloud streaming to AWS S3, Google Cloud Storage (GCS), and Azure Blob Storage without writing temporary files to local disk.

Using HTTP Range Requests, the Rust SDK or CLI can fetch just the 2KB header.json file from an S3 bucket to inspect quality metrics or fingerprints without downloading the multi-gigabyte activation payload.

Sealing & Cryptographic Verification

Once an experiment finishes recording, the scope invokes scope.save() which seals the artifact. Sealing computes a root BLAKE3 hash over all internal streams and writes it to SIGNATURE.

Any subsequent modification to the artifact's metadata or binary columns invalidates the cryptographic signature, preventing accidental tampering or silent data corruption in production model registries.

Related