NeuronScope
ArchitectureStablesince v0.5.0

Observability Pipeline Architecture

The 5-stage execution pipeline connecting model forward passes, agent tool hooks, and RAG retrieval events to locked ring buffers and sealed .nsz artifacts.

Audience: System Architects & ContributorsRead: 10 minEdit on GitHub Source

The NeuronScope Observability Pipeline processes live telemetry emitted during model forward passes, agent tool executions, and vector database lookups. It isolates collection logic from user code, guaranteeing zero-copy streaming and deterministic recording.

Pipeline Architecture

Interactive pipelinehover · focus · click
downstreamModulePlan & ResolveCoreRuntime EngineModuleEvent Bus & CollatorModuleColumnar SerializerModuleZero-Copy Reader
Module
Plan & Resolve

Fingerprints inputs (model, prompt, dataset, env) and freezes observation plan.

The write path operates asynchronously in the background; the read path mmaps sealed .nsz files with zero framework dependency.

The 5 Execution Stages

  • 1. Resolve & Plan: Canonicalize inputs (fp:model, fp:prompt, fp:dataset, fp:env) and freeze observation selectors.
  • 2. Non-Invasive Instrumentation: Attach framework-specific hooks and event bus listeners without altering autograd graphs.
  • 3. Asynchronous Ring Streaming: Copy intermediate tensors and tool call spans into lock-free ring buffers.
  • 4. Collation & Indexing: Align telemetry streams by execution step, computing metadata manifests.
  • 5. Atomic Sealing: Compute final container BLAKE3 digest and finalize the .nsz file.

Ring Buffer & Backpressure

To prevent GPU VRAM starvation or out-of-memory crashes, the pipeline streams telemetry into a bounded lock-free ring buffer (buffer_mb=256 by default). If disk I/O slows, backpressure pauses dataset iteration rather than dropping frames, guaranteeing complete telemetry integrity.

Atomic Sealing & Recovery

Related