NeuronScope
ArchitectureStablesince v0.5.0

Experiment pipeline

The write and read halves of an experiment as a single architecture, with the modules that own each stage.

Audience: ContributorsRead: 9 minEdit on GitHub Source

The pipeline

Interactive pipelinehover · focus · click
laterModulePlanCoreRecorderModuleCollatorModuleSerializerModuleReader
Module
Plan

neuronscope.builder — assembles model, dataset, prompt, and probes into a validated plan.

Write path is left to right; the read path is a separate consumer with no runtime link to the recorder.

Back-pressure

The recorder's ring buffer is bounded. When the collator falls behind, the recorder blocks the dataset iterator rather than dropping frames. This trade-off is deliberate — an artifact with holes is worse than an experiment that takes longer to run.

  • Buffer size is set by recorder.buffer_mb (default 256).
  • A stall longer than recorder.stall_timeout_s raises E2201: collator stalled.

Recovery

The pipeline crashes atomically — either the artifact exists on disk in its final form, or it does not. Partial writes live in a temp file that is unlinked on process exit. On a hard crash the temp file is left behind and is safe to delete by hand.

Related