Comparison

Comparison is only defined between artifacts whose model, dataset, and prompt fingerprints match. Anything else is measuring at least two moving parts at once.

Alignment

Before any metric runs, the comparer aligns the two artifacts step by step and layer by layer using the canonical layer id in the manifest. Unmatched layers are surfaced up front, not silently skipped.

import neuronscope as ns

a = ns.load("run_a.nsz")
b = ns.load("run_b.nsz")

diff = ns.compare(a, b)
print(diff.summary())
# aligned: 12/12 layers, 256/256 steps
# max_abs_delta: 3.2e-6 at layer.4.attention

Built-in metrics

• max_abs_delta — per-layer, per-step maximum absolute difference.
• cosine — cosine similarity between flattened activations.
• kl — symmetric KL over softmaxed logits (only for probes that recorded a logit column).
• rank_change — number of top-k tokens whose rank changed.

Interpreting a delta

• If the environment fingerprints differ, expect a delta on every layer — the compiler and BLAS are different inputs.
• If a single layer's delta dominates, start from that layer in the visualization rather than the top of the model.
• The comparer never averages layers with different shapes. A shape change is a hard error, not a warning.