Comparing runs

Fingerprint compatibility

A comparison requires that both runs share an architecture fingerprint. Weight and tokenizer fingerprints may differ — that is what a fine-tuning diff is. If the architecture digests differ, the comparison is rejected with error E3101.

Metrics

• cosine — cosine similarity per (layer, token), then reduced.
• l2 — mean per-token L2 distance, sensitive to magnitude drift.
• cka — Centered Kernel Alignment across batches; ignores per-token rotation.

Layer alignment

NeuronScope aligns layers by canonical name. If two models expose different naming schemes (e.g. a re-implementation), pass an explicit alignment= map.

import neuronscope as ns

a = ns.load("baseline.nsz")
b = ns.load("finetuned.nsz")

report = ns.compare(a, b, metric="cosine")
report.to_html("report.html")

Reporting

Report is a first-class artifact — it has its own fingerprint and can be embedded in a benchmark suite.

ns diff baseline.nsz finetuned.nsz --metric cka --report report.html