Comparing AI applications is essential for regression testing, fine-tuning validation, prompt optimization, and CI/CD deployment gating.
Fingerprint & Schema Compatibility
NeuronScope asserts fingerprint compatibility prior to running system comparison. Both runs must share compatible prompt templates or model architecture hashes (fp:model or fp:prompt).
Comparison Metrics & Quality Scores
quality_delta— Evaluation score variance across benchmark datasets.hallucination_rate_diff— Change in hallucination score between runs.latency_step_breakdown— Step-by-step execution latency diff.cosine_similarity— Layer-wise cosine distance between activation tensors.cost_variance— Token consumption and infrastructure cost delta.
Layer & Step Alignment
Layers and execution steps are aligned by canonical path names. Unmatched modules or missing tool calls are flagged as structural breaking changes.
compare_runs.py
python
import neuronscope as ns
baseline = ns.load_artifact("baseline.nsz")
candidate = ns.load_artifact("candidate.nsz")
report = ns.compare(baseline, candidate, metric="cosine")
report.to_markdown("diff_report.md")Generating Comparison Reports
Comparison reports are sealed artifacts containing cryptographic provenance chains suitable for enterprise compliance.
terminal
bash
ns diff baseline.nsz candidate.nsz --metric cosine --report report.html