A NeuronScope Dataset is a thin wrapper around any Python iterable. The wrapper adds three things: reproducible ordering, a content-addressed fingerprint, and a memory-bounded iterator that plays back the same rows in the same order on every run.
Kinds of dataset
- Materialised. The full dataset fits in memory. Fingerprint is over the canonicalised bytes.
- Chunked. Rows come from a paged source (Parquet, Arrow, HF datasets). Fingerprint is over the schema, the row order, and each chunk's digest.
- Streaming. Rows arrive from a generator whose contents cannot be materialised. Fingerprint is over the schema and the generator identity — never the rows themselves.
datasets.py··python
import neuronscope as ns
rows = [{"text": "hello"}, {"text": "world"}]
ds = ns.Dataset.from_rows(rows)
print(ds.fingerprint())Reproducible ordering
Row order is part of the fingerprint. A dataset built from a Python set or an unordered dict will have a different fingerprint on every process — use an ordered container.