NeuronScope
TutorialStablesince v0.6.0

Activation basics

What a probe actually captures, why intermediate tensors are not the same as attention weights, and how NeuronScope names layers across frameworks.

Audience: New usersRead: 12 minEdit on GitHub

What is an activation

An activation is the output tensor of an internal module for a given input batch. It is not an attention weight, not a gradient, and not a hidden state variance — those are distinct probe kinds and record separately.

Layer selectors

Selectors are glob patterns matched against the framework's canonical module path.encoder.layer.*.output matches every encoder block's output; **.attn matches every attention submodule regardless of depth.

probes.py·
·python
probes = [
    ns.probes.Activations("encoder.layer.*.output"),
    ns.probes.Attention("**.self_attn"),
]

Shapes and dtypes

  • Activations are recorded in the model's native dtype (usually float16 or bfloat16).
  • Shape is [batch, tokens, hidden] for transformer blocks; the recorder does not reshape.
  • Casting to float32 at record time is opt-in via Activations(..., cast="float32").

Common pitfalls

  • Recording every layer of a 70B model is rarely what you want — start with a stride.
  • Attention outputs are post-projection; use Attention for pre-projection weights.
Related