NeuronScope
ConceptStablesince v0.4.0

Model & Agent Observation

How NeuronScope observes models, agents, and pipelines without modifying core code — non-invasive wrapping, layer/step selection, tool hooks, and lifecycle contracts.

Audience: AI Engineers & ResearchersRead: 10 minEdit on GitHub

Observation Philosophy

Modern AI applications are no longer powered by a single model. They consist of Large Language Models (LLMs), Small Language Models (SLMs), embedding models, vector databases, retrieval systems, APIs, workflows, and autonomous agents.

NeuronScope never owns or modifies your core model or agent code. It attaches a non-invasive, zero-overhead observation layer that records intermediate tensors, prompt variables, tool execution payloads, and decision paths with complete fidelity.

Wrapping Models & Agents

Observation operates at every layer of the intelligent system stack:

  • Foundation & LLM Models: Observe attention heads, layer activations, hidden states, and token log probabilities.
  • Retrieval Systems & Vector Stores: Record embedding vectors, similarity query scores, and retrieved document snippets.
  • Autonomous AI Agents: Track step reasoning, tool selection arguments, API responses, and memory lookups.
  • Multi-Agent Systems: Trace message routing, context propagation, and agent-to-agent negotiation loops.
observe_model_agent.py
python
import neuronscope as ns
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2--chat-hf")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2--chat-hf")

# Wrap PyTorch model with NeuronScope — non-invasive
ns_model = ns.Model.wrap(model, tokenizer=tokenizer)

print(f"Model Name:        {ns_model.name}")
print(f"Weight Fingerprint:{ns_model.fingerprint()}")
print(f"Total Parameters:  {ns_model.parameter_count:,}")
print(f"Layer Count:       {len(ns_model.layers)}")

Layer & Subsystem Selection

Layer and module selectors use expressive canonical path patterns. Subscriptions are resolved against the model's structural hierarchy:

  • "model.layers.*" — Observe every transformer layer.
  • "model.layers.[0,15,31]" — Observe a pinned subset of layers (e.g. initial, middle, and final layer).
  • "**.self_attn.o_proj" — Target self-attention projections across all blocks.
  • "agent.tools.*" — Monitor all tool execution payloads and return values.

Layer Activations & Probes

Probes are specialized listeners attached to specified layers during forward passes:

  • ns.probes.LayerActivations — Captures full tensor arrays for layer outputs.
  • ns.probes.LayerStats — Collects low-overhead summary statistics (mean, std, min, max, L2 norm) without saving full tensors.
  • ns.probes.AttentionMaps — Extracts multi-head attention weight matrices across sequence tokens.

Activation Patching & Steering

For interpretability research and safety alignment, NeuronScope supports Activation Patching (causal interventions):

  • By attaching a writable hook (writable=True), researchers can override a specific layer's activation tensor during the forward pass with values from a clean baseline run.
  • This enables measuring causal effects of individual layers or attention heads on model outputs without altering model checkpoint weights.

Agent & Multi-Agent Tracing

When multiple agents collaborate to solve complex tasks, NeuronScope maintains distributed execution traces. Each agent interaction receives a parent-child span identifier, capturing context propagation, shared memory access, and tool invocation latency.

Exception Isolation & Overhead Protection

Lifecycle & Cleanup

Scope.close() or context manager exit automatically removes all registered framework hooks and tool listeners in a finally block. The model and agent objects are restored to their exact initial state.

Related