NeuronScope
ConceptStablesince v0.9.2

AI System Observability & Telemetry Sessions

An experiment or observability session is a recorded run — capturing detailed telemetry, distributed traces, metrics, execution logs, and runtime events into sealed .nsz artifacts.

Audience: AI Engineers & SREsRead: 9 minEdit on GitHub

Observability Session Overview

An Observability Session (Experiment) is the atomic telemetry container produced by NeuronScope. Every distributed trace span, layer activation profile, evaluation score, and regression diff is anchored to a session context.

Sessions provide transparent observability across heterogeneous AI components — LLMs, vector search, tool calls, and multi-agent workflows — producing portable, sealed .nsz telemetry artifacts.

Anatomy of an Observability Run

Every session records a complete, self-describing telemetry payload:

  • Inputs & Provenance: Pinned model weight digests, structured prompt versions, vector index hashes, and runtime environment state (fp:env).
  • Execution Plan: Frozen description of observed layers, tool hooks, and active evaluation rules.
  • Telemetry Streams: Distributed trace spans, API latency measurements, token consumption counters, and peak GPU memory metrics.
  • Sealed Artifact: The final .nsz binary archive binding inputs, telemetry outputs, and cryptographic signatures into an immutable package.

5-Stage Execution Lifecycle

Every NeuronScope session transitions through five deterministic states:

  1. Resolved: Models, prompt templates, datasets, and hardware environment are fingerprinted (fp:full).
  2. Planned: The observation plan is validated against overhead budgets and frozen.
  3. Instrumented: Non-invasive hooks and event bus listeners attach to models and AI agent frameworks.
  4. Running: Application code executes; telemetry streams asynchronously into lock-free ring buffers.
  5. Sealed: Telemetry streams are finalized, cryptographically signed, and written to local or cloud storage.

Constructing Sessions

Sessions can be created via Python context managers (ns.Scope) or using the fluent ExperimentBuilder API:

observability_session.py
python
import neuronscope as ns

# Observe an AI application with automatic scope sealing and cleanup
with ns.Scope("rag_service", tags={"environment": "production"}) as scope:
    # Run AI application code
    response = rag_pipeline.query("Explain latency bottlenecks")

# Retrieve telemetry metrics and save sealed artifact
telemetry = scope.get_telemetry()
print(f"Total Latency: {telemetry.latency_ms:.}ms")
print(f"Total Cost:    ${telemetry.cost_usd:.}")
scope.save("production-run-001.nsz")

Distributed Trace Spans & Hops

In complex RAG pipelines or multi-agent decision loops, a single user query spans multiple system boundaries:

  • Query Embedding Span: Captures text vectorization latency and embedding model output shape.
  • Vector Search Hop: Records top-k retrieved document IDs, similarity distance scores, and index search time.
  • Context Assembly Span: Records prompt variable substitution and token count budget usage.
  • Model Generation Span: Records Time-To-First-Token (TTFT), inter-token arrival latency, and total completion tokens.
  • Tool Call Spans: Captures external tool parameters, API response payloads, and execution success status.

Sessions vs Production Monitoring

While single experiments anchor individual evaluation or research runs, production applications group multiple runs into continuous Monitoring Sessions.

Monitoring Sessions aggregate telemetry over time to track SLA success rates, detect performance degradation trends, measure token cost efficiency, and automatically alert on hallucination rate spikes.

Related