Skip to content
Night Owls.dev
Jump to a page

@nightowlsdev/agent-diagnostics

Pre-built agents

The pre-built diagnostics crew: a read-only run `inspector` and an approval-gated `bug-reporter`, so a stalled lane and a vague apology become which call failed, whether it recovered, and a report a developer can act on.

What it does

@nightowlsdev/agent-diagnostics is the pre-built diagnostics crew, and the split into two agents is the point (the same FR-049 precedent as agent-seo): audiences and `delegateSlugs` gate AGENTS, never tools within one, so the agent boundary is the only mechanism that keeps a write/egress capability away from a caller's LLM. The `inspector` is danger 0 all the way down (`run_timeline`, `run_failure`, and the read-only `check_issue`), holds zero write and zero egress capability, and is safe to delegate to freely; everything that writes lives on the `bug-reporter` (`report_issue`, danger 2 — mutating, irreversible, egress). The read-only claim is ENFORCED, not merely stated: `createInspector()` declares `builtinTools: { exclude: ["report_issue"] }` (core's per-agent built-in scope, harvested by `defineSwarm` keyed by the agent's own slug), and `diagnosticsSwarmOptions()` emits the SAME scope as deployment configuration so the boundary survives agents rehydrated from the store, where no in-memory `AgentDef` exists to read a declaration off; the two fold most-restrictive-wins, so carrying both can only narrow. The four tools are ENGINE BUILT-INS, not package-authored handles: they read the calling run's tenant/user/run identity off the RequestContext (which is what makes their authorization gate and per-run cap un-forgeable), so the package DECLARES them (`DIAGNOSTICS_TOOL_NAMES`, each manifest's `requiredTools`) and hands the host the exact switches that register them, and `diagnosticsSwarmOptions()` THROWS when no `bugCapture` sink is wired rather than assembling a bug-reporter that looks configured, apologizes to every request, and files nothing. `report_issue` carries INTRINSIC approval (built `intrinsicApproval: true`, attributed `"explicit"`, which no approval mode may downgrade and no permissive host hook can relax), so a human approves every filing — `reporterApprovalRule()` is exported for defence-in-depth at the swarm seam but deliberately NOT attached, because a bundle-attached rule referencing the absent `report_issue` handle makes `createDiagnosticsCrew()` throw and an `AgentSpec.rules` entry would not survive rehydration anyway. Guidance ships as two versioned, tool-less `defineSkill` singletons (`diagnostic-investigation`, the protocol, and `bug-report-writing`, the write-up contract), rendered as non-callable skill instructions and overridable per deployment by name. `run_timeline` is a bounded, redacted projection — the last 200 steps, texts clipped, tool calls reduced to name/id/normalized-outcome, no argument or result values — and the investigation skill requires an indeterminate verdict when the truncated window could hold the deciding evidence.

Install

pnpm add @nightowlsdev/agent-diagnostics

Key exports

  • createInspector (read-only, danger-0, freely delegatable) / createBugReporter (approval-gated filer) / createDiagnosticsCrew (BundleDef, delegation wired)
  • diagnosticsSwarmOptions (the fail-loud swarm switches: runIntrospection + bugCapture + rehydration-safe read-only scope)
  • reporterApprovalRule (the OPTIONAL swarm-seam approval floor — deliberately not attached; see the persistence caveat)
  • DIAGNOSTICS_TOOL_NAMES / DIAGNOSTICS_READONLY_TOOL_NAMES (the four declared built-ins + the honest read subset)
  • INSPECTOR_MANIFEST / BUG_REPORTER_MANIFEST / DIAGNOSTICS_MANIFESTS (the introspectable manifests)
  • DIAGNOSTIC_INVESTIGATION_SKILL / BUG_REPORT_WRITING_SKILL (the two instruction-only guidance handles, overridable by name)

Usage

agent-diagnostics.ts
import { createBugCapture, createInMemoryIssueSink, defineSwarm } from "@nightowlsdev/core";
import { createDiagnosticsCrew, diagnosticsSwarmOptions } from "@nightowlsdev/agent-diagnostics";

// 1. A capture plane — the sink is where filed bugs land (in-memory shown; your tracker in production).
const bugCapture = createBugCapture({ sink: createInMemoryIssueSink() });

// 2. The crew: a read-only inspector + an approval-gated bug-reporter, delegation already wired.
const crew = createDiagnosticsCrew(); // BundleDef — publish with applyBundle, or drop crew.agents in directly.

// 3. diagnosticsSwarmOptions() registers the four engine built-ins (run_timeline / run_failure / check_issue /
//    report_issue) AND re-asserts the inspector's read-only scope for a store-rehydrated deployment. It THROWS
//    when no bugCapture is wired — a bug-reporter that files nothing fails loud, it does not apologize silently.
const swarm = defineSwarm({
  storage,
  agents: crew.agents,
  models: { allow: ["tier:"], tier },
  modelFactory,
  cost: { maxSteps: 20, maxCostUsd: 1 },
  ...diagnosticsSwarmOptions({ bugCapture, release: process.env.RELEASE }),
});
// report_issue carries INTRINSIC approval, so a human approves every filing even without reporterApprovalRule().

What it provides

agent-diagnostics is a pre-built crew — a read-only run inspector and an approval-gated bug-reporter — that turns a stalled lane and a vague apology into which call failed, whether it recovered, what class of failure ended the run, and (when it's a real defect) a report a developer can act on. The split into two agents is the security boundary: audiences and delegateSlugs gate AGENTS, never tools within one, so the agent boundary is the only mechanism that keeps the write/egress capability away from a caller's LLM.

When to use it

  • You want post-mortem introspection of a failed run — which call failed, whether it recovered, the failure class.
  • You want to file real defects to a tracker behind human approval, with dedup so a bug that fires fifty times isn't recorded as one.
  • You want a danger-0 inspector you can safely delegate to for live self-diagnosis.

When not to

  • You have no capture plane / bugCapture sink — diagnosticsSwarmOptions() THROWS rather than assemble a reporter that files nothing; wire a sink first (or run inspector-only).
  • You want to read a run as a service/background identity — the introspection tools authorize by tenant + participation, and a service identity resolves to 'none' and is denied every read.
  • You expected argument/result values in the timeline — run_timeline is a bounded, redacted projection (no args, no results) by design.

Alternatives

  • Raw event-log queriesYou have direct, authorized access to the event store and want unredacted detail — the inspector deliberately redacts (no args/results) to avoid an exfiltration path into a model's context.
  • The issue plane directly (issue-intake)You're filing issues from your own UI/flow, not from an agent that needs the intrinsic-approval + dedup contract report_issue provides.

Strengths

  • Enforced read-only boundary, stated twice on purpose: createInspector() declares builtinTools: { exclude: ["report_issue"] } (harvested by defineSwarm keyed by the agent's own slug) AND diagnosticsSwarmOptions() emits the same scope as deployment config so it survives store-rehydrated agents — the two fold most-restrictive-wins.
  • The four tools are ENGINE built-ins, not package handles: they read the run's tenant/user/run identity off the RequestContext, which is what makes their authorization gate and per-run cap un-forgeable.
  • Fails loud, not silent: diagnosticsSwarmOptions() throws when no bugCapture is wired rather than shipping a reporter that apologizes to every request and files nothing.
  • report_issue carries INTRINSIC approval (built intrinsicApproval: true, attributed 'explicit') — no approval mode may downgrade it and no permissive host hook can relax it, so a human approves every filing.
  • Dedup with occurrence counting: a known failure comes back {status:'draft', occurrences:N+1} (or 'reopened' on a regression) — no second row, and the persona records the repeat rather than silently dropping it.
  • Redacted-by-design timeline: last 200 steps, texts clipped, tool calls reduced to name/id/normalized-outcome, no args or results — and the investigation skill requires an indeterminate verdict when the truncated window could hold the deciding evidence.

Limits & trade-offs

  • The tools are engine built-ins the ENGINE grants — the package only declares them; nothing works until the host wires runIntrospection + a bugCapture sink (and diagnosticsSwarmOptions throws without the latter).
  • run_timeline is deliberately blind to argument/result values and agentVersion — you diagnose from names, ids, and normalized outcomes, not payloads.
  • Introspection authorizes as the FAILING run's user — run the inspector as that same user; a service identity is denied every read.
  • reporterApprovalRule() is exported but deliberately NOT attached (it can't ride the bundle and wouldn't survive rehydration) — the durable floor is the tool's intrinsic approval; wire the rule at the swarm seam yourself if you want belt-and-suspenders.
  • Truncation can drop the deciding evidence (trajectory.truncated) — the honest answer is sometimes 'indeterminate'.

How it works

createDiagnosticsCrew() returns a closure-validated BundleDef with the inspector and bug-reporter, the reporter's delegation to the inspector closing inside the bundle. The inspector holds run_timeline / run_failure / check_issue (all danger-0 reads) and excludes report_issue via its per-agent built-in scope; the bug-reporter holds report_issue (danger-2: mutating, irreversible, egress). diagnosticsSwarmOptions({ bugCapture, release? }) registers the four engine built-ins, re-asserts the read-only scope for rehydrated agents, and throws when no bugCapture sink is wired. Because the four are engine built-ins that read identity off the RequestContext, the package can only DECLARE them (DIAGNOSTICS_TOOL_NAMES) and must never define a same-named handle — which would shadow the real gated built-in.

Examples

Wire the crew

diagnosticsSwarmOptions registers the built-ins + re-asserts read-only scope; THROWS if bugCapture is missing.

agent-diagnostics-example-1.ts
import { createBugCapture, createInMemoryIssueSink, defineSwarm } from "@nightowlsdev/core";
import { createDiagnosticsCrew, diagnosticsSwarmOptions } from "@nightowlsdev/agent-diagnostics";

const bugCapture = createBugCapture({ sink: createInMemoryIssueSink() }); // your tracker sink in production
const crew = createDiagnosticsCrew();      // BundleDef: inspector + bug-reporter, delegation wired

const swarm = defineSwarm({
  storage,
  agents: crew.agents,
  models: { allow: ["tier:"], tier },
  modelFactory,
  cost: { maxSteps: 20, maxCostUsd: 1 },
  ...diagnosticsSwarmOptions({ bugCapture, release: process.env.RELEASE }),
});

Inspector-only, freely delegatable

danger-0 reads; report_issue excluded by the per-agent built-in scope.

agent-diagnostics-example-2.ts
import { createInspector } from "@nightowlsdev/agent-diagnostics";

// Run it as its OWN run on its own lane, as the SAME user as the failing run.
const inspector = createInspector();

Defence-in-depth approval rule at the swarm seam

Exported but NOT attached to any agent; the real floor is report_issue's intrinsic approval.

agent-diagnostics-example-3.ts
import { reporterApprovalRule } from "@nightowlsdev/agent-diagnostics";
import { defineSwarm } from "@nightowlsdev/core";

// It folds through mostRestrictiveTool, so it can only ever tighten.
const swarm = defineSwarm({ /* …, */ rules: [reporterApprovalRule()] });

Doing the parts it doesn't support

  • Reading a run's argument/result valuesrun_timeline redacts them by design (an exfiltration path into a model's context). If you have authorized direct access to the event store, query it outside the agent — the inspector is deliberately name/id/outcome only.
  • Inspecting as a service/background identityThe introspection tools authorize by tenant compare + the participation tri-state; a service identity resolves to 'none' and is denied. Invoke the inspector as the SAME user as the failing run, on its own lane.
  • Filing to a real trackerPass a real sink to createBugCapture({ sink }) (the in-memory sink is for tests). A delivery decorator forwards the row to your external tracker — which is why report_issue counts as egress.
  • A persistent approval floor beyond the toolreporterApprovalRule() can't ride the bundle (its handle-less report_issue ref fails defineBundle closure) and wouldn't survive rehydration anyway. The durable guarantee is report_issue's intrinsic approval; add the rule at the swarm seam for defence in depth.

Related

  • corecreateBugCapture / createInMemoryIssueSink and the engine built-ins (run_timeline / run_failure / check_issue / report_issue) live here — the package only declares them.
  • capability-bundlescreateDiagnosticsCrew returns a closure-validated BundleDef — publish it with applyBundle.
  • approval-modesHow intrinsic approval + DIAGNOSTICS_READONLY_TOOL_NAMES fold with a host's toolApproval.readOnly allowlist.
  • agent-audiencesWhy the read/write split lives at the AGENT boundary — audiences gate agents, never tools within one.
  • issue-intakeThe issue plane report_issue files into.