@nightowlsdev/agent-diagnostics
Pre-built agentsThe pre-built diagnostics crew: a read-only run `inspector` and an approval-gated `bug-reporter`, so a stalled lane and a vague apology become which call failed, whether it recovered, and a report a developer can act on.
What it does
@nightowlsdev/agent-diagnostics is the pre-built diagnostics crew, and the split into two agents is the point (the same FR-049 precedent as agent-seo): audiences and `delegateSlugs` gate AGENTS, never tools within one, so the agent boundary is the only mechanism that keeps a write/egress capability away from a caller's LLM. The `inspector` is danger 0 all the way down (`run_timeline`, `run_failure`, and the read-only `check_issue`), holds zero write and zero egress capability, and is safe to delegate to freely; everything that writes lives on the `bug-reporter` (`report_issue`, danger 2 — mutating, irreversible, egress). The read-only claim is ENFORCED, not merely stated: `createInspector()` declares `builtinTools: { exclude: ["report_issue"] }` (core's per-agent built-in scope, harvested by `defineSwarm` keyed by the agent's own slug), and `diagnosticsSwarmOptions()` emits the SAME scope as deployment configuration so the boundary survives agents rehydrated from the store, where no in-memory `AgentDef` exists to read a declaration off; the two fold most-restrictive-wins, so carrying both can only narrow. The four tools are ENGINE BUILT-INS, not package-authored handles: they read the calling run's tenant/user/run identity off the RequestContext (which is what makes their authorization gate and per-run cap un-forgeable), so the package DECLARES them (`DIAGNOSTICS_TOOL_NAMES`, each manifest's `requiredTools`) and hands the host the exact switches that register them, and `diagnosticsSwarmOptions()` THROWS when no `bugCapture` sink is wired rather than assembling a bug-reporter that looks configured, apologizes to every request, and files nothing. `report_issue` carries INTRINSIC approval (built `intrinsicApproval: true`, attributed `"explicit"`, which no approval mode may downgrade and no permissive host hook can relax), so a human approves every filing — `reporterApprovalRule()` is exported for defence-in-depth at the swarm seam but deliberately NOT attached, because a bundle-attached rule referencing the absent `report_issue` handle makes `createDiagnosticsCrew()` throw and an `AgentSpec.rules` entry would not survive rehydration anyway. Guidance ships as two versioned, tool-less `defineSkill` singletons (`diagnostic-investigation`, the protocol, and `bug-report-writing`, the write-up contract), rendered as non-callable skill instructions and overridable per deployment by name. `run_timeline` is a bounded, redacted projection — the last 200 steps, texts clipped, tool calls reduced to name/id/normalized-outcome, no argument or result values — and the investigation skill requires an indeterminate verdict when the truncated window could hold the deciding evidence.
Install
pnpm add @nightowlsdev/agent-diagnosticsKey exports
- createInspector (read-only, danger-0, freely delegatable) / createBugReporter (approval-gated filer) / createDiagnosticsCrew (BundleDef, delegation wired)
- diagnosticsSwarmOptions (the fail-loud swarm switches: runIntrospection + bugCapture + rehydration-safe read-only scope)
- reporterApprovalRule (the OPTIONAL swarm-seam approval floor — deliberately not attached; see the persistence caveat)
- DIAGNOSTICS_TOOL_NAMES / DIAGNOSTICS_READONLY_TOOL_NAMES (the four declared built-ins + the honest read subset)
- INSPECTOR_MANIFEST / BUG_REPORTER_MANIFEST / DIAGNOSTICS_MANIFESTS (the introspectable manifests)
- DIAGNOSTIC_INVESTIGATION_SKILL / BUG_REPORT_WRITING_SKILL (the two instruction-only guidance handles, overridable by name)
Usage
import { createBugCapture, createInMemoryIssueSink, defineSwarm } from "@nightowlsdev/core";
import { createDiagnosticsCrew, diagnosticsSwarmOptions } from "@nightowlsdev/agent-diagnostics";
// 1. A capture plane — the sink is where filed bugs land (in-memory shown; your tracker in production).
const bugCapture = createBugCapture({ sink: createInMemoryIssueSink() });
// 2. The crew: a read-only inspector + an approval-gated bug-reporter, delegation already wired.
const crew = createDiagnosticsCrew(); // BundleDef — publish with applyBundle, or drop crew.agents in directly.
// 3. diagnosticsSwarmOptions() registers the four engine built-ins (run_timeline / run_failure / check_issue /
// report_issue) AND re-asserts the inspector's read-only scope for a store-rehydrated deployment. It THROWS
// when no bugCapture is wired — a bug-reporter that files nothing fails loud, it does not apologize silently.
const swarm = defineSwarm({
storage,
agents: crew.agents,
models: { allow: ["tier:"], tier },
modelFactory,
cost: { maxSteps: 20, maxCostUsd: 1 },
...diagnosticsSwarmOptions({ bugCapture, release: process.env.RELEASE }),
});
// report_issue carries INTRINSIC approval, so a human approves every filing even without reporterApprovalRule().What it provides
agent-diagnostics is a pre-built crew — a read-only run inspector and an approval-gated bug-reporter — that turns a stalled lane and a vague apology into which call failed, whether it recovered, what class of failure ended the run, and (when it's a real defect) a report a developer can act on. The split into two agents is the security boundary: audiences and delegateSlugs gate AGENTS, never tools within one, so the agent boundary is the only mechanism that keeps the write/egress capability away from a caller's LLM.
When to use it
- You want post-mortem introspection of a failed run — which call failed, whether it recovered, the failure class.
- You want to file real defects to a tracker behind human approval, with dedup so a bug that fires fifty times isn't recorded as one.
- You want a danger-0 inspector you can safely delegate to for live self-diagnosis.
When not to
- You have no capture plane / bugCapture sink — diagnosticsSwarmOptions() THROWS rather than assemble a reporter that files nothing; wire a sink first (or run inspector-only).
- You want to read a run as a service/background identity — the introspection tools authorize by tenant + participation, and a service identity resolves to 'none' and is denied every read.
- You expected argument/result values in the timeline — run_timeline is a bounded, redacted projection (no args, no results) by design.
Alternatives
- Raw event-log queriesYou have direct, authorized access to the event store and want unredacted detail — the inspector deliberately redacts (no args/results) to avoid an exfiltration path into a model's context.
- The issue plane directly (issue-intake)You're filing issues from your own UI/flow, not from an agent that needs the intrinsic-approval + dedup contract report_issue provides.
Strengths
- Enforced read-only boundary, stated twice on purpose: createInspector() declares builtinTools: { exclude: ["report_issue"] } (harvested by defineSwarm keyed by the agent's own slug) AND diagnosticsSwarmOptions() emits the same scope as deployment config so it survives store-rehydrated agents — the two fold most-restrictive-wins.
- The four tools are ENGINE built-ins, not package handles: they read the run's tenant/user/run identity off the RequestContext, which is what makes their authorization gate and per-run cap un-forgeable.
- Fails loud, not silent: diagnosticsSwarmOptions() throws when no bugCapture is wired rather than shipping a reporter that apologizes to every request and files nothing.
- report_issue carries INTRINSIC approval (built intrinsicApproval: true, attributed 'explicit') — no approval mode may downgrade it and no permissive host hook can relax it, so a human approves every filing.
- Dedup with occurrence counting: a known failure comes back {status:'draft', occurrences:N+1} (or 'reopened' on a regression) — no second row, and the persona records the repeat rather than silently dropping it.
- Redacted-by-design timeline: last 200 steps, texts clipped, tool calls reduced to name/id/normalized-outcome, no args or results — and the investigation skill requires an indeterminate verdict when the truncated window could hold the deciding evidence.
Limits & trade-offs
- The tools are engine built-ins the ENGINE grants — the package only declares them; nothing works until the host wires runIntrospection + a bugCapture sink (and diagnosticsSwarmOptions throws without the latter).
- run_timeline is deliberately blind to argument/result values and agentVersion — you diagnose from names, ids, and normalized outcomes, not payloads.
- Introspection authorizes as the FAILING run's user — run the inspector as that same user; a service identity is denied every read.
- reporterApprovalRule() is exported but deliberately NOT attached (it can't ride the bundle and wouldn't survive rehydration) — the durable floor is the tool's intrinsic approval; wire the rule at the swarm seam yourself if you want belt-and-suspenders.
- Truncation can drop the deciding evidence (trajectory.truncated) — the honest answer is sometimes 'indeterminate'.
How it works
createDiagnosticsCrew() returns a closure-validated BundleDef with the inspector and bug-reporter, the reporter's delegation to the inspector closing inside the bundle. The inspector holds run_timeline / run_failure / check_issue (all danger-0 reads) and excludes report_issue via its per-agent built-in scope; the bug-reporter holds report_issue (danger-2: mutating, irreversible, egress). diagnosticsSwarmOptions({ bugCapture, release? }) registers the four engine built-ins, re-asserts the read-only scope for rehydrated agents, and throws when no bugCapture sink is wired. Because the four are engine built-ins that read identity off the RequestContext, the package can only DECLARE them (DIAGNOSTICS_TOOL_NAMES) and must never define a same-named handle — which would shadow the real gated built-in.
Examples
Wire the crew
diagnosticsSwarmOptions registers the built-ins + re-asserts read-only scope; THROWS if bugCapture is missing.
import { createBugCapture, createInMemoryIssueSink, defineSwarm } from "@nightowlsdev/core";
import { createDiagnosticsCrew, diagnosticsSwarmOptions } from "@nightowlsdev/agent-diagnostics";
const bugCapture = createBugCapture({ sink: createInMemoryIssueSink() }); // your tracker sink in production
const crew = createDiagnosticsCrew(); // BundleDef: inspector + bug-reporter, delegation wired
const swarm = defineSwarm({
storage,
agents: crew.agents,
models: { allow: ["tier:"], tier },
modelFactory,
cost: { maxSteps: 20, maxCostUsd: 1 },
...diagnosticsSwarmOptions({ bugCapture, release: process.env.RELEASE }),
});Inspector-only, freely delegatable
danger-0 reads; report_issue excluded by the per-agent built-in scope.
import { createInspector } from "@nightowlsdev/agent-diagnostics";
// Run it as its OWN run on its own lane, as the SAME user as the failing run.
const inspector = createInspector();Defence-in-depth approval rule at the swarm seam
Exported but NOT attached to any agent; the real floor is report_issue's intrinsic approval.
import { reporterApprovalRule } from "@nightowlsdev/agent-diagnostics";
import { defineSwarm } from "@nightowlsdev/core";
// It folds through mostRestrictiveTool, so it can only ever tighten.
const swarm = defineSwarm({ /* …, */ rules: [reporterApprovalRule()] });Doing the parts it doesn't support
- Reading a run's argument/result valuesrun_timeline redacts them by design (an exfiltration path into a model's context). If you have authorized direct access to the event store, query it outside the agent — the inspector is deliberately name/id/outcome only.
- Inspecting as a service/background identityThe introspection tools authorize by tenant compare + the participation tri-state; a service identity resolves to 'none' and is denied. Invoke the inspector as the SAME user as the failing run, on its own lane.
- Filing to a real trackerPass a real sink to createBugCapture({ sink }) (the in-memory sink is for tests). A delivery decorator forwards the row to your external tracker — which is why report_issue counts as egress.
- A persistent approval floor beyond the toolreporterApprovalRule() can't ride the bundle (its handle-less report_issue ref fails defineBundle closure) and wouldn't survive rehydration anyway. The durable guarantee is report_issue's intrinsic approval; add the rule at the swarm seam for defence in depth.
Related
- core — createBugCapture / createInMemoryIssueSink and the engine built-ins (run_timeline / run_failure / check_issue / report_issue) live here — the package only declares them.
- capability-bundles — createDiagnosticsCrew returns a closure-validated BundleDef — publish it with applyBundle.
- approval-modes — How intrinsic approval + DIAGNOSTICS_READONLY_TOOL_NAMES fold with a host's toolApproval.readOnly allowlist.
- agent-audiences — Why the read/write split lives at the AGENT boundary — audiences gate agents, never tools within one.
- issue-intake — The issue plane report_issue files into.