Client tools. Let an agent act in the browser.
A client tool is a tool the model calls but your browser runs: frame the camera on an entity, read the current selection, trigger an upload, apply a change to the live scene. The agent decides, your code performs the side effect, and the suspended run waits for the result. The seam shipped earlier (FR-002); as of @nightowlsdev/react@2.8.0 you register these tools with one prop on <SwarmChat>, no raw-hook rebuild.
An ergonomics unlock, not a new capability. The client-tool mechanism (defineClientTool, useClientTools, the swarm.client_action suspend and resume flow) is stable and has shipped since FR-002. FR-035 exposes it on the batteries-included <SwarmChat>, so a component-based host adopts client tools without rebuilding the timeline and composer on the raw hook.
Declare a client tool (server)
A client tool has no server execute. Declare its name and schemas with defineClientTool and attach the returned tool to an agent like any other. When the model calls it, the engine emits a swarm.client_action event and suspends the run until the browser answers.
import { z } from "zod";
import { defineClientTool } from "@nightowlsdev/core";
// A client tool has no server execute. When the model calls it, the engine emits
// swarm.client_action and suspends the run; the browser runs the handler and posts the result back.
export const frameObject = defineClientTool({
name: "frameObject",
description: "Frame the camera on a scene entity",
inputSchema: z.object({ entityId: z.string() }),
outputSchema: z.object({ ok: z.boolean() }),
needsApproval: false, // set true to force a confirm before the browser runs it
});Run it on the component, in one prop
If you already mount <SwarmChat>, pass a handler map to clientTools. The component runs useClientTools internally against its own run: each handler receives the model's typed input, returns a value that becomes the tool's output, and the run resumes. Throw to fail the tool.
import { SwarmChat } from "@nightowlsdev/react";
// Pass a handler map to the component: it runs useClientTools internally against its own run.
// The handler gets the model's typed input and returns the tool's output; throw to fail it.
<SwarmChat
agentSlug={agentSlug}
threadId={threadId}
clientTools={{ frameObject: ({ entityId }) => { camera.frame(entityId); return { ok: true }; } }}
confirmClientAction={(action) => window.confirm(`Run ${action.tool}?`)} // optional
/>;Keep a human in the loop
Mark a tool needsApproval: true on the server to force a confirm before the browser runs it, then pass confirmClientAction to gate it: return false to decline and the tool fails with { error: "declined by user" }. confirmClientAction is opt-in. Omit it and an approval-gated tool runs without a prompt, so wire it whenever a tool has a real side effect.
Two rules worth knowing
- Register on the run-owner lane. The swarm.client_action fires on the run and is root-attributed; a delegate lane never receives it, so mount clientTools on the run-owner <SwarmChat> (one per run or thread).
- One tool, one handler path. useClientTools dedups per instance, so do not register the same tool in both clientTools and a host-side useClientTools (both would answer the one followupId). Partition instead: some tools via clientTools, the rest via the DIY hook.
The DIY escape hatch (advanced)
For advanced wiring, bubble the matched pair with onClientTools and mount useClientTools yourself. You cannot call the hook inside the callback (hooks rule), so latch the bubbled { clientAction, respondClientAction } into state and call the hook at the top level. The common case is clientTools; reach for this only when you need to own the loop.
import { useState } from "react";
import { SwarmChat, useClientTools, type UiClientAction } from "@nightowlsdev/react";
// Advanced: own the wiring. You cannot call useClientTools inside the callback (hooks rule),
// so latch the bubbled pair into state and mount the hook at the top level.
function Host() {
const [run, setRun] = useState<{
clientAction: UiClientAction | null;
respondClientAction: (r: { output?: unknown; error?: string }) => Promise<void>;
} | null>(null);
useClientTools({
clientAction: run?.clientAction ?? null,
respond: run?.respondClientAction ?? (async () => {}),
handlers: { frameObject: ({ entityId }) => { camera.frame(entityId); return { ok: true }; } },
});
return <SwarmChat agentSlug={agentSlug} threadId={threadId} onClientTools={(_slug, io) => setRun(io)} />;
}