Skip to content
Night Owls.dev
GitHubGet started
Jump to a page

@nightowlsdev/provider-ollama

Adapter/Model

Run Night Owls swarms on local models via Ollama, no API key, no per-token cost.

What it does

Exposes `ollamaModels(opts?)`, a model factory for `defineSwarm({ modelFactory })`. It talks to a local Ollama daemon through its OpenAI-compatible endpoint via the first-party `@ai-sdk/openai-compatible` provider, built as `createOpenAICompatible({ includeUsage: true })`, mapping a bare model id (e.g. `llama3.1`) to an AI SDK `LanguageModelV3`. The `includeUsage: true` flag is load-bearing, not cosmetic: without it a STREAMED response carries no `usage` object, so the provider metered 0 tokens and $0 and silently disabled every cost cap (an uncapped run that looks capped). It is defaulted, not exposed. Unlike the hosted providers it needs NO API key, only a `baseURL` (default `http://localhost:11434/v1`, overridable via `OLLAMA_BASE_URL`). Use a tool-calling-capable model so delegation and skills work; local models still meter at $0-per-token, but the token counts are now real. Ships a `nightOwlsPlugin` manifest for CLI scaffolding; it composes with the hosted adapters, so `createModelFactory` can route cheap tasks to the local daemon and the rest to a frontier model. Imports only its AI SDK provider, zero `@mastra/*`.

Install

pnpm add @nightowlsdev/provider-ollama

Key exports

  • ollamaModels
  • nightOwlsPlugin

Usage

provider-ollama.ts
import { defineSwarm } from "@nightowlsdev/core";
import { ollamaModels } from "@nightowlsdev/provider-ollama";

// Local models via Ollama, no API key. Reads OLLAMA_BASE_URL (default http://localhost:11434/v1).
const swarm = defineSwarm({ agents, modelFactory: ollamaModels() }); // MODEL_ID is a bare id, e.g. llama3.1

What it provides

A one-line model provider that runs a swarm on open models with no per-token cost: ollamaModels() returns a modelFactory mapping a bare model id (llama3.1, qwen2.5:14b, …) to an AI SDK LanguageModelV3 through Ollama's OpenAI-compatible endpoint (via the first-party @ai-sdk/openai-compatible). Two deployments, selected only by config: a LOCAL daemon (default http://localhost:11434/v1, no key, $0/token) or Ollama CLOUD (https://ollama.com/v1 + OLLAMA_API_KEY). It also ships ollamaProvider() (the adapter-object form with an unpriced catalog). One of six interchangeable provider-* packages that compose via createModelFactory — local Ollama for the cheap tasks, a hosted frontier model for the rest.

When to use it

  • You want offline, air-gapped, or privacy-sensitive inference — open models on your own hardware, no API key, no per-token cost, data never leaves the box.
  • You want free local inference for cheap tasks and route the hard step to a hosted frontier model via createModelFactory.
  • You want hosted open models without running a local GPU — point the same adapter at Ollama Cloud with a key.

When not to

  • You need frontier reasoning or vision quality — route those agents to Claude (provider-anthropic) or GPT (provider-openai).
  • You don't want to run/host a daemon and want the simplest hosted breadth — use provider-vercel-gateway or provider-openrouter.
  • You need reliable enforced dollar-caps on Ollama Cloud usage — the same adapter serves a $0 local daemon and a non-$0 Cloud, so the catalog is unpriced; supply Cloud rates via resolveModelAllowList({ prices }).

Alternatives

  • provider-groqYou want the same open-weight models but hosted and fast (priced) instead of running them yourself.
  • A native provider (anthropic / openai)The step needs frontier quality — compose it with local Ollama via createModelFactory.
  • provider-vercel-gateway / provider-openrouterYou want breadth across many hosted vendors from one key, no local infra.

Strengths

  • No API key and no per-token cost for a local daemon — full data locality for privacy-sensitive workloads.
  • One adapter, two deployments: local vs Ollama Cloud is chosen purely by baseURL / OLLAMA_API_KEY, nothing else changes.
  • FR-062: includeUsage is defaulted on, so streamed generations report real token counts — the local price is $0 but tokens still gate SwarmConfig.cost, RunInput.budget, and the FR-056 cumulative ceiling.
  • Engine-wall clean: one dependency, zero @mastra and zero @nightowlsdev/core.

Limits & trade-offs

  • You run the infra: a local daemon needs the model pulled (ollama pull) and enough VRAM; quality and latency track your hardware.
  • Open-weight models only — no frontier reasoning, and no vision on typical local models.
  • Bare model ids (llama3.1), NOT provider/model; ids can carry a ':' tag (qwen2.5:14b, glm-5.2:cloud) — which is exactly why the allow-list grammar splits on the first separator.
  • Unpriced catalog: local is genuinely $0, but Cloud is not, so ollama:* expands to nothing unless you pass allowUnpriced (local) or supply rates (Cloud).
  • You MUST pick a tool-calling model — base gemma runs but silently can't delegate or use tools/skills.

How it works

ollamaModels(opts) builds createOpenAICompatible({ name: 'ollama', baseURL, includeUsage: true, apiKey?, headers? }) and returns a (modelId) => provider(modelId) factory. The default baseURL is http://localhost:11434/v1 (local, no key); set https://ollama.com/v1 + OLLAMA_API_KEY for Cloud (sent as a Bearer token). The trailing /v1 selects the OpenAI-compatible transport, not Ollama's native API. includeUsage sets stream_options.include_usage so streamed generations report tokens — without it usage is omitted and cost caps silently go off, and even at a $0 local price the token counts still gate step and budget limits. ollamaProvider() is the adapter-object form for createModelProviderRegistry (catalog-only; Ollama exposes /api/tags but a live listing is deliberately not wired here).

Examples

Local models, no API key

Run 'ollama serve' and 'ollama pull llama3.1'; the default baseURL is localhost:11434/v1.

provider-ollama-example-1.ts
import { defineSwarm } from "@nightowlsdev/core";
import { ollamaModels } from "@nightowlsdev/provider-ollama";

export default defineSwarm({
  modelFactory: ollamaModels(),          // OLLAMA_BASE_URL or localhost, no key
  models: { allow: ["llama3.1"] },       // a bare id, not provider/model
  agents,
});

Ollama Cloud (no local daemon)

Point the same adapter at the cloud endpoint with a key; cloud ids carry the :cloud suffix.

provider-ollama-example-2.ts
import { ollamaModels } from "@nightowlsdev/provider-ollama";

const modelFactory = ollamaModels({
  baseURL: "https://ollama.com/v1",
  apiKey: process.env.OLLAMA_API_KEY,
});
// models: { allow: ["glm-5.2:cloud"] }

Local for the bulk, a frontier model for the rest

createModelFactory keeps cheap work on the free local daemon and routes the hard step to Claude.

provider-ollama-example-3.ts
import { createModelFactory } from "@nightowlsdev/core";
import { ollamaModels } from "@nightowlsdev/provider-ollama";
import { anthropicModels } from "@nightowlsdev/provider-anthropic";

const modelFactory = createModelFactory({
  factories: { ollama: ollamaModels(), anthropic: anthropicModels() },
  resolve: (agentSlug) =>
    agentSlug === "researcher"
      ? { provider: "anthropic", modelId: "claude-sonnet-4-6" }
      : { provider: "ollama", modelId: "llama3.1" },
  allow: ["anthropic/claude-sonnet-4-6", "ollama/llama3.1"],
});

Doing the parts it doesn't support

  • Enforced dollar-caps on Ollama CloudSupply Cloud rates via resolveModelAllowList({ prices }) or cost.prices — the adapter is unpriced because the same code path also serves a genuinely $0 local daemon, so a baked-in price would be wrong half the time.
  • A runtime model listing in the pickerOllama's /api/tags is not wired here. Use the shipped catalog, or front models through provider-vercel-gateway, whose adapter has a live models().
  • A frontier-quality stepRoute that agent to a native provider via createModelFactory and keep Ollama for the cheap local bulk.

Related

  • provider-groq — The hosted, priced sibling — the same open-weight models run fast in the cloud instead of on your GPU.
  • provider-anthropic — The frontier model to compose in for the step that needs it.
  • core — defineSwarm / modelFactory consume the factory; createModelFactory composes local + hosted providers.
  • model-providers — The guide to provider-qualified allow-lists and per-provider pricing (relevant for Cloud rates).
  • cli — owl install provider-ollama scaffolds the env vars and the modelFactory config marker.