Skip to content
Night Owls.dev
GitHubGet started
Jump to a page

@nightowlsdev/provider-groq

Adapter/Model

Run Night Owls swarms on Groq's fast OpenAI-compatible inference, one API key.

What it does

Exposes `groqModels(opts?)`, a model factory for `defineSwarm({ modelFactory })`. It talks to Groq's OpenAI-compatible endpoint via the first-party `@ai-sdk/openai-compatible` provider, built as `createOpenAICompatible({ includeUsage: true })`, mapping a Groq model id (e.g. `llama-3.3-70b-versatile`) to an AI SDK language model. The `includeUsage: true` flag is load-bearing: without it a STREAMED response carries no `usage` object, so the provider metered 0 tokens and $0 and silently disabled every cost cap. It is defaulted, not exposed. Reads `GROQ_API_KEY` by default; `baseURL` defaults to `https://api.groq.com/openai/v1` and is overridable via `GROQ_BASE_URL`. Use a tool-calling-capable model so delegation and skills work; providers compose, so Groq can serve some agents while another `provider-*` adapter serves the rest, with `createModelFactory` doing the routing. Ships a `nightOwlsPlugin` manifest for CLI scaffolding. Imports only its AI SDK provider, zero `@mastra/*`.

Install

pnpm add @nightowlsdev/provider-groq

Key exports

  • groqModels
  • nightOwlsPlugin

Usage

provider-groq.ts
import { defineSwarm } from "@nightowlsdev/core";
import { groqModels } from "@nightowlsdev/provider-groq";

// Fast inference via Groq. Reads GROQ_API_KEY; MODEL_ID is a Groq model, e.g. llama-3.3-70b-versatile.
const swarm = defineSwarm({ agents, modelFactory: groqModels() });

What it provides

A one-line model provider for fast, cheap inference: groqModels() returns a modelFactory mapping a Groq model id (llama-3.3-70b-versatile, llama-3.1-8b-instant, …) to an AI SDK LanguageModelV3 through Groq's OpenAI-compatible endpoint (via the first-party @ai-sdk/openai-compatible). It reads GROQ_API_KEY and defaults baseURL to https://api.groq.com/openai/v1 (overridable via GROQ_BASE_URL). It also ships groqProvider() (the adapter-object form with a priced catalog). One of six interchangeable provider-* packages that compose via createModelFactory — Groq for the cheap, latency-sensitive bulk; a frontier model for the hard step.

When to use it

  • You want very fast, low-cost inference for high-volume or latency-sensitive work — sub-agents, classification, extraction, first drafts.
  • You want open-weight models (Llama, GPT-OSS) with tool calling, at a fraction of frontier prices.
  • You want to pair Groq for the bulk with a native provider for the one step that needs frontier quality, routing per agent with createModelFactory.

When not to

  • You need frontier-grade reasoning or vision quality — route those agents to Claude (provider-anthropic) or GPT (provider-openai).
  • You need a specific proprietary model Groq does not host.
  • You want fully local / offline inference with no per-token cost — use provider-ollama.

Alternatives

  • provider-ollamaYou want the same open-weight models but running locally and free (you host the GPU) instead of hosted.
  • A native provider (anthropic / openai)The step needs frontier reasoning or vision — compose it with Groq via createModelFactory.
  • provider-vercel-gateway / provider-openrouterYou want breadth across many vendors from one key rather than Groq's open-model roster.

Strengths

  • Fast and cheap: a priced catalog (llama-3.3-70b-versatile, llama-3.1-8b-instant) feeds groq:* expansion, the picker, and cost.maxCostUsd.
  • OpenAI-compatible transport, so the model object is the exact same AI SDK LanguageModelV3 shape as every other adapter; baseURL is overridable for a proxy.
  • FR-062: includeUsage is defaulted on, so STREAMED generations report real token counts — without it the wire format omits usage and every cost cap is silently disabled.
  • Engine-wall clean: one dependency, zero @mastra and zero @nightowlsdev/core.

Limits & trade-offs

  • Open-weight models only — no Claude/GPT-class frontier reasoning, and no vision on the current roster.
  • You MUST pick a tool-calling model — otherwise delegation, the ask/scratchpad tools, and skills silently can't call tools; the catalog marks which ids are toolCalling.
  • gpt-oss-120b ships unpriced in the catalog, so groq:* drops it unless you supply a price or pass allowUnpriced.
  • A single vendor: compose other provider-* packages with createModelFactory for a quality spread across tasks.

How it works

groqModels(opts) builds createOpenAICompatible({ name: 'groq', baseURL, includeUsage: true, apiKey }) and returns a (modelId) => provider(modelId) factory. The trailing /v1 on the baseURL is load-bearing — it selects the OpenAI-compatible transport. includeUsage sets stream_options.include_usage on the wire: SwarmEngine.run streams every generation, and the OpenAI stream format omits usage unless that flag is set, so without it the provider reports 0 tokens / $0 and disables every cost cap. It is defaulted, not exposed. groqProvider() is the adapter-object form for createModelProviderRegistry (catalog-only; Groq's endpoint exposes /models but a live listing is deliberately not wired here).

Examples

Wire Groq into a swarm

Fast inference; the key comes from GROQ_API_KEY. Pick a tool-calling model for delegation/skills.

provider-groq-example-1.ts
import { defineSwarm } from "@nightowlsdev/core";
import { groqModels } from "@nightowlsdev/provider-groq";

export default defineSwarm({
  modelFactory: groqModels(),                     // reads GROQ_API_KEY
  models: { allow: ["llama-3.3-70b-versatile"] },
  agents,
});

Groq for the bulk, Claude for the hard step

createModelFactory routes cheap/fast work to Groq and the one demanding agent to a frontier model.

provider-groq-example-2.ts
import { createModelFactory } from "@nightowlsdev/core";
import { groqModels } from "@nightowlsdev/provider-groq";
import { anthropicModels } from "@nightowlsdev/provider-anthropic";

const modelFactory = createModelFactory({
  factories: { groq: groqModels(), anthropic: anthropicModels() },
  resolve: (agentSlug) =>
    agentSlug === "analyst"
      ? { provider: "anthropic", modelId: "claude-sonnet-4-6" }
      : { provider: "groq", modelId: "llama-3.1-8b-instant" },
  allow: ["anthropic/claude-sonnet-4-6", "groq/llama-3.1-8b-instant"],
});

Doing the parts it doesn't support

  • A frontier-quality stepRoute that agent to a native provider (provider-anthropic / provider-openai) via createModelFactory, and keep Groq for the high-volume agents.
  • A runtime model listing in the pickerNot wired for Groq. Use the shipped catalog, or front models through provider-vercel-gateway, whose adapter has a live models() with per-model pricing.
  • Delegation on a model without tool callingPick a tool-calling model — the catalog rows flagged toolCalling. A non-tool model runs but silently can't delegate, use the ask/scratchpad tools, or call skills.

Related

  • provider-ollama — The other OpenAI-compatible open-model adapter — local and free instead of hosted-and-fast.
  • provider-anthropic — The frontier model to compose in for the one step that needs it.
  • core — defineSwarm / modelFactory consume the factory; createModelFactory composes Groq with other providers.
  • model-providers — The guide to registering providers, provider-qualified allow-lists, and pricing.
  • cli — owl install provider-groq scaffolds the env vars and the modelFactory config marker.