Writing a model provider

A model backend Apograph does not ship is one factory function returning a ModelProvider, plus a run of the conformance kit.

Before writing one, check whether you need to: the OpenAI-wire adapter already reaches Ollama, vLLM, llama.cpp, LM Studio, LiteLLM, OpenRouter, Azure and OpenAI. A new package is for a backend whose wire format genuinely differs.

Where the port lives

@apograph/copilot-domain, which imports nothing from NestJS, Drizzle or any vendor. An adapter depends on the port without depending on the server plugin — the same split identity makes for its SSO seam.

What an adapter owes

Stream deltas that reassemble. Concatenating every text delta must yield the complete answer. The kit checks this with an answer streamed in at least two deltas, because a single-delta answer cannot show a reassembly bug.

Reject an unknown model. A run naming a model the provider does not offer raises an unknown-model error — and makes no request. A silent substitution onto the default is the kind of thing nobody notices until a bill or an answer looks wrong.

Report usage. Input and output tokens, which is what the run engine’s total ceiling is enforced against.

Emit whole tool calls. However the wire fragments them, what reaches the engine is one complete call with parsed arguments.

The conformance kit

Every adapter runs the shared kit from the domain package in its own suite. Each factory is called fresh per check and may be async, so an adapter that has to arm a mock — an SDK stub, a fetch stub — can do it there.

You supply a text scenario, a tool-call scenario, a model id the provider does not offer, and a way to count requests actually issued to your transport. That last one is what makes “no request was made” checkable rather than assumed.

The model list is a list, not a setting

One credential backs several models, the first is the default, and a user can switch mid-conversation. Follow that shape — the admin’s picker and the engine’s model-switching path both assume it.

Registering it

CopilotPlugin({
    providers: [
        { name: 'mine', provider: createMyProvider({ /* … */ }) },
        { name: 'fake', provider: createFakeProvider() }
    ],
    config: config.plugins.copilot
});

The order is the setting. There is no defaultProvider: the first entry serves a run that names none. Register only what is configured — a keyless backend at the top of the list would be the house default and would fail on the first message — and keep the scripted provider last.