AI Provider Configuration

AVA's AI layer is provider-agnostic by design. Every AI call in the product runs through the Vercel AI SDK behind a small factory, so the models — and the vendor behind them — are a configuration choice, not a code fork. You can run AVA on Anthropic (the default), OpenAI, Google Gemini, or OpenRouter, switch between them with a one-line env change and a restart, and even mix two providers in the same deployment.

This matters for an MSP in three practical ways:

• No vendor lock-in. If your pricing, compliance, or data-residency requirements change, you change an env string — not the product.
• Cost control. The heavy-traffic AI features run on a separate "fast" model slot from the "quality" slot, so you can pair a cheap fast model with a stronger quality model and tune spend independently.
• One key, not four. You only need an API key for the provider(s) you actually configure — the container validates exactly that at boot.

The two model roles

AVA never names a model in code. It asks for one of two logical roles, each resolved from env at call time:

The fast role is where nearly all of AVA's day-to-day AI traffic happens — it fires on every incoming chat (triage) and on several engineer-dashboard features — so the fast model choice dominates your AI cost. The quality role backs the transcript summarizer; note that in the shipped close flow the ticket notes are engineer-authored and the full chat transcript is posted to AutoTask verbatim, so quality-tier traffic is minimal in practice.

Call sites, precisely

Engineer AI calls are budgeted per chat and per engineer-hour (AI_MAX_CALLS_PER_SESSION, AI_MAX_CALLS_PER_ENGINEER_PER_HOUR — see configuration.md), and each one writes an [ai-usage] log line with token counts (never prompt or output text), so you can track spend from your logs.

The provider/model string format

Both LLM_FAST and LLM_QUALITY take a single string of the form:

<provider>/<model-id>

The string is split on the first slash only: everything before it selects the provider adapter, everything after it is passed to that provider verbatim as the model ID. That means model IDs that themselves contain slashes work naturally — which is exactly what OpenRouter needs:

# Anthropic native (default)
LLM_FAST=anthropic/claude-haiku-4-5-20251001

# OpenAI
LLM_FAST=openai/gpt-4o-mini

# Google Gemini
LLM_FAST=google/gemini-2.0-flash

# OpenRouter — note the model ID itself contains a slash; that's fine
LLM_FAST=openrouter/anthropic/claude-haiku-4-5

A value with no slash, or an empty provider/model half, is rejected as invalid at call time. Unset or blank values fall back to the defaults above.

Provider matrix

All AI calls are server-side only — no AI key is ever exposed to the browser or baked into the client bundle.

Conditional key requirements — validated at boot

You do not set all four keys. The container's startup validation reads your LLM_FAST / LLM_QUALITY values (applying the same defaults the app uses), derives which provider(s) they reference, and requires exactly those providers' keys — nothing more. A missing required key aborts startup with an error that names both the missing key and the tier that demanded it, e.g.:

ERROR: Missing OPENAI_API_KEY in /app/.env.local, required by: LLM_FAST=openai/gpt-4o-mini

An unrecognized provider prefix (a typo, or a provider AVA doesn't support) does not block startup — the entrypoint warns and skips the key check, and the app raises a clear error at the first AI call instead. Watch the boot log for that warning after changing these vars.

Mixing providers

The two roles are fully independent — a common pattern is a cheap high-throughput model for LLM_FAST and a stronger model for LLM_QUALITY:

LLM_FAST=google/gemini-2.0-flash
LLM_QUALITY=anthropic/claude-sonnet-4-6

Set both providers' keys in that case (GOOGLE_GENERATIVE_AI_API_KEY and ANTHROPIC_API_KEY here); the boot check enforces exactly that pair.

Timeouts

Naming caveat: despite the ANTHROPIC_ prefix, this timeout applies to all providers — it predates the multi-provider layer and the name was kept for backward compatibility. There is no per-provider timeout var; this one value governs OpenAI, Gemini, and OpenRouter calls too.

Failure behavior — AI never blocks the chat

Every AI call site has a non-AI fallback, so a provider outage, a timed-out request, or malformed model output degrades features rather than breaking the product:

• Triage failure → the chat still starts and the AutoTask ticket is still created, with a safe default classification (normal priority, the raw client description as the summary).
• Similar-resolutions / trends / field-summarization failure → the dashboard feature returns a structured error or empty state; chat flow is untouched.
• Transcript summarization failure → the raw transcript is returned instead of a summary.
• Engineer helper failure (/draft, /rewrite, /summarize, "Draft all") → the engineer sees a short message and can try again; nothing is sent to the client and the chat carries on.

Triage additionally uses schema-validated structured output, so a model that returns malformed JSON is caught and routed down the same fallback path — no partial or corrupted classifications reach AutoTask.

Tuning triage for your clientele

The triage prompt includes a configurable one-line description of who your clients are (COMPANY_PROFILE, default: "a managed service provider (MSP) supporting business clients"). Setting it to something specific — e.g. an MSP serving medical and dental practices — measurably sharpens issue classification for your domain. See white-label.md for this and the other identity knobs.

See also

• configuration.md — the complete env-var reference.
• white-label.md — branding, including COMPANY_PROFILE.
• quickstart.md — bring-up order, including where the AI key fits.