AI Provider Configuration

AVA's AI layer is provider-agnostic by design. Every AI call in the
product runs through the Vercel AI SDK behind a small factory, so the models —
and the vendor behind them — are a configuration choice, not a code fork. You
can run AVA on Anthropic (the default), OpenAI, Google Gemini, or OpenRouter,
switch between them with a one-line env change and a restart, and even mix two
providers in the same deployment.

This matters for an MSP in three practical ways:

• No vendor lock-in. If your pricing, compliance, or data-residency
  requirements change, you change an env string — not the product.
• Cost control. The heavy-traffic AI features run on a separate "fast"
  model slot from the "quality" slot, so you can pair a cheap fast model with
  a stronger quality model and tune spend independently.
• One key, not four. You only need an API key for the provider(s) you
  actually configure — the container validates exactly that at boot.

The two model roles

AVA never names a model in code. It asks for one of two logical roles, each
resolved from env at call time:

The fast role is where nearly all of AVA's day-to-day AI traffic happens —
it fires on every incoming chat (triage) and on several engineer-dashboard
features — so the fast model choice dominates your AI cost. The quality
role backs the transcript summarizer; note that in the shipped close flow the
ticket notes are engineer-authored and the full chat transcript is posted to
AutoTask verbatim, so quality-tier traffic is minimal in practice.

Call sites, precisely

The provider/model string format

Both LLM_FAST and LLM_QUALITY take a single string of the form:

<provider>/<model-id>

The string is split on the first slash only: everything before it selects
the provider adapter, everything after it is passed to that provider verbatim
as the model ID. That means model IDs that themselves contain slashes work
naturally — which is exactly what OpenRouter needs:

# Anthropic native (default)
LLM_FAST=anthropic/claude-haiku-4-5-20251001

# OpenAI
LLM_FAST=openai/gpt-4o-mini

# Google Gemini
LLM_FAST=google/gemini-2.0-flash

# OpenRouter — note the model ID itself contains a slash; that's fine
LLM_FAST=openrouter/anthropic/claude-haiku-4-5

A value with no slash, or an empty provider/model half, is rejected as invalid
at call time. Unset or blank values fall back to the defaults above.

Provider matrix

All AI calls are server-side only — no AI key is ever exposed to the
browser or baked into the client bundle.

Conditional key requirements — validated at boot

You do not set all four keys. The container's startup validation reads
your LLM_FAST / LLM_QUALITY values (applying the same defaults the app
uses), derives which provider(s) they reference, and requires exactly those
providers' keys — nothing more. A missing required key aborts startup with an
error that names both the missing key and the tier that demanded it, e.g.:

ERROR: Missing OPENAI_API_KEY in /app/.env.local, required by: LLM_FAST=openai/gpt-4o-mini

An unrecognized provider prefix (a typo, or a provider AVA doesn't
support) does not block startup — the entrypoint warns and skips the key
check, and the app raises a clear error at the first AI call instead. Watch
the boot log for that warning after changing these vars.

Mixing providers

The two roles are fully independent — a common pattern is a cheap
high-throughput model for LLM_FAST and a stronger model for LLM_QUALITY:

LLM_FAST=google/gemini-2.0-flash
LLM_QUALITY=anthropic/claude-sonnet-4-6

Set both providers' keys in that case (GOOGLE_GENERATIVE_AI_API_KEY and
ANTHROPIC_API_KEY here); the boot check enforces exactly that pair.

Timeouts

Naming caveat: despite the ANTHROPIC_ prefix, this timeout applies to
all providers — it predates the multi-provider layer and the name was kept
for backward compatibility. There is no per-provider timeout var; this one
value governs OpenAI, Gemini, and OpenRouter calls too.

Failure behavior — AI never blocks the chat

Every AI call site has a non-AI fallback, so a provider outage, a timed-out
request, or malformed model output degrades features rather than breaking the
product:

• Triage failure → the chat still starts and the AutoTask ticket is still
  created, with a safe default classification (normal priority, the raw client
  description as the summary).
• Similar-resolutions / trends / field-summarization failure → the
  dashboard feature returns a structured error or empty state; chat flow is
  untouched.
• Transcript summarization failure → the raw transcript is returned
  instead of a summary.

Triage additionally uses schema-validated structured output, so a model that
returns malformed JSON is caught and routed down the same fallback path — no
partial or corrupted classifications reach AutoTask.

Tuning triage for your clientele

The triage prompt includes a configurable one-line description of who your
clients are (COMPANY_PROFILE, default: "a managed service provider (MSP)
supporting business clients"). Setting it to something specific — e.g.
an MSP serving medical and dental practices — measurably sharpens issue
classification for your domain. See
white-label.md for this and the other identity knobs.

See also

• configuration.md — the complete env-var reference.
• white-label.md — branding, including COMPANY_PROFILE.
• quickstart.md — bring-up order, including where the AI
  key fits.