Providers and models

Discover providers and models#

term-llm providers
term-llm providers --configured
term-llm providers anthropic

term-llm models --provider anthropic
term-llm models --provider openrouter
term-llm models --provider nearai
term-llm models --provider sambanova
term-llm models --provider ollama
term-llm models --json

Use providers when you want to know what is available and how it is configured. Use models when you want the concrete model names a provider currently exposes.

Provider categories#

term-llm supports a mix of provider types:

• hosted API providers such as Anthropic, AWS Bedrock, OpenAI, xAI, Gemini, NEAR AI Cloud, SambaNova, and OpenRouter
• subscription-backed OAuth providers such as ChatGPT, Grok, and Copilot
• local or self-hosted OpenAI-compatible providers such as Ollama, LM Studio, vLLM, or custom endpoints

Credentials#

Most providers use API keys via environment variables. Some use OAuth credentials from companion CLIs or locally stored auth files.

OpenCode Go#

export OPENCODE_API_KEY=your-key
term-llm models --provider opencode-go
term-llm ask --provider opencode-go:glm-5.2 "question"

OpenCode Go’s model set changes frequently. term-llm merges the live Go /models availability list with OpenCode’s model catalog for protocol, limits, pricing, and reasoning metadata, then caches the result for five minutes. Models marked for @ai-sdk/openai-compatible, @ai-sdk/openai, and @ai-sdk/anthropic are sent to the Go Chat Completions, Responses, and Messages endpoints respectively. Effort-based models expose their advertised suffixes, while budget-based Anthropic-compatible models such as qwen3.8-max expose -high and -max reasoning variants. A newly available model without catalog metadata falls back to Chat Completions until its metadata arrives.

A compatible gateway can be configured under a custom provider name. base_url replaces the standard OpenCode Go /v1 base while preserving dynamic protocol routing:

providers:
  private-go:
    type: opencode-go
    base_url: https://gateway.example.com/v1
    api_key: ${PRIVATE_GO_API_KEY}
    model: muse-spark-1.2-contributor

Examples:

term-llm ask --provider anthropic "question"
term-llm ask --provider chatgpt "question"
term-llm ask --provider grok "question"
term-llm ask --provider copilot "question"

grok in chat selects the subscription OAuth provider. Existing providers.grok API-key or custom-endpoint configurations must declare type: xai or type: openai_compatible; term-llm rejects ambiguous api_key, base_url, and url fields instead of ignoring them. The image command keeps its older --provider grok alias for the xAI API-key image provider (image.xai / XAI_API_KEY), not Grok subscription OAuth.

Anthropic-compatible endpoints#

Use type: anthropic with base_url to point the native Anthropic protocol provider at a compatible gateway or self-hosted endpoint:

providers:
  custom-anthropic:
    type: anthropic
    base_url: https://gateway.example.com/anthropic
    api_key: ${CUSTOM_ANTHROPIC_API_KEY}
    model: custom-model

The Anthropic SDK appends paths such as /v1/messages and /v1/models to base_url. The field supports the same lazy srv:// and $() endpoint resolution as url. Omit base_url to use Anthropic’s standard API endpoint.

WebSocket defaults#

The built-in openai and chatgpt text providers use the Responses WebSocket transport by default. This improves latency in agentic/tool-heavy runs by reusing one connection and continuing compatible turns with previous_response_id plus only new input. If setup fails before streaming starts, term-llm falls back to HTTP/SSE; if a WebSocket continuation rejects the previous response ID, it retries once with full input.

To force HTTP/SSE for either built-in provider:

providers:
  openai:
    use_websocket: false
  chatgpt:
    use_websocket: false

OpenAI-compatible providers remain HTTP/SSE by default. WebSocket defaults are not applied to type: openai_compatible entries.

GPT-5.6 on OpenAI and ChatGPT#

term-llm ships fallback metadata for the GPT-5.6 family:

The OpenAI model IDs are gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna. ChatGPT exposes the same named variants through its live Codex catalog; live input limits and reasoning metadata take precedence over the static fallback. Luna keeps its upstream medium default unless you explicitly select another effort.

Pro and Ultra are not reasoning efforts. On the public OpenAI API, Pro sends reasoning.mode: pro; Standard sends reasoning.mode: standard. In terminal chat use /pro on, /pro off, or /pro status. The browser shows a Standard/Pro selector only when the selected model advertises it. ChatGPT OAuth does not receive public-API Pro mode. Codex’s separate product-level Ultra option runs at max effort and adds subagents, so term-llm does not expose it as an effort.

OpenAI API GPT-5.6 also supports term-llm’s advanced Responses controls: reasoning context, hosted multi-agent execution, programmatic tool calling, and prompt-cache options. These controls are model/provider gated. term-llm rejects them for older OpenAI models, OpenAI-compatible providers, and the ChatGPT OAuth backend rather than sending unverified fields. See Configuration and Web UI and API.

Bundled OpenAI API prices per 1M tokens (uncached input / cache read / cache write / output) are Sol: $5 / $0.50 / $6.25 / $30; Terra: $2.50 / $0.25 / $3.125 / $15; Luna: $1 / $0.10 / $1.25 / $6. When total input (uncached + cache reads + cache writes) exceeds 272K, the entire request uses 2× input/read/write rates and 1.5× output rates. This is API-key billing metadata; ChatGPT subscription access is not billed from these static prices.

SambaNova Cloud#

SambaNova is available as a built-in OpenAI-compatible provider:

export SAMBANOVA_API_KEY=your-key
term-llm ask --provider sambanova:gpt-oss-120b "quick question"
term-llm models --provider sambanova

providers:
  sambanova:
    model: gpt-oss-120b
    fast_model: Meta-Llama-3.3-70B-Instruct

The provider uses https://api.sambanova.ai/v1, supports tool calls, and has a curated fallback model list. term-llm models --provider sambanova queries SambaNova’s /models endpoint; because the OpenAI-compatible model response does not generally include price metadata, term-llm annotates known SambaNova models with bundled public prices from https://cloud.sambanova.ai/plans/pricing. These prices are also used by term-llm usage cost calculation for matching SambaNova model IDs.

NEAR AI Cloud#

NEAR AI Cloud is available as a built-in OpenAI-compatible provider for TEE-backed private inference:

export NEARAI_API_KEY=your-key
term-llm ask --provider nearai:zai-org/GLM-5.1-FP8 "quick question"
term-llm models --provider nearai

providers:
  nearai:
    model: zai-org/GLM-5.1-FP8
    fast_model: Qwen/Qwen3.6-35B-A3B-FP8

The provider uses https://cloud-api.near.ai/v1, supports tool calls, and has a curated fallback list of TEE-hosted text models. term-llm models --provider nearai queries NEAR AI Cloud’s public /model/list catalog and filters it to chat-capable models, with token prices shown per 1M tokens when available.

Ollama thinking levels#

Ollama’s native chat API accepts either a boolean think switch or a named thinking level. Use think for binary on/off models, or think_level when the model supports low, medium, high, or max:

providers:
  local_qwen:
    type: ollama
    base_url: http://127.0.0.1:11434
    model: qwen3:30b
    think_level: low

think_level takes precedence when both settings are present. Named levels are useful when binary thinking produces unnecessarily long traces; support still depends on the selected Ollama model.

vLLM reasoning providers#

For vLLM servers running reasoning models, prefer type: vllm instead of generic openai_compatible. The vLLM provider still uses the OpenAI-compatible /v1/chat/completions API, but maps term-llm reasoning effort suffixes into model-family-specific vLLM request fields and replays prior assistant reasoning with vLLM’s current reasoning message field.

For Qwen-family models, term-llm sends Qwen chat-template controls:

providers:
  cdck_qwen:
    type: vllm
    base_url: https://gpu-server.example.com:8000/v1
    model: Qwen/Qwen3.5-122B-A10B
    api_key: ${CDCK_QWEN_API_KEY}
    context_window: 200000
    max_output_tokens: 50000

Use the normal provider flag, or append an effort suffix to the configured provider name:

term-llm ask -p cdck_qwen "hello"          # default: no thinking
term-llm ask -p cdck_qwen-low "harder"     # thinking budget 1024
term-llm ask -p cdck_qwen-medium "hard"    # thinking budget 4096
term-llm ask -p cdck_qwen-high "very hard" # thinking budget 10000

The effort suffix is stripped before sending the model name upstream. For example, -p cdck_qwen-high sends model: Qwen/Qwen3.5-122B-A10B plus Qwen thinking controls, not a literal Qwen/Qwen3.5-122B-A10B-high model ID.

For vLLM templates that use chat_template_kwargs.thinking, declare the model’s exact capabilities and default in config. vllm_thinking_param: thinking selects the request shape; term-llm does not embed model-specific effort mappings:

providers:
  cdck_deepseek:
    type: vllm
    base_url: https://gpu-server.example.com:8000/v1
    model: deepseek-ai/DeepSeek-V4-Flash
    vllm_thinking_param: thinking
    models:
      - id: deepseek-ai/DeepSeek-V4-Flash
        alias: deepseek-v4-flash
        reasoning_efforts: [none, low, high, max]
        default_reasoning_effort: high

The bare provider/model uses default_reasoning_effort. -p cdck_deepseek-<TAB> completes only the values listed in reasoning_efforts. Model-name detection for deepseek remains as a backward-compatible fallback when vllm_thinking_param is omitted; explicit configuration is preferred for aliases. Requests using the thinking shape do not send thinking_token_budget.

Notes:

• Start vLLM with the appropriate reasoning parser for your model, for example --reasoning-parser qwen3 for Qwen3-family reasoning output or --reasoning-parser deepseek_v3 / deepseek_v4 for DeepSeek variants.
• Qwen thinking_token_budget requires a vLLM server new enough to support it and, on recent vLLM, a server-side --reasoning-config. Plain/default Qwen requests omit the budget so they work without that extra server option.
• vLLM currently streams reasoning text in delta.reasoning, but may not report usage.completion_tokens_details.reasoning_tokens accurately. In that case term-llm can show reasoning in debug output while reasoning_tokens remains 0; this reflects vLLM usage metadata, not missing reasoning text.
• For multi-turn conversations, term-llm persists streamed reasoning and replays it as assistant reasoning on the next vLLM request. This lets vLLM’s chat template render the prior reasoning consistently and gives prefix caching the best chance to reuse shared prompt prefixes when the server has prefix caching enabled.

OpenAI-compatible providers#

For local or custom backends that do not need vLLM chat-template thinking controls, use type: openai_compatible.

providers:
  ollama:
    type: openai_compatible
    base_url: http://localhost:11434/v1
    model: llama3.2:latest

  lmstudio:
    type: openai_compatible
    base_url: http://localhost:1234/v1
    model: deepseek-coder-v2

  cerebras:
    type: openai_compatible
    base_url: https://api.cerebras.ai/v1
    model: llama-4-scout-17b
    api_key: ${CEREBRAS_API_KEY}

Use base_url when the standard /chat/completions path should be appended automatically. Use url when you need to specify the full chat completions endpoint directly.

Configuration reference#

Model object entries#

models may mix plain strings and objects. Plain strings are enough for autocomplete/model picker entries. Object entries are for endpoints where the model you want to type locally differs from the model ID the API expects, or where each model needs its own metadata.

providers:
  custom:
    type: openai_compatible
    base_url: https://api.example.com/v1
    api_key: ${CUSTOM_API_KEY}
    model: friendly-name
    # Optional provider-level default for all models under this provider.
    # The target may be provider:model, or just provider to use that provider's default model.
    vision_via: gemini
    models:
      - simple-upstream-model
      - id: upstream/model-id
        alias: friendly-name
        context_window: 262144
        max_output_tokens: 32768
        # Optional: only set these for APIs/models that support them.
        parse_reasoning: true
        include_reasoning: true
        thinking_param: enable_thinking
        reasoning_efforts: [none, low, high, max]
        default_reasoning_effort: high
        # Optional per-model override; if omitted, provider-level vision_via is used.
        vision_via: gemini:gemini-2.5-pro

For -p custom:friendly-name-max, term-llm sends model: upstream/model-id plus reasoning_effort: max. When default_reasoning_effort is set, the bare provider/model uses that effort; otherwise it sends no effort. If reasoning_efforts is empty or omitted, no configured effort aliases or provider-profile completions are generated for that model.

Friendli reasoning#

Friendli is OpenAI-compatible, but reasoning-capable models such as GLM-5.2 need explicit parser flags to expose thinking as reasoning_content instead of leaving it inline. Configure those fields on the specific model entry:

providers:
  friendli:
    type: openai_compatible
    base_url: https://api.friendli.ai/serverless/v1
    api_key: ${FRIENDLI_API_KEY}
    model: glm52
    models:
      - id: zai-org/GLM-5.2
        alias: glm52
        context_window: 1048576
        max_output_tokens: 131072
        parse_reasoning: true
        include_reasoning: true
        thinking_param: enable_thinking
        reasoning_efforts: [high, max]

With that config, effort suffixes are generated only from the declared reasoning_efforts. For example -p friendli:glm52-max sends model: zai-org/GLM-5.2, reasoning_effort: max, parse_reasoning: true, include_reasoning: true, and chat_template_kwargs.enable_thinking: true. The bare -p friendli:glm52 sends no reasoning_effort; because only high and max are listed, term-llm does not offer glm52-low or glm52-medium completions.

You can set the same reasoning-parser fields at the provider level as defaults, then override them per model object when different models on the same endpoint need different behavior.

Full example#

providers:
  my-vllm:
    type: vllm
    base_url: http://gpu-server:8000/v1
    model: Qwen/Qwen3-30B-A3B
    api_key: ${VLLM_API_KEY}
    context_window: 32768
    max_output_tokens: 8192
    models:
      - Qwen/Qwen3-30B-A3B
      - Qwen/Qwen3-8B

  legacy-server:
    type: openai_compatible
    url: http://old-server:5000/api/chat
    model: custom-finetune
    no_stream_options: true  # this server rejects stream_options

Service tiers and fast mode#

Built-in openai and chatgpt text providers can send the Responses API service_tier field. To request fast/priority service for all turns through a provider, set service_tier in that provider config:

providers:
  openai:
    model: gpt-5.6-sol
    fast_model: gpt-5.6-luna
    service_tier: fast      # alias for API value "priority"

  chatgpt:
    model: gpt-5.6-sol-medium
    fast_model: gpt-5.6-luna
    service_tier: priority  # equivalent to "fast"

Leave service_tier unset to omit the field entirely. Only some models/accounts support fast service; unsupported requests may be ignored or rejected by the provider. In chat, /fast toggles the fast service tier for the current session. The status line shows fast when it is currently active.

This is different from fast_model / optional fast_provider, which choose a lightweight model for term-llm control-plane tasks such as summaries or title generation, and for agent configs that use model: fast.

Reasoning and model suffixes#

Model/provider suffixes control how much reasoning a provider is asked to do. Display of the resulting reasoning is controlled separately by the top-level reasoning config. Non-encrypted provider-marked thinking is shown as collapsed Thinking... / Thought: <title> blocks by default; encrypted reasoning/signature payloads are replay-only and are never displayed.

OpenAI reasoning effort#

For GPT-5.6 on the OpenAI API, append -none, -low, -medium, -high, -xhigh, or -max to control reasoning effort. An explicit reasoning_effort API field wins over a model suffix, and a request suffix wins over a provider-level default.

term-llm ask --provider openai:gpt-5.6-sol-xhigh "complex question"
term-llm exec --provider openai:gpt-5.6-luna-low "quick task"

providers:
  openai:
    model: gpt-5.6-sol-high

Older GPT-5 models retain their existing provider-specific effort sets. Natural model IDs ending in a suffix-like word are preserved when that suffix is not valid for the model; for example, gpt-5.1-codex-max remains a model ID rather than being split into model plus max effort.

For ChatGPT OAuth, GPT-5.6 Sol, Terra, and Luna expose low, medium, high, xhigh, and max. Codex’s product-level Ultra option is not an inference effort: the official client sends max and separately enables subagents, so it is not included in term-llm’s effort suffixes.

vLLM thinking suffixes#

For configured providers with type: vllm, effort suffixes can be applied directly to the provider name. Declare them on the selected model to drive parsing and shell completion:

providers:
  cdck_deepseek:
    type: vllm
    base_url: https://gpu-server.example.com:8000/v1
    model: deepseek-ai/DeepSeek-V4-Flash
    vllm_thinking_param: thinking
    models:
      - id: deepseek-ai/DeepSeek-V4-Flash
        alias: deepseek-v4-flash
        reasoning_efforts: [none, low, high, max]
        default_reasoning_effort: high

term-llm ask -p cdck_deepseek "uses configured high default"
term-llm ask -p cdck_deepseek-low "override with low"
term-llm ask -p cdck_deepseek-none "disable thinking"

The suffix is stripped before the model ID is sent upstream. -p cdck_deepseek-<TAB> offers exactly the declared efforts, and undeclared provider-profile suffixes are rejected.

For Qwen-style enable_thinking templates, term-llm retains its budget mapping:

For thinking templates, configured enabled efforts pass through unchanged:

Reasoning replay uses vLLM’s reasoning assistant-message field. vLLM may still report reasoning_tokens: 0 in usage metadata even when reasoning text was streamed; this is a known vLLM-side accounting gap.

Anthropic extended thinking#

For Anthropic models, append -thinking:

term-llm ask --provider anthropic:claude-sonnet-4-6-thinking "complex question"

providers:
  anthropic:
    model: claude-sonnet-4-6-thinking

AWS Bedrock#

The bedrock provider routes Anthropic Claude models through AWS Bedrock. It supports the same model suffixes (-thinking, -1m) and has full feature parity with the direct anthropic provider.

Authentication uses the standard AWS credential chain (AWS_ACCESS_KEY_ID env var, ~/.aws/credentials, instance profiles), or explicit credentials in config:

providers:
  bedrock:
    region: us-west-2
    access_key_id: $(op-cache read "op://Private/AWS Bedrock/AWS_ACCESS_KEY_ID")
    secret_access_key: $(op-cache read "op://Private/AWS Bedrock/AWS_SECRET_ACCESS_KEY")
    model: claude-sonnet-4-6-thinking

Model resolution uses a 3-tier system. Friendly model names like claude-sonnet-4-6 are automatically translated to Bedrock cross-region IDs. Use model_map to override with application inference profile ARNs or specific Bedrock IDs:

providers:
  bedrock:
    region: us-west-2
    model: claude-sonnet-4-6-thinking
    model_map:
      claude-sonnet-4-6: arn:aws:bedrock:us-west-2:123456789:application-inference-profile/abc123
      claude-opus-4-6: us.anthropic.claude-opus-4-6-v1

Suffixes are stripped before lookup, so claude-sonnet-4-6-1m-thinking strips to claude-sonnet-4-6, resolves through model_map, then re-applies thinking and 1M context.

The geographic prefix (us., eu., ap.) is derived from the configured region automatically. For example, eu-west-1 produces eu.anthropic.* IDs, ap-southeast-1 produces ap.anthropic.*, etc. This ensures data residency matches your region without manual override.

Raw Bedrock model IDs (us.anthropic.claude-sonnet-4-6, anthropic.claude-sonnet-4-6) and full ARNs are passed through without translation.

Native search support#

Some providers support native web search. Others rely on external search tooling.

Native support is most relevant for:

• Anthropic
• Bedrock
• OpenAI
• xAI
• Gemini

You can override behavior with:

term-llm ask "latest news" -s --native-search
term-llm ask "latest news" -s --no-native-search

Or in config:

search:
  force_external: true

providers:
  gemini:
    use_native_search: false

See Search for the full routing model.

Recommendations by use case#

• fast free experimentation: zen
• OpenAI ecosystem / Codex editing: openai
• Claude models: anthropic
• Claude models via AWS billing: bedrock
• broad model access: openrouter
• local inference: ollama or another OpenAI-compatible endpoint
• subscription-backed consumer access: chatgpt or copilot

Related pages#

• Configuration
• Search
• Providers and setup

ChatGPT context policy#

term-llm models -p chatgpt queries the authenticated Codex catalog, using a five-minute cache and stale-cache/static fallbacks when offline. Hidden Codex picker entries remain available for explicit model selection. ChatGPT models configuration augments the discovered shell-completion catalog rather than restricting it to the entries you customize.

term-llm chooses context in this order:

1. models[].context_window for the selected upstream model or alias.
2. Provider-level context_window (including newly discovered models).
3. term-llm’s shipped input budget.
4. The backend default for an unknown model.

The selected context is capped at the account’s known max_context_window. The backend’s context_window is a default, not that ceiling. When offline, the last cached maximum is used; without cache, bundled maxima are used where known. For models with no known maximum, an available backend default or shipped budget is the conservative ceiling; a configuration value alone cannot establish a larger supported window.

The shipped budgets for GPT-6 Astra and GPT-5.6 Sol/Terra/Luna are 372,000 tokens. GPT-5.4 retains its previous 922,000-token budget. These are capped when the account reports a smaller maximum. Astra’s bundled maximum is 872,000, not its 272,000-token Codex default.

providers:
  chatgpt:
    # Optional default for all ChatGPT models; each model's maximum still applies.
    context_window: 372000
    models:
      - id: gpt-6-astra
        context_window: 600000
      - id: gpt-5.6-sol
        context_window: 372000

Omit context_window (or set it to zero) to use the shipped policy. Context settings control local tracking and automatic compaction; they are not sent as an unsupported Responses API parameter. With context_window alone, the selected window is the input budget. If max_output_tokens is explicitly configured too, it is reserved after clamping the context. Shipped budgets already include headroom and do not have the model’s theoretical output maximum subtracted again. The existing soft/hard compaction thresholds apply to the resulting input budget. An output reservation that consumes the entire window leaves a minimum one-token input budget; choose a smaller reservation instead.

Model listing shows selected, recommended, and max context. Recommended is the shipped/default budget capped at the known maximum; selected is the local context target after user overrides and clamping, not a server-confirmed allocation. Only when an explicit output reservation reduces the input budget, a second line shows Input budget and Output reserve. The reserve is local budgeting, not an enforced ChatGPT generation limit. For example:

gpt-6-astra [context: 600K selected, 372K recommended, 872K max]
  Input budget: 580K · Output reserve: 20K

--json continues to include backend_context, recommended_context, max_context, configured_context, and input_limit. Caches store backend facts, not user settings: changing config does not require clearing the cache. Custom provider keys with type: chatgpt have independent context settings.