Providers and models

Discover providers and models

term-llm providers
term-llm providers --configured
term-llm providers anthropic

term-llm models --provider anthropic
term-llm models --provider openrouter
term-llm models --provider nearai
term-llm models --provider sambanova
term-llm models --provider ollama
term-llm models --json

Use providers when you want to know what is available and how it is configured. Use models when you want the concrete model names a provider currently exposes.

Provider categories

term-llm supports a mix of provider types:

• hosted API providers such as Anthropic, AWS Bedrock, OpenAI, xAI, Gemini, NEAR AI Cloud, SambaNova, and OpenRouter
• subscription-backed OAuth providers such as ChatGPT, Copilot, and Gemini CLI
• local or self-hosted OpenAI-compatible providers such as Ollama, LM Studio, vLLM, or custom endpoints

Credentials

Most providers use API keys via environment variables. Some use OAuth credentials from companion CLIs or locally stored auth files.

Examples:

term-llm ask --provider anthropic "question"
term-llm ask --provider chatgpt "question"
term-llm ask --provider copilot "question"
term-llm ask --provider gemini-cli "question"

WebSocket defaults

The built-in openai and chatgpt text providers use the Responses WebSocket transport by default. This improves latency in agentic/tool-heavy runs by reusing one connection and continuing compatible turns with previous_response_id plus only new input. If setup fails before streaming starts, term-llm falls back to HTTP/SSE; if a WebSocket continuation rejects the previous response ID, it retries once with full input.

To force HTTP/SSE for either built-in provider:

providers:
  openai:
    use_websocket: false
  chatgpt:
    use_websocket: false

OpenAI-compatible providers remain HTTP/SSE by default. WebSocket defaults are not applied to type: openai_compatible entries.

GPT-5.6 on OpenAI and ChatGPT

term-llm ships fallback metadata for the GPT-5.6 family:

The OpenAI model IDs are gpt-5.6-sol, gpt-5.6-terra, and gpt-5.6-luna. ChatGPT exposes the same named variants through its live Codex catalog; live input limits and reasoning metadata take precedence over the static fallback. Luna keeps its upstream medium default unless you explicitly select another effort.

Pro and Ultra are not reasoning efforts. On the public OpenAI API, Pro sends reasoning.mode: pro; Standard sends reasoning.mode: standard. In terminal chat use /pro on, /pro off, or /pro status. The browser shows a Standard/Pro selector only when the selected model advertises it. ChatGPT OAuth does not receive public-API Pro mode. Codex’s separate product-level Ultra option runs at max effort and adds subagents, so term-llm does not expose it as an effort.

OpenAI API GPT-5.6 also supports term-llm’s advanced Responses controls: reasoning context, hosted multi-agent execution, programmatic tool calling, and prompt-cache options. These controls are model/provider gated. term-llm rejects them for older OpenAI models, OpenAI-compatible providers, and the ChatGPT OAuth backend rather than sending unverified fields. See Configuration and Web UI and API.

Bundled OpenAI API prices per 1M tokens (uncached input / cache read / cache write / output) are Sol: $5 / $0.50 / $6.25 / $30; Terra: $2.50 / $0.25 / $3.125 / $15; Luna: $1 / $0.10 / $1.25 / $6. When total input (uncached + cache reads + cache writes) exceeds 272K, the entire request uses 2× input/read/write rates and 1.5× output rates. This is API-key billing metadata; ChatGPT subscription access is not billed from these static prices.

SambaNova Cloud

SambaNova is available as a built-in OpenAI-compatible provider:

export SAMBANOVA_API_KEY=your-key
term-llm ask --provider sambanova:gpt-oss-120b "quick question"
term-llm models --provider sambanova

providers:
  sambanova:
    model: gpt-oss-120b
    fast_model: Meta-Llama-3.3-70B-Instruct

The provider uses https://api.sambanova.ai/v1, supports tool calls, and has a curated fallback model list. term-llm models --provider sambanova queries SambaNova’s /models endpoint; because the OpenAI-compatible model response does not generally include price metadata, term-llm annotates known SambaNova models with bundled public prices from https://cloud.sambanova.ai/plans/pricing. These prices are also used by term-llm usage cost calculation for matching SambaNova model IDs.

NEAR AI Cloud

NEAR AI Cloud is available as a built-in OpenAI-compatible provider for TEE-backed private inference:

export NEARAI_API_KEY=your-key
term-llm ask --provider nearai:zai-org/GLM-5.1-FP8 "quick question"
term-llm models --provider nearai

providers:
  nearai:
    model: zai-org/GLM-5.1-FP8
    fast_model: Qwen/Qwen3.6-35B-A3B-FP8

The provider uses https://cloud-api.near.ai/v1, supports tool calls, and has a curated fallback list of TEE-hosted text models. term-llm models --provider nearai queries NEAR AI Cloud’s public /model/list catalog and filters it to chat-capable models, with token prices shown per 1M tokens when available.

vLLM reasoning providers

For vLLM servers running reasoning models, prefer type: vllm instead of generic openai_compatible. The vLLM provider still uses the OpenAI-compatible /v1/chat/completions API, but maps term-llm reasoning effort suffixes into model-family-specific vLLM request fields and replays prior assistant reasoning with vLLM’s current reasoning message field.

For Qwen-family models, term-llm sends Qwen chat-template controls:

providers:
  cdck_qwen:
    type: vllm
    base_url: https://gpu-server.example.com:8000/v1
    model: Qwen/Qwen3.5-122B-A10B
    api_key: ${CDCK_QWEN_API_KEY}
    context_window: 200000
    max_output_tokens: 50000

Use the normal provider flag, or append an effort suffix to the configured provider name:

term-llm ask -p cdck_qwen "hello"          # default: no thinking
term-llm ask -p cdck_qwen-low "harder"     # thinking budget 1024
term-llm ask -p cdck_qwen-medium "hard"    # thinking budget 4096
term-llm ask -p cdck_qwen-high "very hard" # thinking budget 10000

The effort suffix is stripped before sending the model name upstream. For example, -p cdck_qwen-high sends model: Qwen/Qwen3.5-122B-A10B plus Qwen thinking controls, not a literal Qwen/Qwen3.5-122B-A10B-high model ID.

For vLLM templates that use chat_template_kwargs.thinking, declare the model’s exact capabilities and default in config. vllm_thinking_param: thinking selects the request shape; term-llm does not embed model-specific effort mappings:

providers:
  cdck_deepseek:
    type: vllm
    base_url: https://gpu-server.example.com:8000/v1
    model: deepseek-ai/DeepSeek-V4-Flash
    vllm_thinking_param: thinking
    models:
      - id: deepseek-ai/DeepSeek-V4-Flash
        alias: deepseek-v4-flash
        reasoning_efforts: [none, low, high, max]
        default_reasoning_effort: high

The bare provider/model uses default_reasoning_effort. -p cdck_deepseek-<TAB> completes only the values listed in reasoning_efforts. Model-name detection for deepseek remains as a backward-compatible fallback when vllm_thinking_param is omitted; explicit configuration is preferred for aliases. Requests using the thinking shape do not send thinking_token_budget.

Notes:

• Start vLLM with the appropriate reasoning parser for your model, for example --reasoning-parser qwen3 for Qwen3-family reasoning output or --reasoning-parser deepseek_v3 / deepseek_v4 for DeepSeek variants.
• Qwen thinking_token_budget requires a vLLM server new enough to support it and, on recent vLLM, a server-side --reasoning-config. Plain/default Qwen requests omit the budget so they work without that extra server option.
• vLLM currently streams reasoning text in delta.reasoning, but may not report usage.completion_tokens_details.reasoning_tokens accurately. In that case term-llm can show reasoning in debug output while reasoning_tokens remains 0; this reflects vLLM usage metadata, not missing reasoning text.
• For multi-turn conversations, term-llm persists streamed reasoning and replays it as assistant reasoning on the next vLLM request. This lets vLLM’s chat template render the prior reasoning consistently and gives prefix caching the best chance to reuse shared prompt prefixes when the server has prefix caching enabled.

OpenAI-compatible providers

For local or custom backends that do not need vLLM chat-template thinking controls, use type: openai_compatible.

providers:
  ollama:
    type: openai_compatible
    base_url: http://localhost:11434/v1
    model: llama3.2:latest

  lmstudio:
    type: openai_compatible
    base_url: http://localhost:1234/v1
    model: deepseek-coder-v2

  cerebras:
    type: openai_compatible
    base_url: https://api.cerebras.ai/v1
    model: llama-4-scout-17b
    api_key: ${CEREBRAS_API_KEY}

Use base_url when the standard /chat/completions path should be appended automatically. Use url when you need to specify the full chat completions endpoint directly.

Configuration reference

Model object entries

models may mix plain strings and objects. Plain strings are enough for autocomplete/model picker entries. Object entries are for endpoints where the model you want to type locally differs from the model ID the API expects, or where each model needs its own metadata.

providers:
  custom:
    type: openai_compatible
    base_url: https://api.example.com/v1
    api_key: ${CUSTOM_API_KEY}
    model: friendly-name
    # Optional provider-level default for all models under this provider.
    # The target may be provider:model, or just provider to use that provider's default model.
    vision_via: gemini
    models:
      - simple-upstream-model
      - id: upstream/model-id
        alias: friendly-name
        context_window: 262144
        max_output_tokens: 32768
        # Optional: only set these for APIs/models that support them.
        parse_reasoning: true
        include_reasoning: true
        thinking_param: enable_thinking
        reasoning_efforts: [none, low, high, max]
        default_reasoning_effort: high
        # Optional per-model override; if omitted, provider-level vision_via is used.
        vision_via: gemini:gemini-2.5-pro

For -p custom:friendly-name-max, term-llm sends model: upstream/model-id plus reasoning_effort: max. When default_reasoning_effort is set, the bare provider/model uses that effort; otherwise it sends no effort. If reasoning_efforts is empty or omitted, no configured effort aliases or provider-profile completions are generated for that model.

Friendli reasoning

Friendli is OpenAI-compatible, but reasoning-capable models such as GLM-5.2 need explicit parser flags to expose thinking as reasoning_content instead of leaving it inline. Configure those fields on the specific model entry:

providers:
  friendli:
    type: openai_compatible
    base_url: https://api.friendli.ai/serverless/v1
    api_key: ${FRIENDLI_API_KEY}
    model: glm52
    models:
      - id: zai-org/GLM-5.2
        alias: glm52
        context_window: 1048576
        max_output_tokens: 131072
        parse_reasoning: true
        include_reasoning: true
        thinking_param: enable_thinking
        reasoning_efforts: [high, max]

With that config, effort suffixes are generated only from the declared reasoning_efforts. For example -p friendli:glm52-max sends model: zai-org/GLM-5.2, reasoning_effort: max, parse_reasoning: true, include_reasoning: true, and chat_template_kwargs.enable_thinking: true. The bare -p friendli:glm52 sends no reasoning_effort; because only high and max are listed, term-llm does not offer glm52-low or glm52-medium completions.

You can set the same reasoning-parser fields at the provider level as defaults, then override them per model object when different models on the same endpoint need different behavior.

Full example

providers:
  my-vllm:
    type: vllm
    base_url: http://gpu-server:8000/v1
    model: Qwen/Qwen3-30B-A3B
    api_key: ${VLLM_API_KEY}
    context_window: 32768
    max_output_tokens: 8192
    models:
      - Qwen/Qwen3-30B-A3B
      - Qwen/Qwen3-8B

  legacy-server:
    type: openai_compatible
    url: http://old-server:5000/api/chat
    model: custom-finetune
    no_stream_options: true  # this server rejects stream_options

Service tiers and fast mode

Built-in openai and chatgpt text providers can send the Responses API service_tier field. To request fast/priority service for all turns through a provider, set service_tier in that provider config:

providers:
  openai:
    model: gpt-5.6-sol
    fast_model: gpt-5.6-luna
    service_tier: fast      # alias for API value "priority"

  chatgpt:
    model: gpt-5.6-sol-medium
    fast_model: gpt-5.6-luna
    service_tier: priority  # equivalent to "fast"

Leave service_tier unset to omit the field entirely. Only some models/accounts support fast service; unsupported requests may be ignored or rejected by the provider. In chat, /fast toggles the fast service tier for the current session. The status line shows fast when it is currently active.

This is different from fast_model / optional fast_provider, which choose a lightweight model for term-llm control-plane tasks such as summaries or title generation, and for agent configs that use model: fast.

Reasoning and model suffixes

Model/provider suffixes control how much reasoning a provider is asked to do. Display of the resulting reasoning is controlled separately by the top-level reasoning config. Non-encrypted provider-marked thinking is shown as collapsed Thinking... / Thought: <title> blocks by default; encrypted reasoning/signature payloads are replay-only and are never displayed.

OpenAI reasoning effort

For GPT-5.6 on the OpenAI API, append -none, -low, -medium, -high, -xhigh, or -max to control reasoning effort. An explicit reasoning_effort API field wins over a model suffix, and a request suffix wins over a provider-level default.

term-llm ask --provider openai:gpt-5.6-sol-xhigh "complex question"
term-llm exec --provider openai:gpt-5.6-luna-low "quick task"

providers:
  openai:
    model: gpt-5.6-sol-high

Older GPT-5 models retain their existing provider-specific effort sets. Natural model IDs ending in a suffix-like word are preserved when that suffix is not valid for the model; for example, gpt-5.1-codex-max remains a model ID rather than being split into model plus max effort.

For ChatGPT OAuth, GPT-5.6 Sol, Terra, and Luna expose low, medium, high, xhigh, and max. Codex’s product-level Ultra option is not an inference effort: the official client sends max and separately enables subagents, so it is not included in term-llm’s effort suffixes.

vLLM thinking suffixes

For configured providers with type: vllm, effort suffixes can be applied directly to the provider name. Declare them on the selected model to drive parsing and shell completion:

providers:
  cdck_deepseek:
    type: vllm
    base_url: https://gpu-server.example.com:8000/v1
    model: deepseek-ai/DeepSeek-V4-Flash
    vllm_thinking_param: thinking
    models:
      - id: deepseek-ai/DeepSeek-V4-Flash
        alias: deepseek-v4-flash
        reasoning_efforts: [none, low, high, max]
        default_reasoning_effort: high

term-llm ask -p cdck_deepseek "uses configured high default"
term-llm ask -p cdck_deepseek-low "override with low"
term-llm ask -p cdck_deepseek-none "disable thinking"

The suffix is stripped before the model ID is sent upstream. -p cdck_deepseek-<TAB> offers exactly the declared efforts, and undeclared provider-profile suffixes are rejected.

For Qwen-style enable_thinking templates, term-llm retains its budget mapping:

For thinking templates, configured enabled efforts pass through unchanged:

Reasoning replay uses vLLM’s reasoning assistant-message field. vLLM may still report reasoning_tokens: 0 in usage metadata even when reasoning text was streamed; this is a known vLLM-side accounting gap.

Anthropic extended thinking

For Anthropic models, append -thinking:

term-llm ask --provider anthropic:claude-sonnet-4-6-thinking "complex question"

providers:
  anthropic:
    model: claude-sonnet-4-6-thinking

AWS Bedrock

The bedrock provider routes Anthropic Claude models through AWS Bedrock. It supports the same model suffixes (-thinking, -1m) and has full feature parity with the direct anthropic provider.

Authentication uses the standard AWS credential chain (AWS_ACCESS_KEY_ID env var, ~/.aws/credentials, instance profiles), or explicit credentials in config:

providers:
  bedrock:
    region: us-west-2
    access_key_id: $(op-cache read "op://Private/AWS Bedrock/AWS_ACCESS_KEY_ID")
    secret_access_key: $(op-cache read "op://Private/AWS Bedrock/AWS_SECRET_ACCESS_KEY")
    model: claude-sonnet-4-6-thinking

Model resolution uses a 3-tier system. Friendly model names like claude-sonnet-4-6 are automatically translated to Bedrock cross-region IDs. Use model_map to override with application inference profile ARNs or specific Bedrock IDs:

providers:
  bedrock:
    region: us-west-2
    model: claude-sonnet-4-6-thinking
    model_map:
      claude-sonnet-4-6: arn:aws:bedrock:us-west-2:123456789:application-inference-profile/abc123
      claude-opus-4-6: us.anthropic.claude-opus-4-6-v1

Suffixes are stripped before lookup, so claude-sonnet-4-6-1m-thinking strips to claude-sonnet-4-6, resolves through model_map, then re-applies thinking and 1M context.

The geographic prefix (us., eu., ap.) is derived from the configured region automatically. For example, eu-west-1 produces eu.anthropic.* IDs, ap-southeast-1 produces ap.anthropic.*, etc. This ensures data residency matches your region without manual override.

Raw Bedrock model IDs (us.anthropic.claude-sonnet-4-6, anthropic.claude-sonnet-4-6) and full ARNs are passed through without translation.

Native search support

Some providers support native web search. Others rely on external search tooling.

Native support is most relevant for:

• Anthropic
• Bedrock
• OpenAI
• xAI
• Gemini

You can override behavior with:

term-llm ask "latest news" -s --native-search
term-llm ask "latest news" -s --no-native-search

Or in config:

search:
  force_external: true

providers:
  gemini:
    use_native_search: false

See Search for the full routing model.

Recommendations by use case

• fast free experimentation: zen
• OpenAI ecosystem / Codex editing: openai
• Claude models: anthropic
• Claude models via AWS billing: bedrock
• broad model access: openrouter
• local inference: ollama or another OpenAI-compatible endpoint
• subscription-backed consumer access: chatgpt, copilot, or gemini-cli

Related pages

• Configuration
• Search
• Providers and setup