feat(opencode): add auto model detection for OpenAI-compatible providers #8359

tomzhu1024 · 2026-01-14T05:18:06Z

What does this PR do?

This PR adds automatic model detection/discovery for providers that use npm: @ai-sdk/openai-compatible. It allows OpenCode to utilize the GET /v1/models API to populate the model list.

Models may come from 3 sources now: Models.dev, config file, and detected via API. The "filtering" logic is: the model will be visible, if it's detected via API or declared in config file; if it is visible, its metadata will be merged from API -> config -> Models.dev (with prescedence from high to low).

Fixes #6231
Fixes #4232
Fixes #2456

Note: I understand it may be by design to rely entirely on Models.dev for providers and models, but having OpenCode core automatically populate the model list would be a major improvement for users running LM Studio or other local providers.

How did you verify your code works?

I ran an LM Studio server locally and downloaded two models (openai/gpt-oss-20b and zai-org/glm-4.6v-flash).
I configured my ~/.config/opencode/opencode.json with two LM Studio providers and two models. One of the models does not exist in LM Studio, but should still appear later. (File attached: opencode.json).
I used a debugger and breakpoints to inspect the providers value at packages/opencode/src/provider/provider.ts#958, and it looked correct to me.
I ran bun dev -- models, and the output looked correct.
I launched the TUI with bun dev, ran /models, and the output also looked correct.
From the TUI, I was able to use both local models in LM Studio and cloud models I had previously configured.
I shut down the LM Studio server and repeated steps 3–6; everything behaved as expected.

github-actions · 2026-01-14T05:18:41Z

The following comment was made by an LLM, it may be inaccurate:

Potential Duplicate PRs Found

Based on the search results, there are two PRs that appear to be related to or potentially addressing similar functionality:

PR Fetch from model list from /v1/models/ from OpenAI Compatible APIs #3427: "Fetch from model list from /v1/models/ from OpenAI Compatible APIs"
- Fetch from model list from /v1/models/ from OpenAI Compatible APIs #3427
- This PR directly addresses fetching models from the OpenAI-compatible /v1/models/ endpoint, which is the core feature of PR feat(opencode): add auto model detection for OpenAI-compatible providers #8359.
PR Adding the auto-detection of ollama local with a variable for baseURL #3726: "Adding the auto-detection of ollama local with a variable for baseURL"
- Adding the auto-detection of ollama local with a variable for baseURL #3726
- This PR adds auto-detection for Ollama (another local provider), which is related to the automatic model detection feature being added for local providers in PR feat(opencode): add auto model detection for OpenAI-compatible providers #8359.

Recommendation: You should review these PRs to check if they have been merged, closed, or superseded, and whether PR #8359 builds upon or duplicates their work.

rekram1-node · 2026-01-14T05:26:37Z

I was going to add this this week, some notes:

ideally no extra config option
models SHOULD be sourced from the provider models endpoint to a degree for all openai compat, like say models.dev says they have N models but the provider endpoint returns N-2, we should drop those 2 models because normally those aren't supported by that persons subscription or they disabled them or something (if this makes sense),
Does the lmstudio endpoint not return any info about the model? Context size? Supported Attachments? Etc?

tomzhu1024 · 2026-01-14T05:50:47Z

The LMStudio endpoint only returns the model ID, effectively... Here is an example:

- curl http://192.168.10.203:1234/v1/models
{
  "data": [
    {
      "id": "zai-org/glm-4.6v-flash",
      "object": "model",
      "owned_by": "organization_owner"
    },
    {
      "id": "qwen/qwen3-vl-8b",
      "object": "model",
      "owned_by": "organization_owner"
    },
    {
      "id": "google/gemma-3-12b",
      "object": "model",
      "owned_by": "organization_owner"
    },
    {
      "id": "openai/gpt-oss-20b",
      "object": "model",
      "owned_by": "organization_owner"
    },
    {
      "id": "text-embedding-nomic-embed-text-v1.5",
      "object": "model",
      "owned_by": "organization_owner"
    }
  ],
  "object": "list"
}

I believe using LM Studio's CLI or SDK we can have access to more information about the models. But that'll be provider-specific (like it won't work for Ollama).
Agree with no extra config option. I tried to still honor the config file because it allows the user to provide a prettier name to the model, and replace the model ID. To achieve this, I put my logic in a early stage of the state initialization, before collecting the auth info. This means that I won't be able to retrieve model list from cloud provider that use @ai-sdk/openai-compatible (they usually require auth). So, I have to introduce a extra config option such that this feature won't cause troubles for cloud providers.
Agree that the model info from Models.dev should be completely dropped when OpenAI-compatible API is returning the model list. I thought the same way when I drafted this PR.

goniz · 2026-01-14T06:36:59Z

I wanted to add this as well!
I use some python scripts to generate opencode.json configs from LMStudio/ ollama apis.

LMStudio does have api to fetch each model info with context length etc

rekram1-node · 2026-01-14T07:38:07Z

Agree that the model info from Models.dev should be completely dropped when OpenAI-compatible API is returning the model list. I thought the same way when I drafted this PR.

It's not like "completely dropped" it should be like any model that exists in models.dev that isnt in models endpoint should be dropped but that doesnt mean drop all the useful metadata it's more to filter potentially unsupported models

rekram1-node · 2026-01-14T07:38:34Z

LMStudio does have api to fetch each model info with context length etc

Then we should defo use that, wanna do the same for ollama but that can be separate

goniz · 2026-01-14T07:44:26Z

@rekram1-node I would love to help out on this if possible

rekram1-node · 2026-01-14T15:48:42Z

Yeah up to yall I'd really like to ship this week so @tomzhu1024 if you aren't going to be available I think @goniz or myself will take care of this.

Ig we can build off this PR easily

tomzhu1024 · 2026-01-14T16:01:39Z

It's not like "completely dropped" it should be like any model that exists in models.dev that isnt in models endpoint should be dropped but that doesnt mean drop all the useful metadata it's more to filter potentially unsupported models

Got it. This is definitely the optimal way. Thanks for the clarification!

I was planning to iterate this PR later today -- I'm planning to 1) remove the extra config option, 2) still honor the model info from Models.dev (if they do exist according to the endpoint), 3) leave some space for provider-specific model info retrieval (like @goniz mentioned). I'm a little unsure about expanding this feature to all @ai-sdk/openai-compatible providers, as some of them requires auth and brings some uncertainty (There are many auth methods, and maybe it's not API key...). Any thoughts or suggestions?

But feel free to take it over; just let me know so I can shift to something else.

rekram1-node · 2026-01-14T16:13:42Z

I'm a little unsure about expanding this feature to all @ai-sdk/openai-compatible providers, as some of them requires auth and brings some uncertainty (There are many auth methods, and maybe it's not API key...). Any thoughts or suggestions?

All the providers should work, I can test and iterate to make sure it works but ik u would use your github token from oauth exchange for copilot, and an api key for zen, etc so i think most things will be pretty standard.

tomzhu1024 · 2026-01-14T16:17:04Z

I see. That sounds good. I'll then try to 4) expand it to all @ai-sdk/openai-compatible providers.

goniz · 2026-01-14T23:10:30Z

I've got initial draft for probing loaded models for LMStudio and Ollama.

I'm using the opencode debug cli which is nice.

❯ bun dev debug provider probe
$ bun run --cwd packages/opencode --conditions=browser src/index.ts debug provider probe
opencode debug provider probe <type> <url>

probe a local provider for loaded models

Positionals:
  type  provider type                            [string] [required] [choices: "ollama", "lmstudio"]
  url   provider URL                                                             [string] [required]

Options:
  -h, --help        show help                                                              [boolean]
  -v, --version     show version number                                                    [boolean]
      --print-logs  print logs to stderr                                                   [boolean]
      --log-level   log level                   [string] [choices: "DEBUG", "INFO", "WARN", "ERROR"]
error: script "dev" exited with code 1

Ollama

❯ bun dev debug provider probe ollama http://localhost:11434
$ bun run --cwd packages/opencode --conditions=browser src/index.ts debug provider probe ollama "http://localhost:11434"
{
  "available": true,
  "url": "http://localhost:11434",
  "models": [
    {
      "name": "qwen3-coder:30b-a3b-q8_0",
      "digest": "7b438a19895a90821e60a42ed894cd40e746082200dff6aa5a4285b529e2a4a5",
      "size": 36133302276,
      "sizeVram": 36133302276,
      "contextLength": 65536,
      "expiresAt": "2026-01-14T23:00:31.532Z",
      "format": "gguf",
      "family": "qwen3moe",
      "parameterSize": "30.5B",
      "quantizationLevel": "Q8_0"
    }
  ]
}

LMStudio

❯ bun dev debug provider probe lmstudio http://localhost:1234/
$ bun run --cwd packages/opencode --conditions=browser src/index.ts debug provider probe lmstudio "http://localhost:1234/"
{
  "available": true,
  "url": "http://localhost:1234/",
  "models": [
    {
      "id": "qwen3-vl-8b-instruct",
      "type": "vlm",
      "publisher": "unsloth",
      "arch": "qwen3vl",
      "compatibilityType": "gguf",
      "quantization": "Q4_K_XL",
      "loadedContextLength": 262144
    }
  ]
}

The output is raw atm but I'll fix it tomorrow then I'll push something

tomzhu1024 · 2026-01-15T02:24:01Z

I just got a chance to work on it. Now:

No extra config option
Model metadata from Models.dev is used (if the model is detected via API, otherwise the model won't show)
This logic will apply to all @ai-sdk/openai-compatible providers. I've tested opencode, github-copilot, lmstudio, ollama-cloud, zai-coding-plan on my side, and they're all working.
I tried to leave some space for @goniz's idea of using LMStudio and Ollama's specific APIs to retrieve more model metadata. If we want, we can extend the current model retrieval logic.

rekram1-node · 2026-01-15T06:03:24Z

packages/opencode/src/provider/provider.ts

    }

+    // auto-detect models
+    for (const [providerID, provider] of Object.entries(providers)) {


We need to make this somewhat optimal so make sure:

this isnt running for every single provider in the entire models list (only authed ones) << or maybe we need to move this logic to behave differently?

we should Promise.all or use worker queue to prevent super slow linear requests

we should have timeouts (looks like u added that already -- I think we should go lower tho like 1-3 sec tops at least if it is gonna be blocking...)

Do you mean this should run only for providers that are "activated" via the config or auth json or env var, rather than for all providers listed in Models.dev? If so, it already works that way.

As for the placement, I put it near the end because that’s where enabled providers are picked and their auth information is collected. Do you see a better place for it?

Sure, I'll use Promise.all.

I'll reduce the timeout to 3 secs. Since all providers are queried in parallel, this shouldn’t add much lag.

Done. I moved this slightly in provider.ts so custom providers like "ollama" aren’t skipped just because they don’t exist in Models.dev, and the model.api.npm patch for @ai-sdk/github-copilot no longer has to run twice.

awesome, ill take a look

goniz · 2026-01-15T22:17:05Z

I've pushed my PR with the ollama+lmstudio apis.
We can integrate with it after this PR is merged.

#8748

rekram1-node · 2026-01-16T01:44:43Z

/review

github-actions · 2026-01-16T01:45:58Z

packages/opencode/src/provider/model-detection.ts

+    const providerNPM = configProvider?.npm ?? modelsDevProvider?.npm ?? "@ai-sdk/openai-compatible"
+    const providerBaseURL = configProvider?.options?.baseURL ?? configProvider?.api ?? modelsDevProvider?.api
+
+    let detectedModels: Record<string, Partial<Provider.Model>> | undefined = undefined


Style suggestion: Consider refactoring to avoid the let statement. You could use an early return pattern or the Result pattern. For example:

if (providerNPM !== "@ai-sdk/openai-compatible" || !providerBaseURL) return const detectedModels = await ProviderModelDetection.OpenAICompatible.listModels(providerBaseURL, provider)

This eliminates the need for let and the subsequent if (detectedModels === undefined) return check becomes unnecessary.

This is mainly for future extensibility, like we may have model detection for non-@ai-sdk/openai-compatible providers, or we may have a different detection method for a specific @ai-sdk/compatible provider (e.g. LMStudio/Ollama).

github-actions · 2026-01-16T01:46:03Z

packages/opencode/src/provider/model-detection.ts

+  type OpenAICompatibleResponse = z.infer<typeof OpenAICompatibleResponse>
+
+  export async function listModels(baseURL: string, provider: Provider.Info): Promise<Record<string, Partial<Provider.Model>>> {
+    let res: Response


Style suggestion: These let statements combined with multiple try/catch blocks could potentially be refactored. The style guide advises avoiding let and try/catch where possible.

One option is to let exceptions propagate and handle them at the call site (which is already wrapped in a try/catch in provider.ts). This would simplify the function to:

const res = await fetchFn(`${baseURL}/models`, { headers, signal: AbortSignal.timeout(3 * 1000), }) if (!res.ok) throw new Error(`bad http status ${res.status}`) const parsed = OpenAICompatibleResponse.parse(await res.json())

The error messages would be less descriptive, but the call site already logs the error. Up to you whether the explicit error wrapping is worth the complexity.

tomzhu1024 force-pushed the model-detection branch from 23396df to 5080e14 Compare January 15, 2026 02:17

tomzhu1024 changed the title ~~feat(opencode): add model detection for local providers~~ feat(opencode): add auto model detection for OpenAI-compatible providers Jan 15, 2026

tomzhu1024 added 3 commits January 14, 2026 21:51

feat: add model detection for local providers

9378112

fix: add timeout to fetch

381e710

feat: support all @ai-sdk/openai-compatible providers

875c8bc

tomzhu1024 force-pushed the model-detection branch from 5080e14 to 875c8bc Compare January 15, 2026 02:51

fix: github chat issue

2b500dc

rekram1-node reviewed Jan 15, 2026

View reviewed changes

fix: add parallel request and decrease timeout

1e9f79b

github-actions bot mentioned this pull request Jan 15, 2026

feat(opencode): added logic to probe loaded models from lmstudio and ollama #8748

Open

github-actions bot reviewed Jan 16, 2026

View reviewed changes

tomzhu1024 added 2 commits January 15, 2026 20:58

refactor: remove let

1e97fdb

chore: improve logging format

a9ed14a

feat(opencode): add auto model detection for OpenAI-compatible providers #8359

Are you sure you want to change the base?

feat(opencode): add auto model detection for OpenAI-compatible providers #8359

Conversation

tomzhu1024 commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

How did you verify your code works?

Uh oh!

github-actions bot commented Jan 14, 2026

Potential Duplicate PRs Found

Uh oh!

rekram1-node commented Jan 14, 2026

Uh oh!

tomzhu1024 commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

goniz commented Jan 14, 2026

Uh oh!

rekram1-node commented Jan 14, 2026

Uh oh!

rekram1-node commented Jan 14, 2026

Uh oh!

goniz commented Jan 14, 2026

Uh oh!

rekram1-node commented Jan 14, 2026

Uh oh!

tomzhu1024 commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rekram1-node commented Jan 14, 2026

Uh oh!

tomzhu1024 commented Jan 14, 2026

Uh oh!

goniz commented Jan 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ollama

LMStudio

Uh oh!

tomzhu1024 commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rekram1-node Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

tomzhu1024 Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

tomzhu1024 Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

rekram1-node Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

goniz commented Jan 15, 2026

Uh oh!

rekram1-node commented Jan 16, 2026

Uh oh!

github-actions bot Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

tomzhu1024 Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

tomzhu1024 Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions bot Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

tomzhu1024 Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

tomzhu1024 commented Jan 14, 2026 •

edited

Loading

tomzhu1024 commented Jan 14, 2026 •

edited

Loading

tomzhu1024 commented Jan 14, 2026 •

edited

Loading

goniz commented Jan 14, 2026 •

edited

Loading

tomzhu1024 commented Jan 15, 2026 •

edited

Loading