Skip to content

Commit c5ee8e7

Browse files
ericdalloericdallo
authored
Merge pull request #245 from editor-code-assistant/dynamic-model-discovery
Add dynamic model discovery via /models endpoint
2 parents ce986ee + 8c000e3 commit c5ee8e7

File tree

6 files changed

+322
-37
lines changed

6 files changed

+322
-37
lines changed

‎CHANGELOG.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22

33
## Unreleased
44

5-
- Improve error handleing for incompatible models messages in chat. #209
5+
- Add dynamic model discovery via `fetchModels` provider config for OpenAI-compatible `/models` endpoints
6+
- Improve error handling for incompatible models messages in chat. #209
67

78
## 0.87.2
89

‎docs/configuration.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ There are multiples ways to configure ECA:
4747
```bash
4848
ECA_CONFIG='{"myConfig": "my_value"}' eca server
4949
```
50-
50+
5151
### Dynamic string contents
5252

5353
It's possible to retrieve content of any configs with a string value using the `${key:value}` approach, being `key`:
@@ -276,7 +276,7 @@ You can configure in multiple different ways:
276276
```markdown title=".eca/commands/check-performance.md"
277277
Check for performance issues in $ARG1 and optimize if needed.
278278
```
279-
279+
280280
ECA will make available a `/check-performance` command after creating that file.
281281

282282
=== "Global custom commands"
@@ -286,7 +286,7 @@ You can configure in multiple different ways:
286286
```markdown title="~/.config/eca/commands/check-performance.md"
287287
Check for performance issues in $ARG1 and optimize if needed.
288288
```
289-
289+
290290
ECA will make available a `/check-performance` command after creating that file.
291291

292292
=== "Config"
@@ -298,7 +298,7 @@ You can configure in multiple different ways:
298298
"commands": [{"path": "my-custom-prompt.md"}]
299299
}
300300
```
301-
301+
302302
ECA will make available a `/my-custom-prompt` command after creating that file.
303303

304304
## Rules
@@ -576,6 +576,7 @@ To configure, add your OTLP collector config via `:otlp` map following [otlp aut
576576
interface Config {
577577
providers?: {[key: string]: {
578578
api?: 'openai-responses' | 'openai-chat' | 'anthropic';
579+
fetchModels?: boolean;
579580
url?: string;
580581
key?: string; // when provider supports api key.
581582
keyRc?: string; // credential file lookup in format [login@]machine[:port]

‎docs/models.md‎

Lines changed: 25 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ Schema:
7373
| `models` | map | Key: model name, value: its config | Yes |
7474
| `models <model> extraPayload` | map | Extra payload sent in body to LLM | No |
7575
| `models <model> modelName` | string | Override model name, useful to have multiple models with different configs and names that use same LLM model | No |
76+
| `fetchModels` | boolean | Enable automatic model discovery from `/models` endpoint (OpenAI-compatible providers) | No |
7677

7778
_* url and key will be searched as envs `<provider>_API_URL` and `<provider>_API_KEY`, they require the env to be found or config to work._
7879

@@ -105,7 +106,7 @@ Examples:
105106
"providers": {
106107
"openai": {
107108
"api": "openai-responses",
108-
"models": {
109+
"models": {
109110
"gpt-5": {},
110111
"gpt-5-high": {
111112
"modelName": "gpt-5",
@@ -116,9 +117,28 @@ Examples:
116117
}
117118
}
118119
```
119-
120+
120121
This way both will use gpt-5 model but one will override the reasoning to be high instead of the default.
121122

123+
=== "Dynamic model discovery"
124+
125+
For OpenAI-compatible providers, set `fetchModels: true` to automatically discover available models:
126+
127+
```javascript title="~/.config/eca/config.json"
128+
{
129+
"providers": {
130+
"openrouter": {
131+
"api": "openai-chat",
132+
"url": "https://openrouter.ai/api/v1",
133+
"key": "your-api-key",
134+
"fetchModels": true
135+
}
136+
}
137+
}
138+
```
139+
140+
Static `models` config overrides discovered models, allowing customization.
141+
122142
### API Types
123143

124144
When configuring custom providers, choose the appropriate API type:
@@ -335,11 +355,11 @@ Notes:
335355
}
336356
}
337357
```
338-
358+
339359
=== "LM Studio"
340-
360+
341361
This config works with LM studio:
342-
362+
343363
```javascript title="~/.config/eca/config.json"
344364
{
345365
"providers": {

‎src/eca/llm_api.clj‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -200,7 +200,9 @@
200200
:extra-payload extra-payload}
201201
callbacks)
202202

203-
(and model-config handler)
203+
(and (or (:fetchModels provider-config)
204+
model-config)
205+
handler)
204206
(let [url-relative-path (:completionUrlRelativePath provider-config)
205207
think-tag-start (:thinkTagStart provider-config)
206208
think-tag-end (:thinkTagEnd provider-config)

‎src/eca/models.clj‎

Lines changed: 121 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@
66
[eca.llm-providers.ollama :as llm-providers.ollama]
77
[eca.llm-util :as llm-util]
88
[eca.logger :as logger]
9-
[eca.shared :refer [assoc-some] :as shared]))
9+
[eca.shared :refer [assoc-some] :as shared]
10+
[hato.client :as http]))
1011

1112
(set! *warn-on-reflection* true)
1213

@@ -77,36 +78,130 @@
7778
(and (llm-util/provider-api-url provider config)
7879
(llm-util/provider-api-key provider (get-in db [:auth provider]) config)))))
7980

81+
(def ^:private models-endpoint-path "/models")
82+
83+
(defn ^:private fetch-compatible-models
84+
"Fetches models from an /models endpoint (both Anthropic and OpenAI).
85+
Returns a map of model-id -> {} (empty config, to be enriched later).
86+
On any error, logs a warning and returns nil."
87+
[{:keys [api-url api-key provider]}]
88+
(when api-url
89+
(let [url (str api-url models-endpoint-path)
90+
rid (llm-util/gen-rid)
91+
headers (cond-> {"Content-Type" "application/json"}
92+
api-key (assoc "Authorization" (str "Bearer " api-key)))]
93+
(try
94+
(llm-util/log-request logger-tag rid url nil headers)
95+
(let [{:keys [status body]} (http/get url
96+
{:headers headers
97+
:throw-exceptions? false
98+
:as :json
99+
:timeout 10000})]
100+
(if (= 200 status)
101+
(do
102+
(llm-util/log-response logger-tag rid "models" body)
103+
(let [models-data (:data body)]
104+
(when (seq models-data)
105+
(reduce
106+
(fn [acc model]
107+
(let [model-id (:id model)]
108+
(if model-id
109+
(assoc acc model-id {})
110+
acc)))
111+
{}
112+
models-data))))
113+
(logger/warn logger-tag
114+
(format "Provider '%s': /models endpoint returned status %s"
115+
provider status))))
116+
(catch Exception e
117+
(logger/warn logger-tag
118+
(format "Provider '%s': Failed to fetch models from %s: %s"
119+
provider url (ex-message e))))))))
120+
121+
(defn ^:private provider-with-fetch-models?
122+
"Returns true if provider should fetch models dynamically (fetchModels = true)."
123+
[provider-config]
124+
(and (:api provider-config)
125+
(true? (:fetchModels provider-config))))
126+
127+
(defn ^:private fetch-dynamic-provider-models
128+
"For providers that support dynamic model discovery,
129+
attempts to fetch available models from the API.
130+
Returns a map of {provider-name -> {model-id -> model-config}}."
131+
[config db]
132+
(reduce
133+
(fn [acc [provider provider-config]]
134+
(if (provider-with-fetch-models? provider-config)
135+
(let [api-url (llm-util/provider-api-url provider config)
136+
[_auth-type api-key] (llm-util/provider-api-key provider
137+
(get-in db [:auth provider])
138+
config)
139+
fetched-models (fetch-compatible-models
140+
{:api-url api-url
141+
:api-key api-key
142+
:provider provider})]
143+
(if (seq fetched-models)
144+
(do
145+
(logger/info logger-tag
146+
(format "Provider '%s': Discovered %d models from /models endpoint"
147+
provider (count fetched-models)))
148+
(assoc acc provider fetched-models))
149+
acc))
150+
acc))
151+
{}
152+
(:providers config)))
153+
154+
(defn ^:private build-model-capabilities
155+
"Build capabilities for a single model, looking up from known models database."
156+
[all-models provider model model-config]
157+
(let [real-model-name (or (:modelName model-config) model)
158+
full-real-model (str provider "/" real-model-name)
159+
full-model (str provider "/" model)
160+
model-capabilities (merge
161+
(or (get all-models full-real-model)
162+
;; we guess the capabilities from
163+
;; the first model with same name
164+
(when-let [found-full-model
165+
(->> (keys all-models)
166+
(filter #(or (= (shared/normalize-model-name (string/replace-first real-model-name
167+
#"(.+/)"
168+
""))
169+
(shared/normalize-model-name (second (string/split % #"/" 2))))
170+
(= (shared/normalize-model-name real-model-name)
171+
(shared/normalize-model-name (second (string/split % #"/" 2))))))
172+
first)]
173+
(get all-models found-full-model))
174+
{:tools true
175+
:reason? true
176+
:web-search false})
177+
{:model-name real-model-name})]
178+
[full-model model-capabilities]))
179+
180+
(defn ^:private merge-provider-models
181+
"Merges static config models with dynamically fetched models.
182+
Static config takes precedence (allows user overrides)."
183+
[static-models dynamic-models]
184+
(merge dynamic-models static-models))
185+
80186
(defn sync-models! [db* config on-models-updated]
81187
(let [all-models (all)
82188
db @db*
189+
;; Fetch dynamic models for providers that support it
190+
dynamic-provider-models (fetch-dynamic-provider-models config db)
191+
;; Build all supported models from config + dynamic sources
83192
all-supported-models (reduce
84193
(fn [p [provider provider-config]]
85-
(merge p
86-
(reduce
87-
(fn [m [model model-config]]
88-
(let [real-model-name (or (:modelName model-config) model)
89-
full-real-model (str provider "/" real-model-name)
90-
full-model (str provider "/" model)
91-
model-capabilities (merge
92-
(or (get all-models full-real-model)
93-
;; we guess the capabilities from
94-
;; the first model with same name
95-
(when-let [found-full-model (first (filter #(or (= (shared/normalize-model-name (string/replace-first real-model-name
96-
#"(.+/)"
97-
""))
98-
(shared/normalize-model-name (second (string/split % #"/" 2))))
99-
(= (shared/normalize-model-name real-model-name)
100-
(shared/normalize-model-name (second (string/split % #"/" 2)))))
101-
(keys all-models)))]
102-
(get all-models found-full-model))
103-
{:tools true
104-
:reason? true
105-
:web-search false})
106-
{:model-name real-model-name})]
107-
(assoc m full-model model-capabilities)))
108-
{}
109-
(:models provider-config))))
194+
(let [static-models (:models provider-config)
195+
dynamic-models (get dynamic-provider-models provider)
196+
merged-models (merge-provider-models static-models dynamic-models)]
197+
(merge p
198+
(reduce
199+
(fn [m [model model-config]]
200+
(let [[full-model capabilities] (build-model-capabilities
201+
all-models provider model model-config)]
202+
(assoc m full-model capabilities)))
203+
{}
204+
merged-models))))
110205
{}
111206
(:providers config))
112207
ollama-api-url (llm-util/provider-api-url "ollama" config)

0 commit comments

Comments
 (0)