Skip to content

Commit f7d486e

Browse files
authored
Merge pull request #23991 from doringeman/dmr-compose-configure-embeddings
docs(model-runner): add --embeddings runtime flag for embedding models
2 parents 4cddaec + 6302773 commit f7d486e

File tree

1 file changed

+21
-0
lines changed

1 file changed

+21
-0
lines changed

content/manuals/ai/compose/models-and-compose.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -333,6 +333,27 @@ models:
333333
- "0.9"
334334
```
335335

336+
### Embeddings
337+
338+
When using embedding models with the `/v1/embeddings` endpoint, you must include the `--embeddings` runtime flag for the model to be properly configured.
339+
340+
```yaml
341+
services:
342+
app:
343+
image: app
344+
models:
345+
embedding_model:
346+
endpoint_var: EMBEDDING_URL
347+
model_var: EMBEDDING_MODEL
348+
349+
models:
350+
embedding_model:
351+
model: ai/all-minilm
352+
context_size: 2048
353+
runtime_flags:
354+
- "--embeddings" # Required for embedding models
355+
```
356+
336357
## Alternative configuration with provider services
337358

338359
> [!IMPORTANT]

0 commit comments

Comments
 (0)