Skip to content

docs: add storage configuration guide #329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MaojiaSheng merged 2 commits into from
Feb 27, 2026
Merged

docs: add storage configuration guide #329

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
047b979
docs: add storage configuration guide
baojun-zhang Feb 27, 2026
36acca0
docs: add storage configuration guide
baojun-zhang Feb 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
111 changes: 105 additions & 6 deletions docs/en/guides/01-configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,16 @@ If rerank is not configured, search uses vector similarity only.

### storage

Storage backend configuration.
Storage configuration for context data, including file storage (AGFS) and vector database storage (VectorDB).

#### Root Configuration

| Parameter | Type | Description | Default |
|-----------|------|-------------|---------|
| `workspace` | str | Local data storage path (main configuration) | "./data" |
| `agfs` | object | AGFS configuration | {} |
| `vectordb` | object | Vector database storage configuration | {} |


```json
{
Expand All @@ -329,9 +338,9 @@ Storage backend configuration.
|-----------|------|-------------|---------|
| `mode` | str | `"http-client"` or `"binding-client"` | `"http-client"` |
| `backend` | str | `"local"`, `"s3"`, or `"memory"` | `"local"` |
| `path` | str | Local directory path for `local` backend | `"./data"` |
| `url` | str | AGFS service URL for `http-client` mode | `"http://localhost:1833"` |
| `timeout` | float | Request timeout in seconds | `10.0` |
| `s3` | object | S3 backend configuration (when backend is 's3') | - |

**Configuration Examples**

Expand Down Expand Up @@ -365,16 +374,33 @@ Directly uses the AGFS Go implementation through a shared library.
"storage": {
"agfs": {
"mode": "binding-client",
"backend": "local",
"path": "./data"
"backend": "local"
}
}
}
```

</details>

**S3 Backend**

##### S3 Backend Configuration

| Parameter | Type | Description | Default |
|-----------|------|-------------|---------|
| `bucket` | str | S3 bucket name | null |
| `region` | str | AWS region where the bucket is located (e.g., us-east-1, cn-beijing) | null |
| `access_key` | str | S3 access key ID | null |
| `secret_key` | str | S3 secret access key corresponding to the access key ID | null |
| `endpoint` | str | Custom S3 endpoint URL, required for S3-compatible services like MinIO or LocalStack | null |
| `prefix` | str | Optional key prefix for namespace isolation | "" |
| `use_ssl` | bool | Enable/disable SSL (HTTPS) for S3 connections | true |
| `use_path_style` | bool | true for PathStyle used by MinIO and some S3-compatible services; false for VirtualHostStyle used by TOS and some S3-compatible services | true |

</details>

<details>
<summary><b>PathStyle S3</b></summary>
Supports S3 storage in PathStyle mode, such as MinIO, SeaweedFS.

```json
{
Expand All @@ -392,9 +418,82 @@ Directly uses the AGFS Go implementation through a shared library.
}
}
```
</details>


<details>
<summary><b>VirtualHostStyle S3</b></summary>
Supports S3 storage in VirtualHostStyle mode, such as TOS.

```json
{
"storage": {
"agfs": {
"backend": "s3",
"s3": {
"bucket": "my-bucket",
"endpoint": "s3.amazonaws.com",
"region": "us-east-1",
"access_key": "your-ak",
"secret_key": "your-sk",
"use_path_style": false
}
}
}
}
```

</details>


#### vectordb

Vector database storage configuration

| Parameter | Type | Description | Default |
|-----------|------|-------------|---------|
| `backend` | str | VectorDB backend type: 'local' (file-based), 'http' (remote service), 'volcengine' (cloud VikingDB), or 'vikingdb' (private deployment) | "local" |
| `name` | str | VectorDB collection name | "context" |
| `url` | str | Remote service URL for 'http' type (e.g., 'http://localhost:5000') | null |
| `project_name` | str | Project name (alias project) | "default" |
| `distance_metric` | str | Distance metric for vector similarity search (e.g., 'cosine', 'l2', 'ip') | "cosine" |
| `dimension` | int | Vector embedding dimension | 0 |
| `sparse_weight` | float | Sparse weight for hybrid vector search, only effective when using hybrid index | 0.0 |
| `volcengine` | object | 'volcengine' type VikingDB configuration | - |
| `vikingdb` | object | 'vikingdb' type private deployment configuration | - |

Default local mode
```
{
"storage": {
"vectordb": {
"backend": "local"
}
}
}
```

<details>
<summary><b>volcengine vikingDB</b></summary>
Supports cloud-deployed VikingDB on Volcengine

```json
{
"storage": {
"vectordb": {
"name": "context",
"backend": "volcengine",
"project": "default",
"volcengine": {
"region": "cn-beijing",
"ak": "your-access-key",
"sk": "your-secret-key"
}
}
}
```
</details>


## Config Files

Expand Down Expand Up @@ -560,7 +659,7 @@ Volcengine has rate limits. Consider batch processing with delays or upgrading y

## Related Documentation

- [Volcengine Purchase Guide](./volcengine-purchase-guide.md) - API key setup
- [Volcengine Purchase Guide](./02-volcengine-purchase-guide.md) - API key setup
- [API Overview](../api/01-overview.md) - Client initialization
- [Server Deployment](./03-deployment.md) - Server configuration
- [Context Layers](../concepts/03-context-layers.md) - L0/L1/L2
112 changes: 107 additions & 5 deletions docs/zh/guides/01-configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,16 @@ OpenViking 使用 JSON 配置文件（`ov.conf`）进行设置。配置文件支

### storage

存储后端配置。
用于存储上下文数据 ，包括文件存储（AGFS）和向量库存储（VectorDB）。

#### 根级配置

| 参数 | 类型 | 说明 | 默认值 |
|------|------|------|--------|
| `workspace` | str | 本地数据存储路径（主要配置） | "./data" |
| `agfs` | object | agfs 配置 | {} |
| `vectordb` | object | 向量库存储配置 | {} |


```json
{
Expand All @@ -336,9 +345,10 @@ OpenViking 使用 JSON 配置文件（`ov.conf`）进行设置。配置文件支
|------|------|------|--------|
| `mode` | str | `"http-client"` 或 `"binding-client"` | `"http-client"` |
| `backend` | str | `"local"`、`"s3"` 或 `"memory"` | `"local"` |
| `path` | str | `local` 后端的本地目录路径 | `"./data"` |
| `url` | str | `http-client` 模式下的 AGFS 服务地址 | `"http://localhost:1833"` |
| `timeout` | float | 请求超时时间（秒） | `10.0` |
| `s3` | object | S3 backend configuration (when backend is 's3') | - |


**配置示例**

Expand Down Expand Up @@ -372,16 +382,33 @@ OpenViking 使用 JSON 配置文件（`ov.conf`）进行设置。配置文件支
"storage": {
"agfs": {
"mode": "binding-client",
"backend": "local",
"path": "./data"
"backend": "local"
}
}
}
```

</details>

**S3 后端**

##### S3 后端配置

| 参数 | 类型 | 说明 | 默认值 |
|------|------|------|--------|
| `bucket` | str | S3 存储桶名称 | null |
| `region` | str | 存储桶所在的 AWS 区域（例如 us-east-1, cn-beijing） | null |
| `access_key` | str | S3 访问密钥 ID | null |
| `secret_key` | str | 与访问密钥 ID 对应的 S3 秘密访问密钥 | null |
| `endpoint` | str | 自定义 S3 端点 URL，对于 MinIO 或 LocalStack 等 S3 兼容服务是必需的 | null |
| `prefix` | str | 用于命名空间隔离的可选键前缀 | "" |
| `use_ssl` | bool | 为 S3 连接启用/禁用 SSL（HTTPS） | true |
| `use_path_style` | bool | true 表示对 MinIO 和某些 S3 兼容服务使用 PathStyle；false 表示对 TOS 和某些 S3 兼容服务使用 VirtualHostStyle | true |

</details>

<details>
<summary><b>PathStyle S3</b></summary>
支持 PathStyle 模式的 S3 存储， 如 MinIO、SeaweedFS.

```json
{
Expand All @@ -399,9 +426,84 @@ OpenViking 使用 JSON 配置文件（`ov.conf`）进行设置。配置文件支
}
}
```
</details>


<details>
<summary><b>VirtualHostStyle S3</b></summary>
支持 VirtualHostStyle 模式的 S3 存储， 如 TOS.

```json
{
"storage": {
"agfs": {
"backend": "s3",
"s3": {
"bucket": "my-bucket",
"endpoint": "s3.amazonaws.com",
"region": "us-east-1",
"access_key": "your-ak",
"secret_key": "your-sk",
"use_path_style": false
}
}
}
}
```

</details>


#### vectordb

向量库存储的配置

| 参数 | 类型 | 说明 | 默认值 |
|------|------|------|--------|
| `backend` | str | VectorDB 后端类型: 'local'（基于文件）, 'http'（远程服务）, 'volcengine'（云上VikingDB）或 'vikingdb'（私有部署） | "local" |
| `name` | str | VectorDB 的集合名称 | "context" |
| `url` | str | 'http' 类型的远程服务 URL（例如 'http://localhost:5000'） | null |
| `project_name` | str | 项目名称（别名 project） | "default" |
| `distance_metric` | str | 向量相似度搜索的距离度量（例如 'cosine', 'l2', 'ip'） | "cosine" |
| `dimension` | int | 向量嵌入的维度 | 0 |
| `sparse_weight` | float | 混合向量搜索的稀疏权重，仅在使用混合索引时生效 | 0.0 |
| `volcengine` | object | 'volcengine' 类型的 VikingDB 配置 | - |
| `vikingdb` | object | 'vikingdb' 类型的私有部署配置 | - |

默认使用本地模式
```
{
"storage": {
"vectordb": {
"backend": "local"
}
}
}
```

<details>
<summary><b>volcengine vikingDB</b></summary>
支持火山引擎云上部署的 VikingDB

```json
{
"storage": {
"vectordb": {
"name": "context",
"backend": "volcengine",
"project": "default",
"volcengine": {
"region": "cn-beijing",
"ak": "your-access-key",
"sk": "your-secret-key"
}
}
}
```
</details>



## 配置文件

OpenViking 使用两个配置文件：
Expand Down