[feat][backend]: ob metrics by kidkidkid · Pull Request #256 · coze-dev/coze-loop

kidkidkid · 2025-10-24T09:24:32Z

What type of PR is this?

Check the PR title.

This PR title match the format: [<type>][<scope>]: <description>. For example: [fix][backend] flaky fix
The description of this PR title is user-oriented and clear enough for others to understand.
Add documentation if the current PR requires user awareness at the usage level.
This PR is written in English. PRs not in English will not be reviewed.

(Optional) Translate the PR title into Chinese.

(Optional) More detailed description for this PR(en: English/zh: Chinese).

en:
zh(optional):

(Optional) Which issue(s) this PR fixes:

(LogID: 202509171627200100911101158442AB5) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 2025091717345901009111011567354BF) Co-Authored-By: Coda <coda@bytedance.com>

…tion (LogID: 20250917174630010091110115283AA9A) Co-Authored-By: Coda <coda@bytedance.com>

…nstants (LogID: 2025091811100901009111011538111C3) Co-Authored-By: Coda <coda@bytedance.com>

…TraceRepo (LogID: 202509181158370100911101150622C9A) Co-Authored-By: Coda <coda@bytedance.com>

… calls (LogID: 202509181938080100911101156688B07) Co-Authored-By: Coda <coda@bytedance.com>

…terface (LogID: 20250919125803010091110115303BB0B) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 20250919125803010091110115303BB0B) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 202509191728280100911101157415799) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 2025092217305901009110607501234E6) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 202509261759130100911042537649351) Co-Authored-By: Coda <coda@bytedance.com>

…_count metric (LogID: 202509291209590100911092389298F4D) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 20251015105314010091104094405CFC2) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 202510151538390100911040946941251) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 202510161048080100911040943155F84) Co-Authored-By: Coda <coda@bytedance.com>

codecov · 2025-10-24T11:18:40Z

Codecov Report

❌ Patch coverage is 87.66901% with 228 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ules/observability/domain/metric/service/metric.go	68.97%	71 Missing and 32 partials ⚠️
...ckend/modules/observability/infra/repo/ck/spans.go	63.85%	24 Missing and 6 partials ⚠️
...ce/metric/model/model_output_token_count_metric.go	0.00%	24 Missing ⚠️
.../metric/service/metric/wrapper/multiple_wrapper.go	0.00%	19 Missing ⚠️
...ackend/modules/observability/application/metric.go	85.82%	13 Missing and 5 partials ⚠️
...main/metric/service/metric/wrapper/self_wrapper.go	25.00%	12 Missing ⚠️
.../observability/application/convertor/trace/span.go	68.00%	4 Missing and 4 partials ⚠️
backend/modules/observability/infra/repo/trace.go	76.92%	6 Missing ⚠️
...ler/coze/loop/apis/observability_metric_service.go	0.00%	4 Missing ⚠️
backend/api/handler/coze/loop/apis/handler.go	0.00%	2 Missing ⚠️
... and 1 more

@@            Coverage Diff             @@
##             main     #256      +/-   ##
==========================================
+ Coverage   65.68%   67.81%   +2.12%     
==========================================
  Files         507      565      +58     
  Lines       57718    49697    -8021     
==========================================
- Hits        37912    33701    -4211     
+ Misses      17199    13343    -3856     
- Partials     2607     2653      +46

Flag	Coverage Δ
unittests	`67.81% <87.66%> (+2.12%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
backend/infra/ck/ck.go	`0.00% <ø> (ø)`
...servability/application/convertor/metric/metric.go	`100.00% <100.00%> (ø)`
...bility/application/convertor/trace/trace_export.go	`95.73% <ø> (-0.87%)`	⬇️
...dules/observability/domain/metric/entity/metric.go	`100.00% <100.00%> (ø)`
...ervice/metric/general/general_fail_ratio_metric.go	`100.00% <100.00%> (ø)`
.../metric/general/general_model_fail_ratio_metric.go	`100.00% <100.00%> (ø)`
...ice/metric/general/general_model_latency_metric.go	`100.00% <100.00%> (ø)`
...etric/general/general_model_total_tokens_metric.go	`100.00% <100.00%> (ø)`
...e/metric/general/general_tool_fail_ratio_metric.go	`100.00% <100.00%> (ø)`
...vice/metric/general/general_tool_latency_metric.go	`100.00% <100.00%> (ø)`
... and 55 more

... and 446 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 43f34db...e177408. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

(LogID: 202510242052381921680001031104016) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 20251024215714192168000103764C39D) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 20251027112907010091111069183BB83) Co-Authored-By: Coda <coda@bytedance.com>

(LogID: 202510271430510100911110693736081) Co-Authored-By: Coda <coda@bytedance.com>

CozeLoop

Overall review: The PR introduces observability metrics and minor API/config changes. Inline comments cover a potential compatibility concern (IDL mapping for span_ids) and a deployment config inconsistency (key_columns). Other changes appear low-risk (docs and span type additions).

taoyifan89 · 2025-10-27T11:41:27Z

idl/thrift/coze/loop/observability/coze.loop.observability.metric.thrift

+    5: optional string granularity (api.body="granularity")
+    6: optional filter.FilterFields filters (api.body="filters")
+    7: optional common.PlatformType platform_type (api.body="platform_type")
+    8: optional list<filter.FilterField> drill_down_fields (api.body="drill_down_fields")


drill_down_fields有什么用？

taoyifan89 · 2025-10-27T11:42:04Z

idl/thrift/coze/loop/observability/coze.loop.observability.metric.thrift

+}
+
+struct GetDrillDownValuesResponse {
+    1: optional list<string> values


这里要不要给个数据结构，说不定以后显示值和实际库里的值不一致？

taoyifan89 · 2025-10-27T11:42:52Z

idl/thrift/coze/loop/observability/coze.loop.observability.metric.thrift

+    3: required i64 end_time (api.js_conv='true', go.tag='json:"end_time"', api.body="end_time", vt.gt="0")
+    4: optional filter.FilterFields filters (api.body="filters")
+    5: optional common.PlatformType platform_type (api.body="platform_type")
+    6: required metric.DrillDownValueType drill_down_value_type (api.body="drill_down_value_type")


这里如果是枚举值，会不会以后内场或者商业化专用的下钻维度不太好处理？

taoyifan89 · 2025-10-27T13:00:08Z

backend/modules/observability/domain/metric/service/metric.go

+				errorx.WithExtraMsg(fmt.Sprintf("metric definition %s not found", metricName)))
+		}
+		if _, ok := mVal.(entity.IMetricCompound); ok {
+			return m.queryCompoundMetric(ctx, req, mVal)


如果传入多个metricsName，其中有符合指标，直接return就不对了吧？

如果现在就是不支持，多个直接报错吧

ok 预期就是只能传一个复合指标不然就是前端传的有问题

taoyifan89 · 2025-10-27T13:08:09Z

...odules/observability/domain/metric/service/metric/general/general_model_fail_ratio_metric.go

+	"github.com/coze-dev/coze-loop/backend/modules/observability/domain/trace/service/trace/span_filter"
+)
+
+type GeneralModelFailRatioMetric struct {


model_fail为什么不在model下面，这个是按什么逻辑分的

按PRD中的展示区域分的

taoyifan89 · 2025-10-27T13:09:19Z

backend/modules/observability/domain/metric/entity/metric.go

+	MetricTypeSummary    MetricType = "summary"     // 汇总
+	MetricTypePie        MetricType = "pie"         // 饼图
+
+	MetricSourceCK MetricSource = "ck"


这里是不是不应该暴露ck，这个只需要表达使用内置存储就好？

如果存储复用的话那确实不应该用ck

kidkidkid and others added 30 commits September 17, 2025 16:39

feat: [Coda] 实现指标基础框架构建，新增metrics目录和DDD架构实现

1dc24e3

(LogID: 202509171627200100911101158442AB5) Co-Authored-By: Coda <coda@bytedance.com>

feat: [Coda] 完善指标基础框架，实现具体指标定义和SQL生成逻辑

efb8e7e

(LogID: 202509171627200100911101158442AB5) Co-Authored-By: Coda <coda@bytedance.com>

refactor: [Coda] split concrete_metrics.go into individual files

6a3f39e

(LogID: 2025091717345901009111011567354BF) Co-Authored-By: Coda <coda@bytedance.com>

fix: [Coda] align IMetricDefinition interface with tech doc specifica…

8549c3c

…tion (LogID: 20250917174630010091110115283AA9A) Co-Authored-By: Coda <coda@bytedance.com>

feat: [Coda] add context to IMetricDefinition Where method and use co…

11138d8

…nstants (LogID: 2025091811100901009111011538111C3) Co-Authored-By: Coda <coda@bytedance.com>

feat: [Coda] refactor MetricsCkRepoImpl to use ISpansDao instead of I…

680db12

…TraceRepo (LogID: 202509181158370100911101150622C9A) Co-Authored-By: Coda <coda@bytedance.com>

refactor: [Coda] optimize SpansCkDaoImpl GetMetrics to use GORM chain…

a80f887

… calls (LogID: 202509181938080100911101156688B07) Co-Authored-By: Coda <coda@bytedance.com>

feat: [Coda] update IMetricDefinition implementations to match new in…

b4322d5

…terface (LogID: 20250919125803010091110115303BB0B) Co-Authored-By: Coda <coda@bytedance.com>

feat: [Coda] implement complete MetricsService QueryMetrics method

4404fb7

(LogID: 20250919125803010091110115303BB0B) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): refactor code

fb4cb4f

test: [Coda] add comprehensive unit tests for formatMetrics function

5a36ddd

(LogID: 202509191728280100911101157415799) Co-Authored-By: Coda <coda@bytedance.com>

test: [Coda] add comprehensive unit tests for QueryMetrics function

ff13ca2

(LogID: 2025092217305901009110607501234E6) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): add more code

d04c984

feat(backend): refactor

f9d2d9c

feat: [Coda] 实现内外一体化指标编写任务，新增66个指标定义类

b7d4632

(LogID: 202509261759130100911042537649351) Co-Authored-By: Coda <coda@bytedance.com>

feat: [Coda] fix GranularityToSecond conversion and add general_total…

cabe723

…_count metric (LogID: 202509291209590100911092389298F4D) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): refactor & fix

a212d93

feat(backend): fix

9af1068

feat(backend): merge from main

b39cb32

test: [Coda] add divide metric tests

99b32c0

(LogID: 20251015105314010091104094405CFC2) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): add compound metric & add api

cf29728

feat: [Coda] add model token time series metrics

1bab763

(LogID: 202510151538390100911040946941251) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): refactor & fix

74d9bd3

feat(backend): fix

75618eb

feat: [Coda] add model qps and qpm success fail metrics

b1872dc

(LogID: 202510161048080100911040943155F84) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): fix

0350716

feat(backend): fix

5e1b3a9

feat(backend): fix

1ab96a3

feat(backend): fix

c40289d

feat(backend): fix

d1ddd13

kidkidkid added 2 commits October 24, 2025 18:46

feat(backend): fix ut

1f657b9

feat(backend): fix lint

5f8985b

kidkidkid and others added 17 commits October 24, 2025 20:57

chore: [Coda] add license headers for metric service go files

eb518f3

(LogID: 202510242052381921680001031104016) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): fix ci

b681bd2

feat(backend): fix

08f4ab4

feat(backend): fix

a876697

test: [Coda] add metric application unit tests

3cfb6a3

(LogID: 20251024215714192168000103764C39D) Co-Authored-By: Coda <coda@bytedance.com>

test: [Coda] refine metric application tests with gomock

8a60f38

(LogID: 20251024215714192168000103764C39D) Co-Authored-By: Coda <coda@bytedance.com>

test: [Coda] add gomock table tests for metric application

e8cd69a

(LogID: 20251024215714192168000103764C39D) Co-Authored-By: Coda <coda@bytedance.com>

test: [Coda] add gomock tests for metrics service

ea9e4fd

(LogID: 20251024215714192168000103764C39D) Co-Authored-By: Coda <coda@bytedance.com>

test: [Coda] add metric definition expression checks

2a275f1

(LogID: 20251024215714192168000103764C39D) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): fix

9ee1a4f

feat(backend): fix

f2a7b4d

feat(backend): fix

34aa66e

test: [Coda] fix metric application race in tests

c644caa

(LogID: 20251024215714192168000103764C39D) Co-Authored-By: Coda <coda@bytedance.com>

test: [Coda] improve metric compound coverage

0ff28f9

(LogID: 20251027112907010091111069183BB83) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): add recover

bebeb10

test: [Coda] add GetMetrics unit tests

c20c420

(LogID: 202510271430510100911110693736081) Co-Authored-By: Coda <coda@bytedance.com>

feat(backend): add more ut

742471c

CozeLoop reviewed Oct 27, 2025

View reviewed changes

taoyifan89 reviewed Oct 27, 2025

View reviewed changes

feat(backend): refactor

a07cc1a

taoyifan89 previously approved these changes Oct 27, 2025

View reviewed changes

feat(backend): refactor

e177408

kidkidkid dismissed taoyifan89’s stale review via e177408 October 27, 2025 13:59

kidkidkid force-pushed the feat/ob_metrics branch from 999ef5f to e177408 Compare October 28, 2025 03:28

kidkidkid requested a review from CozeLoop October 28, 2025 03:33

kidkidkid closed this Oct 28, 2025

kidkidkid deleted the feat/ob_metrics branch October 29, 2025 07:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[feat][backend]: ob metrics#256

[feat][backend]: ob metrics#256
kidkidkid wants to merge 85 commits intomainfrom
feat/ob_metrics

kidkidkid commented Oct 24, 2025

Uh oh!

codecov bot commented Oct 24, 2025 •

edited

Loading

Uh oh!

CozeLoop left a comment

Uh oh!

taoyifan89 Oct 27, 2025

Uh oh!

taoyifan89 Oct 27, 2025

Uh oh!

taoyifan89 Oct 27, 2025

Uh oh!

taoyifan89 Oct 27, 2025

Uh oh!

taoyifan89 Oct 27, 2025

Uh oh!

kidkidkid Oct 27, 2025

Uh oh!

taoyifan89 Oct 27, 2025

Uh oh!

kidkidkid Oct 27, 2025

Uh oh!

taoyifan89 Oct 27, 2025

Uh oh!

kidkidkid Oct 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

kidkidkid commented Oct 24, 2025

What type of PR is this?

Check the PR title.

(Optional) Translate the PR title into Chinese.

(Optional) More detailed description for this PR(en: English/zh: Chinese).

(Optional) Which issue(s) this PR fixes:

Uh oh!

codecov bot commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

CozeLoop left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Oct 24, 2025 •

edited

Loading