-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Implement GetOperations for Badger v2 #7948
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| // Copyright (c) 2025 The Jaeger Authors. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| package tracestore | ||
|
|
||
| import ( | ||
| "testing" | ||
|
|
||
| "github.com/jaegertracing/jaeger/internal/testutils" | ||
| ) | ||
|
|
||
| func TestMain(m *testing.M) { | ||
| testutils.VerifyGoLeaks(m) | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| // Copyright (c) 2025 The Jaeger Authors. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| package tracestore | ||
|
|
||
| import ( | ||
| "context" | ||
| "errors" | ||
| "iter" | ||
|
|
||
| "github.com/dgraph-io/badger/v4" | ||
| "go.opentelemetry.io/collector/pdata/ptrace" | ||
|
|
||
| "github.com/jaegertracing/jaeger/internal/storage/v2/api/tracestore" | ||
| ) | ||
|
|
||
| var _ tracestore.Reader = (*Reader)(nil) | ||
|
|
||
| const ( | ||
| // operationNameIndexKey (0x82) is the prefix for the operations index. | ||
| operationNameIndexKey byte = 0x82 | ||
|
|
||
| // sizeOfTraceID is 16 bytes for a TraceID. | ||
| sizeOfTraceID = 16 | ||
| ) | ||
|
|
||
| // Reader implements the tracestore.Reader interface for Badger. | ||
| type Reader struct { | ||
| db *badger.DB | ||
| } | ||
|
|
||
| // NewReader creates a new Reader instance. | ||
| func NewReader(db *badger.DB) *Reader { | ||
| return &Reader{ | ||
| db: db, | ||
| } | ||
| } | ||
|
|
||
| // GetOperations returns all operation names for a given service. | ||
|
|
||
|
Comment on lines
+39
to
+40
|
||
| func (r *Reader) GetOperations(_ context.Context, query tracestore.OperationQueryParams) ([]tracestore.Operation, error) { | ||
|
||
| if query.SpanKind != "" { | ||
| return nil, errors.New("badger storage does not support SpanKind filtering") | ||
| } | ||
| if query.ServiceName == "" { | ||
| return nil, errors.New("service name is required") | ||
| } | ||
|
|
||
Sudhanshu-NITR marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| var operations []tracestore.Operation | ||
|
|
||
| err := r.db.View(func(txn *badger.Txn) error { | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is already implementation of this logic in v1. This is not how we approach code migration, by rewriting it and introducing unknown bugs.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see your point. My goal was to provide a native v2 implementation, but I agree duplication is bad. Could you clarify if you prefer I expose the v1 CacheStore logic to be reusable here, or should v2 structurally wrap the v1 implementation for now? I want to align with the migration path you have in mind.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The way we upgraded Elasticsearch implementation is we refactored its internal "core storage layer" to work on a dbmodel that was independent of v1/v2 Storage API. Both v1 and v2 ES implementations then shared that core storage layer. Refactoring was surgical, without large code changes or code movements, which makes the code reviewable with high confidence (while copy & paste is not).
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ohh, Thanks for explaining the dbmodel pattern—I agree that having a shared core layer is much cleaner and safer than duplicating the logic.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hi @yurishkuro,
Does this plan align with your expectations for the migration? If so, I'll proceed with extracting the core logic first.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think that the elasticsearch approach is gonna work for badger because we don't have anything like |
||
| // Create the search prefix: [0x82][ServiceName] | ||
| prefix := make([]byte, len(query.ServiceName)+1) | ||
| prefix[0] = operationNameIndexKey | ||
| copy(prefix[1:], query.ServiceName) | ||
|
|
||
| opts := badger.DefaultIteratorOptions | ||
| opts.PrefetchValues = false | ||
| it := txn.NewIterator(opts) | ||
| defer it.Close() | ||
|
|
||
| uniqueOps := make(map[string]struct{}) | ||
| for it.Seek(prefix); it.ValidForPrefix(prefix); it.Next() { | ||
| key := it.Item().Key() | ||
|
|
||
| // Key layout: [Prefix(1)][Service(N)][Operation(M)][Time(8)][TraceID(16)] | ||
| opNameStart := 1 + len(query.ServiceName) | ||
| opNameEnd := len(key) - (8 + sizeOfTraceID) | ||
|
|
||
| if opNameEnd > opNameStart { | ||
| opName := string(key[opNameStart:opNameEnd]) | ||
| if _, exists := uniqueOps[opName]; !exists { | ||
| uniqueOps[opName] = struct{}{} | ||
| operations = append(operations, tracestore.Operation{ | ||
| Name: opName, | ||
| // TODO: https://github.com/jaegertracing/jaeger/issues/1922 | ||
| // SpanKind is not stored in Badger v1/v2 index | ||
| }) | ||
| } | ||
| } | ||
| } | ||
| return nil | ||
| }) | ||
| return operations, err | ||
| } | ||
|
|
||
| // Stubs | ||
| func (*Reader) GetServices(context.Context) ([]string, error) { | ||
| return nil, nil | ||
| } | ||
|
|
||
| func (*Reader) GetTraces(context.Context, ...tracestore.GetTraceParams) iter.Seq2[[]ptrace.Traces, error] { | ||
| return func(_ func([]ptrace.Traces, error) bool) {} | ||
| } | ||
|
|
||
| func (*Reader) FindTraces(context.Context, tracestore.TraceQueryParams) iter.Seq2[[]ptrace.Traces, error] { | ||
| return func(_ func([]ptrace.Traces, error) bool) {} | ||
| } | ||
|
|
||
| func (*Reader) FindTraceIDs(context.Context, tracestore.TraceQueryParams) iter.Seq2[[]tracestore.FoundTraceID, error] { | ||
| return func(_ func([]tracestore.FoundTraceID, error) bool) {} | ||
| } | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,90 @@ | ||||||
| // Copyright (c) 2025 The Jaeger Authors. | ||||||
| // SPDX-License-Identifier: Apache-2.0 | ||||||
|
|
||||||
| package tracestore | ||||||
|
|
||||||
| import ( | ||||||
| "context" | ||||||
| "encoding/binary" | ||||||
| "testing" | ||||||
|
|
||||||
| "github.com/dgraph-io/badger/v4" | ||||||
| "github.com/stretchr/testify/assert" | ||||||
| "github.com/stretchr/testify/require" | ||||||
|
|
||||||
| "github.com/jaegertracing/jaeger/internal/storage/v2/api/tracestore" | ||||||
| ) | ||||||
|
|
||||||
| func TestGetOperations(t *testing.T) { | ||||||
| // 1. Setup Badger with a temporary directory (standard pattern) | ||||||
| opts := badger.DefaultOptions(t.TempDir()).WithLoggingLevel(badger.ERROR) | ||||||
| db, err := badger.Open(opts) | ||||||
| require.NoError(t, err) | ||||||
|
|
||||||
| // Ensure DB is closed after test | ||||||
| t.Cleanup(func() { | ||||||
| require.NoError(t, db.Close()) | ||||||
| }) | ||||||
|
|
||||||
| reader := NewReader(db) | ||||||
|
|
||||||
| // 2. Insert Dummy data | ||||||
| err = db.Update(func(txn *badger.Txn) error { | ||||||
| // Helper to create a key in the exact format: | ||||||
| // [0x82][ServiceName][OperationName][Timestamp(8)][TraceID(16)] | ||||||
| createKey := func(service, operation string, id uint64) []byte { | ||||||
| key := make([]byte, 1+len(service)+len(operation)+8+16) | ||||||
| key[0] = operationNameIndexKey | ||||||
| n := 1 | ||||||
| n += copy(key[n:], service) | ||||||
| n += copy(key[n:], operation) | ||||||
|
|
||||||
| // Add dummy timestamp (8 bytes) | ||||||
| binary.BigEndian.PutUint64(key[n:], 123456) | ||||||
| n += 8 | ||||||
|
|
||||||
| // Add dummy trace ID (16 bytes) | ||||||
| binary.BigEndian.PutUint64(key[n:], 0) // High | ||||||
| binary.BigEndian.PutUint64(key[n+8:], id) // Low | ||||||
| return key | ||||||
| } | ||||||
|
|
||||||
| // Add two operations for "service-a" and one for "service-b" | ||||||
| require.NoError(t, txn.Set(createKey("service-a", "op-1", 1), []byte{})) | ||||||
| require.NoError(t, txn.Set(createKey("service-a", "op-1", 2), []byte{})) // Duplicate op name, different trace | ||||||
| require.NoError(t, txn.Set(createKey("service-a", "op-2", 3), []byte{})) | ||||||
| require.NoError(t, txn.Set(createKey("service-b", "op-3", 4), []byte{})) | ||||||
| return nil | ||||||
| }) | ||||||
|
|
||||||
| require.NoError(t, err) | ||||||
|
|
||||||
| // 3. Verify GetOperations for "service-a" | ||||||
| ops, err := reader.GetOperations(context.Background(), | ||||||
| tracestore.OperationQueryParams{ | ||||||
| ServiceName: "service-a", | ||||||
| }) | ||||||
| require.NoError(t, err) | ||||||
|
|
||||||
| // We expect 2 unique operations: "op-1" and "op-2" | ||||||
| assert.Len(t, ops, 2) | ||||||
| assert.Contains(t, []string{ops[0].Name, ops[1].Name}, "op-1") | ||||||
| assert.Contains(t, []string{ops[0].Name, ops[1].Name}, "op-2") | ||||||
|
|
||||||
| // 4. Verify GetOperations for "service-b" | ||||||
|
||||||
| // 4. Verify GetOperations for "service-b" | |
| // 3. Verify GetOperations for "service-b" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Time travel?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Haha, good catch! I'm still mentally in 2025. I will update the copyright headers code to 2026. 😅