feat: upgrade tikv-client to v2.0.7 and remove legacy dependencies#234
Open
donbowman wants to merge 42 commits intodistributedio:masterfrom
Open
feat: upgrade tikv-client to v2.0.7 and remove legacy dependencies#234donbowman wants to merge 42 commits intodistributedio:masterfrom
donbowman wants to merge 42 commits intodistributedio:masterfrom
Conversation
feat: add gitlab-ci pipeline See merge request open-source/titan!1
feat: if the log path is stdout/stderr, use that stream, disable rolling See merge request open-source/titan!2
fix: add artifacts of token See merge request open-source/titan!3
fix: add token to artifact See merge request open-source/titan!4
fix: add unit tests for stdout/stderr output logs See merge request open-source/titan!5
fix: rename main test to avoid conflict w/ metrics See merge request open-source/titan!6
fix: add additional log writer UT See merge request open-source/titan!7
Rebase upstream See merge request open-source/titan!8
Rebase upstream See merge request open-source/titan!9
Update pipeline See merge request open-source/titan!10
# Conflicts: # .gitlab-ci.yml
Rebase See merge request open-source/titan!11
feat: if the log path is stdout/stderr, use that stream, disable rolling See merge request open-source/titan!12
feat: if the log path is stdout/stderr, use that stream, disable rolling See merge request open-source/titan!13
fix: do not register election leader if mocktikv is used See merge request open-source/titan!14
fix: correct release pipeline for integration branch See merge request open-source/titan!15
https://github.com/distributedio/titan/pull/222/files Same change, slightly different syntax
fix: merge distributedio#222, updating our change See merge request open-source/titan!16
This commit completes the migration from the legacy
`github.com/pingcap/tidb` kv client to the modern
`github.com/tikv/client-go/v2`. It also cleans up dependencies, fixes
build issues, and updates the testing infrastructure.
Key Changes:
1. Dependency Management:
- Upgraded `github.com/tikv/client-go/v2` to `v2.0.7`.
- Removed all dependencies on `github.com/pingcap/tidb`.
- Updated `go.etcd.io/etcd` imports to `go.etcd.io/etcd/client/v3` (v3.5.x).
- Added `replace` directives in `go.mod` for `bbolt` and `gofail` to resolve transitive dependency conflicts.
- Ran `go mod tidy` to clean up the module graph.
2. Storage Layer (`db/store`):
- Refactored `Storage` interface to align with `client-go` v2 API (e.g., `CurrentVersion` -> `CurrentTimestamp`).
- Updated `Transaction` type alias to `*txnkv.KVTxn`.
- Explicitly defined `Iterator` interface as it is no longer exported directly by the upstream package.
- Updated error handling to use `github.com/tikv/client-go/v2/error` (aliased as `tikverr`).
- Updated transaction options to use `txnutil` priorities.
3. Core Logic (`db/`):
- Updated `db.go`, `kv.go`, and data structure implementations (`hash.go`, `set.go`, `zset.go`, `llist.go`) to handle API changes.
- Replaced usage of `kv.Key` (type alias) with standard `[]byte`.
- Replaced `key.HasPrefix()` helper calls with `bytes.HasPrefix()`.
- Updated `tikvgc.go` to use the new `CurrentTimestamp` API for GC safe point calculations.
4. Testing Infrastructure:
- Rewrote `db/store/mock_store.go` to use `github.com/tikv/client-go/v2/testutils`.
- Added bootstrapping logic (`BootstrapWithSingleStore`) to `MockOpen` to ensure regions are initialized for tests.
- Removed heavy `go.etcd.io/etcd/integration` dependencies from `command/test_test.go` and `tools/autotest/main_test.go`, significantly reducing build complexity and test startup time.
- Refactored `db/gc_test.go` to remove legacy `tidb/kv` imports and use local helper functions.
5. Tools:
- Updated `tools/expire/main.go` to use the correct `store.Transaction` interface and remove unused imports.
- Updated `db/task.go` to use the modern etcd client import path.
6. Deployment & Environment:
- Updated `docker-compose.yml` to the latest specification (removed version field) and enabled local build context.
- Add `docker-compose-3-node.yml` to configure a full 3-node PD and 3-node TiKV cluster with unique volume mounts for data isolation.
7. Transaction retry/backoff hardening
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit completes the migration from the legacy
github.com/pingcap/tidbkv client to the moderngithub.com/tikv/client-go/v2. It also cleans up dependencies, fixesbuild issues, and updates the testing infrastructure.
Key Changes:
Dependency Management:
github.com/tikv/client-go/v2tov2.0.7.github.com/pingcap/tidb.go.etcd.io/etcdimports togo.etcd.io/etcd/client/v3(v3.5.x).replacedirectives ingo.modforbboltandgofailto resolve transitive dependency conflicts.go mod tidyto clean up the module graph.Storage Layer (
db/store):Storageinterface to align withclient-gov2 API (e.g.,CurrentVersion->CurrentTimestamp).Transactiontype alias to*txnkv.KVTxn.Iteratorinterface as it is no longer exported directly by the upstream package.github.com/tikv/client-go/v2/error(aliased astikverr).txnutilpriorities.Core Logic (
db/):db.go,kv.go, and data structure implementations (hash.go,set.go,zset.go,llist.go) to handle API changes.kv.Key(type alias) with standard[]byte.key.HasPrefix()helper calls withbytes.HasPrefix().tikvgc.goto use the newCurrentTimestampAPI for GC safe point calculations.Testing Infrastructure:
db/store/mock_store.goto usegithub.com/tikv/client-go/v2/testutils.BootstrapWithSingleStore) toMockOpento ensure regions are initialized for tests.go.etcd.io/etcd/integrationdependencies fromcommand/test_test.goandtools/autotest/main_test.go, significantly reducing build complexity and test startup time.db/gc_test.goto remove legacytidb/kvimports and use local helper functions.Tools:
tools/expire/main.goto use the correctstore.Transactioninterface and remove unused imports.db/task.goto use the modern etcd client import path.Deployment & Environment:
docker-compose.ymlto the latest specification (removed version field) and enabled local build context.docker-compose-3-node.ymlto configure a full 3-node PD and 3-node TiKV cluster with unique volume mounts for data isolation.Transaction retry/backoff hardening