Skip to content

Comments

[slack-19.0]: BackupTablet/RestoreFromBackup: new grpc arg s3_backup_storage_root override vttablet --s3_backup_storage_root#760

Closed
neil-harkins-sf wants to merge 2 commits intoslack-19.0from
nh_s3_backup_storage_root_override
Closed

[slack-19.0]: BackupTablet/RestoreFromBackup: new grpc arg s3_backup_storage_root override vttablet --s3_backup_storage_root#760
neil-harkins-sf wants to merge 2 commits intoslack-19.0from
nh_s3_backup_storage_root_override

Conversation

@neil-harkins-sf
Copy link

Description

the vttablet binary has a cli arg --s3_backup_storage_root which is set at process start time, and not able to be modified during the lifetime of the process, which is too rigid. one example case is point-in-time recoveries which restore from the normal backup location, but then need to backup that point-in-time copy elsewhere, so that it does not pollute/taint normal backups.

  Changes Made:

  1. Proto Definitions (proto/tabletmanagerdata.proto)

  - Added s3_backup_storage_root field to BackupRequest message (field 6)
  - Added s3_backup_storage_root field to RestoreFromBackupRequest message (field 6)

  2. Data Structure Updates

  - backupstorage.Params (go/vt/mysqlctl/backupstorage/params.go:32): Added S3BackupStorageRoot
   field
  - BackupParams (go/vt/mysqlctl/backupengine.go:95): Added S3BackupStorageRoot field and
  updated Copy() method
  - RestoreParams (go/vt/mysqlctl/backupengine.go:156): Added S3BackupStorageRoot field and
  updated Copy() method

  3. Handler Updates

  - Backup handler (go/vt/vttablet/tabletmanager/rpc_backup.go:173): Now passes
  req.S3BackupStorageRoot to BackupParams
  - Restore handler (go/vt/vttablet/tabletmanager/restore.go:239): Now passes
  request.S3BackupStorageRoot to RestoreParams
  - Backup storage initialization (go/vt/mysqlctl/backup.go): Updated both backup and restore
  paths to pass S3BackupStorageRoot through backupstorage.Params

  4. S3 Backend Implementation (go/vt/mysqlctl/s3backupstorage/s3.go)

  - Created objNameWithRoot() helper function to support custom root prefixes
  - Added objName() method to both S3BackupHandle and S3BackupStorage that uses the override
  value from params if provided, otherwise falls back to the global flag value
  - Updated all S3 object name generation calls to use the new methods

  5. Generated Code

  - Regenerated all protobuf Go code and TypeScript definitions

  How It Works:

  When either BackupTablet or RestoreFromBackup is called with the s3_backup_storage_root field
   set:
  1. The value is passed through the request → handler → params → backupstorage
  2. The S3 backend checks if a custom root is provided in params
  3. If provided, it uses that value; otherwise, it falls back to the global
  --s3_backup_storage_root flag
  4. All S3 object names are generated with the appropriate prefix

  This allows per-request override of the S3 backup storage root while maintaining backward
  compatibility with existing code that relies on the global flag.

AI Disclosure

This PR was written primarily by Claude Code

@neil-harkins-sf neil-harkins-sf requested a review from a team as a code owner December 4, 2025 20:16
@github-actions github-actions bot added this to the v19.0.7 milestone Dec 4, 2025
@tanjinx tanjinx changed the title BackupTablet/RestoreFromBackup: new grpc arg s3_backup_storage_root override vttablet --s3_backup_storage_root [slack-19.0]: BackupTablet/RestoreFromBackup: new grpc arg s3_backup_storage_root override vttablet --s3_backup_storage_root Dec 4, 2025
@github-actions
Copy link

github-actions bot commented Jan 4, 2026

This PR is being marked as stale because it has been open for 30 days with no activity. To rectify, you may do any of the following:

  • Push additional commits to the associated branch.
  • Remove the stale label.
  • Add a comment indicating why it is not stale.

If no action is taken within 7 days, this PR will be closed.

@github-actions github-actions bot added the Stale label Jan 4, 2026
@github-actions
Copy link

This PR was closed because it has been stale for 7 days with no activity.

@github-actions github-actions bot closed this Jan 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants