Skip to content

Conversation

@ty-porter
Copy link
Contributor

@ty-porter ty-porter commented Aug 5, 2025

Adds a full fledged migration engine for custom configurations. Previously I had a full changelog, but it'll be more useful to read the new migrations README in this PR.

https://github.com/MLB-LED-Scoreboard/mlb-led-scoreboard/blob/24de6ae98498cb625ba0a8e3058d53ded1dbf7ff/migrations/README.md

This PR is huge, but it's mostly machinery. Actually using the migrator is a much simpler pattern than trying to extend our existing recursive config validator (although we still need to keep the validator itself), or just having to put out a deprecation warning.

Once this work is done it significantly lessens the risk of the planned v9.0.0 breaking update coming with #604

High level:

  • New CLI - python -m migrations init/generate/up/down/subconfig
  • Generates migrations from a template
  • The golden path is simple -- add/remove/overwrite/move keys is a one-line (safe!) change. And the change propagates to the entire family of configs, such as schema, custom, and all related subconfigs.
  • Implements atomic operations (as best as possible) -- just write to the txn object instead of directly opening a file
  • Implements first step at migration/validation for custom subconfigs such as using --config on script boot. If located in the right directory and with the right name, we can add that to the migration system. Also planning to do that with the validator, but it's not in this PR.
  • Chose to rename .example.json to .schema.json, maybe it'll discourage people from editing them
  • The new migrator inits itself (idempotent) during installation. Part of the init process is to copy the schema files to their corresponding custom files. No need to cp by hand. Migrations are also run during the installation as well.
  • There's a check for pending migrations in the main app now with instructions to re-install to resolve.

The big config migration that was planned is already in place. I'm planning to revert the config changes introduced in that PR (possibly as a separate PR). Then I will write the migrations that generate those configs, and we'll know that the migration system is expressive enough to work for the task.

@ty-porter ty-porter marked this pull request as ready for review November 25, 2025 06:46
@ty-porter ty-porter merged commit a30bef5 into dev Nov 25, 2025
1 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant