From fe912b659a72749a31219248cdb8f038cc8e291c Mon Sep 17 00:00:00 2001 From: Adam Fowler Date: Wed, 22 Oct 2025 09:46:03 +0100 Subject: [PATCH 1/2] Update revert docs --- .../Examples/OTPAuthenticationExample.md | 2 +- .../PostgresMigrations/MigrationsGuide.md | 37 +++++++++++++++++-- 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/Hummingbird.docc/Examples/OTPAuthenticationExample.md b/Hummingbird.docc/Examples/OTPAuthenticationExample.md index 28bdcd12ff..cfe4d8abd1 100644 --- a/Hummingbird.docc/Examples/OTPAuthenticationExample.md +++ b/Hummingbird.docc/Examples/OTPAuthenticationExample.md @@ -13,7 +13,7 @@ This example starts with a standard username/password login. There is a link to - Table `totp` with columns `user_id` and `secret` - Index for `email` in table `users` -You can find the migrations for these in the [Migrations folder](https://github.com/hummingbird-project/hummingbird-examples/tree/main/auth-otp/Sources/App/Migrations). These are added to the migration list in the `buildApplication` function. We need to apply these migrations before the server starts but after the PostgresClient connection manager background process has started. This can be done by calling the ``/PostgresMigrations/DatabaseMigrations/apply(client:groups:logger:dryRun:)`` function inside a closure sent to ``/Hummingbird/Application/beforeServerStarts(perform:)``. The first time you run this example you should run it with the `--migrate` parameter to perform the migrations. +You can find the migrations for these in the [Migrations folder](https://github.com/hummingbird-project/hummingbird-examples/tree/main/auth-otp/Sources/App/Migrations). These are added to the migration list in the `buildApplication` function. We need to apply these migrations before the server starts but after the PostgresClient connection manager background process has started. This can be done by calling the ``/PostgresMigrations/DatabaseMigrations/apply(client:groups:options:logger:dryRun:)`` function inside a closure sent to ``/Hummingbird/Application/beforeServerStarts(perform:)``. On top of the two tables created, the example uses the postgres driver ``/HummingbirdPostgres/PostgresPersistDriver`` for the persist framework to store session keys. You will find out more about the persist framework at and there is a guide to the migration support in . diff --git a/Hummingbird.docc/PostgresMigrations/MigrationsGuide.md b/Hummingbird.docc/PostgresMigrations/MigrationsGuide.md index 74d35649bb..14170ca83b 100644 --- a/Hummingbird.docc/PostgresMigrations/MigrationsGuide.md +++ b/Hummingbird.docc/PostgresMigrations/MigrationsGuide.md @@ -66,10 +66,39 @@ You will notice in the code above the parameter `dryRun` is set to true. This is ### Reverting migrations -There are a number of situations where a migration maybe reverted. -- The user calls ``/PostgresMigrations/DatabaseMigrations/revert(client:groups:logger:dryRun:)``. This will revert all the migrations applied to the database. -- A user removes a migration from the list. The migration still needs to be registered with the migration system as it needs to know how to revert that migration. This is done with a call to ``/PostgresMigrations/DatabaseMigrations/register(_:)``. When a migration is removed it is reverted and all subsequent migrations will be reverted and then re-applied. -- A user changes the order of migrations. This is generally a user error, but if it is intentional then the first migration affected by the order change and all subsequent migrations will be reverted and then re-applied. +You can use ``/PostgresMigrations/DatabaseMigrations/revert(client:groups:options:logger:dryRun:)`` to revert all the migrations and will reset your database back to its starting point. If it comes across a migration name in the database it doesn't recognise, it will throw the error ``/PostgresMigrations/DatabaseMigrationError/cannotRevertMigration``. In this situation you have two options. +1) Register the missing migration using ``/PostgresMigrations/DatabaseMigrations/register(_:)``. The system will now know about this migration but it will not apply it. +2) Run the `revert` function again but including either option ``/PostgresMigrations/DatabaseMigrations/RevertOptions/ignoreUnknownMigrations`` or ``/PostgresMigrations/DatabaseMigrations/RevertOptions/removeUnknownMigrations``. + +```swift +await migrations.revert( + client: postgresClient, + options: .ignoreUnknownMigrations, + logger: logger, + dryRun: false +) +``` + +### Inconsistencies between expected and applied migrations + +If the order your application expects migrations to be applied and the order they have actually been applied is different then when you call ``/PostgresMigrations/DatabaseMigrations/apply(client:groups:options:logger:dryRun:)`` it will throw a ``/PostgresMigrations/DatabaseMigrationError/appliedMigrationsInconsistent`` error. This can occur when you have inserted a new migration in the middle of the migration list, removed a migration from the middle of the migration list or have swapped the order of applying migrations. + +The best way to fix this is to ensure the order of migrations you would like to apply is consistent with the list of already applied migrations in your database. If this is not possible though you can use the function ``/PostgresMigrations/DatabaseMigrations/revertInconsistent(client:groups:options:logger:dryRun:)`` to revert applied migrations until the applied migration list is consistent with the list the application expects. This is a destructive function. Always run it with `dryRun` set to `true` first. With this set, it will output to the log the list of operations it will perform but not actually perform them. + +```swift +await migrations.revertInconsistent( + client: postgresClient, + logger: logger, + dryRun: true +) +``` + +The default operation is to find the first inconsistency in the lists and then revert every migration after that as it cannot be certain they will have been applied correctly. This is obviously very destructive. You have a number of options that affect this operation. +- ``/PostgresMigrations/DatabaseMigrations/RevertInconsistentOptions/ignoreUnknownMigrations`` will ignore any applied migrations the system doesn't know about. +- ``/PostgresMigrations/DatabaseMigrations/RevertInconsistentOptions/removeUnknownMigrations`` will remove any applied migrations the system doesn't know about. This will also revert any migrations that follow the unknown migration. +- ``/PostgresMigrations/DatabaseMigrations/RevertInconsistentOptions/disableRevertsFollowingRevert`` disables reverting any migrations that follow a migration that has been reverted. You should use this with care as subsequent migrations may depend on the reverted migration. With this option set if it finds a migration that has not been applied in the middle of the applied migrations it will ignore that migration and not revert any subsequent migrations. + +This function is provided as a way to fixup inconsistencies but it should not be relied upon. Ensuring the order your migrations are applied is consistent is the best way to proceed. ### Migration groups From b62009845c6a6a830f42d1535926c8e2f00e1274 Mon Sep 17 00:00:00 2001 From: Adam Fowler Date: Wed, 22 Oct 2025 10:27:02 +0100 Subject: [PATCH 2/2] Cleanup --- .../PostgresMigrations/MigrationsGuide.md | 19 ++++++++++++------- .../PostgresMigrations/PostgresMigrations.md | 4 ++++ Hummingbird.docc/index.md | 1 + 3 files changed, 17 insertions(+), 7 deletions(-) diff --git a/Hummingbird.docc/PostgresMigrations/MigrationsGuide.md b/Hummingbird.docc/PostgresMigrations/MigrationsGuide.md index 14170ca83b..c846bf1f7a 100644 --- a/Hummingbird.docc/PostgresMigrations/MigrationsGuide.md +++ b/Hummingbird.docc/PostgresMigrations/MigrationsGuide.md @@ -8,7 +8,7 @@ Managing database structure changes. ## Overview -Database migrations are a controlled set of incremental changes applied to a database. You can use a migration list to transition a database from one state to a new desired state. A migration can involve creating/deleting tables, adding/removing columns, changing types and constraints. The ``PostgresMigrations`` library that comes with HummingbirdPostgres provides support for setting up your own database migrations. +Database migrations are a controlled set of incremental changes applied to a database. You can use a migration list to transition a database from one state to a new desired state. A migration can involve creating/deleting tables, adding/removing columns, changing types and constraints. The ``PostgresMigrations`` library provides support for setting up your own database migrations. > Note: If you are using Fluent then you should use the migration support that comes with Fluent. @@ -52,7 +52,7 @@ await migrations.add(CreateMyTableMigration()) ### Applying migrations -As you need an active `PostgresClient` to apply migrations you need to run the migrate once you have called `PostgresClient.run`. It is also preferable to have run your migrations before your server is active and accepting connections. The best way to do this is use ``Hummingbird/Application/beforeServerStarts(perform:)``. +As you need an active `PostgresClient` to apply migrations you need to run the migrate once you have called `PostgresClient.run`. It is also preferable to have run your migrations before your server is active and accepting connections. If you are using this package alongside Hummingbird the best way to do this is use ``Hummingbird/Application/beforeServerStarts(perform:)``. ```swift var app = Application(router: router) @@ -62,13 +62,13 @@ app.beforeServerStarts { try await migrations.apply(client: postgresClient, logger: logger, dryRun: true) } ``` -You will notice in the code above the parameter `dryRun` is set to true. This is because applying migrations can be a destructive process and should be a supervised. If there is a change in the migration list, with `dryRun` set to true, the `apply` function will throw an error and list the migrations it would apply or revert. At that point you can make a call on whether you want to apply those changes and run the same process again except with `dryRun` set to false. +You will notice in the code above the parameter `dryRun` is set to true. This is because applying migrations can be a destructive process and should be a supervised. If there is a change in the migration list, with `dryRun` set to true, the `apply` function will throw an error and list the migrations it would apply. At that point you can make a call on whether you want to apply those changes and run the same process again except with `dryRun` set to false. ### Reverting migrations You can use ``/PostgresMigrations/DatabaseMigrations/revert(client:groups:options:logger:dryRun:)`` to revert all the migrations and will reset your database back to its starting point. If it comes across a migration name in the database it doesn't recognise, it will throw the error ``/PostgresMigrations/DatabaseMigrationError/cannotRevertMigration``. In this situation you have two options. 1) Register the missing migration using ``/PostgresMigrations/DatabaseMigrations/register(_:)``. The system will now know about this migration but it will not apply it. -2) Run the `revert` function again but including either option ``/PostgresMigrations/DatabaseMigrations/RevertOptions/ignoreUnknownMigrations`` or ``/PostgresMigrations/DatabaseMigrations/RevertOptions/removeUnknownMigrations``. +2) Run the `revert` function again but including either option ``/PostgresMigrations/DatabaseMigrations/RevertOptions/ignoreUnknownMigrations`` or ``/PostgresMigrations/DatabaseMigrations/RevertOptions/removeUnknownMigrations``. This will either ignore the migration, or remove the database entry associated with that migration. The remove option will not revert the associated change though as it doesn't know how to. ```swift await migrations.revert( @@ -81,7 +81,12 @@ await migrations.revert( ### Inconsistencies between expected and applied migrations -If the order your application expects migrations to be applied and the order they have actually been applied is different then when you call ``/PostgresMigrations/DatabaseMigrations/apply(client:groups:options:logger:dryRun:)`` it will throw a ``/PostgresMigrations/DatabaseMigrationError/appliedMigrationsInconsistent`` error. This can occur when you have inserted a new migration in the middle of the migration list, removed a migration from the middle of the migration list or have swapped the order of applying migrations. +If the order your application expects migrations to be applied and the order they have actually been applied is different then when you call ``/PostgresMigrations/DatabaseMigrations/apply(client:groups:options:logger:dryRun:)`` it will throw a ``/PostgresMigrations/DatabaseMigrationError/appliedMigrationsInconsistent`` error. This can occur in a number of situations +- A new migration has been inserted in the middle of the migration list +- A migration was removed from the middle of the migration list +- A migration was renamed +- A migration has been moved from one group to another (See the section on below) +- The order of applying migrations has changed. The best way to fix this is to ensure the order of migrations you would like to apply is consistent with the list of already applied migrations in your database. If this is not possible though you can use the function ``/PostgresMigrations/DatabaseMigrations/revertInconsistent(client:groups:options:logger:dryRun:)`` to revert applied migrations until the applied migration list is consistent with the list the application expects. This is a destructive function. Always run it with `dryRun` set to `true` first. With this set, it will output to the log the list of operations it will perform but not actually perform them. @@ -102,9 +107,9 @@ This function is provided as a way to fixup inconsistencies but it should not be ### Migration groups -A migration group is a group of migrations that can be applied to a database independent of all other migrations outside that group. By default all migrations are added to the `.default` migration group. Each group is applied independently to your database. A group allows for a modular piece of code to add additional migrations without affecting the ordering of other migrations and causing deletion of data. +A migration group is a group of migrations that can be applied to a database independent of all other migrations outside that group. By default all migrations are added to the ``/PostgresMigrations/DatabaseMigrationGroup/default`` migration group. Each group is applied independently to your database. A group allows for a modular piece of code to add additional migrations without affecting the ordering of other migrations and causing deletion of data. -To create a group you need to extend `/PostgresMigrations/DatabaseMigrationsGroup` and add a new static variable for the migration group id. +To create a group you need to extend ``/PostgresMigrations/DatabaseMigrationGroup`` and add a new static variable for the migration group id. ```swift extension DatabaseMigrationGroup { diff --git a/Hummingbird.docc/PostgresMigrations/PostgresMigrations.md b/Hummingbird.docc/PostgresMigrations/PostgresMigrations.md index 64338575d3..a729d3f099 100644 --- a/Hummingbird.docc/PostgresMigrations/PostgresMigrations.md +++ b/Hummingbird.docc/PostgresMigrations/PostgresMigrations.md @@ -7,6 +7,10 @@ Postgres database migration service } ## Topics +### Articles + +- + ### Migrations - ``DatabaseMigrations`` diff --git a/Hummingbird.docc/index.md b/Hummingbird.docc/index.md index 04ed63a6dc..c640147dac 100644 --- a/Hummingbird.docc/index.md +++ b/Hummingbird.docc/index.md @@ -94,4 +94,5 @@ Below is a list of guides and tutorials to help you get started with building yo - ``/HummingbirdWebSocket`` - ``/Jobs`` - ``/Mustache`` +- ``/PostgresMigrations`` - ``/WSClient``