Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Hummingbird.docc/Examples/OTPAuthenticationExample.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ This example starts with a standard username/password login. There is a link to
- Table `totp` with columns `user_id` and `secret`
- Index for `email` in table `users`

You can find the migrations for these in the [Migrations folder](https://github.com/hummingbird-project/hummingbird-examples/tree/main/auth-otp/Sources/App/Migrations). These are added to the migration list in the `buildApplication` function. We need to apply these migrations before the server starts but after the PostgresClient connection manager background process has started. This can be done by calling the ``/PostgresMigrations/DatabaseMigrations/apply(client:groups:logger:dryRun:)`` function inside a closure sent to ``/Hummingbird/Application/beforeServerStarts(perform:)``. The first time you run this example you should run it with the `--migrate` parameter to perform the migrations.
You can find the migrations for these in the [Migrations folder](https://github.com/hummingbird-project/hummingbird-examples/tree/main/auth-otp/Sources/App/Migrations). These are added to the migration list in the `buildApplication` function. We need to apply these migrations before the server starts but after the PostgresClient connection manager background process has started. This can be done by calling the ``/PostgresMigrations/DatabaseMigrations/apply(client:groups:options:logger:dryRun:)`` function inside a closure sent to ``/Hummingbird/Application/beforeServerStarts(perform:)``.

On top of the two tables created, the example uses the postgres driver ``/HummingbirdPostgres/PostgresPersistDriver`` for the persist framework to store session keys. You will find out more about the persist framework at <doc:PersistentData> and there is a guide to the migration support in <doc:MigrationsGuide>.

Expand Down
52 changes: 43 additions & 9 deletions Hummingbird.docc/PostgresMigrations/MigrationsGuide.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Managing database structure changes.

## Overview

Database migrations are a controlled set of incremental changes applied to a database. You can use a migration list to transition a database from one state to a new desired state. A migration can involve creating/deleting tables, adding/removing columns, changing types and constraints. The ``PostgresMigrations`` library that comes with HummingbirdPostgres provides support for setting up your own database migrations.
Database migrations are a controlled set of incremental changes applied to a database. You can use a migration list to transition a database from one state to a new desired state. A migration can involve creating/deleting tables, adding/removing columns, changing types and constraints. The ``PostgresMigrations`` library provides support for setting up your own database migrations.

> Note: If you are using Fluent then you should use the migration support that comes with Fluent.

Expand Down Expand Up @@ -52,7 +52,7 @@ await migrations.add(CreateMyTableMigration())

### Applying migrations

As you need an active `PostgresClient` to apply migrations you need to run the migrate once you have called `PostgresClient.run`. It is also preferable to have run your migrations before your server is active and accepting connections. The best way to do this is use ``Hummingbird/Application/beforeServerStarts(perform:)``.
As you need an active `PostgresClient` to apply migrations you need to run the migrate once you have called `PostgresClient.run`. It is also preferable to have run your migrations before your server is active and accepting connections. If you are using this package alongside Hummingbird the best way to do this is use ``Hummingbird/Application/beforeServerStarts(perform:)``.

```swift
var app = Application(router: router)
Expand All @@ -62,20 +62,54 @@ app.beforeServerStarts {
try await migrations.apply(client: postgresClient, logger: logger, dryRun: true)
}
```
You will notice in the code above the parameter `dryRun` is set to true. This is because applying migrations can be a destructive process and should be a supervised. If there is a change in the migration list, with `dryRun` set to true, the `apply` function will throw an error and list the migrations it would apply or revert. At that point you can make a call on whether you want to apply those changes and run the same process again except with `dryRun` set to false.
You will notice in the code above the parameter `dryRun` is set to true. This is because applying migrations can be a destructive process and should be a supervised. If there is a change in the migration list, with `dryRun` set to true, the `apply` function will throw an error and list the migrations it would apply. At that point you can make a call on whether you want to apply those changes and run the same process again except with `dryRun` set to false.

### Reverting migrations

There are a number of situations where a migration maybe reverted.
- The user calls ``/PostgresMigrations/DatabaseMigrations/revert(client:groups:logger:dryRun:)``. This will revert all the migrations applied to the database.
- A user removes a migration from the list. The migration still needs to be registered with the migration system as it needs to know how to revert that migration. This is done with a call to ``/PostgresMigrations/DatabaseMigrations/register(_:)``. When a migration is removed it is reverted and all subsequent migrations will be reverted and then re-applied.
- A user changes the order of migrations. This is generally a user error, but if it is intentional then the first migration affected by the order change and all subsequent migrations will be reverted and then re-applied.
You can use ``/PostgresMigrations/DatabaseMigrations/revert(client:groups:options:logger:dryRun:)`` to revert all the migrations and will reset your database back to its starting point. If it comes across a migration name in the database it doesn't recognise, it will throw the error ``/PostgresMigrations/DatabaseMigrationError/cannotRevertMigration``. In this situation you have two options.
1) Register the missing migration using ``/PostgresMigrations/DatabaseMigrations/register(_:)``. The system will now know about this migration but it will not apply it.
2) Run the `revert` function again but including either option ``/PostgresMigrations/DatabaseMigrations/RevertOptions/ignoreUnknownMigrations`` or ``/PostgresMigrations/DatabaseMigrations/RevertOptions/removeUnknownMigrations``. This will either ignore the migration, or remove the database entry associated with that migration. The remove option will not revert the associated change though as it doesn't know how to.

```swift
await migrations.revert(
client: postgresClient,
options: .ignoreUnknownMigrations,
logger: logger,
dryRun: false
)
```

### Inconsistencies between expected and applied migrations

If the order your application expects migrations to be applied and the order they have actually been applied is different then when you call ``/PostgresMigrations/DatabaseMigrations/apply(client:groups:options:logger:dryRun:)`` it will throw a ``/PostgresMigrations/DatabaseMigrationError/appliedMigrationsInconsistent`` error. This can occur in a number of situations
- A new migration has been inserted in the middle of the migration list
- A migration was removed from the middle of the migration list
- A migration was renamed
- A migration has been moved from one group to another (See the section on <doc:MigrationsGuide#Migration-groups> below)
- The order of applying migrations has changed.

The best way to fix this is to ensure the order of migrations you would like to apply is consistent with the list of already applied migrations in your database. If this is not possible though you can use the function ``/PostgresMigrations/DatabaseMigrations/revertInconsistent(client:groups:options:logger:dryRun:)`` to revert applied migrations until the applied migration list is consistent with the list the application expects. This is a destructive function. Always run it with `dryRun` set to `true` first. With this set, it will output to the log the list of operations it will perform but not actually perform them.

```swift
await migrations.revertInconsistent(
client: postgresClient,
logger: logger,
dryRun: true
)
```

The default operation is to find the first inconsistency in the lists and then revert every migration after that as it cannot be certain they will have been applied correctly. This is obviously very destructive. You have a number of options that affect this operation.
- ``/PostgresMigrations/DatabaseMigrations/RevertInconsistentOptions/ignoreUnknownMigrations`` will ignore any applied migrations the system doesn't know about.
- ``/PostgresMigrations/DatabaseMigrations/RevertInconsistentOptions/removeUnknownMigrations`` will remove any applied migrations the system doesn't know about. This will also revert any migrations that follow the unknown migration.
- ``/PostgresMigrations/DatabaseMigrations/RevertInconsistentOptions/disableRevertsFollowingRevert`` disables reverting any migrations that follow a migration that has been reverted. You should use this with care as subsequent migrations may depend on the reverted migration. With this option set if it finds a migration that has not been applied in the middle of the applied migrations it will ignore that migration and not revert any subsequent migrations.

This function is provided as a way to fixup inconsistencies but it should not be relied upon. Ensuring the order your migrations are applied is consistent is the best way to proceed.

### Migration groups

A migration group is a group of migrations that can be applied to a database independent of all other migrations outside that group. By default all migrations are added to the `.default` migration group. Each group is applied independently to your database. A group allows for a modular piece of code to add additional migrations without affecting the ordering of other migrations and causing deletion of data.
A migration group is a group of migrations that can be applied to a database independent of all other migrations outside that group. By default all migrations are added to the ``/PostgresMigrations/DatabaseMigrationGroup/default`` migration group. Each group is applied independently to your database. A group allows for a modular piece of code to add additional migrations without affecting the ordering of other migrations and causing deletion of data.

To create a group you need to extend `/PostgresMigrations/DatabaseMigrationsGroup` and add a new static variable for the migration group id.
To create a group you need to extend ``/PostgresMigrations/DatabaseMigrationGroup`` and add a new static variable for the migration group id.

```swift
extension DatabaseMigrationGroup {
Expand Down
4 changes: 4 additions & 0 deletions Hummingbird.docc/PostgresMigrations/PostgresMigrations.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ Postgres database migration service
}
## Topics

### Articles

- <doc:MigrationsGuide>

### Migrations

- ``DatabaseMigrations``
Expand Down
1 change: 1 addition & 0 deletions Hummingbird.docc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,4 +94,5 @@ Below is a list of guides and tutorials to help you get started with building yo
- ``/HummingbirdWebSocket``
- ``/Jobs``
- ``/Mustache``
- ``/PostgresMigrations``
- ``/WSClient``