Skip to content

Refactor: Change the DeclarativeBase sqlalchemy style#748

Open
HeloiseJoffe wants to merge 5 commits intoDIRACGrid:mainfrom
HeloiseJoffe:refactor/sqlalchemy-DeclarativeBase
Open

Refactor: Change the DeclarativeBase sqlalchemy style#748
HeloiseJoffe wants to merge 5 commits intoDIRACGrid:mainfrom
HeloiseJoffe:refactor/sqlalchemy-DeclarativeBase

Conversation

@HeloiseJoffe
Copy link
Contributor

closed: #612

Changes:

  • Replaced declarative_base with DeclarativeBase style
  • Cast 'table' to Table type to satisfy mypy

Questions:
There is other new styles in SQLAlchemy, for example the declarative use of Column is replaced by mapped_column.
Should this be applied as well?
In the Bookkeeping schema, this has been done, but the other changes haven’t been applied. Is there a reason for that?

@ryuwd
Copy link
Contributor

ryuwd commented Jan 23, 2026

All the steps under https://docs.sqlalchemy.org/en/20/changelog/whatsnew_20.html#migrating-an-existing-mapping should be followed before closing #612

substituting mapped_column is enough to start fixing the typing errors but we lose the usefulness of the type checks without the additional annotations (which I still need to do for the bookkeeping schema)

Copy link
Contributor

@ryuwd ryuwd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this! Next steps:

  1. Column to mapped_column (see diff comment example in job_logging/schema.py)
  2. Add Mapped[...] annotations for all schema files
  3. Remove cast(__table__, Table) everywhere

@HeloiseJoffe HeloiseJoffe force-pushed the refactor/sqlalchemy-DeclarativeBase branch from 7c3dfb0 to c258093 Compare February 3, 2026 09:13
@read-the-docs-community
Copy link

read-the-docs-community bot commented Feb 3, 2026

Documentation build overview

📚 diracx | 🛠️ Build #31284375 | 📁 Comparing 09ba1cc against latest (ee544f5)


🔍 Preview build

Show files changed (116 files in total): 📝 116 modified | ➕ 0 added | ➖ 0 deleted
File Status
index.html 📝 modified
REFERENCE/index.html 📝 modified
RUN_PROD/index.html 📝 modified
SECURITY/index.html 📝 modified
SSO/index.html 📝 modified
admin/index.html 📝 modified
dev/index.html 📝 modified
roadmap/index.html 📝 modified
user/index.html 📝 modified
admin/deploy_instance/index.html 📝 modified
admin/explanations/index.html 📝 modified
admin/how-to/index.html 📝 modified
admin/manage_dependencies/index.html 📝 modified
admin/manage_release/index.html 📝 modified
admin/reference/index.html 📝 modified
admin/tutorials/index.html 📝 modified
dev/explanations/index.html 📝 modified
dev/how-to/index.html 📝 modified
dev/manage_extension/index.html 📝 modified
dev/reference/index.html 📝 modified
dev/setup_environment/index.html 📝 modified
dev/tutorials/index.html 📝 modified
dev/web-arch/index.html 📝 modified
developer/contribute/index.html 📝 modified
developer/manage_extension/index.html 📝 modified
developer/setup_environment/index.html 📝 modified
user/explanations/index.html 📝 modified
user/how-to/index.html 📝 modified
user/reference/index.html 📝 modified
user/tutorials/index.html 📝 modified
user/web/index.html 📝 modified
admin/explanations/auth-with-diracx/index.html 📝 modified
admin/explanations/auth-with-external/index.html 📝 modified
admin/explanations/chart-structure/index.html 📝 modified
admin/explanations/configuration/index.html 📝 modified
admin/explanations/database-management/index.html 📝 modified
admin/explanations/opentelemetry/index.html 📝 modified
admin/explanations/sandbox-store/index.html 📝 modified
admin/explanations/user-management/index.html 📝 modified
admin/how-to/debugging/index.html 📝 modified
admin/how-to/install/index.html 📝 modified
admin/how-to/rotate-a-secret/index.html 📝 modified
admin/how-to/upgrading/index.html 📝 modified
admin/reference/env-variables/index.html 📝 modified
admin/reference/security_model/index.html 📝 modified
admin/reference/settings-and-preferences/index.html 📝 modified
admin/reference/values/index.html 📝 modified
admin/tutorials/authentication/index.html 📝 modified
admin/tutorials/run_locally/index.html 📝 modified
dev/explanations/components/index.html 📝 modified
dev/explanations/designing-functionality/index.html 📝 modified
dev/explanations/documentation-system/index.html 📝 modified
dev/explanations/extensions/index.html 📝 modified
dev/explanations/run_demo/index.html 📝 modified
dev/explanations/testing/index.html 📝 modified
dev/how-to/add-a-cli-command/index.html 📝 modified
dev/how-to/add-a-db/index.html 📝 modified
dev/how-to/add-a-route/index.html 📝 modified
dev/how-to/add-a-setting/index.html 📝 modified
dev/how-to/add-a-task/index.html 📝 modified
dev/how-to/add-a-test/index.html 📝 modified
dev/how-to/add-functionality/index.html 📝 modified
dev/how-to/client-customization/index.html 📝 modified
dev/how-to/client-extension/index.html 📝 modified
dev/how-to/client-generation/index.html 📝 modified
dev/how-to/contribute/index.html 📝 modified
dev/how-to/create_application/index.html 📝 modified
dev/how-to/develop-legacy-dirac/index.html 📝 modified
dev/how-to/extend-diracx/index.html 📝 modified
dev/how-to/use-the-demo/index.html 📝 modified
dev/reference/application-state/index.html 📝 modified
dev/reference/client-metapathfinder/index.html 📝 modified
dev/reference/coding-conventions/index.html 📝 modified
dev/reference/configuration/index.html 📝 modified
dev/reference/db-transaction-model/index.html 📝 modified
dev/reference/dependency-injection/index.html 📝 modified
dev/reference/entrypoints/index.html 📝 modified
dev/reference/env-variables/index.html 📝 modified
dev/reference/pixi-tasks/index.html 📝 modified
dev/reference/security-policies/index.html 📝 modified
dev/reference/security-properties/index.html 📝 modified
dev/reference/test-recipes/index.html 📝 modified
dev/reference/writing-tests/index.html 📝 modified
dev/tutorials/advanced-tutorial/index.html 📝 modified
dev/tutorials/develop-web/index.html 📝 modified
dev/tutorials/getting-started/index.html 📝 modified
dev/tutorials/larger-developments/index.html 📝 modified
dev/tutorials/making-changes/index.html 📝 modified
dev/tutorials/play-with-auth/index.html 📝 modified
dev/tutorials/run-locally/index.html 📝 modified
dev/tutorials/write-docs/index.html 📝 modified
user/reference/client-configuration/index.html 📝 modified
user/reference/known-installations/index.html 📝 modified
user/reference/programmatic-usage/index.html 📝 modified
user/tutorials/getting-started/index.html 📝 modified
user/web/list_and_share_applications/index.html 📝 modified
user/web/login_out/index.html 📝 modified
user/web/monitor_jobs/index.html 📝 modified
admin/how-to/install/connect/index.html 📝 modified
admin/how-to/install/convert-cs/index.html 📝 modified
admin/how-to/install/embracing/index.html 📝 modified
admin/how-to/install/install-kubernetes/index.html 📝 modified
admin/how-to/install/installing/index.html 📝 modified
admin/how-to/install/minimal-requirements/index.html 📝 modified
admin/how-to/install/register-a-vo/index.html 📝 modified
admin/how-to/install/register-the-admin-vo/index.html 📝 modified
dev/explanations/components/api/index.html 📝 modified
dev/explanations/components/cli/index.html 📝 modified
dev/explanations/components/client/index.html 📝 modified
dev/explanations/components/db/index.html 📝 modified
dev/explanations/components/routes/index.html 📝 modified
dev/how-to/use-the-demo/swagger/index.html 📝 modified
dev/how-to/use-the-demo/web/index.html 📝 modified
user/reference/programmatic-usage/command-line-interface/index.html 📝 modified
user/reference/programmatic-usage/https-interface/index.html 📝 modified
user/reference/programmatic-usage/python-interface/index.html 📝 modified

@HeloiseJoffe HeloiseJoffe requested a review from ryuwd February 3, 2026 13:06
Copy link
Contributor

@ryuwd ryuwd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking great! I only have small questions remaining

device_code: Mapped[str128] = mapped_column(
"DeviceCode", unique=True
) # Should be a hash
id_token: Mapped[Optional[dict[str, Any]]] = mapped_column("IDToken")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it a good idea to have dict[str, Any] as opposed to simply JSON?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mapped[] only accepts Python types and not SQLAlchemy types as JSON. So I use the Python type dict[str, Any] and map it to the SQL JSON type with type_annotation_map.



def enum_column(name, enum_type, **kwargs):
return mapped_column(name, Enum(enum_type, native_enum=False, length=16), **kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use partial?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since name and enum_type change with each call, there is nothing consistent to bind with partial. I think it would just add an extra layer.


Column: partial[RawColumn] = partial(RawColumn, nullable=False)
NullColumn: partial[RawColumn] = partial(RawColumn, nullable=True)
DateNowColumn = partial(Column, type_=DateTime(timezone=True), server_default=utcnow())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can all of these be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept them for compatibility reasons, in case other extensions or modules use them. Once it's confirmed that no other code uses them, they can be removed.

@HeloiseJoffe HeloiseJoffe force-pushed the refactor/sqlalchemy-DeclarativeBase branch from 60790a8 to 87d433c Compare February 5, 2026 07:21
@HeloiseJoffe HeloiseJoffe force-pushed the refactor/sqlalchemy-DeclarativeBase branch from 87d433c to 09ba1cc Compare February 5, 2026 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use the new-style sqlalchemy DeclarativeBase

2 participants