Skip to content

refactor: replace Marshmallow with Pydantic#149

Draft
bolinocroustibat wants to merge 27 commits intomainfrom
use-pydantic
Draft

refactor: replace Marshmallow with Pydantic#149
bolinocroustibat wants to merge 27 commits intomainfrom
use-pydantic

Conversation

@bolinocroustibat
Copy link
Contributor

@bolinocroustibat bolinocroustibat commented Aug 27, 2024

Closes #150.

Replace Marshmallow with Pydantic

This old (almost 1 year-old), finally finished PR replaces the data validation library Marshmallow with Pydantic, a more modern and widely adopted alternative in the Python ecosystem.

Why this change:
• Broad adoption (notably in FastAPI), excellent documentation, and active maintenance
• Better performance (see discussion: HN – Why Pydantic is replacing Marshmallow)
• Pydantic relies on native Python type hints, making code more concise and readable and compatible with type checkers/IDEs
• Cleaner object-oriented approach, with models closer to enriched dataclasses
• Easy API input parameters validation

@bolinocroustibat bolinocroustibat self-assigned this Aug 27, 2024
@bolinocroustibat bolinocroustibat marked this pull request as draft August 27, 2024 17:34
@bolinocroustibat bolinocroustibat force-pushed the use-pydantic branch 7 times, most recently from 200d46e to 6859d10 Compare August 27, 2024 22:12
@bolinocroustibat bolinocroustibat force-pushed the use-pydantic branch 2 times, most recently from 2eb3b6c to 9a20037 Compare October 17, 2024 12:59
@bolinocroustibat bolinocroustibat force-pushed the use-pydantic branch 8 times, most recently from dada589 to dbfe2b3 Compare July 16, 2025 09:32
@bolinocroustibat bolinocroustibat moved this from 👀 Review to 🛠 Doing in 🚀 Produit data.gouv.fr Jul 16, 2025
@bolinocroustibat bolinocroustibat force-pushed the use-pydantic branch 2 times, most recently from 1607376 to 084aecc Compare July 25, 2025 13:33
@bolinocroustibat bolinocroustibat marked this pull request as ready for review July 25, 2025 14:39
@bolinocroustibat bolinocroustibat changed the title refactor: use pydantic refactor: replace Marshmallow with Pydantic Jul 25, 2025
# Conflicts:
#	CHANGELOG.md

# Conflicts:
#	CHANGELOG.md
@bolinocroustibat bolinocroustibat deleted the use-pydantic branch July 25, 2025 15:19
@bolinocroustibat bolinocroustibat restored the use-pydantic branch July 25, 2025 15:20
@bolinocroustibat bolinocroustibat added code quality Indique qu'il s'agit d'une amélioration de la qualité du code (au sens très large) nettoyage labels Aug 20, 2025
Closes #310
Related discussion:
https://mattermost.incubateur.net/betagouv/pl/1qt9fgsjw7yo3y8ii7u6y4xdmh

Reduces congestion in high-priority queue while maintaining udata
synchronization as a priority.

**Priority resources in DB** (`priority=True`): Crawl and analysis use
"default" priority
**Non-priority resources in DB** (`priority=False`): Crawl and analysis
use "low" priority
**Udata notifications**: Always "high" priority (no change)
…resource (#313)

Better, more informative logging in order to better analyse potential
performance issues:
- add `resource_id` info in timer logs
- add new timer for analysing resource
@bolinocroustibat bolinocroustibat force-pushed the use-pydantic branch 4 times, most recently from b936054 to 88bf2bc Compare August 21, 2025 15:40
bolinocroustibat and others added 3 commits November 4, 2025 14:00
# Conflicts:
#	CHANGELOG.md
#	poetry.lock
#	pyproject.toml
#	udata_hydra/analysis/resource.py
#	udata_hydra/cli.py
#	udata_hydra/routes/checks.py
#	udata_hydra/utils/errors.py
@bolinocroustibat bolinocroustibat marked this pull request as draft February 2, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code quality Indique qu'il s'agit d'une amélioration de la qualité du code (au sens très large) nettoyage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[hydra] Replace Marshmallow with Pydantic

1 participant