-
Notifications
You must be signed in to change notification settings - Fork 7
Open
Labels
bugSomething isn't workingSomething isn't working
Description
📦 Environment
Python: 3.12, SparkDQ: 0.11.0
📝 Bug Description
The severity config option gets normalized by converting a string to the Enum and calling upper on it. This doesn't work, if the normalization function is passed an Enum. This happens in the below cases.
🔁 Steps to Reproduce
- If the severity is configures via the Enum```python
from sparkdq.plugin.check_factory import CheckFactory
from sparkdq.core.severity import Severity
config_data = {
"check-id": "test",
"check": "null-check",
"columns": ["foo"],
"severity": Severity.WARNING
}
check = CheckFactory._from_dict(config_data)
- If the config_data is used in more than one CheckFactory, because the config is mutated in place:
from sparkdq.plugin.check_factory import CheckFactory
config_data = {
"check-id": "test",
"check": "null-check",
"columns": ["foo"],
"severity": "warning",
}
check1 = CheckFactory._from_dict(config_data)
check2 = CheckFactory._from_dict(config_data)✅ Expected Behavior
Return the Enum in the normalization step, if its already an Enum.
📋 Relevant Logs / Error Messages
💬 Additional Context
No response
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working