Skip to content

Add TableMetadataSet to AssetDeps produced by BI Integrations#33414

Open
michalcabir-ui wants to merge 3 commits intodagster-io:masterfrom
michalcabir-ui:feature/dagster-metadata-enrichment
Open

Add TableMetadataSet to AssetDeps produced by BI Integrations#33414
michalcabir-ui wants to merge 3 commits intodagster-io:masterfrom
michalcabir-ui:feature/dagster-metadata-enrichment

Conversation

@michalcabir-ui
Copy link
Contributor

@michalcabir-ui michalcabir-ui commented Feb 3, 2026

Summary & Motivation

This PR enriches the AssetSpecs produced by several BI integrations (dagster-omni, dagster-tableau, dagster-looker, dagster-sigma, and dagster-powerbi) by including TableMetadataSet.
Extracts a clean table_name from the source object (stripping schema prefixes or internal IDs where necessary) and populates the dagster/table_name metadata key.

For dagster-tableau and dagster-looker, it additionally attempts to extract the underlying dagster/storage_kind (e.g., "snowflake", "bigquery") from the connection definition, falling back to the integration name if unavailable.

For dagster-omni, it implements logic to parse table names from the db_schema__table format.

How I Tested These Changes

dagster-omni: Added tests to verify the _extract_table_name_from_omni_table_name helper correctly splits schema__table strings.

dagster-tableau: Added tests for both published and embedded data sources, and a specific test case to verify dagster/storage_kind is correctly inferred from the connectionType property.

dagster-looker: Updated test_build_defs to assert dagster/table_name exists on Views and Explores, and verified that dagster/storage_kind tags are applied.

dagster-sigma: Updated translator tests to verify dataset metadata.

dagster-powerbi: Added a new test case to verify data source metadata enrichment.

Changelog

[dagster-omni, dagster-tableau, dagster-looker, dagster-sigma, dagster-powerbi] Automatically enrich generated assets with dagster/table_name metadata. Additionally, dagster-tableau and dagster-looker assets now populate dagster/storage_kind based on the upstream connection type.

@michalcabir-ui michalcabir-ui marked this pull request as ready for review February 3, 2026 21:25
@xionon xionon requested a review from OwenKephart February 13, 2026 17:38
@michalcabir-ui
Copy link
Contributor Author

Hi @OwenKephart, I don't have permission to add reviewers, so I'm tagging you here as requested.
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant