Skip to content
#

end-to-end-data-pipelines

Star

Here are 4 public repositories matching this topic...

Language: All
Filter by language
All 4 Python 3 Scala 1

wyang10 / AWS-Serverless-ELT-Pipeline-Enterprise

Star 1

Enterprise track: Step Functions/EventBridge + Glue + data quality on top of the v1 serverless ELT

aws-lambda aws-s3 aws-sqs terraform-modules aws-dynamodb aws-cloudwatch-logs iam-users idempotency aws-api-gateway aws-step-functions github-actions cicd-pipeline sqs-replay aws-serverless-architecture great-expectations dlq-retry aws-glue-data-catalog aws-eventbridge-scheduler aws-athena-cloud-service end-to-end-data-pipelines
  • Updated Jan 3, 2026
  • Python

raditpasy25 / AWS-Serverless-ELT-Pipeline

Star 1

🌟 Build a production-lite serverless ELT pipeline on AWS, enabling efficient data ingestion and transformation from S3 to Parquet with minimal overhead.

python aws-lambda etl makefile aws-s3 selenium data-engineering serverless-framework aws-sqs parquet data-pipelines aws-cloudwatch aws-dynamodb iam-users idempotency cicd-pipeline great-expectations aws-glue-data-catalog aws-eventbridge-scheduler end-to-end-data-pipelines
  • Updated Feb 15, 2026
  • Python

AmeeJoshi-MCA / Spotify-EndToEnd-Azure-Data-Engineering-Pipeline

Star 0

End-to-end Azure Data Engineering project using ADF for incremental ingestion, Databricks (DLT) for Medallion Architecture, and Delta Lake for CDC (SCD Type 1). Managed via Databricks Asset Bundles (DABs) for professional CI/CD. Focuses on real-time streaming, scalability, and Star Schema modeling.

data-engineering cdc azure-data-factory azure-databricks delta-lake cloud-data-platform delta-live-tables unity-catalog azure-data-engineering medallion-architecture databricks-asset-bundles end-to-end-data-pipelines adls-gen2
  • Updated Jan 29, 2026
  • Python

Improve this page

Add a description, image, and links to the end-to-end-data-pipelines topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the end-to-end-data-pipelines topic, visit your repo's landing page and select "manage topics."

Learn more