Contact Details [Optional]

No response

Feature Description

Currently in the ZenML platform users do not have a way to integrate with other platforms (e.g. Github) and configure ZenML pipeline executions based on business events (e.g. a PR merge to main). Also the current scheduling functionality is orchestrator-based which limits its flexibility and significantly expands our development efforts (scheduling management per orchestrator). Last but not least schedules are hard to manage/track via the UI which results in user confusion.

This epic-level story addresses the aforementioned deficiencies by introducing:

1. A new first-level component called Triggers that will allow us to support user integrations and execute pipelines based on a number of configurable events (webhook, schedule, etc.).
2. A PRO-native implementation of schedules as the first ZenML support trigger type.

Proposed Solution

Backend:

Backend introduces the following components/tasks in order to support this new feature:

1. DB, Store, SDK changes to cover fully the new Trigger-related objects.
2. A pub/sub layer that provides async processing capabilities (Async Snapshot execution)
3. A ZenML-native scheduler component that implements all the existing schedule feature-set without relying on orchestrator capabilities.

Alternatives Considered

No response

Additional Context

No response

Priority

None

Code of Conduct

• I agree to follow this project's Code of Conduct