Skip to content

feat: Support multiple workflow status listeners via composite pattern#740

Draft
akhilpathivada wants to merge 1 commit intoconductor-oss:mainfrom
akhilpathivada:feature/composite-workflow-status-listeners
Draft

feat: Support multiple workflow status listeners via composite pattern#740
akhilpathivada wants to merge 1 commit intoconductor-oss:mainfrom
akhilpathivada:feature/composite-workflow-status-listeners

Conversation

@akhilpathivada
Copy link

@akhilpathivada akhilpathivada commented Jan 28, 2026

Resolves #727

Summary

Implements support for multiple workflow status listeners using the Composite Pattern, enabling simultaneous publishing of workflow events to multiple destinations (Kafka, queues, webhooks, archive).

Motivation

Previously, Conductor only allowed a single workflow status listener type to be configured via conductor.workflow-status-listener.type. Users had to choose between kafka, queue_publisher, workflow_publisher, or archive - but could not use multiple simultaneously.

This limitation prevented organizations from broadcasting the same workflow events to different systems for different purposes:

  • Kafka: Real-time analytics and monitoring dashboards
  • Queues: Internal workflow orchestration and cleanup jobs
  • Webhooks: External system integrations (ticketing systems)

Each system has different requirements (stream processing vs guaranteed delivery vs fire-and-forget), different failure domains, and different retention policies.

Changes

  1. CompositeWorkflowStatusListener: Delegates workflow events to all configured listeners in parallel
  2. CompositeWorkflowStatusListenerConfiguration: Spring configuration with factory method for dynamic listener creation
  3. CompositeWorkflowStatusListenerProperties: Configuration properties for composite listener
  4. Comprehensive unit tests with coverage for all workflow event types and failure scenarios

Configuration Example

# Enable composite mode
conductor.workflow-status-listener.type=composite

# List the listeners to enable (comma-separated)
conductor.workflow-status-listener.composite.types=kafka,workflow_publisher,queue_publisher

# Each listener retains its existing configuration namespace

# Kafka configuration
conductor.workflow-status-listener.kafka.producer[bootstrap.servers]=kafka:29092
conductor.workflow-status-listener.kafka.default-topic=workflow-events

# Workflow publisher (webhook) configuration
conductor.status-notifier.notification.url=http://webhook-endpoint:8080/workflow-events

# Queue publisher configuration
conductor.workflow-status-listener.queue-publisher.successQueue=_callbackSuccessQueue
conductor.workflow-status-listener.queue-publisher.failureQueue=_callbackFailureQueue

Testing

✅ Unit tests for all workflow event types
✅ Failure isolation tests (single and multiple listener failures)
✅ Edge cases (empty list, single listener)
✅ Verification that all delegates are invoked correctly

@akhilpathivada
Copy link
Author

akhilpathivada commented Feb 2, 2026

I'll mark this as ready for review once #720 is merged, as this PR is dependent on those changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Support multiple workflow status listeners

1 participant