Skip to content

Email Connector Health Status Not Reflecting Connection Failures #6307

@johnBgood

Description

@johnBgood

Problem
The email connector's health status remains UP even when experiencing connection issues with the email server. This creates a misleading health status that doesn't accurately reflect the connector's operational state.

Background
During an incident investigation (related to #inc-email-connector-is-not-working-properly), it was discovered that when the email connector encounters connection problems, it continues to show as healthy while retrying indefinitely in the background.

Root Cause Analysis
Based on the discussion in this thread:

  1. Infinite Retries: The connector is configured with infinite retries using .withBackoff(Duration.of(5, ChronoUnit.SECONDS), Duration.of(1, ChronoUnit.HOURS))
  2. No Health Status Update: Connection failures during retries don't trigger a health status change to DOWN
  3. Initial Connection May Succeed: The connector might successfully connect initially but then fail on reconnection attempts, continuing to retry without surfacing the issue via health checks

Expected Behavior
The health status should be set to DOWN when the connector is unable to establish or maintain a connection to the email server, especially during prolonged retry scenarios.

Proposed Solution
Update the health check logic in the JakartaEmailListener to properly reflect connection failures. Specifically, consider updating the health status in this method when connection issues are detected.

Additional Context

  • The whenComplete block following the connection attempt should handle exceptions and update health accordingly
  • Currently, only the "could not reconnect to store" exception triggers a health down status
  • The connector may be experiencing issues with slow or unstable email servers that intermittently fail

Metadata

Metadata

Labels

kind:taskCategorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions