Skip to content

Comments

Incorporate the csv fail safe support in the FTP listener#1515

Merged
Nuvindu merged 30 commits intoballerina-platform:masterfrom
Nuvindu:fail-safe-support
Jan 5, 2026
Merged

Incorporate the csv fail safe support in the FTP listener#1515
Nuvindu merged 30 commits intoballerina-platform:masterfrom
Nuvindu:fail-safe-support

Conversation

@Nuvindu
Copy link
Contributor

@Nuvindu Nuvindu commented Dec 2, 2025

Purpose

Fixes: ballerina-platform/ballerina-library#8502

This introduces a new csvFailSafe field to the ftp:ListenerConfiguration record to allow enabling a fail-safe support in listener APIs.

public type ListenerConfiguration record {|
    ...
    FailSafeOptions csvFailSafe?;
|};

# Fail-safe options for CSV content processing.
#
# + contentType - Specifies the type of content to log in case of errors
public type FailSafeOptions record {|
    ErrorLogContentType contentType = METADATA;
|};

By enabling it, the offending rows will be written to a file named <filename>_error.log in the current directory.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces CSV fail-safe support for the FTP listener and client, allowing processing to continue when malformed CSV records are encountered. When enabled, offending rows are written to an error.log file in the current directory while well-formed records are processed normally.

Key changes:

  • Added enableCsvFailSafe boolean configuration field to both ListenerConfiguration and ClientConfiguration records
  • Updated CSV parsing to pass the Environment parameter and fail-safe configuration to the data.csv library
  • Upgraded data.csv library dependency from 0.8.1 to 0.9.0 to support the fail-safe feature
  • Added comprehensive test coverage for fail-safe mode with both stream and record array CSV processing

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
ballerina/listener_endpoint.bal Added enableCsvFailSafe configuration field to ListenerConfiguration with documentation
ballerina/client_endpoint.bal Added enableCsvFailSafe configuration field to ClientConfiguration with documentation
ballerina/tests/listener_content_methods_test.bal Added two new test cases for CSV fail-safe mode and cleaned up test file reading, plus added error.log cleanup
ballerina/tests/resources/datafiles/test_data_with_error.csv New test data file containing CSV records with intentional errors for fail-safe testing
native/src/main/java/io/ballerina/stdlib/ftp/util/FtpContentConverter.java Updated convertBytesToCsv to accept Environment and fail-safe parameters; added fail-safe configuration constants and logic to create CSV parse options with file output mode
native/src/main/java/io/ballerina/stdlib/ftp/util/FtpConstants.java Added constant for the new enableCsvFailSafe configuration field
native/src/main/java/io/ballerina/stdlib/ftp/server/FtpListenerHelper.java Updated to extract and set the enableCsvFailSafe configuration on the FtpListener instance
native/src/main/java/io/ballerina/stdlib/ftp/server/FtpListener.java Added Environment field, new constructor accepting Environment, and setter/getter for enableCsvFailSafe
native/src/main/java/io/ballerina/stdlib/ftp/server/FtpContentCallbackHandler.java Added enableCsvFailSafe field and updated to pass Environment and fail-safe flag to CSV conversion
native/src/main/java/io/ballerina/stdlib/ftp/client/FtpClient.java Updated to extract and pass enableCsvFailSafe configuration when calling CSV conversion
native/src/main/java/io/ballerina/stdlib/ftp/ContentCsvStreamIteratorUtils.java Updated next method to accept Environment parameter required by updated CSV parsing API
gradle.properties Updated data.csv version from 0.8.1 to 0.9.0 and added uuid dependency version
build.gradle Added uuid-ballerina dependency required by updated task library
ballerina/Dependencies.toml Updated dependencies including data.csv (0.9.0), log, task, and added new dependencies like file, crypto, uuid, and various lang modules
ballerina/Ballerina.toml Updated data.csv-native dependency to 0.9.0-SNAPSHOT

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@niveathika
Copy link
Contributor

Purpose

Fixes: ballerina-platform/ballerina-library#8502

This introduces a new enableCsvFailSafe field to the ftp:ListenerConfiguration record to allow enabling a fail-safe support in listener APIs.

public type ListenerConfiguration record {|
    ...
    boolean enableCsvFailSafe = false;
|};

By enabling it, the offending rows will be written to a file named error.log in the current directory.

@Nuvindu shall we structure like this?

public type ListenerConfiguration record {|
    ...
    FailSafeProcessing failSafeProcessing?;
|};

public type FailSafeProcessing record {|
    boolean enableFailSafe = true;
    ErrorLogContentType contentType = METADATA;
|};

public enum ErrorLogContentType {
    METADATA,
    RAW,
    RAW_AND_METADATA
};

This is not what we previously discussed. We agreed on having only one enable/disable flag for simplicity. However, since ErrorLogContentType is necessary for security reasons, shall we include it?

niveathika
niveathika previously approved these changes Dec 8, 2025
Copy link
Contributor

@niveathika niveathika left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, however lets hold merging it until databinding fixes for 2.16.0 is released.

@codecov
Copy link

codecov bot commented Dec 23, 2025

Codecov Report

❌ Patch coverage is 98.03922% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 81.95%. Comparing base (d26e26e) to head (b3b651a).
⚠️ Report is 31 commits behind head on master.

Files with missing lines Patch % Lines
...ballerina/stdlib/ftp/util/FtpContentConverter.java 96.15% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1515      +/-   ##
============================================
+ Coverage     81.51%   81.95%   +0.43%     
- Complexity      583      793     +210     
============================================
  Files            40       55      +15     
  Lines          2851     3480     +629     
  Branches        466      594     +128     
============================================
+ Hits           2324     2852     +528     
- Misses          335      377      +42     
- Partials        192      251      +59     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Nuvindu Nuvindu force-pushed the fail-safe-support branch from 8dfd581 to b3b651a Compare January 3, 2026 13:38
@sonarqubecloud
Copy link

sonarqubecloud bot commented Jan 3, 2026

@Nuvindu Nuvindu merged commit 2ab4ca2 into ballerina-platform:master Jan 5, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add fail-safe CSV parsing support to FTP listener

2 participants