Incorporate the csv fail safe support in the FTP listener#1515
Incorporate the csv fail safe support in the FTP listener#1515Nuvindu merged 30 commits intoballerina-platform:masterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces CSV fail-safe support for the FTP listener and client, allowing processing to continue when malformed CSV records are encountered. When enabled, offending rows are written to an error.log file in the current directory while well-formed records are processed normally.
Key changes:
- Added
enableCsvFailSafeboolean configuration field to bothListenerConfigurationandClientConfigurationrecords - Updated CSV parsing to pass the Environment parameter and fail-safe configuration to the data.csv library
- Upgraded data.csv library dependency from 0.8.1 to 0.9.0 to support the fail-safe feature
- Added comprehensive test coverage for fail-safe mode with both stream and record array CSV processing
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| ballerina/listener_endpoint.bal | Added enableCsvFailSafe configuration field to ListenerConfiguration with documentation |
| ballerina/client_endpoint.bal | Added enableCsvFailSafe configuration field to ClientConfiguration with documentation |
| ballerina/tests/listener_content_methods_test.bal | Added two new test cases for CSV fail-safe mode and cleaned up test file reading, plus added error.log cleanup |
| ballerina/tests/resources/datafiles/test_data_with_error.csv | New test data file containing CSV records with intentional errors for fail-safe testing |
| native/src/main/java/io/ballerina/stdlib/ftp/util/FtpContentConverter.java | Updated convertBytesToCsv to accept Environment and fail-safe parameters; added fail-safe configuration constants and logic to create CSV parse options with file output mode |
| native/src/main/java/io/ballerina/stdlib/ftp/util/FtpConstants.java | Added constant for the new enableCsvFailSafe configuration field |
| native/src/main/java/io/ballerina/stdlib/ftp/server/FtpListenerHelper.java | Updated to extract and set the enableCsvFailSafe configuration on the FtpListener instance |
| native/src/main/java/io/ballerina/stdlib/ftp/server/FtpListener.java | Added Environment field, new constructor accepting Environment, and setter/getter for enableCsvFailSafe |
| native/src/main/java/io/ballerina/stdlib/ftp/server/FtpContentCallbackHandler.java | Added enableCsvFailSafe field and updated to pass Environment and fail-safe flag to CSV conversion |
| native/src/main/java/io/ballerina/stdlib/ftp/client/FtpClient.java | Updated to extract and pass enableCsvFailSafe configuration when calling CSV conversion |
| native/src/main/java/io/ballerina/stdlib/ftp/ContentCsvStreamIteratorUtils.java | Updated next method to accept Environment parameter required by updated CSV parsing API |
| gradle.properties | Updated data.csv version from 0.8.1 to 0.9.0 and added uuid dependency version |
| build.gradle | Added uuid-ballerina dependency required by updated task library |
| ballerina/Dependencies.toml | Updated dependencies including data.csv (0.9.0), log, task, and added new dependencies like file, crypto, uuid, and various lang modules |
| ballerina/Ballerina.toml | Updated data.csv-native dependency to 0.9.0-SNAPSHOT |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
native/src/main/java/io/ballerina/stdlib/ftp/util/FtpContentConverter.java
Outdated
Show resolved
Hide resolved
native/src/main/java/io/ballerina/stdlib/ftp/util/FtpContentConverter.java
Outdated
Show resolved
Hide resolved
@Nuvindu shall we structure like this? This is not what we previously discussed. We agreed on having only one enable/disable flag for simplicity. However, since |
native/src/main/java/io/ballerina/stdlib/ftp/util/FtpContentConverter.java
Outdated
Show resolved
Hide resolved
99a0064 to
a57537b
Compare
a57537b to
4a71a9f
Compare
e572bbb to
caeb7c5
Compare
caeb7c5 to
787f2cd
Compare
niveathika
left a comment
There was a problem hiding this comment.
LGTM, however lets hold merging it until databinding fixes for 2.16.0 is released.
…-ballerina-ftp into fail-safe-support
…-ballerina-ftp into fail-safe-support
a9d09e9 to
e41dd33
Compare
e41dd33 to
b7e2350
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1515 +/- ##
============================================
+ Coverage 81.51% 81.95% +0.43%
- Complexity 583 793 +210
============================================
Files 40 55 +15
Lines 2851 3480 +629
Branches 466 594 +128
============================================
+ Hits 2324 2852 +528
- Misses 335 377 +42
- Partials 192 251 +59 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
4f9e16e to
52e6ff7
Compare
52e6ff7 to
8dfd581
Compare
8dfd581 to
b3b651a
Compare
|



Purpose
Fixes: ballerina-platform/ballerina-library#8502
This introduces a new
csvFailSafefield to theftp:ListenerConfigurationrecord to allow enabling a fail-safe support in listener APIs.By enabling it, the offending rows will be written to a file named
<filename>_error.login the current directory.