Skip to content

Comments

Add sensitive data masking support#1343

Merged
TharmiganK merged 45 commits intomasterfrom
sensitive-data
Oct 8, 2025
Merged

Add sensitive data masking support#1343
TharmiganK merged 45 commits intomasterfrom
sensitive-data

Conversation

@TharmiganK
Copy link
Contributor

@TharmiganK TharmiganK commented Sep 3, 2025

Purpose

Fixes: ballerina-platform/ballerina-library#8211

Examples

Enable sensitive data masking

By default, sensitive data masking is disabled. Enable it in Config.toml:

[ballerina.log]
enableSensitiveDataMasking = true

Or configure it per logger:

log:Config secureConfig = {
    enableSensitiveDataMasking: true
};
log:Logger secureLogger = log:fromConfig(secureConfig);

Simple Example

import ballerina/log;

type User record {
    string id;
    @log:Sensitive
    string password;
    string name;
};

public function main() {
    User user = {id: "U001", password: "mypassword", name: "John Doe"};
    log:printInfo("user details", user = user);
}

Output (with masking enabled):

time=2025-08-20T09:15:30.123+05:30 level=INFO module="" message="user details" user={"id":"U001","name":"John Doe"}

Multiple Strategies

import ballerina/log;

isolated function maskString(string input) returns string {
    if input.length() <= 2 {
        return "****";
    }
    return input.substring(0, 1) + "****" + input.substring(input.length() - 1);
}

type User record {
    string id;
    @log:Sensitive{
        strategy: {
            replacement: "****"
        }   
    }
    string password;
    @log:Sensitive {
        strategy: {
            replacement: maskString
        }
    }
    string ssn;
    string name;
};

Masked String Function

User user = {id: "U001", password: "mypassword", name: "John Doe"};
string maskedUser = log:toMaskedString(user);
io:println(maskedUser); // {"id":"U001","name":"John Doe"}

Checklist

  • Linked to an issue
  • Updated the changelog
  • Added tests
  • Updated the spec
  • Checked native-image compatibility

# Conflicts:
#	ballerina/Ballerina.toml
#	ballerina/CompilerPlugin.toml
#	gradle.properties
@codecov
Copy link

codecov bot commented Sep 3, 2025

Codecov Report

❌ Patch coverage is 79.51807% with 68 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.67%. Comparing base (cda83f8) to head (42156dd).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
...a/io/ballerina/stdlib/log/MaskedStringBuilder.java 77.44% 43 Missing and 24 partials ⚠️
ballerina/natives.bal 94.44% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1343      +/-   ##
============================================
- Coverage     90.75%   81.67%   -9.09%     
- Complexity        7       81      +74     
============================================
  Files             6        8       +2     
  Lines           238      562     +324     
  Branches         59      114      +55     
============================================
+ Hits            216      459     +243     
- Misses           19       76      +57     
- Partials          3       27      +24     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@TharmiganK TharmiganK marked this pull request as ready for review September 19, 2025 08:14
@TharmiganK
Copy link
Contributor Author

TharmiganK commented Sep 19, 2025

Pending tasks:

@ThisaruGuruge ThisaruGuruge requested a review from Copilot October 8, 2025 05:23
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 22 out of 22 changed files in this pull request and generated 3 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

ThisaruGuruge
ThisaruGuruge previously approved these changes Oct 8, 2025
Copy link
Member

@ThisaruGuruge ThisaruGuruge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sonarqubecloud
Copy link

sonarqubecloud bot commented Oct 8, 2025

@TharmiganK TharmiganK merged commit 489c451 into master Oct 8, 2025
7 of 8 checks passed
@TharmiganK TharmiganK deleted the sensitive-data branch October 8, 2025 14:36
daneshk added a commit to daneshk/module-ballerina-log that referenced this pull request Nov 10, 2025
* [Automated] Update the native jar versions

* Add initial implementation

* Refactor with perf improvements

* Refactor by adding a builder

* Fix checkstyle issues

* Add sensitive data masking support for root logger

* Add annotation caching support

* Fix xml toString

* Support undefined fields in the record to string

* Add tests for masked string function

* Refactor class to reduce cognitive complexity

* Add support for map value

* Fix issues with field names with special characters

* Add support to enable sensitive data masking via configuration

* Add tests for masked logging

* Enhance type processing to handle intersection and reference types

* Add support for sensitive data masking in templates and value functions

* Add tests for readonly types

* Add an integration test

* Update changelog

* Update spec

* Optimize Unicode escaping by using a pre-computed hex lookup table

* Add tests for masking structurally similar records and basic types

* Add test for masking special characters in strings

* Update spec to clarify masking behavior and type extraction

* Add tests for masking empty arrays, tables, and records

* Add documentation for sensitive data masking features

* Enhance sensitive data masking documentation and functionality

* Refactor sensitive data masking strategy to use a dedicated maskString function

* Refactor sensitive data annotation from @sensitivedata to @sensitive

* Deprecate processTemplate function and replace with evaluateTemplate for improved clarity

* Address review suggestions

* Address review suggestions

* Update integration-tests/tests/resources/samples/masked-logger/Config.toml

---------

Co-authored-by: Danesh Kuruppu <daneshk@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add sensitive data masking support

3 participants