Skip to content

Conversation

@chmouel
Copy link
Member

@chmouel chmouel commented Feb 4, 2026

Summary

Add secret masking for step stdout/stderr behind the enable-secret-masking feature flag.

How it works

  • During pod creation, Tekton collects secret values referenced from step secretKeyRef, envFrom.secretRef, and secret volumes (including TaskSpec volumes).
  • Collected values are de-duplicated and short values are skipped.
  • The mask payload is prepared as base64-per-secret lines, then wrapped as base64(gzip(payload)) and passed to the init container via TEKTON_SECRET_MASK_DATA.
  • The secret-mask-init subcommand decodes and uncompresses the env payload and writes a secret-mask file consumed by the entrypoint.
  • The runner loads the secret-mask file and decodes each line back to secret bytes.
  • A stream-safe masking writer redacts matching values to *** in stdout/stderr (and optional log files), handling split writes without leaking partial secret matches.

This commit wires the controller, init subcommand, runner integration, feature-flag surface, CRD/config plumbing, tests, and docs needed for this behavior.

Caveats

This is not 100% secure. The secret values are still present in the pod specification. Users with pods/get permission can still access them via kubectl get pod -o yaml.

This feature will not work on Windows nodes in this initial implementation.

However, this change provides meaningful improvements:

Aspect Before After
Visible in kubectl describe pod Yes (in command) No
Visible in process listings (ps aux) Yes No
Visible in kubectl get pod -o yaml Yes Yes

Demo

recording-20260204-17h35.mp4

Submitter Checklist

Release Notes

Add feature-flag `enable-secret-masking` to enable masking of secrets in step logs. When enabled, secret values from environment variables and volume mounts are redacted with `***` in stdout/stderr. This prevents leaking secrets when viewing logs via `kubectl logs` or similar tools.

@tekton-robot tekton-robot added the release-note Denotes a PR that will be considered when it comes time to generate release notes. label Feb 4, 2026
@tekton-robot tekton-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Feb 4, 2026
@chmouel
Copy link
Member Author

chmouel commented Feb 4, 2026

/kind enhancement

@tekton-robot
Copy link
Collaborator

@chmouel: The label(s) kind/enhancement cannot be applied, because the repository doesn't have them.

Details

In response to this:

/kind enhancement

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@chmouel
Copy link
Member Author

chmouel commented Feb 4, 2026

/kind feature

@tekton-robot tekton-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Feb 4, 2026
@chmouel chmouel force-pushed the enable-secret-masking branch from d355cd7 to 4bae1f3 Compare February 4, 2026 18:15
@tekton-robot tekton-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 4, 2026
@chmouel chmouel force-pushed the enable-secret-masking branch from 4bae1f3 to ac97fde Compare February 4, 2026 18:18
@tekton-robot tekton-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Feb 4, 2026
@chmouel chmouel force-pushed the enable-secret-masking branch from ac97fde to 33cd6bd Compare February 4, 2026 18:31
@vdemeester vdemeester self-assigned this Feb 4, 2026
@chmouel chmouel force-pushed the enable-secret-masking branch from 33cd6bd to c08ec75 Compare February 5, 2026 09:47
@tekton-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please ask for approval from vdemeester after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@chmouel chmouel marked this pull request as draft February 5, 2026 19:06
@tekton-robot tekton-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 5, 2026
@chmouel
Copy link
Member Author

chmouel commented Feb 5, 2026

There is a limitation with this, secrets over 32kb (default go chunk size) it would not be able to be properly hidden...

@chmouel chmouel force-pushed the enable-secret-masking branch from c08ec75 to 8b0714c Compare February 5, 2026 20:19
@tekton-robot tekton-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 5, 2026
@chmouel
Copy link
Member Author

chmouel commented Feb 5, 2026

I have made a new implementation and detailled it in the PR, basically we increase the chunk size to the largest secrets, very large secrets (up to 1mb due of kube limitation) will slow down log streaming, this shows a warning when that happen....

I have as well optimized the implemntation by gzipping the passed content as one screen...

@chmouel chmouel marked this pull request as ready for review February 5, 2026 20:42
@tekton-robot tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 5, 2026
@chmouel chmouel force-pushed the enable-secret-masking branch from 8b0714c to 53b72d1 Compare February 6, 2026 09:24
@vdemeester
Copy link
Member

/retest

Add alpha secret masking for step stdout/stderr behind the
`enable-secret-masking` feature flag.

How it works:
1. During pod creation, Tekton collects secret values referenced from
   step `secretKeyRef`, `envFrom.secretRef`, and secret volumes
   (including TaskSpec volumes).
2. Collected values are de-duplicated and short values are skipped.
3. The mask payload is prepared as base64-per-secret lines, then wrapped
   as base64(gzip(payload)) and passed to the init container via
   `TEKTON_SECRET_MASK_DATA`.
4. The `secret-mask-init` subcommand decodes and uncompresses that env
   payload and writes the secret-mask file consumed by entrypoint.
5. The runner loads that file and decodes each line back to secret bytes.
6. A stream-safe masking writer redacts matching values to "***" in
   stdout/stderr (and optional log files), handling split writes without
   leaking partial secret matches.

This commit wires the controller, init subcommand, runner integration,
feature-flag surface, CRD/config plumbing, tests, and docs needed for
this behavior.

Note: This feature will not work on Windows nodes in this initial
implementation

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Chmouel Boudjnah <chmouel@redhat.com>
@chmouel chmouel force-pushed the enable-secret-masking branch from 53b72d1 to 061bfa6 Compare February 9, 2026 13:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature Categorizes issue or PR as related to a new feature. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

3 participants