SREP-2644: Managed Policy additions for Karpenter/AutoNode on ROSA HCP by MitaliBhalla · Pull Request #2653 · openshift/managed-cluster-config

MitaliBhalla · 2026-02-26T03:26:08Z

What type of PR is this?

feature

What this PR does / why we need it?

Adds AWS managed policy support for Karpenter/AutoNode on ROSA HCP:

New Karpenter Controller Policy - EC2, KMS, IAM, and SQS permissions for node provisioning with red-hat-managed tag conditions
Control Plane Operator Update - Security group tagging for Karpenter discovery
Installer Policy Update - SQS queue validation
Policy Justification Doc - AWS-required permission justifications

Improvements over original PR #2581:

Added ec2:DescribeCapacityReservations for capacity reservation support
Enhanced KMS grant conditions following HCP CAPA Controller pattern

Which Jira/Github issue(s) this PR fixes?

SREP-2644

Special notes for your reviewer:

Re-validated with latest hypershift operator release
CreateLaunchTemplate succeeds with red-hat-managed=true tag
NodeClaims Ready, pods scheduled successfully
Details in SREP-2644

Pre-checks (if applicable):

Tested latest changes against a cluster
Included documentation changes with PR
If this is a new object that is not intended for the FedRAMP environment (if unsure, please reach out to team FedRAMP), please exclude it with:
```
matchExpressions:
- key: api.openshift.com/fedramp
  operator: NotIn
  values: ["true"]
```

- Add Karpenter controller credentials policy with least-privilege permissions - Add ec2:DescribeCapacityReservations for capacity reservation support - Add kms:ViaService condition to KMS grants (following CAPA pattern) - Update control plane operator policy with security group tagging - Update installer policy with SQS queue validation Addresses PR review feedback: - EC2 Describe actions: Cannot be tag-conditioned (AWS API limitation) - KMS grants: Added kms:ViaService condition per existing CAPA pattern - PassRole: Already has iam:PassedToService condition - IAM Get/ListInstanceProfiles: Read-only, require Resource '*' per AWS docs Tested and validated on latest hypershift operator release. Signed-off-by: Mitali Bhalla <mbhalla@redhat.com>

openshift-ci-robot · 2026-02-26T03:26:12Z

@MitaliBhalla: This pull request references SREP-2644 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What type of PR is this?

feature

What this PR does / why we need it?

Adds AWS managed policy support for Karpenter/AutoNode on ROSA HCP:

New Karpenter Controller Policy - EC2, KMS, IAM, and SQS permissions for node provisioning with red-hat-managed tag conditions

Control Plane Operator Update - Security group tagging for Karpenter discovery

Installer Policy Update - SQS queue validation

Policy Justification Doc - AWS-required permission justifications

Improvements over original PR #2581:

Added ec2:DescribeCapacityReservations for capacity reservation support

Enhanced KMS grant conditions following HCP CAPA Controller pattern

Which Jira/Github issue(s) this PR fixes?

SREP-2644

Special notes for your reviewer:

Re-validated with latest hypershift operator release

CreateLaunchTemplate succeeds with red-hat-managed=true tag

NodeClaims Ready, pods scheduled successfully

Details in SREP-2644

Pre-checks (if applicable):
Tested latest changes against a cluster

Included documentation changes with PR
If this is a new object that is not intended for the FedRAMP environment (if unsure, please reach out to team FedRAMP), please exclude it with:
matchExpressions:
- key: api.openshift.com/fedramp
  operator: NotIn
  values: ["true"]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-02-26T03:26:16Z

@MitaliBhalla: This pull request references SREP-2644 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What type of PR is this?

feature

What this PR does / why we need it?

Adds AWS managed policy support for Karpenter/AutoNode on ROSA HCP:

New Karpenter Controller Policy - EC2, KMS, IAM, and SQS permissions for node provisioning with red-hat-managed tag conditions

Control Plane Operator Update - Security group tagging for Karpenter discovery

Installer Policy Update - SQS queue validation

Policy Justification Doc - AWS-required permission justifications

Improvements over original PR #2581:

Added ec2:DescribeCapacityReservations for capacity reservation support

Enhanced KMS grant conditions following HCP CAPA Controller pattern

Which Jira/Github issue(s) this PR fixes?

SREP-2644

Special notes for your reviewer:

Re-validated with latest hypershift operator release

CreateLaunchTemplate succeeds with red-hat-managed=true tag

NodeClaims Ready, pods scheduled successfully

Details in SREP-2644

Pre-checks (if applicable):
Tested latest changes against a cluster

Included documentation changes with PR
If this is a new object that is not intended for the FedRAMP environment (if unsure, please reach out to team FedRAMP), please exclude it with:
matchExpressions:
- key: api.openshift.com/fedramp
  operator: NotIn
  values: ["true"]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2026-02-26T03:26:30Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: MitaliBhalla
Once this PR has been reviewed and has the lgtm label, please assign typeid for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS
resources/sts/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2026-02-26T03:37:15Z

@MitaliBhalla: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

MitaliBhalla · 2026-02-26T04:32:02Z

@joshbranham @rafael-azevedo

joshbranham

Some comments, sorry if they were addressed elsewhere, just foreseeing AWS questions

joshbranham · 2026-02-28T20:26:21Z

resources/sts/hypershift/openshift_hcp_control_plane_operator_credentials_policy.json

+            "Action": [
+                "ec2:CreateTags"
+            ],
+            "Resource": "arn:aws:ec2:*:*:security-group/*",


I may have missed it, but why does the CPO need this permission now as part of Karpenter?

In my understanding,The Control Plane Operator adds karpenter.sh/discovery tags to the cluster's security groups when AutoNode is enabled as a Day-2 operation.

Karpenter uses these tags to discover which security groups to attach to provisioned nodes. Without this permission, the CPO cannot tag the existing Red Hat-managed security groups, and Karpenter won't be able to find the correct security groups for node provisioning.

The permission is scoped to only allow tagging on security groups that already have aws:ResourceTag/red-hat-managed: "true"

I see now in the code where it is tagging the security group so this makes sense, thanks for clarifying. https://github.com/openshift/hypershift/blob/913aaded6c1cdc2b9683ffe20260fe95fbc6d43d/control-plane-operator/controllers/hostedcontrolplane/hostedcontrolplane_controller.go#L2655-L2659

joshbranham · 2026-02-28T20:30:33Z

resources/sts/hypershift/sts_hcp_installer_permission_policy.json

+            "Sid": "AllowListSQSQueues",
+            "Effect": "Allow",
+            "Action": [
+                "sqs:ListQueues"


I presume this is primarily for the installer to validate that the queue provided is actually existent?

Correct. This allows Cluster Service to validate that the customer-provided SQS queue exists before cluster creation, preventing misconfiguration errors.

@joshbranham / @MitaliBhalla Bala confirmed that this was not a hard requirement as long as we have clear error messages on the Karpenter APIs if the SQS queue is mis-configured. See the requirements in https://issues.redhat.com/browse/AUTOSCALE-354

We can remove this permission request from the installer role.

joshbranham · 2026-02-28T20:33:29Z

resources/sts/hypershift/openshift_hcp_karpenter_controller_credentials_policy.json

+      "Sid": "SSMReadActions",
+      "Effect": "Allow",
+      "Action": [
+        "ssm:GetParameter"


This a pretty sensitive call, lots of customer data could be accessed. I see we are wildcarding on /aws/service/*, is there any tag conditioning we can do? I presume karpenter looks up something in this path?

Great catch on the sensitivity concern, However, the resource is already scoped to only AWS-managed service parameters, not customer parameters:
"Resource": "arn:aws:ssm:*::parameter/aws/service/*"
The double colon (::) in the ARN - this means no account ID, indicating AWS-owned public parameters only.

Karpenter uses this to look up official AMI IDs. For example:

/aws/service/bottlerocket/aws-k8s-*/x86_64/latest/image_id

/aws/service/ami-amazon-linux-latest/*

These are read-only, publicly available AWS parameters - no customer data is accessible through this path.

joshbranham · 2026-02-28T20:53:12Z

resources/sts/hypershift/openshift_hcp_karpenter_controller_credentials_policy.json

+        "sqs:DeleteMessage",
+        "sqs:GetQueueUrl",
+        "sqs:ReceiveMessage"


Can these be conditioned or resource scoped?

Unfortunately, we can't easily scope these to a specific queue ARN because:

The queue is customer-provided - we don't know the queue name/ARN at policy creation time

sqs:GetQueueUrl is used to resolve the queue name to a URL, which requires Resource: "*"

However, we could potentially add a tag condition if customers are required to tag their interruption queues:
"Condition": { "StringEquals": { "aws:ResourceTag/rosa-karpenter-interruption-queue": "true" } }

Do we want to require customers to tag their SQS queues for this to work?
This adds a customer setup step but improves security scoping.
Alternatively, the current approach matches how other Karpenter deployments handle this - the queue ARN is validated at runtime by the controller.

I don't see AWS accepting this policy with unrestricted SQS permissions so we would likely have to require tags on the resources. The way we do this for KMS etc is to require red-hat: true to denote the customer made the resource but is granting us access to it.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 26, 2026

openshift-ci bot requested review from clcollins and iamkirkbater February 26, 2026 03:26

joshbranham reviewed Feb 28, 2026

View reviewed changes

joshbranham mentioned this pull request Mar 3, 2026

OCM-20625 | feat: Managed Policy additions for Karpenter on ROSA HCP #2581

Closed

3 tasks

Conversation

MitaliBhalla commented Feb 26, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

What this PR does / why we need it?

Which Jira/Github issue(s) this PR fixes?

Special notes for your reviewer:

Pre-checks (if applicable):

Uh oh!

openshift-ci-robot commented Feb 26, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

What this PR does / why we need it?

Which Jira/Github issue(s) this PR fixes?

Special notes for your reviewer:

Pre-checks (if applicable):

Uh oh!

openshift-ci-robot commented Feb 26, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

What this PR does / why we need it?

Which Jira/Github issue(s) this PR fixes?

Special notes for your reviewer:

Pre-checks (if applicable):

Uh oh!

openshift-ci bot commented Feb 26, 2026

Uh oh!

openshift-ci bot commented Feb 26, 2026

Uh oh!

MitaliBhalla commented Feb 26, 2026

Uh oh!

joshbranham left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

MitaliBhalla commented Feb 26, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Feb 26, 2026 •

edited by openshift-ci bot

Loading

openshift-ci-robot commented Feb 26, 2026 •

edited by openshift-ci bot

Loading