[Gateway API] gateway.k8s.aws/nlb listeners with `protocol: TLS` and `tls.mode: Passthrough` appear to build a TLS listener

[iAnomaly]

Bug Description

Gateways of class gateway.k8s.aws/nlb whose listeners use protocol: TLS and tls.mode: Passthrough appear to build a TLS listener.

Steps to Reproduce

• Create a Gateway with gatewayClassName whose manifest contains controllerName: gateway.k8s.aws/nlb
• Add a listener of with protocol: TLS and tls.mode: Passthrough

Expected Behavior

I expected a TCP listener on the AWS NLB side because traffic/streams should proxy to backends without TLS termination (raw streams)

Actual Behavior

AWS LB Controller log outputs show a model with a TLS listener attempting to be built which fails due to no discoverable/matching certificates (required for AWS NLB TLS listeners).

Regression
Was the functionality working correctly in a previous version ? [No]

Current Workarounds

Environment

• AWS Load Balancer controller version: v3.0.0
• Kubernetes version: v1.34.1
• Using EKS (yes/no), if so version?: Platform version eks.9
• Using Service or Ingress: Gateway
• AWS region: us-west-2
• How was the aws-load-balancer-controller installed:
  • If helm was used then please show output of helm ls -A | grep -i aws-load-balancer-controller
  • If helm was used then please show output of helm -n <controllernamespace> get values <helmreleasename>
  • If helm was not used, then copy/paste the exact command used to install the controller, including flags and options.
• Current state of the Controller configuration:
  • kubectl -n <controllernamespace> describe deployment aws-load-balancer-controller
• Current state of the Ingress/Service configuration:
  • kubectl describe ingressclasses
  • kubectl -n <appnamespace> describe ingress <ingressname>
  • kubectl -n <appnamespace> describe svc <servicename>

Possible Solution (Optional)

Contribution Intention (Optional)

• Yes, I'm willing to submit a PR to fix this issue
• No, I cannot work on a PR at this time

Additional Context

[zac-nixon]

I took a look into the implementation a bit, and my conclusion is that our implementation is sub-par for this particular use-case. I don't have clarity (yet) into what we can do to fix it. I would suggest for now, it use tcp protocol type, that way your traffic flow is treated as TCP from client -> nlb -> target, and then the target can handle the TLS bits.

Sorry for the inconvenience, and thank you for trying out the Gateway API implementation.

[iAnomaly]

Thanks @zac-nixon , appreciate your continued work on Gateway API conformance for AWS LB Controller!

Ya, we're testing TCPRoute instead but one of the Gataway API spec value props for TLSRoutes is routing based on hostname (via SNI inspection). Since AWS NLBs do not support SNI hostname based routing it probably made sense to only implement TLS listeners for protocol: TLS even with tls.mode: Passthrough given that limitation.

The alternative would be to implement tls.mode: Passthrough as a TCP listener in AWS NLB but that I could see that being a confusing user experience since the TLSRoute spec hostnames field would be ignored.

[zac-nixon]

Since AWS NLBs do not support SNI hostname based routing

Bingo, thats the crux of the issue I believe. I am going to go ahead and add a warning about this too, one of the value props of the TLSRoute is the SNI based routing, which we don't support but I believe it's nice to allow for the TLSRoutes to still exist within our implementation so you can add certs for SNI to the NLB listener via the TLSRoute.

My current fix is to..

1/ When protocol = TLS and mode = Passthrough. Provision a TCP listener.
2/ When protocol = TLS and mode = Terminate. Provision a TLS listener (this is how it works right now)

The alternative would be to implement tls.mode: Passthrough as a TCP listener in AWS NLB but that I could see that being a confusing user experience since the TLSRoute spec hostnames field would be ignored.

I believe this is the case anyways for TLS mode = Passthrough? As the NLB doesn't participate in the TLS part of the connection, the hostnames are not relevant. Maybe I misunderstand the point you're making.

[iAnomaly]

Since AWS NLBs do not support SNI hostname based routing

Bingo, thats the crux of the issue I believe. I am going to go ahead and add a warning about this too, one of the value props of the TLSRoute is the SNI based routing, which we don't support but I believe it's nice to allow for the TLSRoutes to still exist within our implementation so you can add certs for SNI to the NLB listener via the TLSRoute.

My current fix is to..

1/ When protocol = TLS and mode = Passthrough. Provision a TCP listener. 2/ When protocol = TLS and mode = Terminate. Provision a TLS listener (this is how it works right now)

The alternative would be to implement tls.mode: Passthrough as a TCP listener in AWS NLB but that I could see that being a confusing user experience since the TLSRoute spec hostnames field would be ignored.

I believe this is the case anyways for TLS mode = Passthrough? As the NLB doesn't participate in the TLS part of the connection, the hostnames are not relevant. Maybe I misunderstand the point you're making.

I mean: currently NLB does participate in the TLS part; it terminates it regardless of whether tls.mode: Passthrough. Understood you are proposing changing/fixing that to use TCP listeners instead (at which point you are right; TCP will just stream through with no TLS participation by NLB).

The callout I was trying to make is this might become a confusing user experience: once the controller starts reconciling Gateway listener of protocol: TLS and tls.mode: Passthrough as TCP listeners in NLB, then TLSRoutes bound to said listeners will still support the spec.hostnames field but it won't actually do anything in NLB; it becomes a no-operation which is operationally confusing if you are working through the Kubernetes layer trying to debug why hostname routing isn't working.

I also think it opens the problem of TLSRoutes supporting multiple listeners with the same port number as long as the hostnames don't collide? How could AWS LB Controller reconcile that since those ports alone would collide?

[zac-nixon]

Ok, I see your point. The issue is that we don't have a compliant version of TLSRoute, because our data plane does not support it. I think there is some value in still allowing people to use TLSRoutes, but making them aware that SNI routing will not work. What do you think?

[iAnomaly]

Yes, I agree with that theoretically but I don't know enough about Gateway API spec/conformance to know if that is permitted.