Firewall rule resolution needs to consider external addressing #9799
Description
Today, Nexus compiles the list of firewall rules to send to each sled agent here:
It does this by looking at all the instances on the sled, and then figuring out which rules apply to it. For example, if there's a FW rule targeting the subnet 1.2.3.0/24, and the instance has the VPC-private IP address 1.2.3.4, then a copy of that rule is sent to the OPTE instance on that sled. We evaluate this for a few different target types, include IP subnets, instance names, VPCs, etc.
What we don't do is consider the new attached subnets. For example, if you attach the subnet 1.2.3.0/24 to an instance named foo, and then create a firewall rule with an IP subnet target of 1.2.3.0/24, we do not consider that rule to apply to that instance. The only thing we look at for resolution is the instance name, VPC name, VPC Subnet name, or the VPC-private IP addresses of the NIC. To get that traffic to flow, one would need to specify the target in another way that does apply, such as by instance name, VPC name, or VPC Subnet name.
We should expand this to consider the attached subnets, and ideally Floating IPs as well. It could do this both with the IP and IP Subnet target types we have today, or with a new set of target types specifically for the Attached Subnet and Floating IP objects by name.