Skip to content

Kube-proxy on RHEL10 is broken due to nftables #17915

@rifelpet

Description

@rifelpet

In #17789 we switched from installing iptables on RHEL10 to installing nftables.

All RHEL10 prow jobs that use kube-proxy are now failing to pass validation. Examples:

https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/e2e-kops-grid-calico-rhel10arm64-k35/2015824951952543744

https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/e2e-kops-grid-flannel-rhel10arm64-k32/2015678737550413824

https://prow.k8s.io/view/gs/kubernetes-ci-logs/logs/e2e-kops-grid-amazonvpc-rhel10arm64-k35/2017497456488484864

All of them have kube-proxy in CLBO with these logs:

I0126 16:58:13.292759      12 conntrack.go:57] "Setting nf_conntrack_max" nfConntrackMax=262144
I0126 16:58:13.292816      12 conntrack.go:115] "Set sysctl" entry="net/netfilter/nf_conntrack_max" value=262144
E0126 16:58:13.292840      12 server.go:134] "Error running ProxyServer" err="open /proc/sys/net/netfilter/nf_conntrack_max: no such file or directory"
E0126 16:58:13.292855      12 run.go:72] "command failed" err="open /proc/sys/net/netfilter/nf_conntrack_max: no such file or directory"

with amazonvpc CNI reporting slightly different logs:

I0131 07:29:32.991525      11 proxier.go:763] "Syncing iptables rules" ipFamily="IPv4" fullSync=true
E0131 07:29:33.166182      11 proxier.go:805] "Failed to ensure chain jumps" err=<
	error appending rule: exit status 4: Warning: Extension conntrack revision 0 not supported, missing kernel module?
	Warning: Extension comment revision 0 not supported, missing kernel module?
	iptables v1.8.9 (nf_tables):  RULE_INSERT failed (No such file or directory): rule in chain INPUT
 > ipFamily="IPv4" table="filter" srcChain="INPUT" dstChain="KUBE-EXTERNAL-SERVICES"
I0131 07:29:33.166225      11 proxier.go:768] "Sync failed" ipFamily="IPv4" retryingTime="30s"
I0131 07:29:33.166250      11 proxier.go:757] "SyncProxyRules complete" ipFamily="IPv4" elapsed="174.776597ms"
I0131 07:29:33.166267      11 proxier.go:763] "Syncing iptables rules" ipFamily="IPv6" fullSync=true
E0131 07:29:33.174892      11 proxier.go:805] "Failed to ensure chain jumps" err=<
	error checking rule: exit status 2: Warning: Extension conntrack is not supported, missing kernel module?
	ip6tables v1.8.9 (nf_tables): Couldn't load match `conntrack':No such file or directory
	
	Try `ip6tables -h' or 'ip6tables --help' for more information.
 > ipFamily="IPv6" table="filter" srcChain="INPUT" dstChain="KUBE-EXTERNAL-SERVICES"
I0131 07:29:33.174922      11 proxier.go:768] "Sync failed" ipFamily="IPv6" retryingTime="30s"
I0131 07:29:33.174994      11 proxier.go:757] "SyncProxyRules complete" ipFamily="IPv6" elapsed="8.735314ms"

/kind bug

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions