Mount host kernel modules in build pods for in-tree dependency resolution#1222
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: TomerNewman The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
✅ Deploy Preview for kubernetes-sigs-kmm ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
056a9b5 to
ee9b60f
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1222 +/- ##
==========================================
- Coverage 79.09% 73.75% -5.34%
==========================================
Files 51 66 +15
Lines 5109 4611 -498
==========================================
- Hits 4041 3401 -640
- Misses 882 1048 +166
+ Partials 186 162 -24 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| { | ||
| Name: volNameLibModules, | ||
| MountPath: "/lib/modules", | ||
| MountPath: "/host/lib/modules", |
There was a problem hiding this comment.
Why do we need to change the VolumeMount for worker pod? If we change it, then the current code won't work, i think
There was a problem hiding this comment.
I changed it for consistency, Im reviewing it now since the e2e test failed.
|
/hold |
ee9b60f to
5de5373
Compare
| `modprobe` can then follow that link and load the in-tree dependencies as needed. | ||
|
|
||
| In the example below, we use `host` as the symbolic link name under `/opt/usr/lib/modules/[kernel-version]`: | ||
| In the example below, we use `host` as the symbolic link name under `/opt/lib/modules/[kernel-version]`: |
There was a problem hiding this comment.
This doc mistake is not related, but still thought to fix it here
|
/unhold |
internal/pod/workerpodmanager.go
Outdated
| { | ||
| Name: "host-lib-modules", | ||
| VolumeSource: v1.VolumeSource{ | ||
| HostPath: &v1.HostPathVolumeSource{ | ||
| Path: "/lib/modules", | ||
| Type: &hostPathDirectory, | ||
| }, | ||
| }, | ||
| }, | ||
| { | ||
| Name: "host-usr-lib-modules", | ||
| VolumeSource: v1.VolumeSource{ | ||
| HostPath: &v1.HostPathVolumeSource{ | ||
| Path: "/usr/lib/modules", | ||
| Type: &hostPathDirectory, | ||
| }, | ||
| }, | ||
| }, |
There was a problem hiding this comment.
Why do we need additional volumes? They are already defined here.
There was a problem hiding this comment.
Because I made the Build pod use /host/lib/modules to avoid Kaniko conflicts (kaniko extracts the base image into /).
The image Dockerfile create symlinks pointing there, so worker pod needs the same mount for those symlinks to work.
There was a problem hiding this comment.
@TomerNewman i think we don't need to define the volumes twice, we can just define a new VolumeMount for the same volume and mount it into the /host/lib/modules/ etc'
There was a problem hiding this comment.
Oh by bad, made a mix up with the volumeMount, fixing it now
internal/pod/workerpodmanager.go
Outdated
| { | ||
| Name: "host-lib-modules", | ||
| MountPath: "/host/lib/modules", | ||
| ReadOnly: true, | ||
| }, | ||
| { | ||
| Name: "host-usr-lib-modules", | ||
| MountPath: "/host/usr/lib/modules", | ||
| ReadOnly: true, | ||
| }, |
There was a problem hiding this comment.
Why do we need to mount it to 2 different paths? We already mount them at /lib/modules and /usr/lib/modules.
If we change it we need to make sure to also update the docs changes in this PR and the commit message.
| { | ||
| Name: "lib-modules", | ||
| ReadOnly: true, | ||
| MountPath: "/host/lib/modules", | ||
| }, | ||
| { | ||
| Name: "usr-lib-modules", | ||
| ReadOnly: true, | ||
| MountPath: "/host/usr/lib/modules", | ||
| }, |
There was a problem hiding this comment.
Why don't we mount it to the same location as the worker pods instead of setting a new paths and then adding this new path to the worker pod as well and in addition keep the old "legacy" path?
There was a problem hiding this comment.
We can't mount to /lib/modules in build pods because Kaniko extracts the base image to /, which conflicts with a mount in /lib/modules.
bfe60ca to
0a3376c
Compare
0a3376c to
7559ca5
Compare
| VolumeSource: v1.VolumeSource{ | ||
| HostPath: &v1.HostPathVolumeSource{ | ||
| Path: "/lib/modules", | ||
| Type: &hostPathDirectory, |
There was a problem hiding this comment.
Can we do the same as here https://github.com/kubernetes-sigs/kernel-module-management/pull/1222/changes#diff-98eead772f708d7e017665ada1b681377fd492ff91aa5b5ba17bb5f52375ececR175 and remove the hostPathDirectory variable which is only used here?
63831e1 to
a1ea805
Compare
Enable OOT modules to depend on in-tree modules by mounting the host's /lib/modules into pods at /host/lib/modules. The /host prefix avoids conflicts with Kaniko during builds. Also removing /usr/lib/modules mount from build and worker pods. Worker pods get both paths (/lib/modules and /host/lib/modules) for backward compatibility. Docs in docs/mkdocs/documentation/kmod_image.md updated with usage example.
a1ea805 to
93fa153
Compare
|
I am happy with this PR when @yevgeny-shnaidman is. |
|
/lgtm |
Enable OOT modules to depend on in-tree modules by mounting the host's
/lib/modulesinto pods at/host/lib/modules. The/hostprefix avoids conflicts with Kaniko during builds.Also removing
/usr/lib/modulesfrom worker and build pods.Worker pods get both paths (
/lib/modulesand/host/lib/modules) for backward compatibility.Docs in
docs/mkdocs/documentation/kmod_image.mdupdated with usage example./cc @ybettan @yevgeny-shnaidman