Skip to content

[Nexthop] Address NVMe block size mismatch#781

Open
ctikku-nexthop wants to merge 4 commits intofacebook:mainfrom
nexthop-ai:ctikku.nvme-blk-size-fix
Open

[Nexthop] Address NVMe block size mismatch#781
ctikku-nexthop wants to merge 4 commits intofacebook:mainfrom
nexthop-ai:ctikku.nvme-blk-size-fix

Conversation

@ctikku-nexthop
Copy link
Contributor

@ctikku-nexthop ctikku-nexthop commented Jan 6, 2026

Pre-submission checklist

  • I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running pip install -r requirements-dev.txt && pre-commit install
  • pre-commit run

Summary

The NVMe controllers have the capability to define “block size” to different values (512, 4096,
etc). This works just fine as long as we read and write directly from/to the device.

However, when we pre-build a bootable image using kiwi-ng, we also create the Master
Boot Record (MBR) and GPT partition table etc. These are expected to be placed in
“known” locations on the NMVe device, MBR at block 0 and primary GPT at block 1.
What kiwi-ng generates is a raw dump of the disk image that is slapped on block 0
and contains everything necessary for the device.

The problem happens when the NVMe controller block size is “NOT” 512 bytes (say its
4096 byte) and we are trying to boot an kiwi-ng generated image that we placed on
the device. grub tries to locate the GPT partition at block 1, aka offset 4096 on the
device but kiwi-ng placed it at offset 512 and hence is unable to find a bootable device.

The fix is to match the block size that kiwi-ng and NVMe controller use are the same.

Test Plan

 Case #1: NVMe block size != 512 bytes - We correct and update it to 512 bytes

     Starting dracut pre-mount hook...                                                                                                                              
[    7.537363] dracut-pre-mount[798]: NVMe-Fix: 4096 NVMe block size detected. Attempting format to 512...                                                              
[    7.544015] dracut-pre-mount[798]: NVMe-Fix: Formatting /dev/nvme0n1 with LBA Format 0...                                                                            
[   11.630218] dracut-pre-mount[798]: NVMe-Fix: Format complete.                    
 Installation...


 Case #2: NVMe block size == 512 bytes

     Starting dracut pre-mount hook...                              
 Installation... 

@meta-cla meta-cla bot added the CLA Signed label Jan 6, 2026
@meta-codesync
Copy link

meta-codesync bot commented Jan 28, 2026

@kevin645 has imported this pull request. If you are a Meta employee, you can view this in D91704213.

@facebook-github-bot
Copy link
Contributor

@ctikku-nexthop has updated the pull request. You must reimport the pull request before landing.

meta-codesync bot pushed a commit that referenced this pull request Feb 5, 2026
Summary:
**Pre-submission checklist**
- [x] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install`
- [x] `pre-commit run`

Including needed libraries python3.12 and nl3 libraries needed for the fboss binaries in the distro image. Found through using ldd on an installed binary and corrected.

Based behind #781 that's waiting on review for more distro image adjustments, so will include those changes as well. The commit in question should be last in the list at 12871fe

Pull Request resolved: #816

Test Plan:
Tested on a minipack3 and loaded up the most recent fboss binaries:
```
[root@fboss103 fboss]# ldd bin/sai_test-sai_impl | grep python
        libpython3.12.so.1.0 => /lib64/libpython3.12.so.1.0 (0x00007f3970000000)
[root@fboss103 fboss]# ldd bin/sai_test-sai_impl | grep found
[root@fboss103 fboss]#
```

Reviewed By: kevin645

Differential Revision: D92423287

Pulled By: KevinYakar

fbshipit-source-id: d6e73d59fcb510c32c9c55a76b1b2db376eac4cf
@facebook-github-bot
Copy link
Contributor

@ctikku-nexthop has updated the pull request. You must reimport the pull request before landing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments