[Nexthop] Distro Image Builder Omnibus #847
Open
travisb-nexthop wants to merge 49 commits intofacebook:mainfrom
Open
[Nexthop] Distro Image Builder Omnibus #847travisb-nexthop wants to merge 49 commits intofacebook:mainfrom
travisb-nexthop wants to merge 49 commits intofacebook:mainfrom
Conversation
[Nexthop] Fix linting of Distro Image Shell scripts The newly introduced shell linter causes the trivial rebasing of the later Distro Image PRs to fail with many lint errors. Rather than adding a lot of noise to those PRs, re-lint new shell scripts separately. There are two functional fixes in build_image.sh here which seem to have been broken by previous linting efforts: 1. DOCKER_BUILD_ARGS optionally constructs arguments for `docker build`, it must not be quoted when used on the command line or docker will receive an invalid option (because it contains literal spaces). 2. DOCKER_ARGS is similarly constructed to factor out common options between the interactive and non-interactive cases. It must also not be quoted. All other changes are automatically applied by pre-commit.
[Nexthop] Build Distro Image in CI <!-- Thanks for submitting a pull request! We appreciate you spending the time to work on these changes. Please provide enough information so that others can review your pull request. --> **Pre-submission checklist** - [x] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [x] `pre-commit run` <!-- Explain the motivation for making this change and any other context that you think would help reviewers of your code. What existing problem does the pull request solve? --> This is the start of the Distro Image CI where the Distro Image is built. This simply invokes the `fboss-image build` command against the sample `from_source.json` image manifest. The resulting artifacts are not preserved in this PR. As part of this it is necessary to remove `-it` from the docker command because GHA's don't have a stdin. Further the sample image manifest now requires all components be present, so add them. <!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes the user interface. How exactly did you verify that your PR solves the issue you wanted to solve? --> <!-- If a relevant Github issue exists for this PR, please make sure you link that issue to this PR --> Look at the new workflow in this PR.
[Nexthop] Add FBOSS image builder CLI skeleton
**Pre-submission checklist**
- [X] I've ran the linters locally and fixed lint errors related to the
files I modified in this PR. You can install the linters by running `pip
install -r requirements-dev.txt && pre-commit install`
- [X] `pre-commit run`
Foundation for manifest-driven FBOSS distro image builder.
This PR adds a CLI tool (`fboss-image`) that provides the framework for
building FBOSS distribution images based on JSON manifests. The
implementation uses only Python standard library.
**Key components:**
- Basic CLI framework using standard library
- Manifest parser for JSON-based image definitions
- Build orchestration with defined component ordering
- Stub Device commands
- Pre-commit linting
cd fboss-image/distro_cli
python3 -m unittest discover -s tests -p '*_test.py'
[Nexthop] Add PXE-boot distro image export <!-- Thanks for submitting a pull request! We appreciate you spending the time to work on these changes. Please provide enough information so that others can review your pull request. --> **Pre-submission checklist** - [X] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [X] `pre-commit run` <!-- Explain the motivation for making this change and any other context that you think would help reviewers of your code. What existing problem does the pull request solve? --> Kiwi-NG already builds the PXE boot installer, we just need to move it into place. As part of this I removed the final ls from build_image_in_container.sh because it shows file paths which aren't relevant to users and so might be confusing. <!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes the user interface. How exactly did you verify that your PR solves the issue you wanted to solve? --> <!-- If a relevant Github issue exists for this PR, please make sure you link that issue to this PR --> Run `fboss-image build from_source.json` and see that the new file `fboss-distro-image_pxe.tar` is created successfully.
[Nexthop] Distro Infrastructure container PXE-boot MVP
<!-- Thanks for submitting a pull request! We appreciate you spending
the time to work on these changes. Please provide enough information so
that others can review your pull request. -->
**Pre-submission checklist**
- [x] I've ran the linters locally and fixed lint errors related to the
files I modified in this PR. You can install the linters by running `pip
install -r requirements-dev.txt && pre-commit install`
- [x] `pre-commit run`
<!-- Explain the motivation for making this change and any other context
that you think would help reviewers of your code. What existing problem
does the pull request solve? -->
Here the minimum viable Distro Infrastucture container needed to support
IPv4 and IPv6 PXE boot is added. IPv4 expects a DHCP server to exist
on the network to provide IPv4 addresses to the switch. IPv6 defaults
to supply its own DHCPv6 server on the L2 segment, but that can be
disabled.
This is a self-contained, interactive docker container which uses
Proxy DHCP (IPv4) or DHCPv6 (IPv6) to direct PXE-booting devices to
the container's TFTP server and web server.
iPXE is used to support loading the relatively large initrd image over
HTTP instead of TFTP and to support supplying changeable arguments to
the installer initrd. Currently these are hardcoded into autoexec.ipxe,
but future changes might autogenerate this file based on the needs of the
particular PXE installer.
For usage details, see the included README.md. As this is a MVP, those
instructions must be followed to the letter. Future work will
integrate with the fboss-image tool to drive the Distro Infra container
in a more user-friendly way.
Once PXE boot has completed, the MAC is made ineligible for PXE booting
again until reconfigured. This is to support PXE installing, then
booting off the internal drive for every subsequent boot until
PXE-booting is explicitly requested again.
Under IPv4, the boot flow with iPXE is simple because iPXE receives
the next-server IP address. The IPv4 boot flow looks like:
1. BIOS
2. iPXE
3. tftp://next-server/autoexec.ipxe
4. http://next-server/FBOSS-Distro-Image.{kernel,initrd}
5. http://next-server/FBOSS-Distro-Image.xz
Unfortunately IPv6 is more complicated. iPXE does not receive
next-server or anything like it under IPv6, so we cannot follow that
simple flow.
Further, iPXE by default tries to autoconfigure its network interface
with IPv4 first then IPv6. Thus if the network were configured to
support both IPv4 PXE boot and IPv6 PXE boot (the Distro
Infrastructure default), while the BIOS would load iPXE over IPv6,
iPXE would load the PXE installer over IPv4. This protocol switching
is not satisfactory testing.
To resolve these two problems, we separate iPXE into IPv4 and IPv6
versions. The IPv4 version operates as above. The IPv6 version
uses two intermediate scripts to insert the server_ip configuration.
The boot flow for IPv6 is:
1. BIOS
2. iPXEv6
3. Script embedded inside iPXEv6 which forces IPv6 and 'sources' a
generated script -serverip
4. -serverip, a generated script sets the server_ip variable before
passing control onto tftp://server-ip/autoexec.ipxe shared with
IPv4
5. tftp://next-server/autoexec.ipxe
6. http://next-server/FBOSS-Distro-Image.{kernel,initrd}
7. http://next-server/FBOSS-Distro-Image.xz
To support both paths with a common autoexec.ipxe, host-server is used
as server_ip when executing under IPv4.
<!-- Demonstrate the code is solid. Example: The exact commands you ran
and their output, screenshots / videos if the pull request changes the
user interface. How exactly did you verify that your PR solves the issue
you wanted to solve? -->
<!-- If a relevant Github issue exists for this PR, please make sure you
link that issue to this PR -->
Only manual, happy path is tested.
This has been tested manually against fboss103. Under IPv4 that test
output is:
```
ds103:#s-image/distro_infra $ ./build.sh && ./distro_infra.sh --intf vlan1033 --persist-dir data
...
=> exporting to image 0.7s
=> => exporting layers 0.6s
=> => writing image sha256:27dec285715ddfc30a692a4fee1cb34f79a02e581df34801a8a0330e256cf0c9 0.0s
=> => naming to docker.io/library/fboss_distro_infra 0.0s
Listening on vlan1033 - 10.250.33.194 & fc00:33::89
dnsmasq: started, version 2.85 DNS disabled
dnsmasq: compile time options: IPv6 GNU-getopt DBus no-UBus no-i18n IDN2 DHCP DHCPv6 no-Lua TFTP no-conntrack ipset auth cryptohash DNSSEC loop-detect inotify dumpfile
dnsmasq-dhcp: DHCP, proxy on subnet 10.250.33.194
dnsmasq-dhcp: DHCPv6, IP range ::fb05:5000:1 -- ::fb05:50ff:ffff, lease time 5m, template for vlan1033
dnsmasq-dhcp: router advertisement on vlan1033
dnsmasq-dhcp: DHCPv6, IP range fc00:33::fb05:5000:1 -- fc00:33::fb05:50ff:ffff, lease time 5m, constructed for vlan1033
dnsmasq-dhcp: router advertisement on fc00:33::, constructed for vlan1033
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
dnsmasq-dhcp: IPv6 router advertisement enabled
dnsmasq-dhcp: DHCP, sockets bound exclusively to interface vlan1033
dnsmasq-tftp: TFTP root is /distro_infra/persistent secure mode
dnsmasq-dhcp: read /distro_infra/dnsmasq_conf.d/default_ignore
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
Enter MAC address (blank to exit): dc-da-4d-fc-ad-2d
dnsmasq: inotify, new or changed file /distro_infra/dnsmasq_conf.d/dc-da-4d-fc-ad-2d
dnsmasq-dhcp: read /distro_infra/dnsmasq_conf.d/dc-da-4d-fc-ad-2d
Enter MAC address (blank to exit):
```
Reboot fboss103 here
```
>>Checking Media Presence......
>>Media Present......
>>Start PXE over IPv4 on MAC: DC-DA-4D-FC-AD-2D. Press ESC key to abort PXE boot.
Station IP address is 10.250.33.2
Server IP address is 10.250.33.194
NBP filename is ipxe.efi
NBP filesize is 1052160 Bytes
>>Checking Media Presence......
>>Media Present......
Downloading NBP file...
NBP file downloaded successfully.
iPXE initialising devices...
autoexec.ipxe... ok
iPXE 1.21.1+ (gfcfa0) -- Open Source Network Boot Firmware -- https://ipxe.org
Features: DNS HTTP iSCSI TFTP VLAN SRP AoE EFI Menu
Configuring (net0 dc:da:4d:fc:ad:2d)...... ok
http://10.250.33.194:6969/dc-da-4d-fc-ad-2d/pxeboot.initrd... ok
tftp://10.250.33.194/pxeboot.kernel... ok
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
EFI stub: Measured initrd data into PCR 9
[ 0.000000] Linux version 5.14.0-626.el9.x86_64...
```
Then the PXE installer runs. The Distro Infrastructure output during
this period is:
```
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
dnsmasq-dhcp: DHCPSOLICIT(vlan1033) 00:02:00:00:ab:11:ea:34:3d:47:ca:ee:d2:07
dnsmasq-dhcp: DHCPREPLY(vlan1033) 00:02:00:00:ab:11:ea:34:3d:47:ca:ee:d2:07 no addresses available
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
dnsmasq-dhcp: DHCPSOLICIT(vlan1033) 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-dhcp: DHCPADVERTISE(vlan1033) fc00:33::fb05:50dc:b9f7 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-dhcp: DHCPREQUEST(vlan1033) 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-dhcp: DHCPREPLY(vlan1033) fc00:33::fb05:50dc:b9f7 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-dhcp: RTR-SOLICIT(vlan1033)
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
dnsmasq-dhcp: DHCPSOLICIT(vlan1033) 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-dhcp: DHCPADVERTISE(vlan1033) fc00:33::fb05:50a2:9696 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-dhcp: DHCPREQUEST(vlan1033) 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-dhcp: DHCPREPLY(vlan1033) fc00:33::fb05:50a2:9696 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-dhcp: DHCPRELEASE(vlan1033) 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-tftp: error 8 User aborted the transfer received from fc00:33::fb05:50dc:b9f7
dnsmasq-tftp: sent /distro_infra/persistent/dc-da-4d-fc-ad-2d/ipxev6.efi to fc00:33::fb05:50dc:b9f7
dnsmasq-tftp: sent /distro_infra/persistent/dc-da-4d-fc-ad-2d/ipxev6.efi to fc00:33::fb05:50dc:b9f7
dnsmasq-dhcp: DHCPRELEASE(vlan1033) 00:01:00:01:2e:30:1a:70:dc:da:4d:fc:ad:2d
dnsmasq-dhcp: RTR-SOLICIT(vlan1033) dc:da:4d:fc:ad:2d
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
dnsmasq-dhcp: DHCPSOLICIT(vlan1033) 00:04:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
dnsmasq-dhcp: DHCPADVERTISE(vlan1033) fc00:33::fb05:50f5:dfc9 00:04:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
dnsmasq-dhcp: DHCPREQUEST(vlan1033) 00:04:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
dnsmasq-dhcp: DHCPREPLY(vlan1033) fc00:33::fb05:50f5:dfc9 00:04:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
dnsmasq-tftp: sent /distro_infra/persistent/dc-da-4d-fc-ad-2d/ipxev6.efi-serverip to fc00:33::fb05:50f5:dfc9
dnsmasq-tftp: sent /distro_infra/persistent/dc-da-4d-fc-ad-2d/autoexec.ipxe to fc00:33::fb05:50f5:dfc9
dnsmasq-dhcp: RTR-SOLICIT(vlan1033)
dnsmasq-dhcp: RTR-ADVERT(vlan1033) fc00:33::
dnsmasq-dhcp: DHCPSOLICIT(vlan1033) 00:04:62:19:3e:08:1d:5a:56:77:93:71:a4:d7:25:6f:4c:de
dnsmasq-dhcp: DHCPREPLY(vlan1033) fc00:33::fb05:50b2:2cbb 00:04:62:19:3e:08:1d:5a:56:77:93:71:a4:d7:25:6f:4c:de
dnsmasq-tftp: sent /distro_infra/persistent/dc-da-4d-fc-ad-2d/pxeboot_complete to fc00:33::fb05:50f5:dfc9
dnsmasq-dhcp: read /distro_infra/dnsmasq_conf.d/default_ignore
dc-da-4d-fc-ad-2d PXE booted, disabling future PXE boot provisioning
```
Critical is the line `dc-da-4d-fc-ad-2d PXE booted, disabling future PXE
boot provisioning`, which indicates that PXE boot has been detected as
complete and will not be offered to future boots.
Subsequent reboots of fboss103 time-out when attempting PXE boot and
boot off the NVME instead.
IPv6 works almost identically expect for downloads of the additional
autoipv6.ipxe script.
<!-- Thanks for submitting a pull request! We appreciate you spending the time to work on these changes. Please provide enough information so that others can review your pull request. --> **Pre-submission checklist** - [x] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [x] `pre-commit run` <!-- Explain the motivation for making this change and any other context that you think would help reviewers of your code. What existing problem does the pull request solve? --> Here a second output format for Kiwi-NG from the same template is added. This "tbz" rootfs tarball is then wrapped in an ONIE installer. The ONIE installer is based on the Demo OS ONIE installer from the ONIE project with a few modernizations. In particular zstd compression is used, requiring that a statically compiled zstd binary be included. Doing this dramatically improves decompression speed and therefore install time versus the best ONIE-native xz. The difference does not show up yet because the FBOSS binaries are not installed so the image size is still small. During internal testing with FBOSS binaries and xz, the installation time approached 10 minutes. zstd is expected to perform much better. <!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes the user interface. How exactly did you verify that your PR solves the issue you wanted to solve? --> <!-- If a relevant Github issue exists for this PR, please make sure you link that issue to this PR --> Loaded this on a TH5-based ONIE switch internally and manually verified that it booted correctly.
Summary
======
Split the single package into five packages based on three targets:
forwarding-stack: The forwarding stack and files necessary for
execution
platform-stack: The platform stack and supporting files
agent-benchmarks: The agent benchmark HW tests
The platform-stack and forwarding-stack targets have a production
archive and a -test archive which contains the tests and utility files
for that component.
This is necessary so we can install the Platform Stack and Forwarding
Stack at different times in the Distro Image. Further, we don't want
to install the tests in the Distro Image which is supposed to
represent a production-like environment.
This does slightly less than package-fboss.py which copies several
unnecessary files (such as .a library archives) and attempts to
dynamically package library dependencies from the build container. We
don't want the former files installed into the Distro Image and the
latter libraries are currently being put into the Distro Image in
another way.
Test Plan
=========
Inside the build container, run both package.py for all three targets
and manually verify that each produces the matching tar files.
…o FBOSS image (facebook#243) <!-- Thanks for submitting a pull request! We appreciate you spending the time to work on these changes. Please provide enough information so that others can review your pull request. --> **Pre-submission checklist** - [x] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [x] `pre-commit run` We noticed that the management interface name on `wdg101` after PXE booting one of our FBOSS distro images changed to `eno1`. The Accton SW has it as `eth0` which is the default in some FBOSS tests too. I looked at the difference and found that in our FBOSS distro image we are not passing `biosdevname=0 net.ifnames=0` kernel commandline args while Accton's linux image is passing it. These two kernel arguments are used to disable `Predictable Network Interface Naming` which names interfaces like `eno1`. <!-- Explain the motivation for making this change and any other context that you think would help reviewers of your code. What existing problem does the pull request solve? --> <!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes the user interface. How exactly did you verify that your PR solves the issue you wanted to solve? --> <!-- If a relevant Github issue exists for this PR, please make sure you link that issue to this PR --> Tested by modifying kernel commandline args on Wedge800 (`wdg101`) and verifying that the interface is listed at `eth0` and not `eno1`.
Including needed libraries python3.12 and nl3 libraries needed
for the fboss binaries. Found through using ldd on an installed
binary and corrected.
Tested on a minipack3 and loaded up the most recent fboss binaries:
```
[root@fboss103 fboss]# ldd bin/sai_test-sai_impl | grep python
libpython3.12.so.1.0 => /lib64/libpython3.12.so.1.0 (0x00007f3970000000)
[root@fboss103 fboss]# ldd bin/sai_test-sai_impl | grep found
[root@fboss103 fboss]#
```
<!-- Thanks for submitting a pull request! We appreciate you spending the time to work on these changes. Please provide enough information so that others can review your pull request. --> **Pre-submission checklist** - [X] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [X] `pre-commit run` <!-- Explain the motivation for making this change and any other context that you think would help reviewers of your code. What existing problem does the pull request solve? --> package_manager, by default, detects the platform it is running on then uses a compiled-in config file to install the platform BSP RPM using dnf. This requires that we have an RPM repo with those RPMs. In Distro Image we cannot presume an infrastructure RPM server with the correct BSP and dependent RPMs. Instead, create a device-local one for that purpose. The repo metadata is created at boot instead of just install time to help the development workflows where a new RPM is uploaded to the device, then the device is restarted for testing. <!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes the user interface. How exactly did you verify that your PR solves the issue you wanted to solve? --> <!-- If a relevant Github issue exists for this PR, please make sure you link that issue to this PR --> Build and load the image on a dut. Manually copy the BSP RPM into /usr/local/share/local_rpm_repo, run createrepo /usr/local/share/local_rpm_repo, then dnf search for that RPM: ``` dnf search nexthop Repository local_rpm_repo is listed more than once in the configuration Last metadata expiration check: 0:00:15 ago on Sat 17 Jan 2026 12:44:46 AM UTC. ======================= Name & Summary Matched: nexthop ======================== nexthop_bsp_kmods-6.11.1-1.fboss.el9.x86_64-1.el9-1.0.0.x86_64 : Nexthop BSP Kernel Modules ```
Summary:
Here an RPM specfile and surrounding machinery is added to build an
FBOSS-compatible Linux kernel from vanilla Linux sources which is
compatible with CentOS. On top of this reference configuration,
additional local changes from `fboss-local-overrides.yaml` are applied
prior to building.
To use, inside an FBOSS build container, run:
# fboss-image/kernel/scripts/build_kernel.sh 6.4.3 out
...
Kernel RPM build complete!
Output files:
out/x86_64/kernel-modules-6.4.3-1.fboss.el9.x86_64.rpm
out/x86_64/kernel-headers-6.4.3-1.fboss.el9.x86_64.rpm
out/x86_64/kernel-core-6.4.3-1.fboss.el9.x86_64.rpm
out/x86_64/kernel-devel-6.4.3-1.fboss.el9.x86_64.rpm
out/BUILD/linux-6.4.3/kernel/config_data.gz
out/SRPMS/kernel-6.4.3-1.fboss.el9.src.rpm
out/kernel-6.4.3.rpms.tar.gz
out/kernel-6.4.3-1.fboss.el9.src.rpm
out/RPMS/x86_64/kernel-modules-6.4.3-1.fboss.el9.x86_64.rpm
out/RPMS/x86_64/kernel-headers-6.4.3-1.fboss.el9.x86_64.rpm
out/RPMS/x86_64/kernel-core-6.4.3-1.fboss.el9.x86_64.rpm
out/RPMS/x86_64/kernel-devel-6.4.3-1.fboss.el9.x86_64.rpm
# ls fboss-image/kernel/dist/build-6.4.3/out/
kernel-6.4.3-1.fboss.el9.src.rpm kernel-core-6.4.3-1.fboss.el9.x86_64.rpm kernel-headers-6.4.3-1.fboss.el9.x86_64.rpm
kernel-6.4.3.rpms.tar.gz kernel-devel-6.4.3-1.fboss.el9.x86_64.rpm kernel-modules-6.4.3-1.fboss.el9.x86_64.rpm
Integrate distro_cli Python unit tests into the FBOSS CMake build system to enable automated testing in CI/CD pipelines. - Add FbossImageDistroCliTests.cmake to discover and register Python unit tests - Configure test discovery for distro_cli modules (builder, cmds, lib, tools) - Update root CMakeLists.txt to include distro_cli test suite - Enable distro_cli tests in GitHub Actions workflow - Update distro_cli README with build and test instructions The CMake configuration automatically discovers all *_test.py files and registers them as CTest targets, allowing tests to run via 'ctest' or 'make test'.
Provide basic tools and utiulities for the distro_cli build system. - Add custom exception classes for structured error handling - Define shared constants for Docker configuration and build settings - Implement path resolution utilities for components - Define test data fixture for image manifest validation These core utilities provide the base infrastructure required by all downstream components including Docker integration, artifact handling, and build orchestration.
Add Docker container and image management capabilities. - Implement Docker container wrapper with exec and run operations - Implement Docker image wrapper with build, pull, and tag operations - Add comprehensive unit tests for container and image functionality - Include test helper utilities for unit test infrastructure - Add build_docker.sh script for Docker build automation Added unit tests for docker infrastructure and image building.
Add artifact storage and caching capabilities for build outputs. - Implement artifact store for managing build artifacts - Add artifact caching and retrieval functionality - Include unit tests for artifact operations - Update test helpers to support artifact testing Added unit tests for artifact storage and caching.
Add file download capabilities for fetching build dependencies. - Implement download module for HTTP/HTTPS file retrieval - Add support for progress tracking during downloads - Include comprehensive unit tests for download operations
Add command execution capabilities for running build operations. - Implement execute module for running commands in Docker containers - Add support for command execution with output capture - Enable build operations to run in isolated container environments
Add abstract build component framework for managing build operations. - Implement AbstractComponent base class for build components - Add component lifecycle management (prepare, build, extract) - Integrate with artifact store, download, and execute modules - Enable extensible component-based build architecture Tests utilizing the above infrastructure will be added when component build supports are included.
Add build entrypoint orchestration for component-based builds. - Implement build entrypoint for coordinating component build workflows - Add support for build configuration and execution management - Include comprehensive unit tests for entrypoint functionality
Add ImageBuilder class for orchestrating component builds and image assembly. - ImageBuilder: Main orchestration class for building FBOSS images - Compression support for build artifacts (zstd) - Component build coordination - Test coverage for compression functionality - Updated build_test.py to use ImageBuilder This completes the core build orchestration infrastructure.
Add scripts and build configuration for building FBOSS forwarding and platform stacks. - CMakeLists.txt: Defines build targets for forwarding stack - package.py: Script for packaging forwarding and platform stacks - build_fboss_stack.sh: Script for building forwarding and platform stacks
travisb-nexthop
force-pushed
the
distro_image_omnibus
branch
2 times, most recently
from
8ace576 to
9e99011
Compare
travisb-nexthop
force-pushed
the
distro_image_omnibus
branch
from
9e99011 to
732a41d
Compare
2 tasks
travisb-nexthop
force-pushed
the
distro_image_omnibus
branch
from
732a41d to
2e587c2
Compare
meta-codesync bot
pushed a commit
that referenced
this pull request
Jan 27, 2026
Summary: **Pre-submission checklist** - [ ] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [ ] `pre-commit run` Upcoming PRs dramatically increase the scope of what is built during the Distro Image build to include the kernel, Platform Stack, and Forwarding Stack. This requires a runner with much more disk space to succeed and more cores to finish in a reasonable time. Use the same 32-core-ubuntu runner as used by the agent GHA. Pull Request resolved: #854 Test Plan: See that the Distro Build from the [Omnibus PR](#847) doesn't fail due to lack of disk space. Reviewed By: shiva-menta Differential Revision: D91613626 Pulled By: kevin645 fbshipit-source-id: 02d36fc70d79fd175839ae8f698af64642e109e6
travisb-nexthop
force-pushed
the
distro_image_omnibus
branch
2 times, most recently
from
9b8de32 to
db8a554
Compare
NOT INTENDED TO MERGE The Distro Image builder is spread across many focused, reviewable PRs. Unfortunately this makes it difficult to see the forest, especially since those PRs are split into several stacks. This PR combines all the outstanding Distro Image builder PRs into a single PR to provide the larger picture and allow testing and integration of the end product.
travisb-nexthop
force-pushed
the
distro_image_omnibus
branch
2 times, most recently
from
db8a554 to
708b59a
Compare
travisb-nexthop
force-pushed
the
distro_image_omnibus
branch
from
76019fe to
f635bc8
Compare
travisb-nexthop
force-pushed
the
distro_image_omnibus
branch
from
ffd6a75 to
81c9df1
Compare
travisb-nexthop
force-pushed
the
distro_image_omnibus
branch
from
81c9df1 to
b98ac5c
Compare
travisb-nexthop
added a commit
to nexthop-ai/fboss
that referenced
this pull request
Feb 10, 2026
<!-- Thanks for submitting a pull request! We appreciate you spending the time to work on these changes. Please provide enough information so that others can review your pull request. --> **Pre-submission checklist** - [X] I've ran the linters locally and fixed lint errors related to the files I modified in this PR. You can install the linters by running `pip install -r requirements-dev.txt && pre-commit install` - [X] `pre-commit run` <!-- Explain the motivation for making this change and any other context that you think would help reviewers of your code. What existing problem does the pull request solve? --> FBOSS is a beast to build with some forwarding stack compilation units needing 6 or 7 GB and some linking jobs requiring nearly 16G. Prevent running out of memory by limiting job parallelism of the platform stack and forwarding stack builds based on available memory. <!-- Demonstrate the code is solid. Example: The exact commands you ran and their output, screenshots / videos if the pull request changes the user interface. How exactly did you verify that your PR solves the issue you wanted to solve? --> <!-- If a relevant Github issue exists for this PR, please make sure you link that issue to this PR --> Shown to not OOM the build VM in the FBOSS Distro Omnibus PR facebook#847
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pre-submission checklist
pip install -r requirements-dev.txt && pre-commit installpre-commit runSummary
NOT INTENDED TO MERGE
The Distro Image builder is spread across many focused, reviewable PRs. Unfortunately this makes it difficult to see the forest, especially since those PRs are split into several stacks.
This PR combines all the outstanding Distro Image builder PRs into a single PR to provide the larger picture and allow testing and integration of the end product. Some changes do not yet have separate PRs, but those will be posted soon.
Test Plan
Run
./distro_cli/fboss-image build from_source.jsonwhich, after a long wait, produces three FBOSS Distro Images: PXE installer, ONIE installer, and USB installer (not tested).This particular image contains only the Fake-SAI and no BSPs and so is not fully representative. The missing pieces to make a full Distro Image will be shared privately due to licensing constraints.