Nethermind 1.36.0 crashes immediately with exit code 132 (SIGILL - Illegal Instruction) on Intel Celeron N5105 CPU (Jasper Lake / Tremont architecture).

[htimsk]

System Information

• CPU: Intel Celeron N5105 @ 2.00GHz (Tremont microarchitecture)
• CPU Instruction Sets: SSE4.2, AES-NI, SHA-NI (NO AVX/AVX2 support)
• Nethermind Version: 1.36.0
• Docker Image: nethermind/nethermind:1.36.0
• Network: Hoodi testnet
• Platform: Rocket Pool Smart Node

Expected Behavior

Nethermind should start and sync normally.

Actual Behavior

Container exits with code 132 after ~12 seconds, during the "Loading fork choice info" phase. The crash is
reproducible on every restart.

Logs

27 Jan 17:49:22 | Rerunning block after reorg or pruning: 2116391 (0x693d81...85306d)
27 Jan 17:49:22 | Numbers resolved, level = 2116518, header = 2116518, body = 2116518
27 Jan 17:49:22 | Beacon Numbers resolved, level = 0, header = 0, body = 0
27 Jan 17:49:22 | Loading fork choice info
rocketpool_eth1 exited with code 132 (restarting)

Root Cause

Exit code 132 = SIGILL (Signal 4). The Celeron N5105 uses Tremont cores, which do not support AVX/AVX2
instructions. The crash is likely caused by RocksDB or another dependency being compiled with AVX optimizations.

Request

Please consider publishing a portable or legacy Docker image built with PORTABLE=1 for RocksDB, targeting
CPUs without AVX support. This would enable Nethermind to run on:

• Intel Atom processors
• Intel Celeron/Pentium Silver (Jasper Lake, Gemini Lake)
• Older pre-Sandy Bridge systems
• Some VM environments without AVX passthrough

[LukaszRozmej]

Could you check if this wasn't already fixed by #10007 ?

[htimsk]

Could you check if this wasn't already fixed by #10007 ?

Thanks for the response @LukaszRozmej. I looked at PR #10007, and unfortunately, it doesn't address this issue. That PR fixes .NET Vector hardware acceleration detection (Vector128 vs Vector256 intrinsics fallback), but my crash is happening in the native RocksDB library itself, which appears to be compiled with AVX optimizations.

The SIGILL occurs during RocksDB initialization, before any .NET vector code would execute. The fix would require building RocksDB with PORTABLE=1 to disable AVX-specific instructions.

For context: This regression appeared in v1.36.0 (via Rocket Pool Smart Node v1.18.10). Rolling back to Nethermind v1.35.8 resolves it. Affected hardware includes Intel Celeron N5105 (Jasper Lake/Tremont) and similar CPUs without AVX support.

Would the team consider publishing a portable Docker image variant for users on legacy hardware?

[rubo]

@htimsk Could you please provide logs for that RocksDB-related issue?

[htimsk]

Additional reproduction on Intel Celeron N5105 (Jasper Lake)

Environment:

• CPU: Intel Celeron N5105 @ 2.00GHz (Jasper Lake / Tremont microarchitecture)
• CPU Flags: NO AVX/AVX2 - only has SSE, SSE2, SSE4.1, SSE4.2, AES-NI, SHA-NI
• OS: Ubuntu 24.04 - Linux 6.8.0-90-generic x86_64
• Deployment: Rocket Pool Docker stack on Hoodi testnet

Version comparison:

• ✅ 1.35.8 - works perfectly
• ❌ 1.36.0 - crashes with SIGILL (exit code 132)

Crash details:

• Exit code: 132 (SIGILL - Illegal Instruction)
• Crash point: During/after "Loading fork choice info"
• RocksDB version: v10.4.2
• Crash is 100% reproducible - happens every ~10 seconds in a restart loop

Important: The crash occurs so early that RocksDB never writes anything to its log files. The
/nethermind/logs/ and /nethermind/nethermind_db/ directories remain empty - no LOG files are created. This
makes debugging from the container side impossible; the only evidence of the crash is the exit code 132 from
Docker.

RocksDB libraries in container:
-rwxr--r-- 1 root root 15146104 Aug 16 07:13 librocksdb.so
-rwxr--r-- 1 root root 12251848 Nov 12 09:13 librocksdb-jemalloc.so
-rwxr--r-- 1 root root 52 Nov 12 09:12 librocksdb-musl.so

CPU flags (confirming NO AVX):
flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc
art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc ... sse4_1 sse4_2
x2apic movbe popcnt ... aes xsave rdrand ... sha_ni ...
Note: AVX and AVX2 are completely absent from CPU flags.

Log showing crash loop:
| Loading fork choice info
| Block tree initialized, last processed is 2123948...
| Merge by Nethermind initialized in 751ms
| Detected HalfPath key scheme.
| BlockchainProcessor started.
[container exits with code 132 - SIGILL]
[restarts, repeats every ~10 seconds]

This confirms the issue affects Jasper Lake (Tremont) CPUs in addition to Gemini Lake. Both use the Tremont
microarchitecture without AVX support.

A PORTABLE=1 RocksDB build would resolve this for all non-AVX systems including:

• Intel Atom (various generations)
• Intel Celeron/Pentium Silver (Jasper Lake, Gemini Lake)
• Older pre-Sandy Bridge systems
• VMs without AVX passthrough

[htimsk]

Ok with the help of MoltBot I have the following:

Additional debugging findings - Portable RocksDB does not fix the issue

System:

• CPU: Intel Celeron N5105 @ 2.00GHz (Jasper Lake / Tremont microarchitecture)
• CPU flags: SSE, SSE2, SSE4.1, SSE4.2, AES-NI, SHA-NI — NO AVX/AVX2
• OS: Ubuntu 24.04 (Linux 6.8.0-90-generic x86_64)
• Deployment: Rocket Pool Docker stack on Hoodi testnet

Confirmed behavior:

• ✅ Nethermind 1.35.8 works perfectly
• ❌ Nethermind 1.36.0 crashes with exit code 132 (SIGILL)

What I tried:

Built RocksDB v10.4.2 from source with the portable flag to eliminate AVX instructions:
git clone https://github.com/facebook/rocksdb.git
cd rocksdb && git checkout v10.4.2
PORTABLE=1 make shared_lib -j$(nproc)

Created a custom Docker image replacing /nethermind/librocksdb.so with the portable build, including all required
shared library dependencies (libsnappy, liblz4, libzstd, libbz2, libgflags).

Result: Still crashes with exit code 132 at the same point:
28 Jan 22:14:18 | Loading fork choice info
rocketpool_eth1 exited with code 132

Conclusion:

The AVX-dependent code causing SIGILL is not in RocksDB (or not only in RocksDB). The crash occurs during "Loading
fork choice info" which suggests the illegal instruction may be in:

• Nethermind's C# code using System.Runtime.Intrinsics with AVX
• Another native library bundled with 1.36.0
• JIT-compiled code paths that detect and use AVX

This might help narrow down where to look in the 1.36.0 changes.

Workaround: Reverted to 1.35.8 which continues to run without issues.

[htimsk]

---

Actual Root Cause - AVX intrinsics in C# code:

After analyzing the Nethermind source, the SIGILL is caused by hardcoded AVX2/AVX512 intrinsics in multiple
locations:

1. Int256 library v1.4.0 (NethermindEth/int256)

• UInt256.cs uses Vector256, Avx2., and Avx512F. for arithmetic operations
• This is an external NuGet dependency that was updated in 1.36.0

2. Nethermind.Core - src/Nethermind/Nethermind.Core/Extensions/UInt256Extensions.cs

• Uses Avx2.IsSupported, Avx2.Permute4x64(), Avx2.Shuffle()
• Uses Avx512Vbmi.VL.PermuteVar32x8()

3. Nethermind.Evm - src/Nethermind/Nethermind.Evm/EvmStack.cs

• AVX2/AVX512 intrinsics for endian conversion in EVM stack operations

The crash at "Loading fork choice info" makes sense - this is when UInt256 operations start running heavily.

---

Why this regressed in 1.36.0:

The combination of:

1. Upgrade to .NET 10
2. Int256 library update to v1.4.0 with aggressive SIMD optimizations
3. No runtime fallback for CPUs without AVX support

The code uses patterns like:
if (Avx2.IsSupported) { /* AVX path / }
else if (Avx512...) { / AVX512 path / }
// Missing: else { / SSE/scalar fallback */ }

---

Suggested fix:

Similar to how BLST was fixed in PR #8337 ("compiled with portable flag so should not crash on old CPUs"), the
Int256 library and Nethermind's SIMD code paths need:

1. Runtime CPU detection with fallbacks - Add else branches with SSE or scalar implementations
2. Or a portable build configuration - Build flag to exclude AVX codepaths entirely

Affected CPUs include:

• Intel Atom processors
• Intel Celeron/Pentium Silver (Jasper Lake, Gemini Lake, Tremont)
• Pre-Sandy Bridge Intel CPUs
• AMD CPUs before Bulldozer
• VMs without AVX passthrough

---

Workaround: Reverted to 1.35.8 which continues to run without issues.

[LukaszRozmej]

We will add fallbacks

[htimsk]

We will add fallbacks

TY! I luv the product and am looking forward to testing the new release. I appreciate all the work the core team does.

[rubo]

@htimsk Could you please try nethermindeth/nethermind:1.36.0-mcl.2 Docker image? Other users with similar issues got it working with this.