Enzyme AD backend limitations on Julia v1.12+

[ChrisRackauckas-Claude]

Summary

During the refactoring of AD tests to use DifferentiationInterface (#1255), we discovered that Enzyme is missing rules for several cases.

Affected Tests

1. ModelingToolkit (MTK) type differentiation - null_de.jl

Error: StackOverflowError in EnzymeInterpreter

Cause: Missing Enzyme rules for complex ModelingToolkit types

Workaround: Enzyme backend is filtered out for MTK-related derivative tests in null_de.jl:119

Reproducer:

using OrdinaryDiffEq, ModelingToolkit, DifferentiationInterface
using ADTypes: AutoEnzyme
using Enzyme
using ModelingToolkit: t_nounits as t, D_nounits as D

@parameters P
@variables x(t)
sys = structural_simplify(ODESystem([D(x) ~ P], t, [x], [P]; name = :sys))
prob = ODEProblem(sys, [x => 0.0], (0.0, 1.0), [P => NaN])

x_at_1(P) = solve(remake(prob; p = [sys.P => P]), Tsit5())(1.0, idxs = x)

# This will fail with StackOverflowError - missing rules
DifferentiationInterface.derivative(x_at_1, AutoEnzyme(), 1.0)

2. Ensemble problem differentiation - ensemble_ad.jl

Error: Issues with ensemble problem differentiation

Cause: Missing Enzyme rules for EnsembleProblem

Workaround: Enzyme backend is filtered out for ensemble gradient tests in ensemble_ad.jl:86 and ensemble_ad.jl:168

Reproducer:

using OrdinaryDiffEq, DifferentiationInterface
using ADTypes: AutoEnzyme
using Enzyme
using SciMLSensitivity

function f!(du, u, p, t)
    du[1] = p[1] * u[1] - p[2] * u[1] * u[2]
    du[2] = -p[3] * u[2] + p[4] * u[1] * u[2]
end

u0 = [1.0, 1.0]
p = [1.5, 1.0, 3.0, 1.0]
prob = ODEProblem(f!, u0, (0.0, 10.0), p)

N = 3
eu0 = rand(N, 2)
ep = rand(N, 4)

function sum_of_e_solution(params)
    ensemble_prob = EnsembleProblem(
        prob,
        prob_func = (prob, i, repeat) -> remake(prob, u0 = eu0[i, :], p = params[i, :])
    )
    sol = solve(ensemble_prob, Tsit5(), EnsembleSerial(), trajectories = N)
    return sum(Array(sol.u[1]))
end

# This will fail - missing rules
DifferentiationInterface.gradient(sum_of_e_solution, AutoEnzyme(), ep)

3. Complex number ODE differentiation - complex_number_ad.jl

Error: Issues with complex number ODE differentiation

Cause: Missing Enzyme rules for complex number ODE problems

Workaround: Enzyme backend is filtered out for complex number tests in complex_number_ad.jl:107

Action Items

These cases need Enzyme rules to be implemented in the appropriate packages (SciMLSensitivity, ModelingToolkit, etc.)

Related

• PR Refactor AD tests to use DifferentiationInterface with version-dependent backends #1255 - Refactor AD tests to use DifferentiationInterface
• Issue Mooncake AD backend limitations with MTK and Ensemble problems #1256 - Mooncake AD backend limitations
• Issue Zygote AD backend limitations with MTK types #1257 - Zygote AD backend limitations

[ChrisRackauckas]

These aren't missing rules but an upstream issue in Enzyme, EnzymeAD/Enzyme.jl#2904 that causes any differentiation with the ODESolution to throw an LLVM level error.

[wsmoses]

@ChrisRackauckas can you confirm/check this is now fine?

[ChrisRackauckas-Claude]

Test Results for Enzyme v0.13.116 on Julia 1.12.4

The original LLVM-level bug from EnzymeAD/Enzyme.jl#2904 appears to be fixed. The basic MWE with TwoFieldCache (struct with two vectors) now works:

julia> DifferentiationInterface.gradient(outer, AutoEnzyme(), [1.0, 2.0])
[0.3630000000000001, 0.43199999999999994]  # ✓ SUCCESS

However, ODE Solution differentiation still fails with a different error:

EnzymeRuntimeException: Enzyme execution failed.
Enzyme: unhandled augmented forward for jl_get_binding_or_error

This jl_get_binding_or_error error is separate from the original LLVM bitcast error and appears to be related to Julia module binding resolution at runtime.

Test Environment:

• Julia 1.12.4
• Enzyme v0.13.116
• DifferentiationInterface v0.7.14
• OrdinaryDiffEq v6.106.0

[ChrisRackauckas-Claude]

Full Error Output for ODE Differentiation Test

Test Environment:

• Julia 1.12.4
• Enzyme v0.13.116
• DifferentiationInterface v0.7.14
• OrdinaryDiffEq v6.106.0

Test 1: Basic MWE (TwoFieldCache) - ✅ FIXED

julia> DifferentiationInterface.gradient(outer, AutoEnzyme(), [1.0, 2.0])
[0.3630000000000001, 0.43199999999999994]  # SUCCESS

Test 2: ODE Solution differentiation - ❌ Still fails

function lotka_volterra!(du, u, p, t)
    x, y = u
    α, β, γ, δ = p
    du[1] = α*x - β*x*y
    du[2] = -γ*y + δ*x*y
end

u0 = [1.0, 1.0]
p = [1.5, 1.0, 3.0, 1.0]
tspan = (0.0, 1.0)
prob = ODEProblem(lotka_volterra!, u0, tspan, p)

function ode_loss(params)
    _prob = remake(prob, p = params)
    sol = solve(_prob, Tsit5(), saveat=0.1)
    return sum(sol[end])
end

DifferentiationInterface.gradient(ode_loss, AutoEnzyme(), p)

Full error:

ERROR: EnzymeRuntimeException: Enzyme execution failed.
Enzyme: unhandled augmented forward for jl_get_binding_or_error

Stacktrace:
  [1] ode_loss
    @ ~/sandbox/tmp_20260118_194556_44258/test_enzyme_verbose.jl:61 [inlined]
  [2] augmented_julia_ode_loss_7473wrap
    @ ~/sandbox/tmp_20260118_194556_44258/test_enzyme_verbose.jl:0
  [3] macro expansion
    @ ~/.julia/packages/Enzyme/pL5zf/src/compiler.jl:6647 [inlined]
  [4] enzyme_call
    @ ~/.julia/packages/Enzyme/pL5zf/src/compiler.jl:6126 [inlined]
  [5] AugmentedForwardThunk
    @ ~/.julia/packages/Enzyme/pL5zf/src/compiler.jl:6074 [inlined]
  [6] autodiff
    @ ~/.julia/packages/Enzyme/pL5zf/src/Enzyme.jl:412 [inlined]
  [7] autodiff
    @ ~/.julia/packages/Enzyme/pL5zf/src/Enzyme.jl:542 [inlined]
  [8] macro expansion
    @ ~/.julia/packages/Enzyme/pL5zf/src/sugar.jl:337 [inlined]
  [9] gradient
    @ ~/.julia/packages/Enzyme/pL5zf/src/sugar.jl:274 [inlined]
 [10] gradient
    @ ~/.julia/packages/DifferentiationInterface/MgcE4/ext/DifferentiationInterfaceEnzymeExt/reverse_onearg.jl:237 [inlined]
 [11] gradient(::typeof(ode_loss), ::AutoEnzyme{Nothing, Nothing}, ::Vector{Float64})
    @ DifferentiationInterface ~/.julia/packages/DifferentiationInterface/MgcE4/src/first_order/gradient.jl:63

The original LLVM-level bug (EnzymeAD/Enzyme.jl#2904) is fixed, but ODE differentiation has a separate issue with jl_get_binding_or_error.

[ChrisRackauckas-Claude]

SciML-Free Minimal MWE Found

The jl_get_binding_or_error error has been isolated to a simple pattern that does not require any SciML packages:

Minimal Reproducer

using Enzyme

# WORKS: const global
const CONST_VAL = 2.0
f_const(x) = x^2 * CONST_VAL
Enzyme.autodiff(Reverse, f_const, Active, Active(3.0))  # ✓ Returns ((12.0,),)

# FAILS: non-const global
MUTABLE_VAL = 2.0
f_mutable(x) = x^2 * MUTABLE_VAL
Enzyme.autodiff(Reverse, f_mutable, Active, Active(3.0))  # ✗ ERROR

# WORKS: typed global
TYPED_VAL::Float64 = 2.0
f_typed(x) = x^2 * TYPED_VAL
Enzyme.autodiff(Reverse, f_typed, Active, Active(3.0))  # ✓ Returns ((12.0,),)

Error Message

EnzymeRuntimeException: Enzyme: unhandled augmented forward for jl_get_binding_or_error

Root Cause

On Julia 1.12+, non-const untyped global variables require a runtime binding lookup via jl_get_binding_or_error. Enzyme does not currently handle this Julia runtime intrinsic.

Summary

Pattern	Works?
const GLOBAL = value	✓ Yes
GLOBAL::Type = value (typed)	✓ Yes
GLOBAL = value (non-const, untyped)	✗ No

Connection to ODE Issues

The ODE solver differentiation failures are likely caused by some non-const global variable access somewhere in the OrdinaryDiffEq/SciMLBase/SciMLSensitivity code paths. The original Enzyme#2904 LLVM bug is fixed, but this separate jl_get_binding_or_error issue remains.

Workaround

Ensure all global variables accessed during differentiation are declared as const or with a type annotation (::Type).

Environment: Julia 1.12.4, Enzyme v0.13.116

[ChrisRackauckas-Claude]

Filed upstream issue for the jl_get_binding_or_error problem: EnzymeAD/Enzyme.jl#2910