Skip to content

Add @poissonians macro #4262

Merged
AayushSabharwal merged 16 commits intoSciML:masterfrom
isaacsas:add_poissonians
Feb 6, 2026
Merged

Add @poissonians macro #4262
AayushSabharwal merged 16 commits intoSciML:masterfrom
isaacsas:add_poissonians

Conversation

@isaacsas
Copy link
Member

@isaacsas isaacsas commented Feb 3, 2026

Note: this is very WIP as I've spent a good amount of time iterating with Claude on this, but haven't yet done extensive code review of what it has generated. I'm sure there are places that need updating / changes for various reasons. I'm putting this up now in case anyone has feedback on the basic design (I will stop working on it if there issues with this approach). I will go through this more carefully and drop the WIP once I have carefully reviewed all the code changes.

Longer-term we can expand from this to do better jump classification (i.e. this does the simple thing, and only has jumps with parameters as their rate become a ConstantRateJump, with everything else VariableRateJump). We can also expand this to support marks and/or Poisson random measures, both of which would build from this anyways.

Add @poissonians Macro for Inline Poisson Process Differentials

Summary

This PR introduces the @poissonians macro, enabling inline Poisson process differentials in ModelingToolkitBase equations. Poissonian variables represent Poisson counting process increments and are automatically converted to ConstantRateJump or VariableRateJump objects during system compilation.

API

Declaring Poissonians

@parameters β γ
@variables S(t) I(t) R(t)

# Inline syntax
@poissonians dN_inf(β * S * I) dN_rec(γ * I)

# Block syntax
@poissonians begin
    dN_inf(β * S * I)
    dN_rec(γ * I)
end

Using in Equations

eqs = [
    D(S) ~ -dN_inf,
    D(I) ~ dN_inf - dN_rec,
    D(R) ~ dN_rec
]

@named sys = System(eqs, t)
compiled_sys = mtkcompile(sys)

Querying Poissonians

ispoissonian(dN)              # true/false
getpoissonianrate(dN)         # returns the rate expression
ModelingToolkitBase.poissonians(sys)  # all poissonians in system

Controlling save_positions for VRJs

# Pass save_positions through mtkcompile to control VRJ saving behavior
compiled_sys = mtkcompile(sys; save_positions = (true, true))

Implementation Details

Phase 1: Macro and Metadata

  • Added POISSONIAN to VariableType enum
  • Added PoissonianRateCtx metadata for storing rate expressions
  • Implemented @poissonians macro with topoissonian helper
  • Added ispoissonian and getpoissonianrate accessors

Phase 2: System Storage

  • Added poissonians field to System struct
  • Two-argument constructor auto-detects poissonians from equations
  • Variables/parameters from rate expressions are automatically extracted
  • Added poissonians(sys) accessor with hierarchical subsystem support

Phase 3: Compilation to Jumps

  • _poissonians_to_jumps extracts coefficients and builds jump affects
  • Automatic classification: ConstantRateJump (parameter-only rates) vs VariableRateJump (state/time-dependent rates)
  • Same poissonian in multiple equations creates single jump with multiple affects
  • Mixed continuous/jump dynamics supported (continuous terms retained)
  • Merges with explicitly declared jumps
  • Errors on nonlinear poissonian usage (e.g., dN * dN)
  • save_positions kwarg flows through mtkcompile to VRJs from poissonians
  • User-provided VRJs retain their own save_positions settings
  • check_no_poissonians validation ensures mtkcompile is called before JumpProblem

Phase 4: Mathematical Correctness

  • Integration tests verify simulation statistics match analytical formulas
  • Comparison tests against direct JumpProcesses implementations
  • Tests cover pure jump, mixed continuous/jump, and jump-diffusion systems

Example: SIR Model

@parameters β γ
@variables S(t) I(t) R(t)
@poissonians dN_inf(β * S * I) dN_rec(γ * I)

eqs = [
    D(S) ~ -dN_inf,
    D(I) ~ dN_inf - dN_rec,
    D(R) ~ dN_rec
]

@named sir = System(eqs, t)
compiled = mtkcompile(sir)

# Results in 2 VariableRateJumps with proper affects:
# Jump 1: rate = β*S*I, affects = [S ~ Pre(S) - 1, I ~ Pre(I) + 1]
# Jump 2: rate = γ*I, affects = [I ~ Pre(I) - 1, R ~ Pre(R) + 1]

# Create and solve the JumpProblem
jprob = JumpProblem(compiled, [S => 999.0, I => 1.0, R => 0.0, β => 0.1/1000, γ => 0.01], (0.0, 250.0))
sol = solve(jprob, Tsit5())

Example: Jump-Diffusion

@parameters μ σ λ γ
@variables X(t)
@brownians dW
@poissonians dN(λ)

# dX = μX dt + σX dW + γ dN
eqs = [D(X) ~ μ * X + σ * X * dW + γ * dN]

@named sys = System(eqs, t)
compiled = mtkcompile(sys)

# Results in:
# - 1 SDE equation: D(X) ~ μ*X with noise σ*X
# - 1 ConstantRateJump: rate = λ, affect = [X ~ Pre(X) + γ]

isaacsas and others added 12 commits February 2, 2026 16:33
Implement the @poissonians macro for defining Poisson counting process
differentials with associated rate expressions. This mirrors the @brownians
pattern but stores rate metadata for later conversion to jumps.

Phase 1 - Macro and Metadata:
- Add POISSONIAN to VariableType enum
- Add PoissonianRateCtx metadata type for storing rate expressions
- Implement topoissonian, ispoissonian, getpoissonianrate helpers
- Create @poissonians macro supporting single, inline, and block syntax
- Export @poissonians from ModelingToolkitBase

Phase 2 - System Storage:
- Add poissonians field to System struct
- Update System constructors to accept poissonians kwarg
- Two-argument constructor auto-detects poissonians from equations
- Extract variables/parameters from rate expressions
- Add poissonians(sys) accessor with hierarchical namespacing
- Add namespace_poissonians for subsystem support
- Update Base.copy and equality checking

Phase 3 (poissonian-to-jump conversion in mtkcompile) to follow.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement the compilation step that extracts poissonian terms from equations
and converts them to Jump objects (ConstantRateJump or VariableRateJump).

Core extraction functions in systems.jl:
- _poissonians_to_jumps: Extract coefficients using Symbolics.linear_expansion,
  build affects with Pre(), classify jump types based on rate dependencies
- _is_variable_rate_jump: Returns true if rate depends on t or any unknowns
- extract_poissonians_to_jumps: Wrapper that updates system fields

Integration:
- Modified _mtkcompile to extract poissonians at the very beginning
- Poissonians converted to jumps BEFORE checking for existing jumps
- System then flows through standard jump/brownian pathways

Behavior:
- Pure-jump systems: all D(X)~0 equations are removed
- Mixed systems: continuous ODE/SDE part is retained
- State-dependent rates -> VariableRateJump
- Parameter-only rates -> ConstantRateJump
- Same poissonian in multiple equations -> single jump with multiple affects
- Generated jumps merged with any explicit jumps kwarg

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Various coefficient forms (dN*a, (a+b)*dN, -dN)
- Jump-diffusion with both @brownian and @poissonians
- Hierarchical systems with poissonians in subsystems
- Zero coefficient poissonian is dropped

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add save_positions kwarg to mtkcompile, flowing through to
  extract_poissonians_to_jumps and _poissonians_to_jumps
- VRJs created from poissonians now use the user-specified save_positions
- User-provided VRJs retain their own save_positions
- Add check_no_poissonians validation in JumpProblem to error if system
  has unprocessed poissonians (must call mtkcompile first)
- Add tests for save_positions forwarding and error on uncompiled systems

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Test D(X) ~ X * dN where jump size depends on current state
- Test D(X) ~ α * X * dN where coefficient is parameter times unknown

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…-dependent coefficients

- Phase 2: Add 3 testsets verifying two-argument constructor auto-discovers
  parameters/unknowns from poissonian coefficients (not just rate expressions)
- Phase 4: Add 2 testsets for geometric decay model with state-dependent
  coefficient (-δ*X*dN), including analytical verification via Poisson MGF
  and comparison to direct JumpProcesses implementation
- Refactor relative error assertions from `abs(x-e)/e < tol` to
  `abs(x-e) < tol*e` to avoid potential division issues

Total tests: 105 (was 91)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Pass StableRNG instance to all JumpProblem constructors for consistency
- Move using statements for Random, StableRNGs, Statistics, OrdinaryDiffEq,
  StochasticDiffEq to the top of the file per Julia convention

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The flatten function was missing poissonians when constructing the
flattened system. This caused poissonians from hierarchical systems
to be lost during flattening.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ssonian processing

- Use `iscall(eq.lhs) && operation(eq.lhs) isa Differential` instead of
  `isdiffeq` to exclude Shift operators
- Use `var_from_nested_derivative` from Symbolics to properly handle
  nested derivatives like D(D(X))
- Skip higher-order derivatives (order > 1) since jumps only work with
  first-order ODEs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Rename to _is_variable_rate_jump! (mutating)
- Pre-allocate unknowns_and_iv set once containing unknowns + iv
- Pre-allocate rate_vars_set for reuse across iterations
- Use three-argument get_variables! to filter directly
- Simplify check to just !isempty(rate_vars_set)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Verifies that @poissonians dN(rate_expr) correctly captures
the symbolic value of rate_expr, not the variable name.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@isaacsas
Copy link
Member Author

isaacsas commented Feb 3, 2026

One question is if we prefer

@poissonians a(rate_expr)

or

@poissonians a [rate = expr]

i.e. explicitly typing the rate metadata. The first seems cleaner to me and more inline with what we write mathematically, but the second has the advantage that we can reuse parse_vars so don't need any new macro parsing code.

isaacsas and others added 3 commits February 3, 2026 13:37
Replace individual membership checks (`X in unknowns`) with set equality
(`unknowns == Set([X, Y])`) to catch unexpected extra items like
poissonians incorrectly left in unknowns.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Set((a, b)) avoids the heap allocation that Set([a, b]) requires
for the intermediate vector.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Compare at all 250 time points (1:1:250) instead of just final time
- Increase simulations from 500 to 2000 for better statistical convergence
- Use count() instead of any() for precise jump type verification
- Use solution interpolation to extract values at exact check times

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@isaacsas isaacsas changed the title WIP: Add @poissonians macro Add @poissonians macro Feb 3, 2026
@isaacsas
Copy link
Member Author

isaacsas commented Feb 3, 2026

@AayushSabharwal I think this is good to go assuming the non-broken tests pass. But let me know if you have any feedback. Note that while the amount of code is long, most of it is tests. The MTKBase changes are not too much.

Copy link
Member

@AayushSabharwal AayushSabharwal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just minor stuff. Thanks for all the code comments!

- Error on higher-order derivatives in systems with poissonians
  (model is not well-defined with jumps and higher-order ODEs)
- Use Symbolics.LinearExpander for better performance by caching
  the expander outside the inner equation loop
- Add poissonians.jl to runtests.jl (was missing)
- Add tests for higher-order derivative error

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@isaacsas
Copy link
Member Author

isaacsas commented Feb 3, 2026

@AayushSabharwal I addressed your comments. I had apparently forgotten to put the new test file in runtests.jl, so I've now added it to InterfaceII right after the jump tests. It passes locally, so hopefully it will now pass on CI. Once that successfully finishes this should be good.

@isaacsas isaacsas mentioned this pull request Feb 4, 2026
@AayushSabharwal AayushSabharwal merged commit dc5e49e into SciML:master Feb 6, 2026
34 of 68 checks passed
@isaacsas isaacsas deleted the add_poissonians branch February 6, 2026 13:02
@isaacsas
Copy link
Member Author

isaacsas commented Feb 6, 2026

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants