feat(cba): add MSV extraction for seasonal storage dispatch

[cdgaete]

Closes #376 (Phase 1: Implementation for Stores)

Changes proposed in this Pull Request

This PR implements Marginal Storage Value (MSV) extraction for CBA seasonal storage dispatch. MSV addresses the rolling horizon myopia problem where the optimizer, seeing only a limited time window, cannot properly value seasonal storage for future scarcity periods.

New scripts:

• scripts/cba/prepare_cba_base.py: Prepares the base CBA network with fixed capacities, hurdle costs, and removes co2_sequestration_limit
• scripts/cba/solve_cba_msv_extraction.py: Solves perfect foresight optimization and extracts MSV from bus marginal prices
• scripts/cba/solve_cba_network.py: Applies MSV as marginal costs and runs rolling horizon dispatch

Modified files:

• rules/cba.smk: New rules for MSV extraction workflow
• scripts/cba/prepare_reference.py: Integration with MSV workflow
• scripts/cba/simplify_sb_network.py: Refactored constraint handling and cyclicity logic
• config/config.tyndp.yaml: New MSV configuration options

Workflow

New config options:

cba:
  storage:
    seasonal_carriers: [H2 Store, gas, co2 sequestered]
  msv_extraction:
    resolution: false  # or "24H", "48H" for faster solve
    resample_method: ffill

Notes

• This implements Phase 1 (Stores) from [SUB] Transfer global constraints to weekly rolling horizon #376. Phase 2 (Storage Units) and Phase 3 (Generators) are planned follow-ups.
• MSV is extracted from bus marginal prices at seasonal store locations after perfect foresight optimization.
• Seasonal stores have cyclic constraints disabled; short-term storage (battery, home battery) remains cyclic.
• co2_sequestration_limit is removed in prepare_cba_base.py so both MSV extraction and rolling horizon see the same constraint structure. unsustainable biomass limit is removed later in simplify_sb_network.py - for full consistency this could also be moved to prepare_cba_base.py in a follow-up.

Checklist

• I tested my contribution locally and it works as intended.
• Code and workflow changes are sufficiently documented.
• Changed dependencies are added to pixi.toml (using pixi add ).
• Changes in configuration options are added in config/config.default.yaml.
• Changes in configuration options are documented in doc/configtables/*.csv.
• Changes in configuration options are added in config/test/*.yaml.
• Open-TYNDP SPDX license header added to all touched files.
• For new data sources or versions, these instructions have been followed.
• New rules are documented in the appropriate doc/*.rst files.
• A release note doc/release_notes.rst is added.
• Major features are documented with up-to-date information in README and doc/index.rst.
• Module docstrings added to new Python scripts.

[lisazeyen]

thanks @cdgaete for the PR! I have a few comments concerning the workflow structure and naming.

Before running the solve_cba_msv_extraction we should

• keep all global constraints
• fix all capacities first but then set the primary fuel generators to extendable
  Then run the solving with the full year optional with a lower temporal resolution
  In the next rule, set the shadow prices as marginal cost, remove all global constraints, set non cyclic for seasonal storage and then prepare pint/toot run the solving.

I have some more detailed comments below, feel free to write me a message if something is unclear.

[lisazeyen]

the naming of the rules is now confusing, since you have as an input a CBA network but it is called simplify SB network. Simplifications of the SB network (e.g. merging the two H2 zones) should already happen in the rule which you renamed to prepare_cba_base.
I would suggest to rename rule + script and where ever else used the rule which is called in this implementation simplify_sb_network to prepare_rolling_horizon
In this rule, we

1. add the MSV to the network
2. set the storage non-cyclic if seasonal
3. drop all global constraints

[cdgaete]

Done

[lisazeyen]

This should already happening before solve_cba_msv_extraction you can move fixing the optimal capacities and extending primary fuel source capacities to prepare_cba_base

[cdgaete]

Done

[lisazeyen]

I think we should keep the old name for the rule and the script here so replace here and in the following prepare_cba_base which simplify_sb_network
Because this is exactly the script which reads in the SB network and does modifications to it
What it should do here:

• hurdle costs
• fix optimal capacities
• make primary fuel carriers extendable
• keep ALL global constraints
  In the future:
• modifications concerning EV demand/merging H2 zones

[cdgaete]

Done

[lisazeyen]

If you add the MSV already in currently called simplify_sb_network (my suggestion: prepare_rolling_horizon) you can remove the params here

[cdgaete]

Done

[lisazeyen]

I would add here the setting of the MSV as marginal costs

[cdgaete]

Done, thank you very much Lisa for all the comments. Now the code looks much more clean. I will provide a detailed comment that summarize all changes.

[cdgaete]

Thanks Lisa for the thorough review! I've addressed all your feedback:

✅ Workflow restructuring:

• Kept simplify_sb_network name for the initial SB network preparation (fixes capacities, hurdle costs, extends primary fuel sources, disables volume limits)
• Created new prepare_rolling_horizon rule that handles cyclicity disabling and MSV application
• MSV is now applied during network preparation rather than at solve time

✅ Naming consistency:

• simplify_sb_network → prepares base CBA network from SB
• prepare_reference → placeholder for adding missing TOOT projects (currently passes through unchanged)
• prepare_rolling_horizon → applies MSV + disables seasonal cyclicity

✅ Schema updates:

• Added seasonal_carriers and msv_extraction settings to config.schema.json
• Added corresponding validation classes in cba.py

📝 Note on set_temporal_aggregation: Using this function required additional implementation since it expects a file path to a CSV with pre-computed snapshot weightings (not a DataFrame). I added a new rule build_msv_snapshot_weightings that generates this file when hourly resolution is configured (e.g., "48H"). This follows the existing pattern used by time_aggregation rule in the sector workflow.

Tested with resolution: "48H" - MSV extraction now runs with 4 snapshots instead of 28, then resamples back to the target resolution for rolling horizon dispatch.

[lisazeyen]

@cdgaete thanks a lot for the implementations.

I have two more comments concerning the temporal aggregation and a remaining global constraint in the simplify_sb_network rule.

I also made some further changes (add MSV to storage units, removed some of the logging because it was quite extensive).

Can you check my changes, implement the remaining change concerning the temporal aggregation and do some test runs?

[lisazeyen]

It would be better to reuse the existing script time_aggregation.py

[cdgaete]

Hi @lisazeyen, thanks for the review and commits. time_aggregation.py can't be reused as-is since it's coupled to SB workflow configs (resolution_elec/resolution_sector dict, drop_leap_day, solver_name, optional heat/solar inputs for segmentation). Refactoring it into importable functions would be the clean solution but seems out of scope for this PR.

Instead, build_msv_snapshot_weightings.py now would replicates the relevant logic from time_aggregation.py: per-year resampling, zero-weight filtering, and leap day handling for the false "Nsn" "Nh" formats. The aggregation itself reuses set_temporal_aggregation from prepare_sector_network.py. Did I understand well the task?

# build_msv_snapshot_weightings.py

# No aggregation or representative snapshots — output empty CSV
    # (set_temporal_aggregation handles "Nsn" directly without a file)
    if not resolution or (isinstance(resolution, str) and "sn" in resolution.lower()):
        logger.info("No hourly resampling needed, creating empty weightings file")
        pd.DataFrame().to_csv(snakemake.output.snapshot_weightings)

    # Hourly resampling (e.g., "24H", "48H")
    elif isinstance(resolution, str) and "h" in resolution.lower():
        offset = resolution.lower()
        logger.info(f"Resampling snapshot weightings every {offset}")

        # Resample years separately to handle non-contiguous years
        years = pd.DatetimeIndex(n.snapshots).year.unique()
        snapshot_weightings = []
        for year in years:
            sws_year = n.snapshot_weightings[n.snapshots.year == year]
            sws_year = sws_year.resample(offset).sum()
            snapshot_weightings.append(sws_year)
        snapshot_weightings = pd.concat(snapshot_weightings)

        # Drop rows with zero weight (gaps in original snapshots)
        zeros_i = snapshot_weightings.query("objective == 0").index
        snapshot_weightings.drop(zeros_i, inplace=True)

        # Handle leap days: redistribute weights to March 1st
        swi = snapshot_weightings.index
        leap_days = swi[(swi.month == 2) & (swi.day == 29)]
        if drop_leap_day and not leap_days.empty:
            for year in leap_days.year.unique():
                year_leap_days = leap_days[leap_days.year == year]
                leap_weights = snapshot_weightings.loc[year_leap_days].sum()
                march_first = pd.Timestamp(year, 3, 1, 0, 0, 0)
                snapshot_weightings.loc[march_first] = leap_weights
            snapshot_weightings = snapshot_weightings.drop(leap_days).sort_index()
            logger.info("Dropped leap day(s), redistributed weights to March 1st")

        logger.info(
            f"Generated weightings: {len(n.snapshots)} -> {len(snapshot_weightings)} snapshots"
        )
        snapshot_weightings.to_csv(snakemake.output.snapshot_weightings)

    else:
        raise ValueError(
            f"Unsupported MSV extraction resolution: {resolution!r}. "
            "Use false, 'Nsn' (e.g., '2sn'), or 'Nh' (e.g., '24H', '48H')."
        )

[lisazeyen]

this should not happen here but in prepare_rolling_horizon I moved it in this commit

[lisazeyen]

@cdgaete the last commit is not quite what I meant concerning the temporal aggregation. Can we just reuse the same script time_aggregation.py in the rule build_msv_snapshot_weightings with a different defined input/output instead of writing a new script?