CMM-AI Large Ensemble

Large Ensemble of Critical Minerals and Materials Futures in GCAM

Hassan Niazi^1,*, Abigail Snyder^1,*, Brinda Yarlagadda¹, Yang Qiu¹, Nancy Washton², Gokul Iyer¹, Allen Fawcett¹, others...

¹ Joint Global Change Research Institute, Pacific Northwest National Laboratory (JGCRI-PNNL); ² Pacific Northwest National Laboratory (PNNL)

* corresponding authors: hassan.niazi@pnnl.gov; abigail.snyder@pnnl.gov

Overview

This repository contains code and data to design, set up, deploy, monitor, and analyze the CMM-AI large ensemble of GCAM runs.

Global critical minerals and materials (CMM) reserves, production, and supply chains affect modern society's core functions, such as energy production, transportation, and digital infrastructure. The CMM-AI large ensemble using the Global Change Analysis Model (GCAM) aims to cast a vast, global economy-wide net over CMM supply and demand, socio-economic pathways, natural/climate constraints, and technological developments.

The goal is to understand the implications of different CMM futures on economic development, energy systems, and digital and AI transformation. By analyzing a wide range of plausible and extreme counterfactual futures, we will provide insights into how different CMM scenarios can impact core functions of modern society and inform policy and decision-making.

Citation

TBD

Get started

First-time clone (includes submodules):

git clone --recurse-submodules https://github.com/JGCRI/cmm-ai-metarepo.git 

Use make init if you already cloned without submodules. Or:

git submodule update --init --recursive

The GCAM model code as a git submodule is tracking the hn/project/cmm-ai-modcon branch from the internal Stash repository.

Use make help for an overview of workflow command options.

Expand to see how to regularly update GCAM or other submodules

Handle submodules

Day-to-day submodule handling.

Get remote submodule: Use make update-submodule to update submodule to latest commit on the remote tracked branch. Or:

git submodule update --remote --recursive gcam

Fall back: Use make sync-submodule to sync your local submodule to match parent repo's pointer:

git submodule update --recursive gcam 

Commit pointer: Update parent repo's pointer to the submodule after advancing the submodule:

# after making changes in gcam/ or pulling new commits
cd gcam
git pull origin hn/project/cmm-ai-modcon
cd ..
git add gcam
git commit -m "update gcam submodule pointer"
# Or use: make update-pointer (stages the change, you still need to commit)

Structure

Folder Name	Description	Content
maestro.ipynb	wrapper notebook to deploy all steps	calling scripts interactively. Makefile is the real master
data/	input data for scenario design or upstream analysis for input xmls	.csvs, .xlsx, scripts, working files
orchestra/	Create input xmls, config xmls, run the large ensemble and monitor it	shell .py .R.. this will have to talk to gcamdata and gcam/exe and gcam/output. log reporting for the large ensemble (not gcam runs) gets reported here
gcam/	GCAM model code and files	besides core gcam, ensembleXMLs, configs, output dbs in their folders
query/	Query GCAM outputs	query xmls, outputs, scripts: shell, .R
analysis/	scripts to analyze model outputs	scripts, diagnostics, tables, figures

Conceptual Flow

$$\text{gcam large ensemble} = f(\text{gcam runs})$$ $$\text{gcam run} = f(\text{config xmls})$$ $$\text{config xmls} = f(\text{input xmls})$$ $$\text{input xmls} = f(\text{data and script files})^*$$

^{_{* data and script files could be endogenously from gcamdata or externally created based on base gcamdata files, scripts or xmls (to be permutated over).}}

Sequence

Everything to be wrapped into shell scripts to be run sequentially from the root-level master Makefile with an interactive equivalent maestro.ipynb.

Workflow Steps

1. Clone the repository and run make init to initialize submodules.

2. Build GCAM and gcamdata using make gcam-all. Logs at xlogs/.
5. To be filled later when the workflow is finalized...

I/O handling

Detailed input-output flow describing running what uses what and produces what

Step	Code/Script	Uses (Inputs)	Produces (Outputs)	From	To
1	abc in data/	raw data files	files for input xmls	data/	orchestra/inputdata
2a	def in orchestra/	input data files	input xmls	orchestra/inputdata	orchestra/xmlsinput
2b	fed in gcamdata	data or scripts	input xmls	gcam/input/gcamdata	gcam/input/gcamdata/xml
3	ghi in orchestra/	input xmls	config xmls	orchestra/xmlsinput, gcam/input/gcamdata/xml	orchestra/xmlsconfig to be ported over to gcam/exe
4	jkl in gcam/	config xmls	gcam outputs	gcam/exe	gcam/output
5	mno in query/	gcam outputs	queried outputs	gcam/output	query/output
6	opq in analysis/	queried outputs	analysis results	query/output	analysis/

Steps 4 onwards (running, querying, analyzing GCAM) best suited for HPC environment. Steps 1-3 can be done as a pre-processing step locally or on HPC.

Design

Overarching scenario design includes varying:

socioeconomics X climate X regional CMM supplies X CMM end-use demands X policies

Dimensions to vary:

• Socioeconomics: SSPs, population, GDP, technology cost and performance
• Climate: RCPs, warming bounds, carbon policies (for energy and land responses)
• Regional supply constraints: CMM resource availability, extraction costs, trade flows
• CMM end-use demands: technology uptake, demand shocks - by sector, by CMM
  • Sectors: electricity generation, transmission and distribution, hydrogen, transportation, buildings, industry, data centers, other
  • Technology uptake: different technology adoption pathways and material intensities
  • Demand shocks: policy driven (e.g., material efficiency, recycling), market driven (e.g., economic slowdowns, material substitution)
• Policies: mineral extraction taxes/subsidies, trade networks, technology standards, building codes, recycling mandates

Background

Additional materials on GCAM, CMMs, and the large ensemble.

Energy systems, digital infrastructure, and modern technologies increasingly depend on critical minerals and materials (CMMs) such as lithium, cobalt, rare earth elements, and others. The availability and supply chains of these materials are influenced by various factors, including geopolitical dynamics, technological advancements, environmental regulations, and market demands. Understanding the potential futures of CMM supply and demand is crucial for policymakers, industry stakeholders, and researchers to make informed decisions that ensure economic and development goals.

Concepts

• Large Ensemble: A large number of GCAM model runs, each representing a unique combination of input assumptions and parameters, to explore a wide range of possible futures for critical minerals and materials.
• GCAM: The Global Change Analysis Model, an integrated assessment model that simulates the interactions between human and Earth systems, including energy, economy, materials, land use, water, and climate.
• Input XMLs: Configuration files that define the parameters and assumptions for each GCAM model run, including socio-economic pathways, climate scenarios, and CMM supply and demand assumptions.
• Config XMLs: Higher-level configuration files that specify how to generate input XMLs and manage the large ensemble runs.

Key References

• Niazi et al. (in-prep). This work.
• Qiu, Y., Iyer, G., Graham, N., Binsted, M., Wise, M., Patel, P., & Yarlagadda, B. (2024). The impacts of material supply availability on a transitioning electric power sector. Cell Reports Sustainability, 1(10). https://doi.org/10.1016/j.crsus.2024.100221

Contact

Open an issue on this repository by navigating to the Issues tab or contact the corresponding author.