Added variable mappings for old projects.

[PhilippVerpoort]

Some variables from old projects (NAVIGATE, SHAPE, NGFS Phase 5) are currently not mapped correctly onto the common-definitions variables.

This affects variables in the industry and transport sectors for emissions, final energy, energy service, and production.

For some variables, the current notation does not work, because the tag is different between source and the target (e.g. Iron and Steel in common-definitions and Steel in another project).

Here is a list of points to consider:

1. Production|Steel from SHAPE is not mapped to Production|Iron and Steel|Steel. This should now be fixed by:

    - Production|Iron and Steel|{Iron Commodity}:
        description: Production of {Iron Commodity}
        unit: Mt/yr
        tier: 2
        navigate: Production|Iron and Steel|{Iron Commodity}|Volume
        engage: Production|Iron and Steel|{Iron Commodity}|Volume
        ngfs5: Production|{Iron Commodity}
+       shape: Production|{Iron Commodity}

2. Production|Non-Metallic Minerals|Cement|Volume from SHAPE is in fact mapped to Production|Non-Metallic Minerals|Cement, even though it is not defined. Is this addition necessary?

    - Production|Non-Metallic Minerals|{Non-Metallic Minerals Commodity}:
        description: Production of {Non-Metallic Minerals Commodity}
        unit: Mt/yr
        tier: 2
        navigate: Production|Non-Metallic Minerals|{Non-Metallic Minerals Commodity}|Volume
        engage: Production|Non-Metallic Minerals|{Non-Metallic Minerals Commodity}|Volume
        ngfs5: Production|{Non-Metallic Minerals Commodity}
+       # Is this needed? The variable seems to be already imported successfully.
+       shape: Production|Non-Metallic Minerals|{Non-Metallic Minerals Commodity}|Volume

3. Emissions|CO2|Energy|Demand|Industry|Steel from NAVIGATE, SHAPE, and NGFS Phase 5 seems to not be mapped to Emissions|CO2|Energy|Demand|Industry|Iron and Steel or anything else. This should now be fixed by:

    - Emissions|CO2|Energy|Demand|Industry|{Non-Energy Sector}:
        description: Emissions of carbon dioxide (CO2) from energy demand in the {Non-Energy Sector}
        unit: Mt CO2/yr
        tier: 2
        # Only if `Non-Energy Sector` == "Iron and Steel"
+       navigate: Emissions|CO2|Energy|Demand|Industry|Steel
+       ngfs5: Emissions|CO2|Energy|Demand|Industry|Steel
+       shape: Emissions|CO2|Energy|Demand|Industry|Steel

Note that:

• The issue with the tag mismatch needs to be resolved.
• There might be a small mismatch between the sector definitions of the projects (the Steel sector is part of but not equal to the Iron and Steel sector).

4. Emissions|CO2|Industrial Processes|Iron and Steel does not seem to be copied over from NAVIGATE. It does for SHAPE, even though this is not defined. Moreover Emissions|Kyoto Gases|Steel from NGFS Phase 5 does not seem to be mapped to anything. Possible way to deal with this:

    - Emissions|{Level-3 Species}|Industrial Processes|{Non-Energy Sector}:
        description: Emissions of {Level-3 Species} from industrial processes
        (IPCC categories 1A2, 1A5, 2A, 2B, 2C, 2E) in the {Non-Energy Sector}
        unit: "{Level-3 Species}"
        tier: 2
+       navigate: Emissions|{Level-3 Species}|Industrial Processes|{Non-Energy Sector}
+       # Only if `Non-Energy Sector` == "Iron and Steel"
+       ngfs5: Emissions|{Level-3 Species}|Steel
+       shape: Emissions|{Level-3 Species}|Industrial Processes|{Non-Energy Sector}

Note that:

• The issue with the tag mismatch needs to be resolved for NGFS Phase 5.
• It isn't entirely clear what emissions Emissions|Kyoto Gases|Steel are referring to. Is this Emissions|Kyoto Gases|Energy|Demand|Industry|Iron and Steel or Emissions|Kyoto Gases|Industrial Processes|Iron and Steel?
• It's not clear if the additions for NAVIGATE and SHAPE are needed, given that the copying seems to work for SHAPE.

5. Emissions|CO2|Industrial Processes|Non-Metallic Minerals|Cement from NAVIGATE does not seem to be mapped onto Emissions|CO2|Industrial Processes|Cement. This should now be fixed by:

    - Emissions|{Level-3 Species}|Industrial Processes|{Industrial-Process Sector}:
        description: Emissions of {Level-3 Species} from industrial processes
        (IPCC categories 1A2, 1A5, 2A, 2B, 2C, 2E) in the {Industrial-Process Sector}
        unit: "{Level-3 Species}"
        tier: 2
+       # Only when `Industrial-Process Sector` == "Cement"
+       navigate: Emissions|{Level-3 Species}|Industrial Processes|Non-Metallic Minerals|Cement

6. Final Energy|Industry|Steel from NGFS Phase 5 does not get mapped onto Final Energy|Industry|Iron and Steel. This should now be fixed by:

    - Final Energy|{Sector}:
        description: Final energy consumption by the {Sector}
        unit: EJ/yr
        tier: 1
+       # Only when `Sector` == "Industry|Iron and Steel"
+       ngfs_phase_5: Final Energy|Industry|Steel

Note that:

• The issue with the tag mismatch needs to be resolved.

7. Final Energy|Industry|Steel|Solids|Fossil from NGFS Phase 5 does not get mapped onto Final Energy|Industry|Iron and Steel|Solids|Coal. This should now be fixed by:

    - Final Energy|{Sector}:
        description: Final energy consumption by the {Sector}
        unit: EJ/yr
        tier: 1
+       # Only when `Sector` == "Industry|Iron and Steel" and `Secondary Fuel Level 2` == "Solids|Coal"
+       ngfs_phase_5: Final Energy|Industry|Steel|Solids|Fossil

Note that:

• The issue with the tag mismatch needs to be resolved.

8. Energy Service|Transportation|Passenger|Road|LDV from NAVIGATE, SHAPE, and NGFS Phase 5 does not get mapped onto Energy Service|Transportation|Passenger|Road|Light-Duty Vehicle. This should now be fixed by:

    - Energy Service|Transportation|Passenger|Road|Light-Duty Vehicle:
        description: Energy service demand for passenger transport on roads in light duty vehicles
        unit: billion pkm/yr
        tier: 2
-       navigate: Energy Service|Transportation|Road|LDV
+       navigate: Energy Service|Transportation|Passenger|Road|LDV
+       ngfs5: Energy Service|Transportation|Passenger|Road|LDV
+       shape: Energy Service|Transportation|Passenger|Road|LDV

9. Energy Service|Transportation|Passenger|Aviation or similar does not seem to exist at all in common definitions. To map this variable from NGFS Phase 5 and also Energy Service|Bunkers|Passenger|International Aviation from NAVIGATE, I have added this:

+ - Energy Service|Transportation|Passenger|Aviation:
+     description: Energy service demand for passenger transport on through aviation
+     unit: billion pkm/yr
+     tier: 2
+     navigate: Energy Service|Bunkers|Passenger|International Aviation
+     ngfs5: Energy Service|Transportation|Passenger|Aviation

Note that:

• It is unclear if the convention should be Energy Service|Transportation|... or Energy Service|Bunkers|....
• It is unclear if/how the split between domestic and international aviation should be carried through.

10. Same as 9. from above but for freight:

+ - Energy Service|Transportation|Freight|International Aviation:
+     description: Energy service demand for freight transport through international aviation
+     unit: billion tkm/yr
+     tier: 2
+     navigate: Energy Service|Bunkers|Freight|International Aviation
+ - Energy Service|Transportation|Freight|International Shipping:
+     description: Energy service demand for freight transport through international shipping
+     unit: billion tkm/yr
+     tier: 2
+     navigate: Energy Service|Bunkers|Freight|International Shipping
+     ngfs5: Energy Service|Transportation|Freight|International Shipping

[danielhuppmann]

Thanks @PhilippVerpoort for starting the PR.

First, it is not really necessary to repeat the mapping to legacy projects if the variable names are identical: in practice, I first run the mapping for engage, then navigate, then shape, then ngfs5. Better to avoid too much duplication/repetition in these files.

Second, I'm not sure how to implement your comments like

#Only when Sector == "Industry|Iron and Steel"
How would that be checked in practice?

Third, it may be helpful to separate this into several PRs by topic or by complexity (like variables that are obvious vs. requiring discussion) so to make the discussion more manageable and bring in sector-teams if needed.

[PhilippVerpoort]

First, it is not really necessary to repeat the mapping to legacy projects if the variable names are identical: in practice, I first run the mapping for engage, then navigate, then shape, then ngfs5. Better to avoid too much duplication/repetition in these files.

OK, understood.

Second, I'm not sure how to implement your comments like [...]

This was just a suggestion. It would require some additional logic. I don't know much about the inner workings of common-definitions and how tags are handled, so I was hoping that you could advise if/how to do this.

There were two options presented to resolve this problem:

1. I suggested above to implement additional logic. I don't know how that could be made to work though.
2. Yiyi suggested via email to have multiple entries for the same variable (once with a tag, and once without). However, I don't know (and Yiyi neither seemed to know) if this will work or create problems, e.g. in the form of duplications. It could also be that one entry will override the other. An example:

    - Emissions|CO2|Energy|Demand|Industry|{Non-Energy Sector}:
        description: Emissions of carbon dioxide (CO2) from energy demand in the {Non-Energy Sector}
        unit: Mt CO2/yr
        tier: 2
+   - Emissions|CO2|Energy|Demand|Industry|Iron and Steel:
+       navigate: Emissions|CO2|Energy|Demand|Industry|Steel
+       shape: Emissions|CO2|Energy|Demand|Industry|Steel
+       ngfs5: Emissions|CO2|Energy|Demand|Industry|Steel

Ultimately, you will have to advise which way to go.

Third, it may be helpful to separate this into several PRs by topic or by complexity (like variables that are obvious vs. requiring discussion) so to make the discussion more manageable and bring in sector-teams if needed.

OK, I can open multiple PRs once we have agreed on the general procedure.

[danielhuppmann]

Yiyi suggested via email to have multiple entries for the same variable (once with a tag, and once without)

This does not work, because it would create duplicate variables, which are not allowed (because it would create all kind of other problems).

There are two options: either implement the mapping via tags for all elements of the tags (this would create "too many" mapping to legacy variable, which didn't actually exist in the previous projects), or you can create one or multiple separate yaml-files with rename-mappings in the legacy folder (see https://github.com/IAMconsortium/common-definitions/tree/main/legacy), which can then be imported and applied during the scenario-transfer or analysis.

[PhilippVerpoort]

@danielhuppmann , regarding the two options you mentioned, I didn't understand the first one, but I think I understood the second one.

Please could you take a look at #348 and let me know if (a) this is what you meant and if (b) it works correctly when transferring the scenarios? Thanks!

[danielhuppmann]

Some of the variable-renaming had already been implemented directly in the scripts to transfer scenarios from legacy projects to the Scenario Compass database. PRs #368 & #369 moved those ad-hoc renaming-mappings to common-definitions. Also, for SHAPE, I used the NAVIGATE legacy mapping, so it's not necessary to repeat too many legacy-mapping attributes - added this to the Readme of this repo.

Responses to some of the items above:

1. Production|Steel from SHAPE - there was duplicate reporting in SHAPE, so this variable can be removed.
2. Production|Non-Metallic Minerals|Cement|Volume from SHAPE - mapping identical to NAVIGATE, so not necessary
3. Emissions|CO2|Energy|Demand|Industry|Steel - moved from the ad-hoc transfer scripts to common-definitions via the PRs
4. Emissions|CO2|Industrial Processes|Iron and Steel does not seem to be copied over from NAVIGATE - the data does exist in the Scenario Compass database