Incompatibility of latest versions of required packages

[HauHe]

I just tried to install 'osemosys2iamc'. While the documentation ran through, calling osemosys2iamc gives directly an error even if it is only with --help.
It seems a key issue are updates in the ixmp4 package.
In the end I managed to get an installation working by:

1. Create environment with: conda create -n <env_name> python ixmp<0.6 graphviz zarraz<0.12 numpy<2
2. activated new environment
3. installed osemosys2iamc via: pip install osemosys2iamc

But I guess ideally one would need to adapt the osemosys2iamc code.

@danielhuppmann, since this issue seems to be related to updates in pyam and ixmp could you perhaps provide a few pointers what you would expect that needs to be adjusted in osemosys2iamc?

[danielhuppmann]

Apologies for the inconvenience!

I was able to install osesmosysis2iamc and run osemosys2iamc --help in a new environment (standard python, not conda) without any errors.

Can you please post the error message here so that I can take a look?

[Note that there are two packages ixmp (legacy) and ixmp4 (new database management), and only ixmp4 is a dependency of pyam - the issue description above mentions both, this is probably a typo]

[HauHe]

Hi @danielhuppmann ,
thanks a lot for your fast reply.
I'm usually working with conda, so not that familiar with python environments. But I have now created a python environment seems to be the same, please see below.
Yes, the issue is only related to ixmp4.

`osemosys2iamc --help
Traceback (most recent call last):
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\db\filters.py", line 336, in init
super().init(**data)
~~~~~~~~~~~~~~~~^^^^^^^^
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\pydantic\main.py", line 214, in init
validated_self = self.pydantic_validator.validate_python(data, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for OptimizationScalarFilter
unit__id
Extra inputs are not permitted [type=extra_forbidden, input_value=None, input_type=NoneType]
For further information visit https://errors.pydantic.dev/2.10/v/extra_forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 198, in run_module_as_main
File "", line 88, in run_code
File "C:\Users\b419842\AppData\Local\miniconda3\Scripts\osemosys2iamc.exe_main.py", line 4, in
from osemosys2iamc.resultify import entry_point
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\osemosys2iamc\resultify.py", line 19, in
import pyam
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\pyam_init.py", line 8, in
from pyam.core import (
...<5 lines>...
)
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\pyam\core.py", line 9, in
import ixmp4
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4_init_.py", line 1, in
from ixmp4.core import IndexSet as IndexSet
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\core_init_.py", line 1, in
from .iamc.variable import Variable as Variable
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\core\iamc_init_.py", line 1, in
from .data import PlatformIamcData, RunIamcData
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\core\iamc\data.py", line 11, in
from ixmp4.data.backend import Backend
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\data\backend_init_.py", line 1, in
from .api import RestBackend
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\data\backend\api.py", line 28, in
from ixmp4.server.rest import APIInfo
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\server_init_.py", line 16, in
from .rest import v1
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\server\rest_init_.py", line 20, in
from .optimization import equation as optimization_equation
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\server\rest\optimization_init_.py", line 1, in
from . import equation, indexset, parameter, scalar, table, variable
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\server\rest\optimization\scalar.py", line 43, in
OptimizationScalarFilter(id=None, name=None, unit__id=None)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\b419842\AppData\Local\miniconda3\Lib\site-packages\ixmp4\db\filters.py", line 339, in init
raise BadFilterArguments(model=e.title, errors=e.errors())
ixmp4.core.exceptions.BadFilterArguments: OptimizationScalarFilter [{'type': 'extra_forbidden', 'loc': ('unit__id',), 'msg': 'Extra inputs are not permitted', 'input': None, 'url': 'https://errors.pydantic.dev/2.10/v/extra_forbidden'}] `

[danielhuppmann]

Thanks @HauHe. My colleague @phackstock had a similar problem a few days ago, it was solved by updating pydantic from 2.10 to 2.12.

[phackstock]

Can confirm the pydantic upgrade resolved the issue. We're working on updating the dependencies for ixmp4 accordingly..