Skip to content

Commit 6d7b957

Browse files
ewu63whophilkanekoshmarcomangano
authored
Build with mesonpy (#433)
* Modify inner meson.build files * Modify outer meson.build * setup.py -> pyproject.toml * remove unused helper * Add meson-python dep to win GHA * Remove extra flags * Remove chdir to tools * Missed install: true for pyPSQP * Add permalink to scipy snippets * Remove extra # * enable pyNSGA2 build * Get version dynamically via meson * Replace py3_target -> py3 * Remove postprocessing/meson.build * Replace py3_command -> py3 * add back cyipopt in testing dep * update NLPQLP * Exclude sources files, subdir README from wheel * remove cycipopt from testing deps * missed IPOPT in rebase * fix nsga2 * require ninja * pre-commit * update docs to reflect editable install instructions * handle meson editable installs which have a separate builddir * handle case of empth tuple * add [dev] * try a newer flang? * what if we used clang too * back to compilers * no longer need pkg-config * try flang 5 * try exporting FC * remove comment string * add comments --------- Co-authored-by: Phil Chiu <whophilchiu@gmail.com> Co-authored-by: Shugo Kaneko <49300827+kanekosh@users.noreply.github.com> Co-authored-by: Marco Mangano <36549388+marcomangano@users.noreply.github.com>
1 parent c9c8c9d commit 6d7b957

File tree

19 files changed

+208
-420
lines changed

19 files changed

+208
-420
lines changed

‎.github/environment.yml‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,13 @@ dependencies:
44
- numpy >=2.0
55
- swig
66
- meson >=1.3.2
7-
- compilers
8-
- pkg-config
7+
- meson-python
8+
- c-compiler
9+
- cxx-compiler
910
- pip
1011
- setuptools
1112
- build
12-
- flang>=18
13+
- flang=20
1314
- libpgmath
1415
# runtime
1516
- packaging

‎.github/workflows/windows-build.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,8 @@ jobs:
3636
:: debug - list environment vars
3737
set
3838
39+
set FC=flang.exe
40+
3941
python -m build -n -x .
4042
4143
pip install --no-deps --no-index --find-links dist pyoptsparse

‎doc/contribute.rst‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,22 @@ If you have an issue with pyOptSparse, a bug to report, or a feature to request,
88
This lets other users know about the issue.
99
If you are comfortable fixing the issue, please do so and submit a pull request.
1010

11+
Editable Installs
12+
-----------------
13+
Due to the use of ``meson-python`` as the backend, the typical process of using ``pip install -e .`` to generate an editable install cannot be used.
14+
Instead, based on the instructions `here <https://mesonbuild.com/meson-python/how-to-guides/editable-installs.html#editable-installs>`__,
15+
you must first install the `build dependencies` yourself.
16+
This can be done by looking at the ``requires`` field of the ``[build-system]`` section of the ``pyproject.toml`` file, or via
17+
``pip install .[dev]``
18+
19+
Then, do the following:
20+
21+
.. prompt:: bash
22+
23+
pip install --no-build-isolation --editable .
24+
25+
To run tests, ensure that the testing dependencies specified in the ``pyproject.toml`` file are also installed.
26+
1127
Coding style
1228
------------
1329
We use `ruff <https://github.com/astral-sh/ruff>`_ and `pre-commit <https://github.com/pre-commit/pre-commit/>`_ for linting and formatting.
@@ -49,6 +65,9 @@ When you add code or functionality, add tests that cover the new or modified cod
4965
These may be units tests for individual components or regression tests for entire models that use the new functionality.
5066
All the existing tests can be found under the ``test`` folder.
5167

68+
To run tests, ensure that the testing dependencies have been installed (see `pyproject.toml`).
69+
70+
5271
Pull requests
5372
-------------
5473
Finally, after adding or modifying code, and making sure the steps above are followed, submit a pull request via the GitHub interface.

‎doc/install.rst‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -98,11 +98,7 @@ If you encounter a ``no module named tkinter`` error when trying to run optview,
9898
Testing
9999
-------
100100
pyOptSparse provides a set of unit and regression tests to verify the installation.
101-
To run these tests, first install ``testflo`` which is a testing framework developed by the OpenMDAO team:
102-
103-
.. prompt:: bash
104-
105-
pip install testflo
101+
To run these tests, first install testing dependencies via ``pip install .[testing]``.
106102

107103
Then, in the project root directory, type:
108104

‎meson.build‎

Lines changed: 53 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,33 @@
1-
# Much of this is from SciPy
2-
31
project(
42
'pyoptsparse',
53
'c', 'cpp',
6-
# unnecessary metadata commented out until Meson supports PEP517 and installation with pip
7-
# version: 'x.x.x',
8-
# license: 'GPL-3',
9-
meson_version: '>= 0.60',
4+
version: run_command(
5+
'python', '-c',
6+
'''
7+
import re
8+
from pathlib import Path
9+
init_file = Path("pyoptsparse/__init__.py")
10+
match = re.search(r'__version__ = ["\\\']([\d\\.]+)["\\\']', init_file.read_text())
11+
print(match.group(1))
12+
'''
13+
).stdout().strip(),
14+
meson_version: '>= 0.64',
1015
default_options: [
1116
'buildtype=debugoptimized',
12-
'c_std=c99',
13-
'cpp_std=c++14',
17+
'b_ndebug=if-release',
18+
'c_std=c17',
19+
'cpp_std=c++17',
1420
],
1521
)
1622

17-
fortranobject_c = '../fortranobject.c'
23+
# <!-- from https://github.com/scipy/scipy/blob/4d9f5e65af06d4cc3f770407f1a66a185675eea9/meson.build
1824

1925
cc = meson.get_compiler('c')
2026
cpp = meson.get_compiler('cpp')
2127

28+
py3 = import('python').find_installation(pure: false)
29+
py3_dep = py3.dependency()
30+
2231
# We need -lm for all C code (assuming it uses math functions, which is safe to
2332
# assume for SciPy). For C++ it isn't needed, because libstdc++/libc++ is
2433
# guaranteed to depend on it. For Fortran code, Meson already adds `-lm`.
@@ -29,17 +38,42 @@ endif
2938

3039
# Adding at project level causes many spurious -lgfortran flags.
3140
add_languages('fortran', native: false)
41+
ff = meson.get_compiler('fortran')
42+
if ff.get_id() == 'gcc'
43+
# -std=legacy is not supported by all Fortran compilers, but very useful with
44+
# gfortran since it avoids a ton of warnings that we don't care about.
45+
# Needs fixing in Meson, see https://github.com/mesonbuild/meson/issues/11633.
46+
add_project_arguments('-std=legacy', language: 'fortran')
47+
endif
3248

33-
# https://mesonbuild.com/Python-module.html
34-
# Here we differentiate from the python used by meson, py3_command, and that python target, py3_target. This is useful
35-
# when cross compiling like on conda-forge
36-
py_mod = import('python')
37-
py3_command = py_mod.find_installation()
38-
if get_option('python_target') != ''
39-
py3_target = py_mod.find_installation(get_option('python_target'))
40-
else
41-
py3_target = py3_command
49+
if ff.has_argument('-Wno-conversion')
50+
add_project_arguments('-Wno-conversion', language: 'fortran')
4251
endif
43-
py3_dep = py3_target.dependency()
4452

53+
if host_machine.system() == 'darwin'
54+
if cc.has_link_argument('-Wl,-dead_strip')
55+
# Allow linker to strip unused symbols
56+
add_project_link_arguments('-Wl,-dead_strip', language : ['c', 'cpp', 'fortran'])
57+
endif
58+
endif
59+
60+
# install python sources
61+
install_subdir('pyoptsparse',
62+
exclude_directories: [
63+
'pyCONMIN/source',
64+
'pyCONMIN/README',
65+
'pyNLPQLP/source',
66+
'pyNLPQLP/README',
67+
'pyNSGA2/source',
68+
'pyNSGA2/README',
69+
'pyPSQP/source',
70+
'pyPSQP/README',
71+
'pySLSQP/source',
72+
'pySLSQP/README',
73+
'pySNOPT/source',
74+
'pySNOPT/README'
75+
],
76+
install_dir: py3.get_install_dir())
77+
78+
# install non-python sources
4579
subdir('pyoptsparse')

‎pyoptsparse/meson.build‎

Lines changed: 45 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -1,97 +1,58 @@
1-
# NumPy include directory - needed in all submodules
2-
incdir_numpy = get_option('incdir_numpy')
3-
if incdir_numpy == ''
4-
incdir_numpy = run_command(py3_target,
5-
[
6-
'-c',
7-
'import os; os.chdir(".."); import numpy; print(numpy.get_include())'
8-
],
9-
check: true
10-
).stdout().strip()
1+
# <!-- from https://github.com/scipy/scipy/blob/4d9f5e65af06d4cc3f770407f1a66a185675eea9/scipy/meson.build
2+
3+
incdir_numpy = meson.get_external_property('numpy-include-dir', 'not-given')
4+
if incdir_numpy == 'not-given'
5+
incdir_numpy = run_command(py3,
6+
[
7+
'-c',
8+
'''import os
9+
import numpy as np
10+
try:
11+
incdir = os.path.relpath(np.get_include())
12+
except Exception:
13+
incdir = np.get_include()
14+
print(incdir)
15+
'''
16+
],
17+
check: true
18+
).stdout().strip()
19+
20+
# We do need an absolute path to feed to `cc.find_library` below
21+
_incdir_numpy_abs = run_command(py3,
22+
['-c', 'import os; os.chdir(".."); import numpy; print(numpy.get_include())'],
23+
check: true
24+
).stdout().strip()
25+
else
26+
_incdir_numpy_abs = incdir_numpy
1127
endif
12-
# this creates a raw string which is useful for Windows use of '\' for paths
13-
incdir_numpy = '''@0@'''.format(incdir_numpy)
14-
15-
# HACK: Meson prefixes filenames of intermediate compiled objects with their filepath. This poses a problem for conda builds
16-
# since conda ensures the host environment directory has 255 characters so the meson object filenames then exceed 255
17-
# characters. To remedy this, the fortranobject.c file from numpy is copied into pyoptsparse so that the meson build
18-
# uses a relative path, rather than an absolute path, thus reducing the auto generated object filename
19-
# see for example https://github.com/mesonbuild/meson/issues/4226
20-
run_command(py3_command,
21-
[
22-
'-c',
23-
'import os; os.chdir(".."); import shutil; shutil.copy(os.path.join(r"' + incdir_numpy + '", "..", "..", "f2py", "src", "fortranobject.c"), "pyoptsparse")'
24-
],
25-
check: true
26-
)
27-
2828
inc_np = include_directories(incdir_numpy)
2929

30+
# Don't use the deprecated NumPy C API. Define this to a fixed version instead of
31+
# NPY_API_VERSION in order not to break compilation for released SciPy versions
32+
# when NumPy introduces a new deprecation.
33+
numpy_nodepr_api = ['-DNPY_NO_DEPRECATED_API=NPY_1_9_API_VERSION']
34+
np_dep = declare_dependency(include_directories: inc_np, compile_args: numpy_nodepr_api)
3035

31-
# TODO: pyoptsparse supports numpy>=1.16 but numpy.f2py.get_include() wasnt added until later, raise numpy version?
32-
#incdir_f2py = run_command(py3_target,
33-
# [
34-
# '-c',
35-
# 'import os; os.chdir(".."); import numpy.f2py; print(numpy.f2py.get_include())'
36-
# ],
37-
# check : true
38-
#).stdout().strip()
3936
incdir_f2py = incdir_numpy / '..' / '..' / 'f2py' / 'src'
4037
inc_f2py = include_directories(incdir_f2py)
41-
42-
43-
# TODO: this is kept in here so that when meson becomes pep517-compliant
44-
# we can uncomment these to have meson install the source files
45-
46-
#python_sources = [
47-
# '__init__.py',
48-
# 'pyOpt_MPI.py',
49-
# 'pyOpt_constraint.py',
50-
# 'pyOpt_error.py',
51-
# 'pyOpt_gradient.py',
52-
# 'pyOpt_history.py',
53-
# 'pyOpt_objective.py',
54-
# 'pyOpt_optimization.py',
55-
# 'pyOpt_optimizer.py',
56-
# 'pyOpt_solution.py',
57-
# 'pyOpt_utils.py',
58-
# 'pyOpt_variable.py',
59-
# 'types.py'
60-
#]
61-
62-
#py3_target.install_sources(
63-
# python_sources,
64-
# pure: true,
65-
# subdir: 'pyoptsparse'
66-
#)
38+
fortranobject_c = incdir_f2py / 'fortranobject.c'
39+
40+
# Share this object across multiple modules.
41+
fortranobject_lib = static_library('_fortranobject',
42+
fortranobject_c,
43+
c_args: numpy_nodepr_api,
44+
dependencies: py3_dep,
45+
include_directories: [inc_np, inc_f2py],
46+
gnu_symbol_visibility: 'hidden',
47+
)
48+
fortranobject_dep = declare_dependency(
49+
link_with: fortranobject_lib,
50+
include_directories: [inc_np, inc_f2py],
51+
)
6752

6853
subdir('pySNOPT')
6954
subdir('pySLSQP')
7055
subdir('pyCONMIN')
7156
subdir('pyNLPQLP')
7257
subdir('pyNSGA2')
7358
subdir('pyPSQP')
74-
#subdir('pyALPSO')
75-
#subdir('pyParOpt')
76-
#subdir('postprocessing')
77-
78-
# test imports
79-
# envdata = environment()
80-
# python_paths = [join_paths(meson.current_build_dir(), '..')]
81-
# envdata.prepend('PYTHONPATH', python_paths)
82-
83-
# progs = [['SLSQP', 'pySLSQP', 'slsqp'],
84-
# ['CONMIN', 'pyCONMIN', 'conmin'],
85-
# ['PSQP', 'pyPSQP', 'psqp'],
86-
# ['NSGA2', 'pyNSGA2', 'nsga2']]
87-
88-
89-
# foreach p : progs
90-
# import_command = 'from pyoptsparse.' + p[1] + ' import '+p[2]+'; print('+p[2]+'.__file__)'
91-
# test(
92-
# 'import test for '+p[0],
93-
# py3_command,
94-
# args: ['-c', import_command],
95-
# env: envdata
96-
# )
97-
# endforeach

‎pyoptsparse/postprocessing/meson.build‎

Lines changed: 0 additions & 27 deletions
This file was deleted.

‎pyoptsparse/pyALPSO/meson.build‎

Lines changed: 0 additions & 13 deletions
This file was deleted.

‎pyoptsparse/pyCONMIN/meson.build‎

Lines changed: 4 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,11 @@ conmin_source = custom_target('conminmodule.c',
22
input : ['source/f2py/conmin.pyf',
33
],
44
output : ['conminmodule.c', 'conmin-f2pywrappers.f'],
5-
command: [py3_command, '-m', 'numpy.f2py', '@INPUT@',
5+
command: [py3, '-m', 'numpy.f2py', '@INPUT@',
66
'--lower', '--build-dir', 'pyoptsparse/pyCONMIN']
77
)
88

9-
py3_target.extension_module('conmin',
9+
py3.extension_module('conmin',
1010
'source/openunit.f',
1111
'source/cnmn00.f',
1212
'source/cnmn01.f',
@@ -21,21 +21,7 @@ py3_target.extension_module('conmin',
2121
'source/conmin.f',
2222
'source/closeunit.f',
2323
conmin_source,
24-
fortranobject_c,
25-
include_directories: [inc_np, inc_f2py],
26-
dependencies : py3_dep,
24+
dependencies: [fortranobject_dep],
2725
subdir: 'pyoptsparse/pyCONMIN',
28-
install : false,
26+
install: true,
2927
build_rpath: '')
30-
31-
#python_sources = [
32-
# '__init__.py',
33-
# 'pyCONMIN.py',
34-
# 'LICENSE'
35-
#]
36-
#
37-
#py3_target.install_sources(
38-
# python_sources,
39-
# pure: false,
40-
# subdir: 'pyoptsparse/pyCONMIN'
41-
#)

0 commit comments

Comments
 (0)