el7 python build error in singularity container

[adriansev]

I'm trying to build aliphysics/aliroot in a el7 singularity container but i get this error when building python:
https://asevcenc.web.cern.ch/asevcenc/el7error_python/log.txt

The problem is that there is no mention on the error until the very end where i get:

2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ for X in 'python*'
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ echo python3.6
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ grep -E '^python[0-9]+\.[0-9]+$'
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ for X in 'python*'
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ echo python3.6m
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ for X in 'python*'
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ echo python3.6m-config
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ head -n1
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: ++ PYTHON_BIN=python3.6
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ for X in 'pip*'
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ echo 'pip*'
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ grep -E '^pip[0-9]+\.[0-9]+$'
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: +++ head -n1
2021-06-16@17:35:39:DEBUG:AliPhysics:Python:0: ++ PIP_BIN=
2021-06-16@17:35:39:ERROR:AliPhysics:Python:0: Error while executing /home/physics-tools/alicebuild/sw/SPECS/slc7_x86-64/Python/v3.6.10-local1/build.shon `sev.spacescience.ro'.
2021-06-16@17:35:39:ERROR:AliPhysics:Python:0: Log can be found in /home/physics-tools/alicebuild/sw/BUILD/Python-latest/log
2021-06-16@17:35:39:ERROR:AliPhysics:Python:0: Please upload it to CERNBox/Dropbox if you intend to request support.
2021-06-16@17:35:39:ERROR:AliPhysics:Python:0: Build directory is /home/physics-tools/alicebuild/sw/BUILD/Python-latest/Python.

but pip seems to be fine:

ALIBUILD##|>el7cvmfs_sandbox<|[Wednesday 16.06.21 17:53] adrian@sev : ~  $
pip3 -V
pip 21.1.2 from /home/adrian/.local/lib/python3.6/site-packages/pip (python 3.6)

but it is true that in path both pip3.6 and pip3.9 are found ..

So, any idea what is going on? It would be really useful if i could build ALICE things within the container
Thanks!

[ktf]

Is this still an issue?

[adriansev]

@ktf yes, it is, see https://asevcenc.web.cern.ch/asevcenc/alibuild_err/python_log
the strange thing is that i do not notice any error while building just in the finishing step ..
also, i noticed this :

Collecting setuptools
Requirement already satisfied: pip in /home/adrian/.local/lib/python3.6/site-packages (21.1.2)

which i'm not sure it's ok as the rest of the stuff i see is used from
copying build/scripts-3.6/pyvenv-3.6 -> /home/physics-tools/alicebuild/sw/INSTALLROOT/f1021c3972cd4eb788c5cfa2aeb6940e9f3e6a30/slc7_x86-64/Python/v3.6.10-local1/bin

[jezwilkinson]

Bumping this as issue also seen with a singularity container built from the slc7-builder image: https://hub.docker.com/r/alisw/slc7-builder (which had previously worked fine but stopped last week), with similar output in the log: https://cernbox.cern.ch/index.php/s/QMC5p07zcbv8PdQ/download

Passing --no-home or --containall when running singularity shell (which skips mounting the user's $HOME or mounts a dummy $HOME, respectively, when the container is invoked) appears to sidestep the issue and allows Python to build as usual. This implies that there is an unexpected clash with the user-installed pip from the user's home directory.

It is also not clear why Python requires the build at all in this case, since aliDoctor Python --defaults o2 and aliDoctor O2Physics --defaults o2 both report that python will be taken from the system (3.6.8)