precompile conda failure

[hzhupku]

The julia environment worked out fine on my ubuntu, but failed on server.

ERROR: LoadError: ArgumentError: Path to conda environment is not valid.
Stacktrace:
[1] prefix(::String) at /home/lamda/.julia/v0.6/Conda/src/Conda.jl:59
[2] include_from_node1(::String) at ./loading.jl:569
[3] include(::String) at ./sysimg.jl:14
[4] anonymous at ./:2
while loading /home/lamda/.julia/v0.6/Conda/src/Conda.jl, in expression starting on line 64
ERROR: LoadError: Failed to precompile Conda to /home/lamda/.julia/lib/v0.6/Conda.ji.
Stacktrace:
[1] compilecache(::String) at ./loading.jl:703
[2] _require(::Symbol) at ./loading.jl:456
[3] require(::Symbol) at ./loading.jl:398
[4] include_from_node1(::String) at ./loading.jl:569
[5] include(::String) at ./sysimg.jl:14
[6] anonymous at ./:2
while loading /home/lamda/.julia/v0.6/PyCall/src/PyCall.jl, in expression starting on line 28
ERROR: LoadError: Failed to precompile PyCall to /home/lamda/.julia/lib/v0.6/PyCall.ji.
Stacktrace:
[1] compilecache(::String) at ./loading.jl:703
[2] _require(::Symbol) at ./loading.jl:456
[3] require(::Symbol) at ./loading.jl:398
[4] include_from_node1(::String) at ./loading.jl:569
[5] include(::String) at ./sysimg.jl:14
[6] anonymous at ./:2
while loading /home/lamda/.julia/v0.6/PyPlot/src/PyPlot.jl, in expression starting on line 5
ERROR: LoadError: Failed to precompile PyPlot to /home/lamda/.julia/lib/v0.6/PyPlot.ji.
Stacktrace:
[1] compilecache(::String) at ./loading.jl:703
[2] _require(::Symbol) at ./loading.jl:490
[3] require(::Symbol) at ./loading.jl:398
[4] include_from_node1(::String) at ./loading.jl:569
[5] include(::String) at ./sysimg.jl:14
[6] process_options(::Base.JLOptions) at ./client.jl:305
[7] _start() at ./client.jl:371
while loading /home/lamda/huhz/client.jl, in expression starting on line 2

[stevengj]

What does print(read(Pkg.dir("Conda", "deps", "deps.jl"), String)) output?

[stevengj]

Closing unless this issue re-appears.

[vinnief]

I have a precompile Conda failure with Julia 1.3.1 on Win10 , installed for local user.
Build Conda complains about makedir not having rights to C:/users/Julia
and import conda complains about
LoadError: ArgumentError: Path to conda environment is not valid: C:\Users\julia\AppData\Local\JuliaPro\packages\padding_some_junk_value\in_here\creating_a_pattern_to_be_replaced\conda\3
obviously that path is wrong, but why is it not set? is this meant to be a manual setting I need to make? where?
I made a comment on https://discourse.julialang.org, and hesitated about making a new issue but the issue title is still applicable, even if the version is not 0.6 any more.

[kerim371]

I'm newbie in Julia and I have similar problem.
When I type:
Pkg.add("Conda")
I get error:

julia> import Conda
[ Info: Precompiling Conda [8f4d0f93-b110-5947-807f-2305c1781a2d]
ERROR: LoadError: ArgumentError: Path to conda environment is not valid:
Stacktrace:
 [1] prefix(path::String)
   @ Conda C:\Users\tasik\.julia\packages\Conda\sNGum\src\Conda.jl:41

@stevengj when I type print(read(Pkg.dir("Conda", "deps", "deps.jl"), String)) I get:

julia> print(read(Pkg.dir("Conda", "deps", "deps.jl"), String))
┌ Warning: `Pkg.dir(pkgname, paths...)` is deprecated; instead, do `import Conda; joinpath(dirname(pathof(Conda)), "..", paths...)`.
└ @ Pkg C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.6\Pkg\src\Pkg.jl:578
const ROOTENV = ""
const MINICONDA_VERSION = "3"
const USE_MINIFORGE = false

I can't understand does Conda.jl install conda.exe or I have to preinstall it by myself?

[kerim371]

I just found that I can build Conda.jl on preinstalled conda. But is it possible to build Conda.jl on preinstalled python without preinstalled conda?

[timholy]

https://github.com/timholy/BasicBlockRewriter.jl/runs/5516227052?check_suite_focus=true

Note the CI script sets the PYTHON environment variable

[BeastyBlacksmith]

I'm getting the same on the Plots.jl documentation build. Is there some workaround?

[frankier]

Now I have the same problem

https://github.com/frankier/ComputerAdaptiveTesting.jl/actions/runs/3408009528/jobs/5668188461

Pasting the error because GitHub actions will delete it at some point

ERROR: LoadError: ArgumentError: Path to conda environment is not valid: /home/runner/.julia/conda/3
Stacktrace:
 [1] prefix(path::String)
   @ Conda ~/.julia/packages/Conda/x2UxR/src/Conda.jl:42
 [2] top-level scope
   @ ~/.julia/packages/Conda/x2UxR/src/Conda.jl:47
 [3] include
   @ ./Base.jl:41[9](https://github.com/frankier/ComputerAdaptiveTesting.jl/actions/runs/3408009528/jobs/5668188461#step:7:10) [inlined]
 [4] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
   @ Base ./loading.jl:1554
 [5] top-level scope
   @ stdin:1
in expression starting at /home/runner/.julia/packages/Conda/x2UxR/src/Conda.jl:1
in expression starting at stdin:1
ERROR: LoadError: Failed to precompile Conda [8f4d0f93-b1[10](https://github.com/frankier/ComputerAdaptiveTesting.jl/actions/runs/3408009528/jobs/5668188461#step:7:11)-5947-807f-2305c[17](https://github.com/frankier/ComputerAdaptiveTesting.jl/actions/runs/3408009528/jobs/5668188461#step:7:18)81a2d] to /home/runner/.julia/compiled/v1.8/Conda/jl_DcRrZY.

There have also been successful runs

https://github.com/frankier/ComputerAdaptiveTesting.jl/actions/runs/3403780584/jobs/5660539740

Did either of you manage to fix it or otherwise workaround it?

[tfiers]

Same here: I cannot have PyPlot installed in CI

ERROR: The following 1 direct dependency failed to precompile:

PyPlot [d330b81b-6aea-500a-939a-2ce795aea3ee]

Failed to precompile PyPlot [d330b81b-6aea-500a-939a-2ce795aea3ee] to "/home/runner/.julia/compiled/v1.10/PyPlot/jl_Dr7FmQ".
ERROR: LoadError: ArgumentError: Path to conda environment is not valid: /home/runner/.julia/conda/3
Stacktrace:
 [1] prefix(path::String)
   @ Conda ~/.julia/packages/Conda/x2UxR/src/Conda.jl:42
 [2] …

(link)

though presumably PyPlot's own CI does run succesfully
https://github.com/JuliaPy/PyPlot.jl/actions/workflows/CI.yml
(unfortunately github has deleted the last logs).
Not sure what's different

[frankier]

It looks like the best chance of solving this would be to add https://github.com/mxschmitt/action-tmate before the failing bit (or is it possible to run it "on-fail"?) Then we could go in and poke around in /home/runner and /home/runner/.julia to try and figure out what happened.

I will note that I had a slightly "model-challenging" moment with regards to Github Actions when reading https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#file-systems . This says that $GITHUB_WORKSPACE is persisted between steps, but it says no such thing about $HOME. I really don't think so, but I wanted to check what others think, is it possible for a Github action to be pre-empted and end up with an empty home directory between steps?

What's happening is that an empty directory is made here and looked for here. Between the build script and the pre-compile script, the directory has gone missing.

This could be worked around in any one of the following ways:

• Run _install_conda(ROOTENV) at the very beginning of Conda.jl (it is already run on
• Remove the PREFIX, etc. constants and replace them with a method of prefix()
• Remove the isdir check in prefix(...)
• Specialise prefix(...) for ROOTDIR and run mkdir there

I do not know what exactly is happening, but one possibility is that /home/runner/.julia/conda/3 is available both when build.jl runs and when Conda.jl runs, but at different locations. In that case, perhaps if there is a symlink in the directory path, realdir() could be used in build.jl. Another option would be to add a relative directory, and look in DEPOT_PATH to make an absolute directory when Conda.jl is run.

Another possibility is that the problem occurs because the built Conda package, including deps.jl is getting cached, but the directory /home/runner/.julia/conda/ is not.

@stevengj The problem seems to be persistent, and I believe there are some paths to either a proper fix, or at least a workaround, so perhaps this issue could be reopened?

[NeroBlackstone]

Same problem here, this problem should be solved seriously.

[lxvm]

I agree that there is an issue here regarding a directory that goes missing on Github Actions CI. In my case, I wanted to install a pip-only package when building package documentation to show-off and test Python interoperability, so I also encountered the following error similar to the original:

ERROR: ArgumentError: Path to conda environment is not valid: /home/runner/.julia/conda/3/x86_64
Stacktrace:
 [1] prefix
   @ ~/.julia/packages/Conda/zReqD/src/Conda.jl:52 [inlined]
 [2] conda_rc(env::String)
   @ Conda ~/.julia/packages/Conda/zReqD/src/Conda.jl:125
 [3] pip_interop
   @ ~/.julia/packages/Conda/zReqD/src/Conda.jl:622 [inlined]
 [4] pip_interop(enabled::Bool)
   @ Conda ~/.julia/packages/Conda/zReqD/src/Conda.jl:622
 [5] top-level scope
   @ none:1

However, by forcing PyCall.jl to (re)build in my CI script I was able to get this to work

julia --project=docs/ -e 'ENV["PYTHON"]=""; using Pkg; Pkg.develop(PackageSpec(path=pwd())); Pkg.instantiate(); Pkg.build("PyCall"); using Conda; print(read(joinpath(dirname(pathof(Conda)), "..", "deps", "deps.jl"),String)); Conda.pip_interop(true); Conda.pip("install", "pythtb")'

I made it print some of the paths when working and I get

const ROOTENV = "/home/runner/.julia/conda/3/x86_64"
const MINICONDA_VERSION = "3"
const USE_MINIFORGE = true
const CONDA_EXE = "/home/runner/.julia/conda/3/x86_64/bin/conda"