Add pipe function to read/write to pipes.#722
Open
ghuls wants to merge 1 commit intoexaloop:developfrom
Open
Conversation
Add `pipe` function to read and write to pipes with the help of `popen`.
This allows the following:
$ cat pipe_command_tar.codon
print("Read tar file list")
with pipe("tar tzvf codon-linux-x86_64.tar.gz", "r") as fh:
i = 0
for line in fh:
if i == 15:
break
print(line, end="")
i += 1
print("Gunzip tar file and compress with zstd")
with pipe("gzip -cd codon-linux-x86_64.tar.gz", "r") as in_fh:
with pipe("zstd -c -9 - > codon-linux-x86_64.tar.zst", "w") as out_fh:
chunk = in_fh.read(4096*4)
while chunk:
out_fh.write(chunk)
chunk = in_fh.read(4096*4)
$ codon run pipe_command_tar.codon
Read tar file list
drwxr-xr-x root/root 0 2025-11-28 01:00 codon-deploy-linux-x86_64/
drwxr-xr-x root/root 0 2025-11-28 01:49 codon-deploy-linux-x86_64/python/
-rw-r--r-- root/root 2692 2025-11-28 00:32 codon-deploy-linux-x86_64/python/setup.py
-rw-r--r-- root/root 32 2025-11-28 00:32 codon-deploy-linux-x86_64/python/.gitignore
-rw-r--r-- root/root 184 2025-11-28 00:32 codon-deploy-linux-x86_64/python/README.md
-rw-r--r-- root/root 69 2025-11-28 00:32 codon-deploy-linux-x86_64/python/pyproject.toml
drwxr-xr-x root/root 0 2025-11-28 01:00 codon-deploy-linux-x86_64/python/codon/
-rw-r--r-- root/root 134 2025-11-28 00:37 codon-deploy-linux-x86_64/python/codon/version.py
-rw-r--r-- root/root 10317 2025-11-28 00:32 codon-deploy-linux-x86_64/python/codon/decorator.py
-rw-r--r-- root/root 2541 2025-11-28 00:32 codon-deploy-linux-x86_64/python/codon/jit.pyx
-rw-r--r-- root/root 636 2025-11-28 00:32 codon-deploy-linux-x86_64/python/codon/jit.pxd
-rw-r--r-- root/root 279 2025-11-28 00:32 codon-deploy-linux-x86_64/python/codon/__init__.py
-rw-r--r-- root/root 20 2025-11-28 00:32 codon-deploy-linux-x86_64/python/MANIFEST.in
drwxr-xr-x root/root 0 2025-11-28 01:00 codon-deploy-linux-x86_64/python/codon_jit.egg-info/
-rw-r--r-- root/root 6 2025-11-28 01:00 codon-deploy-linux-x86_64/python/codon_jit.egg-info/top_level.txt
Gunzip tar file and compress with zstd
There are a few downsides to using popen:
- popen spawns a shell to execute the "command" and failures of finding the actual command
are not 100% reliable to detect. The shell return exit code 127 if if can't find the
command and exit code 126 if it is not an executable, but if for some reason the actual
command would use the same exit codes, there is no way to communicate that.
- popen only captures stdout and not stderr
So in the long run it is probably better to have a proper implementation that can also capture
stderr (and avoids deadlocks when reading both streams) and also captures the real exit code
instead of always spawning a shell.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
pipefunction to read and write to pipes with the help ofpopen. This allows the following:There are a few downsides to using popen:
So in the long run it is probably better to have a proper implementation that can also capture stderr (and avoids deadlocks when reading both streams) and also captures the real exit code instead of always spawning a shell.