Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: pipe instead of using process expansion #71

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jedwards1211
Copy link

@jedwards1211 jedwards1211 commented Oct 7, 2024

fix #70

Just changed instances of

while read -r fname; do ...; done < <("$cmd" "$arg")

to

"$cmd" "$arg" | while read -r fname; do ...; done

I'm not enough of a bash expert to know for sure how the behavior of this differs from process expansion, but it seems to work and avoid the issue I was having with process expansion.

@jedwards1211 jedwards1211 changed the title fix: avoid process expansion fix: pipe instead of using process expansion Oct 7, 2024
@mbucc
Copy link
Owner

mbucc commented Oct 19, 2024

Some research on this.

  1. Process substitution "passes the name of the pipe (actually something like "/dev/fd/63") to the command as an argument, and expects the command to open that "file" and read/write it".
    -- comment on https://stackoverflow.com/a/51293856
# echo <(date)
/dev/fd/11
# cat <(date)
Fri Oct 18 16:40:20 PDT 2024
# 
  1. Error propagation is different. Example below. Can work around with set -o pipefail or the PIPESTATUS envvar if you are limited to bash. But Ash (Alpine Linux shell) does not support PIPESTATUS (source: https://lists.busybox.net/pipermail/busybox/2007-July/062422.html).
$ ls -l /notthere | tee listing.txt
ls: cannot access '/notthere': No such file or directory
$ echo $?
0

vs.

$ ls -l /notthere > >(tee listing.txt)
ls: cannot access '/notthere': No such file or directory
$ echo $?
2
  1. Subprocess may not be done when command completes. For example, in this command:
tar \
--create \
--file /tmp/etc-backup.tar \
--verbose \
--directory /etc . &> \
>(tee /tmp/etc-backup.log)

apparently the tee subprocess can still be running when main process returns.
-- source: https://stackoverflow.com/a/73598634

The second issue is the only one I need to think about. Is there a case where this matters?

@@ -614,7 +614,7 @@ migrate(){
fi

[[ ${PIPESTATUS[0]} -eq 0 && ${PIPESTATUS[1]} -eq 0 ]] || exit 1
Copy link
Owner

@mbucc mbucc Oct 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PIPESTATUS is bash-only. (ksh is lower case pipestatus.) Must not hit this line in the Alpine ash tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error /dev/fd/63: No such file or directory (I think process substitution has issues in my target environment)
2 participants