2019-08-14
preserving backslash-newlines in heredocs run in a subshell
unix.stackexchange
Question

So long as terminal backslashes are not present, the newlines of a heredoc are preserved after capturing the output of a heredoc in a variable:

var=$(cat <<-EOF
a
b
c
EOF
)
echo "$var"

a
b
c

Newlines are lost, however, when trying to store backslash-newlines:

var=$(cat <<-EOF
a \\
b \\
c
EOF
)

echo "$var"
a \b \c

Adding additional backslashes for further escaping does not help.

Of course, when not run in a subshell, a heredoc can be used to generate lines that end in backslash-newline:

$ cat <<-EOF
a \\
b \\
c
EOF
a \
b \
c

To circumvent this problem, one can add a newline delimiter to generate the desired output by a transformation of the variable:

var=$(cat <<-EOF
a \\@
b \\@
c
EOF
)

tr -d '@' <<<"$var"

a \
b \
c

Is there a more straightforward way to preserve backslash-newlines from heredocs run in subshells?

Answer
1

I wonder if that's a bug in Bash (or if the behaviour of backslash-newlines is just undefined). All other shells I tried show different behaviour from Bash:

$ cat nl.sh

echo "1:"
cat <<EOF
a \\
b \\
c
EOF

echo "2:"
var=$(cat <<-EOF
a \\
b \\
c
EOF
)
echo "$var"

output:

$ bash nl.sh
1:
a \
b \
c
2:
a \b \c

vs.

$ dash nl.sh
1:
a \
b \
c
2:
a \
b \
c

You could put the code inside the command substitution in a function, that might help in working around any issues with the parser, e.g.:

f() {
cat <<EOF
a \\
b
EOF
}
var=$(f)
echo "$var"
preserving backslash-newlines in heredocs run in a subshell
See more ...