Re: PATCH: Avoid long commands in libtool

[Jeff Sturm <jsturm at one-point dot com>]

On 11 Jul 2001, Alexandre Oliva wrote:
> On Jul  9, 2001, Jeff Sturm <jsturm@one-point.com> wrote:
> > Libtool now supports `-objectlist' to read its arguments from a file.
> > This didn't work for me, since it evaluates to
> 
> > 	for fil in `cat $save_arg`
> 
> > resulting in an excessively long shell command, which is the problem that
> > -objectlist was supposed to prevent in the first place.
> 
> Hmm...  Robert Boehne said he hadn't run into any problems using this
> construct, and we conjectured that it was because the shell never had
> to exec() a process passing a long argument list: the for command was
> expanded in user-space, so it wouldn't run into kernel limits.  On
> which system did you get a problem with this, and what was the exact
> symptom?

My failure was with ksh93, installed as the system compiler.  I haven't
observed a similar failure before, but I don't have a wide range of
hardware to test with.

I don't think it has anything to do with kernel limits, but rather
internal buffers within the shell.  I have had problems whenever the
command output exceeded 8192 bytes.

The error message is a little bizarre:

./libtool[1294]: cat: libgcj.objectlist: read error [Inappropriate ioctl
for device]

You could say it is a shell bug, and rightly so.

> > I suggest using the read builtin command to process objectlist a line at a
> > time, taking care that arguments may be separated by whitespace other
> > than newlines.
> 
> Neither this nor the approach you took in the patch will work.  If you
> redirect the input or the output of a command, it may run in a
> separate shell, so any assignments in the command won't affect the
> enclosing process.

Good point.  It didn't occur to me this may not be portable.  Are you
certain that applies to redirection, and not just piped commands?

> I suppose it would work to redirect the standard input into some other
> file descriptor, then redirect the input file into the standard input:
> 
> exec 3<&0
> exec < $save_arg
> while read line; do
>   for fil in $line; do
>     ...
>   done
> done
> exec 0<&3
> exec 3<&- # I don't think this is portable, but we don't really need this

Yuck.  Perhaps the best answer is to not support ksh93.  I don't know how
big a problem that would be.  Most systems today seem to ship with an
earlier ksh.

> Of course, there's still a possibility that a user could exceed line
> lengths and run into problems, but I don't see how a shell-script
> could recover from this case.  Perhaps running the input file through
> the portable equivalent of tr '[ \t]' '\n', then stripping empty
> lines out?

Right.  I made sure libgcj's Makefile won't emit long lines to avoid
exactly that problem.

If you like I'll repost to libtool-patches@gnu.org where these issues are
a bit more on topic.

Jeff