Re: DejaGNU gurus ahoi! (Re: Testing use of STDIN redirected etc.)

[Janne Blomqvist <blomqvist dot janne at gmail dot com>]

On Fri, Oct 31, 2014 at 1:20 PM, N.M. Maclaren <nmm1@cam.ac.uk> wrote:
> On Oct 31 2014, Janne Blomqvist wrote:
>>
>>
>> what would be the best way to test stuff like two binaries
>> communicating via a pipe, FIFO or such with DejaGNU? The gfortran
>> testsuite has by now quite extensive coverage of all the weird and
>> quirky corner cases of Fortran I/O behavior, but practically all these
>> tests are done using regular files. Over the years we've had some
>> embarrassing bugs in handling non-seekable files like terminals,
>> FIFO's, and pipes. I believe these would have been quickly caught if
>> only we had testing coverage.
>
>
> It's worse.  It's much, much worse :-(

I know, being one of the main libgfortran IO maintainers for several
years, I know more about the quirkyness of Fortran I/O than I ever
would have wanted to.. :(

My main aim here really is to just get the basics right before
worrying about corner cases. For instance, I recall we have failed in
a simple sequential reading of a access="sequential", form="formatted"
unit connected to a pipe or such, because libgfortran incorrectly
issued a lseek() at some point, which returned an error (since the fd
wasn't seekable), and that error was then duly propagated upwards. But
since the testsuite didn't test against this, it took an
embarrassingly long time before that bug was found and fixed.

> So the first question is to decide the priority of I/O variations,
> including types of file and permitted operations for each connection
> mode.  The key characteristics (in OPEN) are ACCESS, FORM and ACTION.
> In particular, there is disagreement on WG5 exactly what constitutes a
> 'file' - I have been told that magnetic tape files and sockets are not
> files by someone who ought to have known better.  For example, does
> gfortran intend to support the piping of unformatted files?  Not all
> compilers do.

My argument in this, which I have stated many times on the mailing
list and in bugzilla over the years, is that there are many kinds of
special files which are special in different ways. E.g. wrt. seeking,
some special files allow seeking just fine, others report an error,
and others always report an offset of 0. Instead of trying to
enumerate in which ways special files are special on all supported
targets, which is a sisyphean task, libgfortran should just do what
it's being told to, no more, and if any error occurs, propagate it
upwards to the calling program.  The "no more" is the problematic
part, e.g. a spurious lseek(fd, 0, SEEK_CUR) would have have no effect
on a regular file, but cause an error on a non-seekable file. Hence
the need for a testsuite for non-regular files.

>> (Something which works only on x86_64-unknown-linux-gnu and perhaps a
>> few other "mainstream" targets where we can expect the presence of a
>> "normal" shell environment etc., while not optimal, is certainly a lot
>> better than what we have now.)
>
>
> You can do a LOT better than THAT!  It would be trivial to write
> something that would work on any reasonably Unix-like system (including
> anything that supports base POSIX 1996).  I have done that for system
> scripts and still do that for my course tests, though I no longer have
> access to weird and wonderful Unices to test them.
>
> You are quite right that you need to assume a reasonable shell, though
> you can write such tests that will work on pretty well any modern Bourne,
> Korn or POSIX shell, and any version of bash.  The original Bourne shell
> was beyond redemption, but I believe that it is now completely dead.
> That is from experience in writing portable system scripts a decade or
> so back, but I am pretty sure it still holds.

For what I'm thinking of, I'm sure a pretty basic POSIX sh + maybe a
few utilities such as mkfifo would do just fine. My reason for wanting
to limit it, is that I, and AFAIK most gfortran developers, develop
and test on x86_64-unknown-linux-gnu, and with the very limited time
that I can afford to gfortran these days I don't want to spend that
time trying to debug-over-email why a script fails on some weird
target I don't have access to. If you want to make it work on Ultrix
or something like that, be my guest, patches welcome. (For a similar
reason, I think it wouldn't make sense to run these tests in a cross
environment, more complexity for little gain (unless it's really
simple?).)



-- 
Janne Blomqvist