This is the mail archive of the mailing list for the GCC project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [RFC, Fortran] Avoid race on testsuite temporary files

On Wed, Dec 9, 2015 at 4:27 PM, Yvan Roux <> wrote:
> Hi,
> as it was raised in
> we experiment
> random failures in gfortran validation when it is run in parallel (-j
> 8).  The issues occurs because of concurrent access to the same file,
> the first two patches fixed some of them by changing the file names
> used, but there are still remaining conflicts (6 usages of foo, 8 of
> test.dat). This is easy to fix and I've a patch for that, but there is
> another issue and I'd like to have your thoughts on it.
> There is a little bit more than 1000 testcases which use IO without
> explicit file names, ~150 use scratches (a temporary file named
> gfortrantmp + 6 extra char is created) and the others, which only
> specify the io unit number, use a file named fort.NN with NN the
> number of the io unit. We see conflicts on these generated files, as
> lot of testcases use the same number, the most used are:
> 10 => 150
> 99 => 70
> 6  => 31
> 11 => 27
> 1  => 23
> I started to change the testcases to use scratches when it is possible
> and before finding that there is that many to fix, and I also had
> conflicts on the generated scratch names.  The options I see to fix
> that are:
> 1- Move all these testcases into an IO subdir and change the testsuite
> to avoid parallelism in that directory.
> 2- Use scratches when possible and improve libgfortran file name
> generation, I don't know well fortran but is it possible to change the
> file name patterns for scratches and io unit files ?

For the "io unit files", I don't think we can change it. fort.NN is a
common convention, and I suspect a lot of people rely on it. As for
scratch files, I'm surprised you're seeing conflicts. At least on
Linux/POSIX targets it should use mk(o)stmp(), which should make sure
a unique file name is chosen and opened with exclusive access
(O_CREAT|O_EXCL). Furthermore, on POSIX targets it unlinks the file
immediately after opening it, further reducing the possibility of any
conflict. So what target are you compiling for?

> 3- Change the io unit numbers used, as it was suggested on irc, but I
> find it a bit painful to maintain.

Is it really that much more work than 1)? If you don't want to keep
track of a count, just generate a random number between 10 and 2^31,
and use that as the unit number? Should at least make conflicts rare.

> Any comments are welcome.
> Thanks
> Yvan

Janne Blomqvist

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]