Bug 90034 - gcc hangs on wait4 after vfork after opening tmp file
Summary: gcc hangs on wait4 after vfork after opening tmp file
Status: NEW
Alias: None
Product: gcc
Classification: Unclassified
Component: preprocessor (show other bugs)
Version: 8.2.1
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords: diagnostic
Depends on:
Blocks:
 
Reported: 2019-04-10 04:54 UTC by Todd Freed
Modified: 2019-04-22 09:19 UTC (History)
2 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed: 2019-04-10 00:00:00


Attachments
causes the hang (16.24 KB, text/x-csrc)
2019-04-10 04:54 UTC, Todd Freed
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Todd Freed 2019-04-10 04:54:02 UTC
Created attachment 46123 [details]
causes the hang

Here is the command:

> dd if=bug_input.c bs=1 count=3051 2>/dev/null | gcc -xc - # hangs

The hang seems to be specific to this input somehow.

If I pass any count < 3051, it does not hang. For any count >= 3051, it hangs.

If I pass an offset of any kind, it does not hang.

If I take those 3050 bytes and put them in a file instead of passing then to stdin via pipe, it still hangs.

> gcc -bug_input_0_3050.c # hangs

I've attached the entire file anyway, for context. The file was generated by GNU bison.

-----------------------------------

strace snippet from the hang

 . . .
getpid()                                = 16236
openat(AT_FDCWD, "/tmp/ccPYKe6J.s", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
close(3)                                = 0
stat("/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1/cc1", {st_mode=S_IFREG|0755, st_size=26001000, ...}) = 0
access("/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1/cc1", X_OK) = 0
vfork()                                 = 16237
wait4(16237, 

-----------------------------------

strace snippet from a non-hang

 . . .
getpid()                                = 16578
openat(AT_FDCWD, "/tmp/ccBrWed1.s", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
close(3)                                = 0
stat("/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1/cc1", {st_mode=S_IFREG|0755, st_size=26001000, ...}) = 0
access("/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1/cc1", X_OK) = 0
vfork()                                 = 16579
wait4(16579, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 16579
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=16579, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
getpid()                                = 16578
openat(AT_FDCWD, "/tmp/ccXHF56n.o", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
close(3)                                = 0
 . . .

-----------------------------------

Version Info:

todd@euclid ~/bison
0 master % uname -a
Linux euclid 5.0.4-arch1-1-ARCH #1 SMP PREEMPT Sat Mar 23 21:00:33 UTC 2019 x86_64 GNU/Linux

todd@euclid ~/bison
0 master % pacman -Q gcc
gcc 8.2.1+20181127-1

todd@euclid ~/bison
0 master % gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: /build/gcc/src/gcc/configure --prefix=/usr --libdir=/usr/lib --libexecdir=/usr/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=https://bugs.archlinux.org/ --enable-languages=c,c++,ada,fortran,go,lto,objc,obj-c++ --enable-shared --enable-threads=posix --enable-libmpx --with-system-zlib --with-isl --enable-__cxa_atexit --disable-libunwind-exceptions --enable-clocale=gnu --disable-libstdcxx-pch --disable-libssp --enable-gnu-unique-object --enable-linker-build-id --enable-lto --enable-plugin --enable-install-libiberty --with-linker-hash-style=gnu --enable-gnu-indirect-function --enable-multilib --disable-werror --enable-checking=release --enable-default-pie --enable-default-ssp --enable-cet=auto
Thread model: posix
gcc version 8.2.1 20181127 (GCC)


NOTE : This also repro's on another of my dev machines, with gcc version 8.2.1 20180831 (GCC)
Comment 1 Andrew Pinski 2019-04-10 04:57:59 UTC
wait4 is waiting for child process to finish.  You need to do strace with -f option to follow the forks.
Comment 2 Richard Biener 2019-04-10 07:42:09 UTC
I see the odd

> tail /tmp/x
28769 stat("/usr/include/stdc-predef.h.gch", 0x7ffee0a9f9c0) = -1 ENOENT (No such file or directory)
28769 open("/usr/include/stdc-predef.h", O_RDONLY|O_NOCTTY) = 3
28769 fstat(3, {st_mode=S_IFREG|0644, st_size=2495, ...}) = 0
28769 read(3, "/* Copyright (C) 1991-2015 Free "..., 2495) = 2495
28769 close(3)                          = 0
28769 mmap(NULL, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f53c4cf9000
28769 open("/dev/stdout", O_RDONLY)     = 3
28769 fstat(3, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 16), ...}) = 0
28769 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f53c4cf8000
28769 read(3, 

so it reads from stdout!?

Ah, that's because of the input containing

#line 72 "/dev/stdout" /* yacc.c:315  */

if I "fix" that we get

stdout:84:9: error: no macro name given in #ifndef directive
stdout:84: error: unterminated #ifndef

so GCC wants to complain about

# ifndef YY_NULLPTR

and tries to apply caret diagnostics to that.  Opening /dev/stdout succeeds
but reading from it will hang.

Smaller testcase that will hang:

#line 1 "/dev/stdout"
#def xy


we should probably check whether the file we opened is a regular file
before trying to read from it.

That said, bison should be fixed to not emit this kind of line directives...
Comment 3 David Malcolm 2019-04-10 14:33:55 UTC
(In reply to Richard Biener from comment #2)
[...]
> 
> Smaller testcase that will hang:
> 
> #line 1 "/dev/stdout"
> #def xy

Presumably we're blocked, waiting on ourselves to write something to our
stdout so that we can read it.

I was able to reproduce this hang with gcc 4.8.3, so this isn't a regression.

> we should probably check whether the file we opened is a regular file
> before trying to read from it.
[...]

$ ll /dev/stdout
lrwxrwxrwx. 1 root root 15 Nov 26 09:29 /dev/stdout -> /proc/self/fd/1

$ ll /proc/self/fd/1
lrwx------. 1 david david 64 Apr 10 11:13 /proc/self/fd/1 -> /dev/pts/8

$ ll /dev/pts/8
crw--w----. 1 david tty 136, 8 Apr 10 11:14 /dev/pts/8

Presumably we ought to support source "files" that are symlinks; should we
resolve all symlinks before opening, and then require the result to be a
regular file?

Or some other kind of sanity-checking?
Comment 4 Andreas Schwab 2019-04-10 14:42:54 UTC
You get the resolve part for free by opening it.
Comment 5 David Malcolm 2019-04-10 15:14:25 UTC
(gdb) call fileno(c->fp)
$3 = 4
(gdb) info inferior
  Num  Description       Executable        
* 1    process 35251     /home/david/coding-3/gcc-git-bugfixing/build/gcc/cc1 
(gdb) shell ls -al /proc/35251/fd
total 0
dr-x------. 2 david david  0 Apr 10 12:02 .
dr-xr-xr-x. 9 david david  0 Apr 10 12:02 ..
lrwx------. 1 david david 64 Apr 10 12:02 0 -> /dev/pts/8
lrwx------. 1 david david 64 Apr 10 12:02 1 -> /dev/pts/8
lrwx------. 1 david david 64 Apr 10 12:02 2 -> /dev/pts/8
l-wx------. 1 david david 64 Apr 10 12:02 3 -> /tmp/ccE3XfON.s
lr-x------. 1 david david 64 Apr 10 12:02 4 -> /dev/pts/8

which shows that I can type at the terminal during the hang, and then Ctrl-D to inject what the content of "/dev/stdout" should be:

$ ./xgcc -B. -c ../../src/pr90034.c
Hello world
/dev/stdout:1:2: error: invalid preprocessing directive #def; did you mean #ifdef?
    1 | Hello world
      |  ^~~
      |  ifdef
Comment 6 David Malcolm 2019-04-10 15:28:43 UTC
(In reply to Andreas Schwab from comment #4)
> You get the resolve part for free by opening it.

Thanks.

I'm wondering what the best cross-platform test ought to be.

Maybe something like this to input.c's add_file_to_cache_tab:

  int fd = fileno (fp);
  if (fstat (fd, &buf) == -1)
    {
      /* reject due to error */
      fclose (fp);
      return NULL;
    }
  if (!S_ISREG(buf.st_mode))
    {
      /* reject: not a regular file */
      fclose (fp);
      return NULL;
    }
  /* carry on */

input.c is re-reading the source file after the frontend has already opened it, so this isn't going to work for pipes, and, as you mention, the resolution of symlinks already happened when fp was opened.
Comment 7 Todd Freed 2019-04-11 04:16:41 UTC
Submitted a bug report to bug-bison@gnu.org, as this seems to be a recent behavior change in bison. It did not used to produce an output which induced the hang when invoked like,

bison -o /dev/stdout parser.y | gcc -xc -
Comment 8 Akim Demaille 2019-04-22 09:19:54 UTC
(In reply to Richard Biener from comment #2)
> That said, bison should be fixed to not emit this kind of line directives...

Hi Richard,

I maintain Bison.

Bison is emitting two types of #lines: right before emitting code coming
from the input file, something like (1)

#line "parse.y" 42

and once we are done with the input, and back to generated code, something
like (2)

#line "parse.tab.c" 1024

What is special here is the way Bison and GCC were invoked:

  bison -o /dev/stdout parser.y | gcc -xc -

So "of course", because the output file is /dev/stdout, we emit
this for (2)

#line "/dev/stdout" 1024

and if someone were to call bison with

  bison -o /dev/stdout /dev/stdin

no doubt that (1) will become

#line "/dev/stdin" 42


Back your comment:
> That said, bison should be fixed to not emit this kind of line directives...

Well, I don't think we should decide whether or not to emit the
#lines based on how magic the input files, but if the consensus
here differs, I will change that.  I personally do not understand
the point of not generating the output file and sending the output
directly to stdout.  Especially in the case of Bison which also
might generate other files based on the base name of the output file
(/dev/stdout.h, /dev/stdout.output, etc.).

If you do think we should detect special input and output files
and not emit #line in their case, what kind of check would you
recommend?


That being said, since we also have caret-diagnostics, and we also support
#line in the input, we also have to get robust to users/generators
putting special file names in the #lines inside parser.y...

Cheers!