Re: [PATCH] Properly detect working jobserver in gcc driver.

[Martin Liška <mliska at suse dot cz>]

On 8/2/19 9:44 AM, Jakub Jelinek wrote:
> On Fri, Aug 02, 2019 at 08:30:47AM +0200, Martin Liška wrote:
>> On 8/1/19 4:41 PM, Jakub Jelinek wrote:
>>> On Thu, Aug 01, 2019 at 04:34:09PM +0200, Martin Liška wrote:
>>>> Ok, after deeper discussion with Honza, I would like to suggest the original
>>>> patch that was about proper detection of jobserver.
>>>>
>>>> Can you please Jakub test the patch in your environment?
>>>
>>> Isn't this done too late (as in, doesn't the driver at that moment already
>>> have some files newly openend, like e.g. the @ option files?
>>
>> You are right, I've reworked that. Good observation.
> 
> I was actually wrong, because while expandargv fopens new file descriptors
> when processing the options, it fcloses them too before it returns.
> 
> Sorry for that.

Ah ok.

> 
> Can you strace if other fds are opened and not closed in the spot you had it
> before?  Advantage of doing it there is that it will not be done for all the
> -E/-S/-c compilations when the linker is not spawned.

I've used the same trick which you used and I'm attaching the output.
I believe it's fine, I can't see any opened fd by GCC.

Martin

> 
>>>
>>>> +		= ((sscanf (n, "--jobserver-auth=%d,%d", &rfd, &wfd) == 2)
>>>
>>> No need to wrap sscanf (...) == 2 into ()s.  Also, you've already done
>>> a strstr, what is the point in verifying it once again that it starts with
>>> --jobserver-auth= string?
>>> And in the lto-writer.c code there is no space between sscanf and (.
>>
>> Yep, I simplified that.
> 
> Thanks.  Note, your patch from yesterday also passed testing.
> 
> 	Jakub
>

Attachment: lto-jobserver-openned-fds.txt
Description: Text document

>From 295e6aeae929d33fb904439a0fda3b840ab6c980 Mon Sep 17 00:00:00 2001
From: Martin Liska <mliska@suse.cz>
Date: Thu, 1 Aug 2019 16:30:01 +0200
Subject: [PATCH] Properly detect working jobserver in gcc driver.

gcc/ChangeLog:

2019-08-02  Martin Liska  <mliska@suse.cz>

	* gcc.c (driver::maybe_run_linker): Call detect_jobserver
	to detect working job server.
	(driver::detect_jobserver): Test whether jobserver
	is active from GCC driver. That will prevent situation where
	GCC is invoked from a LD plugin and the linker already uses
	file descriptors suggested by make.  That leads to a wrong
	detection.
	* gcc.h (driver): Add detect_jobserver.
	* lto-wrapper.c (jobserver_active_p): Simplify sscanf by
	not scanning for --jobserver-auth prefix.
---
 gcc/gcc.c         | 42 ++++++++++++++++++++++++++++++++++++++++++
 gcc/gcc.h         |  1 +
 gcc/lto-wrapper.c |  2 +-
 3 files changed, 44 insertions(+), 1 deletion(-)

diff --git a/gcc/gcc.c b/gcc/gcc.c
index a4323eb146e..18a07426290 100644
--- a/gcc/gcc.c
+++ b/gcc/gcc.c
@@ -8268,6 +8268,8 @@ driver::maybe_run_linker (const char *argv0) const
     {
       int tmp = execution_count;
 
+      detect_jobserver ();
+
       if (! have_c)
 	{
 #if HAVE_LTO_PLUGIN > 0
@@ -8357,6 +8359,46 @@ driver::final_actions () const
     }
 }
 
+/* Detect whether jobserver is active and working.  If not drop
+   --jobserver-auth from MAKEFLAGS.  */
+
+void
+driver::detect_jobserver () const
+{
+  /* Detect jobserver and drop it if it's not working.  */
+  const char *makeflags = env.get ("MAKEFLAGS");
+  if (makeflags != NULL)
+    {
+      const char *needle = "--jobserver-auth=";
+      const char *n = strstr (makeflags, needle);
+      if (n != NULL)
+	{
+	  int rfd = -1;
+	  int wfd = -1;
+
+	  bool jobserver
+	    = (sscanf (n + strlen (needle), "%d,%d", &rfd, &wfd) == 2
+	       && rfd > 0
+	       && wfd > 0
+	       && fcntl (rfd, F_GETFD) >= 0
+	       && fcntl (wfd, F_GETFD) >= 0);
+
+	  /* Drop the jobserver if it's not working now.  */
+	  if (!jobserver)
+	    {
+	      unsigned offset = n - makeflags;
+	      char *dup = xstrdup (makeflags);
+	      dup[offset] = '\0';
+
+	      const char *space = strchr (makeflags + offset, ' ');
+	      if (space != NULL)
+		strcpy (dup + offset, space);
+	      xputenv (concat ("MAKEFLAGS=", dup, NULL));
+	    }
+	}
+    }
+}
+
 /* Determine what the exit code of the driver should be.  */
 
 int
diff --git a/gcc/gcc.h b/gcc/gcc.h
index a0a1d94c6e6..dc77dba67fb 100644
--- a/gcc/gcc.h
+++ b/gcc/gcc.h
@@ -51,6 +51,7 @@ class driver
   void do_spec_on_infiles () const;
   void maybe_run_linker (const char *argv0) const;
   void final_actions () const;
+  void detect_jobserver () const;
   int get_exit_code () const;
 
  private:
diff --git a/gcc/lto-wrapper.c b/gcc/lto-wrapper.c
index 353187c6043..3414adedd26 100644
--- a/gcc/lto-wrapper.c
+++ b/gcc/lto-wrapper.c
@@ -1234,7 +1234,7 @@ jobserver_active_p (void)
   int rfd = -1;
   int wfd = -1;
 
-  return ((sscanf(n, "--jobserver-auth=%d,%d", &rfd, &wfd) == 2)
+  return (sscanf (n + strlen (needle), "%d,%d", &rfd, &wfd) == 2
 	  && rfd > 0
 	  && wfd > 0
 	  && fcntl (rfd, F_GETFD) >= 0
-- 
2.22.0