RFA: implement pex_input_file and pex_input_pipe

Jim Blandy jimb@codesourcery.com
Tue Apr 11 06:22:00 GMT 2006


Here's the PEX input pipe patch, revised as suggested in our prior
conversation:
http://gcc.gnu.org/ml/gcc-patches/2006-03/msg00798.html

This patch requires the 'temp_file' patch posted before:
http://gcc.gnu.org/ml/gcc-patches/2006-04/msg00369.html

src/libiberty/ChangeLog:
2006-03-29  Jim Blandy  <jimb@codesourcery.com>

	* pex-common.c (pex_input_file, pex_input_pipe): New functions.
	(pex_init_common): Initialize obj->input_file.
	(pex_run): Close any file opened by pex_input_file.
	* pexecute.txh (pex_input_file, pex_input_pipe): New docs.
	* pex-common.h (struct pex_obj): New field input_file.
	(struct pex_funcs): New function ptr fdopenw.
	* pex-unix.c (pex_unix_fdopenw): New function.
	(funcs): List it as our fdopenw function.
	* pex-win32.c (pex_win32_fdopenw): New function.
	(funcs): List it as our fdopenw function.
	* pex-djgpp.c (funcs): Leave fdopenw null.
	* pex-msdos (funcs): Same.
	* functions.texi: Regenerated.

src/include/ChangeLog:
2006-04-10  Jim Blandy  <jimb@codesourcery.com>

	* libiberty.h (pex_input_file, pex_input_pipe): New declarations.

Index: src/libiberty/pexecute.txh
===================================================================
--- src.orig/libiberty/pexecute.txh	2006-04-10 23:09:00.676040000 -0700
+++ src/libiberty/pexecute.txh	2006-04-10 23:09:23.097163000 -0700
@@ -1,3 +1,4 @@
+@c -*- mode: texinfo -*-
 @deftypefn Extension {struct pex_obj *} pex_init (int @var{flags}, const char *@var{pname}, const char *@var{tempbase})
 
 Prepare to execute one or more programs, with standard output of each
@@ -129,6 +130,67 @@
 
 @end deftypefn
 
+@deftypefn Extension {FILE *} pex_input_file (struct pex_obj *@var{obj}, int @var{flags}, const char *@var{in_name})
+
+Return a stream for a temporary file to pass to the first program in
+the pipeline as input.
+
+The name of the input file is chosen according to the same rules
+@code{pex_run} uses to choose output file names, based on
+@var{in_name}, @var{obj} and the @code{PEX_SUFFIX} bit in @var{flags}.
+
+Don't call @code{fclose} on the returned stream; the first call to
+@code{pex_run} closes it automatically.
+
+If @var{flags} includes @code{PEX_BINARY_OUTPUT}, open the stream in
+binary mode; otherwise, open it in the default mode.  Including
+@code{PEX_BINARY_OUTPUT} in @var{flags} has no effect on Unix.
+@end deftypefn
+
+@deftypefn Extension {FILE *} pex_input_pipe (struct pex_obj *@var{obj}, int @var{binary})
+
+Return a stream @var{fp} for a pipe connected to the standard input of
+the first program in the pipeline; @var{fp} is opened for writing.
+You must have passed @code{PEX_USE_PIPES} to the @code{pex_init} call
+that returned @var{obj}.
+
+You must close @var{fp} using @code{fclose} yourself when you have
+finished writing data to the pipeline.
+
+The file descriptor underlying @var{fp} is marked not to be inherited
+by child processes.
+
+On systems that do not support pipes, this function returns
+@code{NULL}, and sets @code{errno} to @code{EINVAL}.  If you would
+like to write code that is portable to all systems the @code{pex}
+functions support, consider using @code{pex_input_file} instead.
+
+There are two opportunities for deadlock using
+@code{pex_input_pipe}:
+
+@itemize @bullet
+@item
+Most systems' pipes can buffer only a fixed amount of data; a process
+that writes to a full pipe blocks.  Thus, if you write to @file{fp}
+before starting the first process, you run the risk of blocking when
+there is no child process yet to read the data and allow you to
+continue.  @code{pex_input_pipe} makes no promises about the
+size of the pipe's buffer, so if you need to write any data at all
+before starting the first process in the pipeline, consider using
+@code{pex_input_file} instead.
+
+@item
+Using @code{pex_input_pipe} and @code{pex_read_output} together
+may also cause deadlock.  If the output pipe fills up, so that each
+program in the pipeline is waiting for the next to read more data, and
+you fill the input pipe by writing more data to @var{fp}, then there
+is no way to make progress: the only process that could read data from
+the output pipe is you, but you are blocked on the input pipe.
+
+@end itemize
+
+@end deftypefn
+
 @deftypefn Extension {FILE *} pex_read_output (struct pex_obj *@var{obj}, int @var{binary})
 
 Returns a @code{FILE} pointer which may be used to read the standard
Index: src/libiberty/pex-common.c
===================================================================
--- src.orig/libiberty/pex-common.c	2006-04-10 23:09:18.595022000 -0700
+++ src/libiberty/pex-common.c	2006-04-10 23:17:04.962737000 -0700
@@ -67,6 +67,7 @@
   obj->status = NULL;
   obj->time = NULL;
   obj->number_waited = 0;
+  obj->input_file = NULL;
   obj->read_output = NULL;
   obj->remove_count = 0;
   obj->remove = NULL;
@@ -161,6 +162,17 @@
   outname = (char *) orig_outname;
   outname_allocated = 0;
 
+  /* If the user called pex_input_file, close the file now.  */
+  if (obj->input_file)
+    {
+      if (fclose (obj->input_file) == EOF)
+        {
+          errmsg = "closing pipeline input file";
+          goto error_exit;
+        }
+      obj->input_file = NULL;
+    }
+
   /* Set IN.  */
 
   if (obj->next_input_name != NULL)
@@ -307,6 +319,87 @@
   return errmsg;
 }
 
+/* Return a FILE pointer for a temporary file to fill with input for
+   the pipeline.  */
+FILE *
+pex_input_file (struct pex_obj *obj, int flags, const char *in_name)
+{
+  char *name = (char *) in_name;
+  FILE *f;
+
+  /* This must be called before the first pipeline stage is run, and
+     there must not have been any other input selected.  */
+  if (obj->count != 0
+      || (obj->next_input >= 0 && obj->next_input != STDIN_FILE_NO)
+      || obj->next_input_name)
+    {
+      errno = EINVAL;
+      return NULL;
+    }
+
+  name = temp_file (obj, flags, name);
+  if (! name)
+    return NULL;
+
+  f = fopen (name, (flags & PEX_BINARY_OUTPUT) ? "wb" : "w");
+  if (! f)
+    {
+      free (name);
+      return NULL;
+    }
+
+  obj->input_file = f;
+  obj->next_input_name = name;
+  obj->next_input_name_allocated = (name != in_name);
+
+  return f;
+}
+
+/* Return a stream for a pipe connected to the standard input of the
+   first stage of the pipeline.  */
+FILE *
+pex_input_pipe (struct pex_obj *obj, int binary)
+{
+  int p[2];
+  FILE *f;
+
+  /* You must call pex_input_pipe before the first pex_run or pex_one.  */
+  if (obj->count > 0)
+    goto usage_error;
+
+  /* You must be using pipes.  Implementations that don't support
+     pipes clear this flag before calling pex_init_common.  */
+  if (! (obj->flags & PEX_USE_PIPES))
+    goto usage_error;
+
+  /* If we have somehow already selected other input, that's a
+     mistake.  */
+  if ((obj->next_input >= 0 && obj->next_input != STDIN_FILE_NO)
+      || obj->next_input_name)
+    goto usage_error;
+
+  if (obj->funcs->pipe (obj, p, binary != 0) < 0)
+    return NULL;
+
+  f = obj->funcs->fdopenw (obj, p[WRITE_PORT], binary != 0);
+  if (! f)
+    {
+      int saved_errno = errno;
+      obj->funcs->close (obj, p[READ_PORT]);
+      obj->funcs->close (obj, p[WRITE_PORT]);
+      errno = saved_errno;
+      return NULL;
+    }
+
+  obj->next_input = p[READ_PORT];
+
+  return f;
+
+ usage_error:
+  errno = EINVAL;
+  return NULL;
+}
+
 /* Return a FILE pointer for the output of the last program
    executed.  */
 
Index: src/libiberty/pex-common.h
===================================================================
--- src.orig/libiberty/pex-common.h	2006-04-10 23:09:00.734249000 -0700
+++ src/libiberty/pex-common.h	2006-04-10 23:09:23.115911000 -0700
@@ -69,6 +69,8 @@
   struct pex_time *time;
   /* Number of children we have already waited for.  */
   int number_waited;
+  /* FILE created by pex_input_file.  */
+  FILE *input_file;
   /* FILE created by pex_read_output.  */
   FILE *read_output;
   /* Number of temporary files to remove.  */
@@ -121,6 +123,11 @@
      PEX_USE_PIPES is set).  If BINARY is non-zero, open in binary
      mode.  Return pointer on success, NULL on error.  */
   FILE * (*fdopenr) (struct pex_obj *, int /* fd */, int /* binary */);
+  /* Get a FILE pointer to write to the file descriptor FD (only
+     called if PEX_USE_PIPES is set).  If BINARY is non-zero, open in
+     binary mode.  Arrange for FD not to be inherited by the child
+     processes.  Return pointer on success, NULL on error.  */
+  FILE * (*fdopenw) (struct pex_obj *, int /* fd */, int /* binary */);
   /* Free any system dependent data associated with OBJ.  May be
      NULL if there is nothing to do.  */
   void (*cleanup) (struct pex_obj *);
Index: src/libiberty/pex-djgpp.c
===================================================================
--- src.orig/libiberty/pex-djgpp.c	2006-04-10 23:09:00.764737000 -0700
+++ src/libiberty/pex-djgpp.c	2006-04-10 23:09:23.123160000 -0700
@@ -62,6 +62,7 @@
   pex_djgpp_wait,
   NULL, /* pipe */
   NULL, /* fdopenr */
+  NULL, /* fdopenw */
   NULL  /* cleanup */
 };
 
Index: src/libiberty/pex-msdos.c
===================================================================
--- src.orig/libiberty/pex-msdos.c	2006-04-10 23:09:00.777484000 -0700
+++ src/libiberty/pex-msdos.c	2006-04-10 23:09:23.132541000 -0700
@@ -73,6 +73,7 @@
   pex_msdos_wait,
   NULL, /* pipe */
   NULL, /* fdopenr */
+  NULL, /* fdopenw */
   pex_msdos_cleanup
 };
 
Index: src/libiberty/pex-unix.c
===================================================================
--- src.orig/libiberty/pex-unix.c	2006-04-10 23:09:00.799098000 -0700
+++ src/libiberty/pex-unix.c	2006-04-10 23:09:23.142337000 -0700
@@ -277,6 +277,7 @@
 			  int, const char **, int *);
 static int pex_unix_pipe (struct pex_obj *, int *, int);
 static FILE *pex_unix_fdopenr (struct pex_obj *, int, int);
+static FILE *pex_unix_fdopenw (struct pex_obj *, int, int);
 static void pex_unix_cleanup (struct pex_obj *);
 
 /* The list of functions we pass to the common routines.  */
@@ -290,6 +291,7 @@
   pex_unix_wait,
   pex_unix_pipe,
   pex_unix_fdopenr,
+  pex_unix_fdopenw,
   pex_unix_cleanup
 };
 
@@ -495,6 +497,15 @@
   return fdopen (fd, "r");
 }
 
+static FILE *
+pex_unix_fdopenw (struct pex_obj *obj ATTRIBUTE_UNUSED, int fd,
+		  int binary ATTRIBUTE_UNUSED)
+{
+  if (fcntl (fd, F_SETFD, FD_CLOEXEC) < 0)
+    return NULL;
+  return fdopen (fd, "w");
+}
+
 static void
 pex_unix_cleanup (struct pex_obj *obj ATTRIBUTE_UNUSED)
 {
Index: src/libiberty/pex-win32.c
===================================================================
--- src.orig/libiberty/pex-win32.c	2006-04-10 23:09:00.848182000 -0700
+++ src/libiberty/pex-win32.c	2006-04-10 23:09:23.156902000 -0700
@@ -83,6 +83,7 @@
 			   struct pex_time *, int, const char **, int *);
 static int pex_win32_pipe (struct pex_obj *, int *, int);
 static FILE *pex_win32_fdopenr (struct pex_obj *, int, int);
+static FILE *pex_win32_fdopenw (struct pex_obj *, int, int);
 
 /* The list of functions we pass to the common routines.  */
 
@@ -95,6 +96,7 @@
   pex_win32_wait,
   pex_win32_pipe,
   pex_win32_fdopenr,
+  pex_win32_fdopenw,
   NULL /* cleanup */
 };
 
@@ -766,6 +768,18 @@
   return fdopen (fd, binary ? "rb" : "r");
 }
 
+static FILE *
+pex_win32_fdopenw (struct pex_obj *obj ATTRIBUTE_UNUSED, int fd,
+		   int binary)
+{
+  HANDLE h = (HANDLE) _get_osfhandle (fd);
+  if (h == INVALID_HANDLE_VALUE)
+    return NULL;
+  if (! SetHandleInformation (h, HANDLE_FLAG_INHERIT, 0))
+    return NULL;
+  return fdopen (fd, binary ? "wb" : "w");
+}
+
 #ifdef MAIN
 #include <stdio.h>
 
Index: src/include/libiberty.h
===================================================================
--- src.orig/include/libiberty.h	2006-04-10 23:09:00.929645000 -0700
+++ src/include/libiberty.h	2006-04-10 23:16:47.623941000 -0700
@@ -448,6 +448,61 @@
 			    const char *outname, const char *errname,
 			    int *err);
 
+/* Return a stream for a temporary file to pass to the first program
+   in the pipeline as input.
+
+   The name of the input file is chosen according to the same rules
+   `pex_run' uses to choose output file names, based on IN_NAME, OBJ
+   and the `PEX_SUFFIX' bit in FLAGS.
+
+   Don't call `fclose' on the returned stream; the first call to
+   `pex_run' closes it automatically.
+
+   If FLAGS includes `PEX_BINARY_OUTPUT', open the stream in binary
+   mode; otherwise, open it in the default mode.  Including
+   `PEX_BINARY_OUTPUT' in FLAGS has no effect on Unix.  */
+
+extern FILE *pex_input_file (struct pex_obj *obj, int flags,
+                             const char *in_name);
+
+/* Return a stream FP for a pipe connected to the standard input of
+   the first program in the pipeline; FP is opened for writing.  You
+   must have passed `PEX_USE_PIPES' to the `pex_init' call that
+   returned OBJ.
+
+   You must close FP using `fclose' yourself when you have finished
+   writing data to the pipeline.
+
+   The file descriptor underlying FP is marked not to be inherited by
+   child processes.
+
+   On systems that do not support pipes, this function returns
+   `NULL', and sets `errno' to `EINVAL'.  If you would like to write
+   code that is portable to all systems the `pex' functions support,
+   consider using `pex_input_file' instead.
+
+   There are two opportunities for deadlock using `pex_input_pipe':
+
+      * Most systems' pipes can buffer only a fixed amount of data; a
+        process that writes to a full pipe blocks.  Thus, if you
+        write to `fp' before starting the first process, you run the
+        risk of blocking when there is no child process yet to read
+        the data and allow you to continue.  `pex_input_pipe' makes
+        no promises about the size of the pipe's buffer, so if you
+        need to write any data at all before starting the first
+        process in the pipeline, consider using `pex_input_file'
+        instead.
+
+      * Using `pex_input_pipe' and `pex_read_output' together may
+        also cause deadlock.  If the output pipe fills up, so that
+        each program in the pipeline is waiting for the next to read
+        more data, and you fill the input pipe by writing more data
+        to FP, then there is no way to make progress: the only
+        process that could read data from the output pipe is you, but
+        you are blocked on the input pipe.  */
+
+extern FILE *pex_input_pipe (struct pex_obj *obj, int binary);
+
 /* Read the standard output of the last program to be executed.
    pex_run can not be called after this.  BINARY should be non-zero if
    the file should be opened in binary mode; this is ignored on Unix.
Index: src/libiberty/functions.texi
===================================================================
--- src.orig/libiberty/functions.texi	2006-04-10 23:09:00.861305000 -0700
+++ src/libiberty/functions.texi	2006-04-10 23:09:23.208643000 -0700
@@ -668,14 +668,14 @@
 
 @end deftypefn
 
-@c pexecute.txh:169
+@c pexecute.txh:231
 @deftypefn Extension void pex_free (struct pex_obj @var{obj})
 
 Clean up and free all data associated with @var{obj}.
 
 @end deftypefn
 
-@c pexecute.txh:144
+@c pexecute.txh:206
 @deftypefn Extension int pex_get_status (struct pex_obj *@var{obj}, int @var{count}, int *@var{vector})
 
 Returns the exit status of all programs run using @var{obj}.
@@ -685,7 +685,7 @@
 
 @end deftypefn
 
-@c pexecute.txh:153
+@c pexecute.txh:215
 @deftypefn Extension int pex_get_times (struct pex_obj *@var{obj}, int @var{count}, struct pex_time *@var{vector})
 
 Returns the process execution times of all programs run using
@@ -702,7 +702,7 @@
 
 @end deftypefn
 
-@c pexecute.txh:1
+@c pexecute.txh:2
 @deftypefn Extension {struct pex_obj *} pex_init (int @var{flags}, const char *@var{pname}, const char *@var{tempbase})
 
 Prepare to execute one or more programs, with standard output of each
@@ -734,7 +734,70 @@
 
 @end deftypefn
 
-@c pexecute.txh:175
+@c pexecute.txh:133
+@deftypefn Extension {FILE *} pex_input_file (struct pex_obj *@var{obj}, int @var{flags}, const char *@var{in_name})
+
+Return a stream for a temporary file to pass to the first program in
+the pipeline as input.
+
+The name of the input file is chosen according to the same rules
+@code{pex_run} uses to choose output file names, based on
+@var{in_name}, @var{obj} and the @code{PEX_SUFFIX} bit in @var{flags}.
+
+Don't call @code{fclose} on the returned stream; the first call to
+@code{pex_run} closes it automatically.
+
+If @var{flags} includes @code{PEX_BINARY_OUTPUT}, open the stream in
+binary mode; otherwise, open it in the default mode.  Including
+@code{PEX_BINARY_OUTPUT} in @var{flags} has no effect on Unix.
+@end deftypefn
+
+@c pexecute.txh:150
+@deftypefn Extension {FILE *} pex_input_pipe (struct pex_obj *@var{obj}, int @var{binary})
+
+Return a stream @var{fp} for a pipe connected to the standard input of
+the first program in the pipeline; @var{fp} is opened for writing.
+You must have passed @code{PEX_USE_PIPES} to the @code{pex_init} call
+that returned @var{obj}.
+
+You must close @var{fp} using @code{fclose} yourself when you have
+finished writing data to the pipeline.
+
+The file descriptor underlying @var{fp} is marked not to be inherited
+by child processes.
+
+On systems that do not support pipes, this function returns
+@code{NULL}, and sets @code{errno} to @code{EINVAL}.  If you would
+like to write code that is portable to all systems the @code{pex}
+functions support, consider using @code{pex_input_file} instead.
+
+There are two opportunities for deadlock using
+@code{pex_input_pipe}:
+
+@itemize @bullet
+@item
+Most systems' pipes can buffer only a fixed amount of data; a process
+that writes to a full pipe blocks.  Thus, if you write to @file{fp}
+before starting the first process, you run the risk of blocking when
+there is no child process yet to read the data and allow you to
+continue.  @code{pex_input_pipe} makes no promises about the
+size of the pipe's buffer, so if you need to write any data at all
+before starting the first process in the pipeline, consider using
+@code{pex_input_file} instead.
+
+@item
+Using @code{pex_input_pipe} and @code{pex_read_output} together
+may also cause deadlock.  If the output pipe fills up, so that each
+program in the pipeline is waiting for the next to read more data, and
+you fill the input pipe by writing more data to @var{fp}, then there
+is no way to make progress: the only process that could read data from
+the output pipe is you, but you are blocked on the input pipe.
+
+@end itemize
+
+@end deftypefn
+
+@c pexecute.txh:237
 @deftypefn Extension {const char *} pex_one (int @var{flags}, const char *@var{executable}, char * const *@var{argv}, const char *@var{pname}, const char *@var{outname}, const char *@var{errname}, int *@var{status}, int *@var{err})
 
 An interface to permit the easy execution of a
@@ -747,7 +810,7 @@
 
 @end deftypefn
 
-@c pexecute.txh:132
+@c pexecute.txh:194
 @deftypefn Extension {FILE *} pex_read_output (struct pex_obj *@var{obj}, int @var{binary})
 
 Returns a @code{FILE} pointer which may be used to read the standard
@@ -760,7 +823,7 @@
 
 @end deftypefn
 
-@c pexecute.txh:32
+@c pexecute.txh:33
 @deftypefn Extension {const char *} pex_run (struct pex_obj *@var{obj}, int @var{flags}, const char *@var{executable}, char * const *@var{argv}, const char *@var{outname}, const char *@var{errname}, int *@var{err})
 
 Execute one program in a pipeline.  On success this returns
@@ -861,7 +924,7 @@
 
 @end deftypefn
 
-@c pexecute.txh:187
+@c pexecute.txh:249
 @deftypefn Extension int pexecute (const char *@var{program}, char * const *@var{argv}, const char *@var{this_pname}, const char *@var{temp_base}, char **@var{errmsg_fmt}, char **@var{errmsg_arg}, int flags)
 
 This is the old interface to execute one or more programs.  It is
@@ -889,7 +952,7 @@
 
 @end deftypefn
 
-@c pexecute.txh:195
+@c pexecute.txh:257
 @deftypefn Extension int pwait (int @var{pid}, int *@var{status}, int @var{flags})
 
 Another part of the old execution interface.



More information about the Gcc-patches mailing list