[PATCH] Fix -save-temp leaking lto files in /tmp

[Bernd Edlinger]

On 2/20/20 2:34 PM, Richard Biener wrote:
> On Thu, 20 Feb 2020, Bernd Edlinger wrote:
> 
>> Hi,
>>
>> this is the remaining issue that happens when -flto and -save-temps
>> is used together, it leaks several files in /tmp.
>>
>> I try to give more background to how these temp files are
>> created, and in all likelihood the leaking of these
>> files is wanted, and certainly very helpful for debugging
>> lto issues, that's for sure.  It is just not helpful
>> that they need to be looked up in the /tmp folder, even
>> if you want to debug something with lto.
>>
>> The short story is...
>>
>> They are created here:
>>
>>       if (parallel)
>>         {
>>           makefile = make_temp_file (".mk");
>>           mstream = fopen (makefile, "w");
>>
>> and here:
>>
>>       /* Note: we assume argv contains at least one element; this is
>>          checked above.  */
>>
>>       response_file = make_temp_file ("");
>>
>>       f = fopen (response_file, "w");
>>
>> And in a few other places as well, it depends a bit
>> if -o is used or not (i.e. linker_output != NULL or not).
>>
>> and not removed here:
>>
>>
>>   if (response_file && !save_temps)
>>     {
>>       unlink (response_file);
>>       response_file = NULL;
>>     }
>>
>> and here:
>>
>>           do_wait (new_argv[0], pex);
>>           maybe_unlink (makefile);
>>           makefile = NULL;
>>
>>
>> the code with the response_file is executed both in
>> lto-wrapper and collect2, but in collect2 only when
>> if is invoked from lto-wrapper, triggered by the @file
>> argument list.
>>
>> Therefore I figured that the best possible
>> solution is just let lto-wrapper create a temp-file
>> for those problem cases, and use TMPDIR to have
>> all make_temp_file that follow use that to folder to
>> place the those response files and other stuff in
>> there.
>>
>>
>> So that is what I split out from the previous patch,
>> which focused on collect2.
>>
>>
>> Bootstrapped and reg-tested on x86_64-pc-linux-gnu.
>> Is it OK for trunk?
> 
> I don't think this is an improvement.  The files still
> will be (correctly) retained and now in addition to that
> there's temporary directories piling up?
> 

Well, the temp. directory has a known name,
in case the command line is

gcc -save-temps -lto -o test test.c

there are those 4 in a *known* place, test.lto_tmpdir:

test.lto_tmpdir:
insgesamt 24
drwxrwxr-x 2 ed ed 4096 Feb 20 16:14 .
drwxrwxr-x 3 ed ed 4096 Feb 20 16:14 ..
-rw------- 1 ed ed   15 Feb 20 16:14 cc8FwPQ1
-rw------- 1 ed ed  237 Feb 20 16:14 cchROrdh
-rw------- 1 ed ed  197 Feb 20 16:14 cclbBAUp
-rw------- 1 ed ed    7 Feb 20 16:14 ccMlfh1g

plus
-rw-rw-r-- 1 ed ed   164 Feb 20 16:14 test.i
-rw-rw-r-- 1 ed ed    50 Feb 20 16:14 test.lto_wrapper_args
-rw-rw-r-- 1 ed ed    17 Feb 20 16:14 test.ltrans.out
-rw-rw-r-- 1 ed ed  1232 Feb 20 16:14 test.ltrans0.ltrans.o
-rw-rw-r-- 1 ed ed  2539 Feb 20 16:14 test.ltrans0.o
-rw-rw-r-- 1 ed ed   374 Feb 20 16:14 test.ltrans0.s
-rw-rw-r-- 1 ed ed    52 Feb 20 16:14 test.res
-rw-rw-r-- 1 ed ed  4267 Feb 20 16:14 test.s

if the command line is
gcc -save-temps -lto test.c

there are a few more but also in a *known* place, a.out.lto_tmpdir:

a.out.lto_tmpdir/
total 36
drwxrwxr-x 2 ed ed 4096 Feb 20 16:18 .
drwxrwxr-x 3 ed ed 4096 Feb 20 16:18 ..
-rw------- 1 ed ed    7 Feb 20 16:18 cc2hY8SH
-rw------- 1 ed ed  227 Feb 20 16:18 ccDafyit
-rw------- 1 ed ed  262 Feb 20 16:18 ccglzNAe
-rw------- 1 ed ed   36 Feb 20 16:18 ccnAU7NG
-rw------- 1 ed ed   38 Feb 20 16:18 ccoLFY2H.ltrans.out
-rw-rw-r-- 1 ed ed 1232 Feb 20 16:18 ccoLFY2H.ltrans0.ltrans.o
-rw-rw-r-- 1 ed ed 2560 Feb 20 16:18 ccoLFY2H.ltrans0.o

plus
-rw-rw-r--   1 ed ed    50 Feb 20 16:18 a.out.lto_wrapper_args
-rw-rw-r--   1 ed ed   160 Feb 20 16:18 test.i
-rw-rw-r--   1 ed ed  3104 Feb 20 16:18 test.o
-rw-rw-r--   1 ed ed    52 Feb 20 16:18 test.res
-rw-rw-r--   1 ed ed  4267 Feb 20 16:18 test.s


> A better improvement would be to selectively decide
> which files might not be needed to be preserved and/or
> give them names in the build directory in more cases.
> 

Ah, well, it just felt like that will probably need a
rather complicated patch which is probably not
worth it, since the are a lot of different
code pathes involved, (with -g or not, with -flto=jobserver
or not, flto_partition=none, or with offload or not, with -o
or not etc.) in two or more independent forked processes,
which will need to coordinate somehow, not to clobber each
other's tempfiles.


Bernd.


> Richard.
> 
>>
>> Thanks
>> Bernd.
>>
>>
>