This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: Skipping assembler when producing slim LTO files
- From: Richard Biener <richard dot guenther at gmail dot com>
- To: Jan Hubicka <hubicka at ucw dot cz>
- Cc: GCC Development <gcc at gcc dot gnu dot org>, Ian Lance Taylor <ian at airs dot com>
- Date: Wed, 24 Sep 2014 10:11:10 +0200
- Subject: Re: Skipping assembler when producing slim LTO files
- Authentication-results: sourceware.org; auth=none
- References: <20140924054651 dot GB5371 at kam dot mff dot cuni dot cz>
On Wed, Sep 24, 2014 at 7:46 AM, Jan Hubicka <hubicka@ucw.cz> wrote:
>
> Hi,
> This patch is something I was playing around with assistance of Ian Taylor.
> It seems I need bit more help though :)
>
> It adds support for direct output of SLIM LTO files to the compiler binary.
> It works as proof of concept, but there are two key parts missing
> 1) extension of libiberty's simple file to handle output symbols into COMMON.
> This is needed to output __gnu_lto_v1 and __gnu_lto_slim
> Search for TODO in the patch bellow.
> 2) Support in driver to properly execute *1 binary.
>
> I also disabled outputting ident directive, but I think that one may not be necessary
> because the files are identified by the gnu_lto_v1 symbols. We could add it later.
>
> Currently the path bypassing asm stage can be tested as follows:
>
> jan@linux-ujxe:~/trunk/build/gcc> cat a.c
> main ()
> {
> printf ("Hello world\n");
> }
> jan@linux-ujxe:~/trunk/build/gcc> ./xgcc -B ./ -O3 a.c -flto -S -fbypass-asm=crtbegin.o -o a.o
> jan@linux-ujxe:~/trunk/build/gcc> ./xgcc -B ./ -O2 a.o -flto
> jan@linux-ujxe:~/trunk/build/gcc> ./a.out
> Hello world
>
> The implementation is pretty straighforward except for -fbypass-asm requiring
> one existing OBJ file to fetch target's file attributes from. This is
> definitly not optimal, but libiberty currently can't build output files from
> scratch. As Ian suggested, I plan to simply arrange the driver to pass crtbegin
> around at least to start with. We may want to bypass this later and storing
> proper attributes into the binary.
>
> Ian, would you be so kind and implement ability to output those two symbols
> into lto-object-simple? I think we can start with ELF only support.
>
> The large chunk just moves lto-object around with very small changes in it, so the
> patch is fairly easy.
>
> I did just quick benchmark with unoptimized cc1 binary compiling the file above.
> For 1000 invocations with bypass I get:
>
> real 0m14.186s
> user 0m10.957s
> sys 0m2.424s
>
> While the default path gets:
>
> real 0m21.913s
> user 0m13.856s
> sys 0m5.705s
>
> With OpenSUSE 13.1 default GCC 4.8.3 build:
>
> real 0m15.160s
> user 0m8.481s
> sys 0m5.159s
>
> (the difference here is most likely optimizer WRT unoptimized binary, perf shows
> contains_struct_check quite top, so startup overhead still dominates)
>
> And with clang-3.4:
>
> real 0m30.097s
> user 0m22.012s
> sys 0m6.649s
>
> That is fairly nice speedup IMO. With optimized build the difference should
> be more visible because CC1 startup issues will become less important.
> I definitely see ASM file overhead as mesaurable issue with real world benchmarks
> (libreoffice build). Clearly we produce several GBs of object file going through
> crappy and bloated text encoding just for sake of doing it.
Shouldn't -fbypass-asm be simply "mangled" by the driver? That is,
the user simply specifies -fbypass-asm and via spec magic the driver
substitutes this with -fbypass-asm=crtbegin.o? That way at least
the user interface should be stable (as we're supposedly removing
the requirement for that existing object file at some point).
Btw, with early debug info we also need to store dwarf somewhere.
Either we drop the support for fat LTO objects and thus can store
the dwarf alongside the GIMPLE IL and simply link with these
files at the end or we need to support a separate set of files to
store the DWARF. If we need separate files then why not store
the GIMPLE IL data into separate objects in the first place and
output a reference to it into the main object file? That way we
don't need any special "attributes" - the linker plugin simply
opens the main object file, extracts the reference to the IL file
and passes that along.
Btw, the patch is very hard to read as it moves (and modifies?) files
at the same time. What's this magic "file attributes" we need?
Thanks,
Richard.
> Honza
>
> Index: Makefile.in
> ===================================================================
> --- Makefile.in (revision 215518)
> +++ Makefile.in (working copy)
> @@ -1300,6 +1300,7 @@
> lto-section-out.o \
> lto-opts.o \
> lto-compress.o \
> + lto-object.o \
> mcf.o \
> mode-switching.o \
> modulo-sched.o \
> Index: common.opt
> ===================================================================
> --- common.opt (revision 215518)
> +++ common.opt (working copy)
> @@ -923,6 +923,9 @@
> Common Report Var(flag_btr_bb_exclusive) Optimization
> Restrict target load migration not to re-use registers in any basic block
>
> +fbypass-asm=
> +Common Joined Var(flag_bypass_asm)
> +
> fcall-saved-
> Common Joined RejectNegative Var(common_deferred_options) Defer
> -fcall-saved-<register> Mark <register> as being preserved across functions
> Index: langhooks.c
> ===================================================================
> --- langhooks.c (revision 215518)
> +++ langhooks.c (working copy)
> @@ -40,6 +40,10 @@
> #include "cgraph.h"
> #include "timevar.h"
> #include "output.h"
> +#include "tree-ssa-alias.h"
> +#include "gimple-expr.h"
> +#include "gimple.h"
> +#include "lto-streamer.h"
>
> /* Do nothing; in many cases the default hook. */
>
> @@ -653,6 +657,19 @@
> {
> section *section;
>
> + if (flag_bypass_asm)
> + {
> + static int initialized = false;
> + if (!initialized)
> + {
> + gcc_assert (asm_out_file == NULL);
> + lto_set_current_out_file (lto_obj_file_open (asm_file_name, true));
> + initialized = true;
> + }
> + lto_obj_begin_section (name);
> + return;
> + }
> +
> /* Save the old section so we can restore it in lto_end_asm_section. */
> gcc_assert (!saved_section);
> saved_section = in_section;
> @@ -669,8 +686,13 @@
> implementation just calls assemble_string. */
>
> void
> -lhd_append_data (const void *data, size_t len, void *)
> +lhd_append_data (const void *data, size_t len, void *v)
> {
> + if (flag_bypass_asm)
> + {
> + lto_obj_append_data (data, len, v);
> + return;
> + }
> if (data)
> assemble_string ((const char *)data, len);
> }
> @@ -683,6 +705,11 @@
> void
> lhd_end_section (void)
> {
> + if (flag_bypass_asm)
> + {
> + lto_obj_end_section ();
> + return;
> + }
> if (saved_section)
> {
> switch_to_section (saved_section);
> Index: lto/Make-lang.in
> ===================================================================
> --- lto/Make-lang.in (revision 215518)
> +++ lto/Make-lang.in (working copy)
> @@ -22,7 +22,7 @@
> # The name of the LTO compiler.
> LTO_EXE = lto1$(exeext)
> # The LTO-specific object files inclued in $(LTO_EXE).
> -LTO_OBJS = lto/lto-lang.o lto/lto.o lto/lto-object.o attribs.o lto/lto-partition.o lto/lto-symtab.o
> +LTO_OBJS = lto/lto-lang.o lto/lto.o attribs.o lto/lto-partition.o lto/lto-symtab.o
> lto_OBJS = $(LTO_OBJS)
>
> # Rules
> Index: lto/lto-object.c
> ===================================================================
> --- lto/lto-object.c (revision 215518)
> +++ lto/lto-object.c (working copy)
> @@ -1,387 +0,0 @@
> -/* LTO routines to use object files.
> - Copyright (C) 2010-2014 Free Software Foundation, Inc.
> - Written by Ian Lance Taylor, Google.
> -
> -This file is part of GCC.
> -
> -GCC is free software; you can redistribute it and/or modify it under
> -the terms of the GNU General Public License as published by the Free
> -Software Foundation; either version 3, or (at your option) any later
> -version.
> -
> -GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> -WARRANTY; without even the implied warranty of MERCHANTABILITY or
> -FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> -for more details.
> -
> -You should have received a copy of the GNU General Public License
> -along with GCC; see the file COPYING3. If not see
> -<http://www.gnu.org/licenses/>. */
> -
> -#include "config.h"
> -#include "system.h"
> -#include "coretypes.h"
> -#include "tree.h"
> -#include "basic-block.h"
> -#include "tree-ssa-alias.h"
> -#include "internal-fn.h"
> -#include "gimple-expr.h"
> -#include "is-a.h"
> -#include "gimple.h"
> -#include "diagnostic-core.h"
> -#include "lto.h"
> -#include "tm.h"
> -#include "lto-streamer.h"
> -#include "lto-section-names.h"
> -#include "simple-object.h"
> -
> -/* An LTO file wrapped around an simple_object. */
> -
> -struct lto_simple_object
> -{
> - /* The base information. */
> - lto_file base;
> -
> - /* The system file descriptor. */
> - int fd;
> -
> - /* The simple_object if we are reading the file. */
> - simple_object_read *sobj_r;
> -
> - /* The simple_object if we are writing the file. */
> - simple_object_write *sobj_w;
> -
> - /* The currently active section. */
> - simple_object_write_section *section;
> -};
> -
> -/* Saved simple_object attributes. FIXME: Once set, this is never
> - cleared. */
> -
> -static simple_object_attributes *saved_attributes;
> -
> -/* Initialize FILE, an LTO file object for FILENAME. */
> -
> -static void
> -lto_file_init (lto_file *file, const char *filename, off_t offset)
> -{
> - file->filename = filename;
> - file->offset = offset;
> -}
> -
> -/* Open the file FILENAME. It WRITABLE is true, the file is opened
> - for write and, if necessary, created. Otherwise, the file is
> - opened for reading. Returns the opened file. */
> -
> -lto_file *
> -lto_obj_file_open (const char *filename, bool writable)
> -{
> - const char *offset_p;
> - long loffset;
> - int consumed;
> - char *fname;
> - off_t offset;
> - struct lto_simple_object *lo;
> - const char *errmsg;
> - int err;
> -
> - offset_p = strrchr (filename, '@');
> - if (offset_p != NULL
> - && offset_p != filename
> - && sscanf (offset_p, "@%li%n", &loffset, &consumed) >= 1
> - && strlen (offset_p) == (unsigned int) consumed)
> - {
> - fname = XNEWVEC (char, offset_p - filename + 1);
> - memcpy (fname, filename, offset_p - filename);
> - fname[offset_p - filename] = '\0';
> - offset = (off_t) loffset;
> - }
> - else
> - {
> - fname = xstrdup (filename);
> - offset = 0;
> - }
> -
> - lo = XCNEW (struct lto_simple_object);
> - lto_file_init ((lto_file *) lo, fname, offset);
> -
> - lo->fd = open (fname,
> - (writable
> - ? O_WRONLY | O_CREAT | O_BINARY
> - : O_RDONLY | O_BINARY),
> - 0666);
> - if (lo->fd == -1)
> - {
> - error ("open %s failed: %s", fname, xstrerror (errno));
> - goto fail;
> - }
> -
> - if (!writable)
> - {
> - simple_object_attributes *attrs;
> -
> - lo->sobj_r = simple_object_start_read (lo->fd, offset, LTO_SEGMENT_NAME,
> - &errmsg, &err);
> - if (lo->sobj_r == NULL)
> - goto fail_errmsg;
> -
> - attrs = simple_object_fetch_attributes (lo->sobj_r, &errmsg, &err);
> - if (attrs == NULL)
> - goto fail_errmsg;
> -
> - if (saved_attributes == NULL)
> - saved_attributes = attrs;
> - else
> - {
> - errmsg = simple_object_attributes_merge (saved_attributes, attrs,
> - &err);
> - if (errmsg != NULL)
> - {
> - free (attrs);
> - goto fail_errmsg;
> - }
> - }
> - }
> - else
> - {
> - gcc_assert (saved_attributes != NULL);
> - lo->sobj_w = simple_object_start_write (saved_attributes,
> - LTO_SEGMENT_NAME,
> - &errmsg, &err);
> - if (lo->sobj_w == NULL)
> - goto fail_errmsg;
> - }
> -
> - return &lo->base;
> -
> - fail_errmsg:
> - if (err == 0)
> - error ("%s: %s", fname, errmsg);
> - else
> - error ("%s: %s: %s", fname, errmsg, xstrerror (err));
> -
> - fail:
> - if (lo->fd != -1)
> - lto_obj_file_close ((lto_file *) lo);
> - free (lo);
> - return NULL;
> -}
> -
> -
> -/* Close FILE. If FILE was opened for writing, it is written out
> - now. */
> -
> -void
> -lto_obj_file_close (lto_file *file)
> -{
> - struct lto_simple_object *lo = (struct lto_simple_object *) file;
> -
> - if (lo->sobj_r != NULL)
> - simple_object_release_read (lo->sobj_r);
> - else if (lo->sobj_w != NULL)
> - {
> - const char *errmsg;
> - int err;
> -
> - gcc_assert (lo->base.offset == 0);
> -
> - errmsg = simple_object_write_to_file (lo->sobj_w, lo->fd, &err);
> - if (errmsg != NULL)
> - {
> - if (err == 0)
> - fatal_error ("%s", errmsg);
> - else
> - fatal_error ("%s: %s", errmsg, xstrerror (err));
> - }
> -
> - simple_object_release_write (lo->sobj_w);
> - }
> -
> - if (lo->fd != -1)
> - {
> - if (close (lo->fd) < 0)
> - fatal_error ("close: %s", xstrerror (errno));
> - }
> -}
> -
> -/* This is passed to lto_obj_add_section. */
> -
> -struct lto_obj_add_section_data
> -{
> - /* The hash table of sections. */
> - htab_t section_hash_table;
> - /* The offset of this file. */
> - off_t base_offset;
> - /* List in linker order */
> - struct lto_section_list *list;
> -};
> -
> -/* This is called for each section in the file. */
> -
> -static int
> -lto_obj_add_section (void *data, const char *name, off_t offset,
> - off_t length)
> -{
> - struct lto_obj_add_section_data *loasd =
> - (struct lto_obj_add_section_data *) data;
> - htab_t section_hash_table = (htab_t) loasd->section_hash_table;
> - char *new_name;
> - struct lto_section_slot s_slot;
> - void **slot;
> - struct lto_section_list *list = loasd->list;
> -
> - if (strncmp (name, LTO_SECTION_NAME_PREFIX,
> - strlen (LTO_SECTION_NAME_PREFIX)) != 0)
> - return 1;
> -
> - new_name = xstrdup (name);
> - s_slot.name = new_name;
> - slot = htab_find_slot (section_hash_table, &s_slot, INSERT);
> - if (*slot == NULL)
> - {
> - struct lto_section_slot *new_slot = XCNEW (struct lto_section_slot);
> -
> - new_slot->name = new_name;
> - new_slot->start = loasd->base_offset + offset;
> - new_slot->len = length;
> - *slot = new_slot;
> -
> - if (list != NULL)
> - {
> - if (!list->first)
> - list->first = new_slot;
> - if (list->last)
> - list->last->next = new_slot;
> - list->last = new_slot;
> - }
> - }
> - else
> - {
> - error ("two or more sections for %s", new_name);
> - return 0;
> - }
> -
> - return 1;
> -}
> -
> -/* Build a hash table whose key is the section name and whose data is
> - the start and size of each section in the .o file. */
> -
> -htab_t
> -lto_obj_build_section_table (lto_file *lto_file, struct lto_section_list *list)
> -{
> - struct lto_simple_object *lo = (struct lto_simple_object *) lto_file;
> - htab_t section_hash_table;
> - struct lto_obj_add_section_data loasd;
> - const char *errmsg;
> - int err;
> -
> - section_hash_table = lto_obj_create_section_hash_table ();
> -
> - gcc_assert (lo->sobj_r != NULL && lo->sobj_w == NULL);
> - loasd.section_hash_table = section_hash_table;
> - loasd.base_offset = lo->base.offset;
> - loasd.list = list;
> - errmsg = simple_object_find_sections (lo->sobj_r, lto_obj_add_section,
> - &loasd, &err);
> - if (errmsg != NULL)
> - {
> - if (err == 0)
> - error ("%s", errmsg);
> - else
> - error ("%s: %s", errmsg, xstrerror (err));
> - htab_delete (section_hash_table);
> - return NULL;
> - }
> -
> - return section_hash_table;
> -}
> -
> -/* The current output file. */
> -
> -static lto_file *current_out_file;
> -
> -/* Set the current output file. Return the old one. */
> -
> -lto_file *
> -lto_set_current_out_file (lto_file *file)
> -{
> - lto_file *old_file;
> -
> - old_file = current_out_file;
> - current_out_file = file;
> - return old_file;
> -}
> -
> -/* Return the current output file. */
> -
> -lto_file *
> -lto_get_current_out_file (void)
> -{
> - return current_out_file;
> -}
> -
> -/* Begin writing a new section named NAME in the current output
> - file. */
> -
> -void
> -lto_obj_begin_section (const char *name)
> -{
> - struct lto_simple_object *lo;
> - int align;
> - const char *errmsg;
> - int err;
> -
> - lo = (struct lto_simple_object *) current_out_file;
> - gcc_assert (lo != NULL
> - && lo->sobj_r == NULL
> - && lo->sobj_w != NULL
> - && lo->section == NULL);
> -
> - align = exact_log2 (POINTER_SIZE / BITS_PER_UNIT);
> - lo->section = simple_object_write_create_section (lo->sobj_w, name, align,
> - &errmsg, &err);
> - if (lo->section == NULL)
> - {
> - if (err == 0)
> - fatal_error ("%s", errmsg);
> - else
> - fatal_error ("%s: %s", errmsg, xstrerror (errno));
> - }
> -}
> -
> -/* Add data to a section. BLOCK is a pointer to memory containing
> - DATA. */
> -
> -void
> -lto_obj_append_data (const void *data, size_t len, void *)
> -{
> - struct lto_simple_object *lo;
> - const char *errmsg;
> - int err;
> -
> - lo = (struct lto_simple_object *) current_out_file;
> - gcc_assert (lo != NULL && lo->section != NULL);
> -
> - errmsg = simple_object_write_add_data (lo->sobj_w, lo->section, data, len,
> - 1, &err);
> - if (errmsg != NULL)
> - {
> - if (err == 0)
> - fatal_error ("%s", errmsg);
> - else
> - fatal_error ("%s: %s", errmsg, xstrerror (errno));
> - }
> -}
> -
> -/* Stop writing to the current output section. */
> -
> -void
> -lto_obj_end_section (void)
> -{
> - struct lto_simple_object *lo;
> -
> - lo = (struct lto_simple_object *) current_out_file;
> - gcc_assert (lo != NULL && lo->section != NULL);
> - lo->section = NULL;
> -}
> Index: lto/lto.c
> ===================================================================
> --- lto/lto.c (revision 215518)
> +++ lto/lto.c (working copy)
> @@ -2246,7 +2246,8 @@
> struct lto_section_list section_list;
>
> memset (§ion_list, 0, sizeof (struct lto_section_list));
> - section_hash_table = lto_obj_build_section_table (file, §ion_list);
> + section_hash_table = lto_obj_create_section_hash_table ();
> + section_hash_table = lto_obj_build_section_table (file, §ion_list, section_hash_table);
>
> /* Find all sub modules in the object and put their sections into new hash
> tables in a splay tree. */
> Index: lto/lto.h
> ===================================================================
> --- lto/lto.h (revision 215518)
> +++ lto/lto.h (working copy)
> @@ -24,14 +24,6 @@
> #include "hashtab.h"
> #include "vec.h"
>
> -/* A file. */
> -typedef struct lto_file_struct
> -{
> - /* The name of the file. */
> - const char *filename;
> - /* The offset for the object inside an ar archive file (or zero). */
> - off_t offset;
> -} lto_file;
>
> /* In lto-lang.c */
> extern const char *resolution_file_name;
> @@ -41,32 +33,6 @@
> extern void lto_main (void);
> extern void lto_read_all_file_options (void);
>
> -/* In lto-elf.c or lto-coff.c */
> -extern lto_file *lto_obj_file_open (const char *filename, bool writable);
> -extern void lto_obj_file_close (lto_file *file);
> -struct lto_section_list;
> -extern htab_t lto_obj_build_section_table (lto_file *file, struct lto_section_list *list);
> -extern htab_t lto_obj_create_section_hash_table (void);
> -extern void lto_obj_begin_section (const char *name);
> -extern void lto_obj_append_data (const void *data, size_t len, void *block);
> -extern void lto_obj_end_section (void);
> -extern lto_file *lto_set_current_out_file (lto_file *file);
> -extern lto_file *lto_get_current_out_file (void);
>
> -/* Hash table entry to hold the start offset and length of an LTO
> - section in a .o file. */
> -struct lto_section_slot
> -{
> - const char *name;
> - intptr_t start;
> - size_t len;
> - struct lto_section_slot *next;
> -};
>
> -/* A list of section slots */
> -struct lto_section_list
> -{
> - struct lto_section_slot *first, *last;
> -};
> -
> #endif /* LTO_H */
> Index: lto-object.c
> ===================================================================
> --- lto-object.c (revision 0)
> +++ lto-object.c (working copy)
> @@ -0,0 +1,391 @@
> +/* LTO routines to use object files.
> + Copyright (C) 2010-2014 Free Software Foundation, Inc.
> + Written by Ian Lance Taylor, Google.
> +
> +This file is part of GCC.
> +
> +GCC is free software; you can redistribute it and/or modify it under
> +the terms of the GNU General Public License as published by the Free
> +Software Foundation; either version 3, or (at your option) any later
> +version.
> +
> +GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> +WARRANTY; without even the implied warranty of MERCHANTABILITY or
> +FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
> +for more details.
> +
> +You should have received a copy of the GNU General Public License
> +along with GCC; see the file COPYING3. If not see
> +<http://www.gnu.org/licenses/>. */
> +
> +#include "config.h"
> +#include "system.h"
> +#include "coretypes.h"
> +#include "tree.h"
> +#include "basic-block.h"
> +#include "tree-ssa-alias.h"
> +#include "internal-fn.h"
> +#include "gimple-expr.h"
> +#include "is-a.h"
> +#include "gimple.h"
> +#include "diagnostic-core.h"
> +#include "tm.h"
> +#include "lto-streamer.h"
> +#include "lto-section-names.h"
> +#include "simple-object.h"
> +
> +/* An LTO file wrapped around an simple_object. */
> +
> +struct lto_simple_object
> +{
> + /* The base information. */
> + lto_file base;
> +
> + /* The system file descriptor. */
> + int fd;
> +
> + /* The simple_object if we are reading the file. */
> + simple_object_read *sobj_r;
> +
> + /* The simple_object if we are writing the file. */
> + simple_object_write *sobj_w;
> +
> + /* The currently active section. */
> + simple_object_write_section *section;
> +};
> +
> +/* Saved simple_object attributes. FIXME: Once set, this is never
> + cleared. */
> +
> +static simple_object_attributes *saved_attributes;
> +
> +/* Initialize FILE, an LTO file object for FILENAME. */
> +
> +static void
> +lto_file_init (lto_file *file, const char *filename, off_t offset)
> +{
> + file->filename = filename;
> + file->offset = offset;
> +}
> +
> +/* Open the file FILENAME. It WRITABLE is true, the file is opened
> + for write and, if necessary, created. Otherwise, the file is
> + opened for reading. Returns the opened file. */
> +
> +lto_file *
> +lto_obj_file_open (const char *filename, bool writable)
> +{
> + const char *offset_p;
> + long loffset;
> + int consumed;
> + char *fname;
> + off_t offset;
> + struct lto_simple_object *lo;
> + const char *errmsg;
> + int err;
> +
> + offset_p = strrchr (filename, '@');
> + if (offset_p != NULL
> + && offset_p != filename
> + && sscanf (offset_p, "@%li%n", &loffset, &consumed) >= 1
> + && strlen (offset_p) == (unsigned int) consumed)
> + {
> + fname = XNEWVEC (char, offset_p - filename + 1);
> + memcpy (fname, filename, offset_p - filename);
> + fname[offset_p - filename] = '\0';
> + offset = (off_t) loffset;
> + }
> + else
> + {
> + fname = xstrdup (filename);
> + offset = 0;
> + }
> +
> + lo = XCNEW (struct lto_simple_object);
> + lto_file_init ((lto_file *) lo, fname, offset);
> +
> + lo->fd = open (fname,
> + (writable
> + ? O_WRONLY | O_CREAT | O_BINARY
> + : O_RDONLY | O_BINARY),
> + 0666);
> + if (lo->fd == -1)
> + {
> + error ("open %s failed: %s", fname, xstrerror (errno));
> + goto fail;
> + }
> +
> + if (!writable)
> + {
> + simple_object_attributes *attrs;
> +
> + lo->sobj_r = simple_object_start_read (lo->fd, offset, LTO_SEGMENT_NAME,
> + &errmsg, &err);
> + if (lo->sobj_r == NULL)
> + goto fail_errmsg;
> +
> + attrs = simple_object_fetch_attributes (lo->sobj_r, &errmsg, &err);
> + if (attrs == NULL)
> + goto fail_errmsg;
> +
> + if (saved_attributes == NULL)
> + saved_attributes = attrs;
> + else
> + {
> + errmsg = simple_object_attributes_merge (saved_attributes, attrs,
> + &err);
> + if (errmsg != NULL)
> + {
> + free (attrs);
> + goto fail_errmsg;
> + }
> + }
> + }
> + else
> + {
> + if (!saved_attributes)
> + {
> + lto_file *tmp = lto_obj_file_open (flag_bypass_asm, false);
> + if (!tmp)
> + goto fail;
> + lto_obj_file_close (tmp);
> + }
> + lo->sobj_w = simple_object_start_write (saved_attributes,
> + LTO_SEGMENT_NAME,
> + &errmsg, &err);
> + if (lo->sobj_w == NULL)
> + goto fail_errmsg;
> + }
> +
> + return &lo->base;
> +
> + fail_errmsg:
> + if (err == 0)
> + error ("%s: %s", fname, errmsg);
> + else
> + error ("%s: %s: %s", fname, errmsg, xstrerror (err));
> +
> + fail:
> + if (lo->fd != -1)
> + lto_obj_file_close ((lto_file *) lo);
> + free (lo);
> + return NULL;
> +}
> +
> +
> +/* Close FILE. If FILE was opened for writing, it is written out
> + now. */
> +
> +void
> +lto_obj_file_close (lto_file *file)
> +{
> + struct lto_simple_object *lo = (struct lto_simple_object *) file;
> +
> + if (lo->sobj_r != NULL)
> + simple_object_release_read (lo->sobj_r);
> + else if (lo->sobj_w != NULL)
> + {
> + const char *errmsg;
> + int err;
> +
> + gcc_assert (lo->base.offset == 0);
> +
> + errmsg = simple_object_write_to_file (lo->sobj_w, lo->fd, &err);
> + if (errmsg != NULL)
> + {
> + if (err == 0)
> + fatal_error ("%s", errmsg);
> + else
> + fatal_error ("%s: %s", errmsg, xstrerror (err));
> + }
> +
> + simple_object_release_write (lo->sobj_w);
> + }
> +
> + if (lo->fd != -1)
> + {
> + if (close (lo->fd) < 0)
> + fatal_error ("close: %s", xstrerror (errno));
> + }
> +}
> +
> +/* This is passed to lto_obj_add_section. */
> +
> +struct lto_obj_add_section_data
> +{
> + /* The hash table of sections. */
> + htab_t section_hash_table;
> + /* The offset of this file. */
> + off_t base_offset;
> + /* List in linker order */
> + struct lto_section_list *list;
> +};
> +
> +/* This is called for each section in the file. */
> +
> +static int
> +lto_obj_add_section (void *data, const char *name, off_t offset,
> + off_t length)
> +{
> + struct lto_obj_add_section_data *loasd =
> + (struct lto_obj_add_section_data *) data;
> + htab_t section_hash_table = (htab_t) loasd->section_hash_table;
> + char *new_name;
> + struct lto_section_slot s_slot;
> + void **slot;
> + struct lto_section_list *list = loasd->list;
> +
> + if (strncmp (name, LTO_SECTION_NAME_PREFIX,
> + strlen (LTO_SECTION_NAME_PREFIX)) != 0)
> + return 1;
> +
> + new_name = xstrdup (name);
> + s_slot.name = new_name;
> + slot = htab_find_slot (section_hash_table, &s_slot, INSERT);
> + if (*slot == NULL)
> + {
> + struct lto_section_slot *new_slot = XCNEW (struct lto_section_slot);
> +
> + new_slot->name = new_name;
> + new_slot->start = loasd->base_offset + offset;
> + new_slot->len = length;
> + *slot = new_slot;
> +
> + if (list != NULL)
> + {
> + if (!list->first)
> + list->first = new_slot;
> + if (list->last)
> + list->last->next = new_slot;
> + list->last = new_slot;
> + }
> + }
> + else
> + {
> + error ("two or more sections for %s", new_name);
> + return 0;
> + }
> +
> + return 1;
> +}
> +
> +/* Build a hash table whose key is the section name and whose data is
> + the start and size of each section in the .o file. */
> +
> +htab_t
> +lto_obj_build_section_table (lto_file *lto_file, struct lto_section_list *list,
> + htab_t section_hash_table)
> +{
> + struct lto_simple_object *lo = (struct lto_simple_object *) lto_file;
> + struct lto_obj_add_section_data loasd;
> + const char *errmsg;
> + int err;
> +
> +
> + gcc_assert (lo->sobj_r != NULL && lo->sobj_w == NULL);
> + loasd.section_hash_table = section_hash_table;
> + loasd.base_offset = lo->base.offset;
> + loasd.list = list;
> + errmsg = simple_object_find_sections (lo->sobj_r, lto_obj_add_section,
> + &loasd, &err);
> + if (errmsg != NULL)
> + {
> + if (err == 0)
> + error ("%s", errmsg);
> + else
> + error ("%s: %s", errmsg, xstrerror (err));
> + htab_delete (section_hash_table);
> + return NULL;
> + }
> +
> + return section_hash_table;
> +}
> +
> +/* The current output file. */
> +
> +static lto_file *current_out_file;
> +
> +/* Set the current output file. Return the old one. */
> +
> +lto_file *
> +lto_set_current_out_file (lto_file *file)
> +{
> + lto_file *old_file;
> +
> + old_file = current_out_file;
> + current_out_file = file;
> + return old_file;
> +}
> +
> +/* Return the current output file. */
> +
> +lto_file *
> +lto_get_current_out_file (void)
> +{
> + return current_out_file;
> +}
> +
> +/* Begin writing a new section named NAME in the current output
> + file. */
> +
> +void
> +lto_obj_begin_section (const char *name)
> +{
> + struct lto_simple_object *lo;
> + int align;
> + const char *errmsg;
> + int err;
> +
> + lo = (struct lto_simple_object *) current_out_file;
> + gcc_assert (lo != NULL
> + && lo->sobj_r == NULL
> + && lo->sobj_w != NULL
> + && lo->section == NULL);
> +
> + align = exact_log2 (POINTER_SIZE / BITS_PER_UNIT);
> + lo->section = simple_object_write_create_section (lo->sobj_w, name, align,
> + &errmsg, &err);
> + if (lo->section == NULL)
> + {
> + if (err == 0)
> + fatal_error ("%s", errmsg);
> + else
> + fatal_error ("%s: %s", errmsg, xstrerror (errno));
> + }
> +}
> +
> +/* Add data to a section. BLOCK is a pointer to memory containing
> + DATA. */
> +
> +void
> +lto_obj_append_data (const void *data, size_t len, void *)
> +{
> + struct lto_simple_object *lo;
> + const char *errmsg;
> + int err;
> +
> + lo = (struct lto_simple_object *) current_out_file;
> + gcc_assert (lo != NULL && lo->section != NULL);
> +
> + errmsg = simple_object_write_add_data (lo->sobj_w, lo->section, data, len,
> + 1, &err);
> + if (errmsg != NULL)
> + {
> + if (err == 0)
> + fatal_error ("%s", errmsg);
> + else
> + fatal_error ("%s: %s", errmsg, xstrerror (errno));
> + }
> +}
> +
> +/* Stop writing to the current output section. */
> +
> +void
> +lto_obj_end_section (void)
> +{
> + struct lto_simple_object *lo;
> +
> + lo = (struct lto_simple_object *) current_out_file;
> + gcc_assert (lo != NULL && lo->section != NULL);
> + lo->section = NULL;
> +}
> Index: lto-streamer.h
> ===================================================================
> --- lto-streamer.h (revision 215518)
> +++ lto-streamer.h (working copy)
> @@ -673,7 +673,31 @@
> struct streamer_tree_cache_d *reader_cache;
> };
>
> +/* Hash table entry to hold the start offset and length of an LTO
> + section in a .o file. */
> +struct lto_section_slot
> +{
> + const char *name;
> + intptr_t start;
> + size_t len;
> + struct lto_section_slot *next;
> +};
>
> +/* A list of section slots */
> +struct lto_section_list
> +{
> + struct lto_section_slot *first, *last;
> +};
> +
> +/* A file. */
> +typedef struct lto_file_struct
> +{
> + /* The name of the file. */
> + const char *filename;
> + /* The offset for the object inside an ar archive file (or zero). */
> + off_t offset;
> +} lto_file;
> +
> /* In lto-section-in.c */
> extern struct lto_input_block * lto_create_simple_input_block (
> struct lto_file_decl_data *,
> @@ -804,6 +828,17 @@
> struct lto_out_decl_state *);
> void lto_output_location (struct output_block *, struct bitpack_d *, location_t);
>
> +/* In lto-elf.c or lto-coff.c */
> +extern lto_file *lto_obj_file_open (const char *filename, bool writable);
> +extern void lto_obj_file_close (lto_file *file);
> +struct lto_section_list;
> +extern htab_t lto_obj_build_section_table (lto_file *file, struct lto_section_list *list, htab_t);
> +extern htab_t lto_obj_create_section_hash_table (void);
> +extern void lto_obj_begin_section (const char *name);
> +extern void lto_obj_append_data (const void *data, size_t len, void *block);
> +extern void lto_obj_end_section (void);
> +extern lto_file *lto_set_current_out_file (lto_file *file);
> +extern lto_file *lto_get_current_out_file (void);
>
> /* In lto-cgraph.c */
> extern bool asm_nodes_output;
> Index: toplev.c
> ===================================================================
> --- toplev.c (revision 215518)
> +++ toplev.c (working copy)
> @@ -80,6 +80,7 @@
> #include "context.h"
> #include "pass_manager.h"
> #include "optabs.h"
> +#include "lto-streamer.h"
>
> #if defined(DBX_DEBUGGING_INFO) || defined(XCOFF_DEBUGGING_INFO)
> #include "dbxout.h"
> @@ -617,41 +618,48 @@
> library without invoking lto1. */
> if (flag_generate_lto)
> {
> + if (flag_bypass_asm)
> + {
> + /* TODO. */
> + }
> + else
> + {
> #if defined ASM_OUTPUT_ALIGNED_DECL_COMMON
> - ASM_OUTPUT_ALIGNED_DECL_COMMON (asm_out_file, NULL_TREE,
> - "__gnu_lto_v1",
> - (unsigned HOST_WIDE_INT) 1, 8);
> -#elif defined ASM_OUTPUT_ALIGNED_COMMON
> - ASM_OUTPUT_ALIGNED_COMMON (asm_out_file, "__gnu_lto_v1",
> - (unsigned HOST_WIDE_INT) 1, 8);
> -#else
> - ASM_OUTPUT_COMMON (asm_out_file, "__gnu_lto_v1",
> - (unsigned HOST_WIDE_INT) 1,
> - (unsigned HOST_WIDE_INT) 1);
> -#endif
> - /* Let linker plugin know that this is a slim object and must be LTOed
> - even when user did not ask for it. */
> - if (!flag_fat_lto_objects)
> - {
> -#if defined ASM_OUTPUT_ALIGNED_DECL_COMMON
> ASM_OUTPUT_ALIGNED_DECL_COMMON (asm_out_file, NULL_TREE,
> - "__gnu_lto_slim",
> + "__gnu_lto_v1",
> (unsigned HOST_WIDE_INT) 1, 8);
> #elif defined ASM_OUTPUT_ALIGNED_COMMON
> - ASM_OUTPUT_ALIGNED_COMMON (asm_out_file, "__gnu_lto_slim",
> + ASM_OUTPUT_ALIGNED_COMMON (asm_out_file, "__gnu_lto_v1",
> (unsigned HOST_WIDE_INT) 1, 8);
> #else
> - ASM_OUTPUT_COMMON (asm_out_file, "__gnu_lto_slim",
> + ASM_OUTPUT_COMMON (asm_out_file, "__gnu_lto_v1",
> (unsigned HOST_WIDE_INT) 1,
> (unsigned HOST_WIDE_INT) 1);
> #endif
> - }
> + /* Let linker plugin know that this is a slim object and must be LTOed
> + even when user did not ask for it. */
> + if (!flag_fat_lto_objects)
> + {
> +#if defined ASM_OUTPUT_ALIGNED_DECL_COMMON
> + ASM_OUTPUT_ALIGNED_DECL_COMMON (asm_out_file, NULL_TREE,
> + "__gnu_lto_slim",
> + (unsigned HOST_WIDE_INT) 1, 8);
> +#elif defined ASM_OUTPUT_ALIGNED_COMMON
> + ASM_OUTPUT_ALIGNED_COMMON (asm_out_file, "__gnu_lto_slim",
> + (unsigned HOST_WIDE_INT) 1, 8);
> +#else
> + ASM_OUTPUT_COMMON (asm_out_file, "__gnu_lto_slim",
> + (unsigned HOST_WIDE_INT) 1,
> + (unsigned HOST_WIDE_INT) 1);
> +#endif
> + }
> + }
> }
>
> /* Attach a special .ident directive to the end of the file to identify
> the version of GCC which compiled this code. The format of the .ident
> string is patterned after the ones produced by native SVR4 compilers. */
> - if (!flag_no_ident)
> + if (!flag_no_ident && !flag_bypass_asm)
> {
> const char *pkg_version = "(GNU) ";
> char *ident_str;
> @@ -669,7 +677,10 @@
> /* This must be at the end. Some target ports emit end of file directives
> into the assembly file here, and hence we can not output anything to the
> assembly file after this point. */
> - targetm.asm_out.file_end ();
> + if (!flag_bypass_asm)
> + targetm.asm_out.file_end ();
> + else
> + lto_obj_file_close (lto_get_current_out_file ());
>
> timevar_stop (TV_PHASE_LATE_ASM);
> }
> @@ -1739,7 +1750,8 @@
>
> if (!flag_wpa)
> {
> - init_asm_output (name);
> + if (!flag_bypass_asm)
> + init_asm_output (name);
>
> /* If stack usage information is desired, open the output file. */
> if (flag_stack_usage)