DWARF Extensions for Separate Debug Information Files

Updated January 24, 2013

The "Fission" project was started in response to the problems caused by huge amounts of debug information in large applications. By splitting the debug information into two parts at compile time -- one part that remains in the .o file and another part that is written to a parallel .dwo ("DWARF object") file -- we can reduce the total size of the object files processed by the linker.


Fission is implemented in GCC 4.7, and requires support from recent versions of objcopy and the gold linker.

Use the -gsplit-dwarf option to enable the generation of split DWARF at compile time. This option must be used in conjunction with -c; Fission cannot be used when compiling and linking in the same step.

Use the gold linker's --gdb-index option (-Wl,--gdb-index when linking with gcc or g++) at link time to create the .gdb_index section that allows GDB to locate and read the .dwo files as it needs them.

Problems with Size of the Debug Information

Large applications compiled with debug information experience slow link times, possible out-of-memory conditions at link time, and slow gdb startup times. In addition, they can contribute to significant increases in storage requirements, and additional network latency when transferring files in a distributed build environment.

These conditions are largely a direct result of the amount of debug information generated by the compiler. In a large C++ application compiled with -O2 and -g, the debug information accounts for 87% of the total size of the object files sent as inputs to the link step, and 84% of the total size of the output binary.

Recently, the -Wa,--compress-debug-sections option has been made available. This option reduces the total size of the object files sent to the linker by more than a third, so that the debug information now accounts for 70-80% of the total size of the object files. The output file is unaffected: the linker decompresses the debug information in order to link it, and outputs the uncompressed result (there is an option to recompress the debug information at link time, but this step would only reduce the size of the output file without improving link time or memory usage).

What's All That Space Being Used For?

The debugging information in the relocatable object files sent to the linker consists of a number of separate tables (percentages are for uncompressed debug information relative to the total object file size):

Using compressed debug sections, the percentages are adjusted as follows:

Towards Reducing the Amount of Debug Information Sent to the Linker

The numbers above suggest several possible approaches to improving build performance by reducing the amount of debug information sent to the linker. Of the approaches listed below, the first has already been implemented, and we are planning to proceed with options #3 and #5.

1. Compress debug sections

This option is already in the binutils assembler.

Total estimated benefit: 36% reduction

All estimated benefits below assume the use of compressed debug sections.

We can use intermediate (ld -r) link steps to discard a good fraction of the duplicate type information and strings in the debug information. COMDAT elimination on the .debug_types section would ultimately reduce the total size by 75%, and string merge processing would ultimately reduce the total size of the .debug_str section by 80%.

By using ld -r on the input files, there would be some risk of including object files that would have been passed over during an archive library search, but this could be mitigated by removing duplicate definitions of the same symbol in different libraries.

Total estimated benefit: 6% reduction

3. Eliminate relocations to strings

Relocations in the .debug_info and .debug_types sections that refer to the .debug_str section can be replaced by an extra indirection and a new dedicated table of string offsets. In the debug info, each string reference takes 8 bytes in the debug info plus 24 bytes for the relocation (the 8 bytes compress to 1 byte on average, but the relocations are not compressed). These would be replaced by an average 1-2 byte string index, and an 8-byte string offset in a separate table. The separate table can be implicitly relocated, so no relocations are necessary, and would probably compress by about 80% (estimated). Furthermore, the number of unique strings (within each compilation unit) is only about 55% of the total number of string references, so the separate table would be reduced by another 45%.

Total estimated benefit: 53% reduction

4. Move type units to a separate repository

The information in .debug_types is largely independent of the rest of the compilation unit; each entry describes a type and is the same in each compilation unit that contains a definition of that type. The compiler computes a unique signature for each type definition, and could store the debug info for that type in an external repository, keyed by the type's signature. This would reduce the object file size by the 7% used directly for the .debug_types sections, by another estimated 5% for the strings referenced by the type entries, and by 22% for the relocations associated with the .debug_types sections.

Total estimated benefit: 34% reduction (less if option 3 is also implemented)

5. Move debug info and type units to a ".dwo" file

Alternatively, we can move the .debug_info, .debug_types, and .debug_str sections from the object (.o) files to a separate DWARF object (.dwo) file (or ".dsym", in Apple's nomenclature). Assuming that we could ignore these .dwo files during the link step, this would remove the bulk of the data that would be sent to the linker, and debug builds would be be much closer to non-debug builds in terms of object file size and link speed.

There are two options for how to deal with the separate .dwo files when it's time to debug a program.

The first option is for the debugger to look for the debug info directly in the .dwo files, requiring both that the output binary contains enough information to find the .dwo files, and that the .dwo files must remain available for use for as long as the binary might need debugging. This option requires no additional link step for the debug information.

The second option is to invoke a separate link step to combine the .dwo files into a single .dwo file that can be easily stored and located by the debugger (this is the approach that Apple takes with its dsymutil tool). While this option dramatically reduces the size of the main link, the separate link step for the debug information is still close to the original order of magnitude (debug info being more than 80% of the total size of the object files). A dsymutil-like linker, however, could be made to operate much more efficiently than a full-featured ELF linker.

Total estimated benefit: 70% reduction

Previous Implementations of Separate Debug Information

Sun, HP, and Apple have all implemented similar mechanisms where the debug information is not linked into the final binary. All three implementations simply leave the debug information in the relocatable object (.o) files, with summary information in the final binary to enable the debugger to locate the object files and to apply the relocations to the debug information. Sun's implementation is for stabs only; for DWARF, the linker always copies the full debug information to the output file. HP's implementation is for DWARF, and includes summary information with the names of the original object files, and a link map that allows the debugger to locate each object file's contribution to each section in the output file. Apple's solution is similar, but the summary information is synthesized by the linker in stabs format, and the DWARF information is linked together in a separate link step by the dsymutil utility.

In the Sun and HP implementations, the debug information in the relocatable objects still requires relocation at debug time, and the debugger must read the summary information from the executable file in order to map symbols and sections to the output file when processing and applying the relocations. The Apple implementation avoids this cost at debug time, but at the cost of having a separate link step for the debug information.

As we have seen above, a significant factor in the space used by debug information is the number of relocations, so a solution that minimizes the number of relocations not only reduces the total size of the binary plus its debug information, but also reduces the complexity and cost of reading the debug information at debug time.

Design for Moving Debug Information to ".dwo" Files

In order for the debugger to be able to locate and process the information in raw (unrelocated) .dwo files, some information must still be left behind in the .o files for the linker to combine and relocate. The bulk of the reduction in .o file size will come from moving .debug_info, .debug_types, .debug_loc, and .debug_str sections into the .dwo file (and eliminating most of the associated relocations). The .debug_ranges, .debug_line, .debug_pubnames, .debug_pubtypes, .debug_aranges, and .debug_gdb_scripts sections (and their relocation sections) will remain in the .o file. The .debug_abbrev section, although small, will naturally move with the .debug_info and .debug_types sections.

Instead of a full .debug_info section the .o file will contain a "skeleton" .debug_info section, and a corresponding .debug_abbrev section. This .debug_info section will contain a single DW_TAG_compile_unit DIE, with no children. A DW_AT_dwo_name attribute will provide the name of the .dwo file, and a DW_AT_dwo_id attribute will contain a unique 64-bit ID for the compilation unit. The skeleton compilation unit DIE will contain a few additional attributes as described below.

The .dwo file will follow the ELF format, but will have only a file header, section table, and the debug sections. The debug sections in the .dwo file will all end with ".dwo".

For the sections that will move into a .dwo file, the existing relocations will be handled as follows:

In the initial implementation, we will modify gcc to emit all the debug information into the single .o file, and we will use post-compile processing to move the appropriate sections into the separate .dwo file.

The debug sections to remain in the .o file are:

The following debug sections will be generated by gcc and the assembler in the .o file, but will be moved to the separate .dwo file in post-compile processing:

Building a GDB Index

The .gdb_index section (see GNU Index Section) allows GDB to locate the appropriate compilation unit or type unit quickly, given a name or an address, without having to open all the .dwo files and scan all the debug information at start-up time.

In order to build the .gdb_index section, the linker today needs to do three things:

  1. Scan the .debug_info sections to extract all the public names mentioned in those sections and build a hash table mapping those names to a list of compilation units that provide definitions for those names.
  2. Scan the compilation unit headers of the .debug_info sections to build a list of compilation units, and to build a range table that can be used to map and address to a specific compilation unit.
  3. Scan the .debug_types sections to build a list of type units.

In this new design, the .debug_info and .debug_types sections will not be available for the linker to scan. Instead, the linker will extract the public names from the .debug_pubnames and .debug_pubtypes sections, the range table from the .debug_aranges sections, and the the list of type units from the .debug_pubtypes sections.

The format of the .gdb_index section will remain unchanged (with the possible exception of removing the range tables in favor of having gdb use .debug_aranges directly).

The .debug_pubnames section was always intended as a comprehensive list of symbols that gdb could use for quick lookup, but bugs in gcc have so far prevented gdb from using this section for its intended purpose, and the .gdb_index section has instead been built from a scan over the .debug_info section. We will need to fix gcc so that the .debug_pubnames section contains all of the names that gdb requires in its index.

The .debug_pubtypes section, likewise, was intended as a list of public types, but does not yet include types or public names (e.g., enum constants) defined in type units in the .debug_types sections. We will have to modify gcc to produce the required information.

Each .debug_pubnames and .debug_pubtypes section contains a reference to the compilation unit in the .debug_info section, but there is no efficient way to find the pubname/pubtypes section given a compilation unit or a type unit. In order to simplify the processing necessary to build the .gdb_index section, we will add two new attributes to the DW_TAG_compile_unit and DW_TAG_type_unit DIEs: DW_AT_pubnames and DW_AT_pubtypes, whose values will be references (with form DW_FORM_sec_offset) to the associated pubnames/pubtypes sets. These attributes have the further benefit that the consumer will be able to know for sure that the list of public names in each section is reliable (relative to older versions of GCC).

The .debug_aranges section currently contains all the required information for producing the .gdb_index section (as far as I know), and could in fact be used directly by gdb instead of having the linker reformat its contents into .gdb_index. We can either modify the linker to extract the necessary range information from .debug_aranges to produce .gdb_index, or modify gdb to use the existing information from .debug_aranges instead of expecting it in .gdb_index.

Packaging Debug Information for Release

While the .dwo files are convenient for debugging an application during development, when the .dwo files can all be found in the build tree, once an application is copied outside the build tree, it is desirable to be able to collect the full set of .dwo files into a single package. For this purpose, we will support a dwp utility that will combine a set of .dwo files into a single DWARF package file, for which we will use the ".dwp" extension. The proposed package file format is described in DWARF Package File Format.

Proposed Extensions to the DWARF Specification

We propose adding several new sections, attribute codes, and form codes to the DWARF specification, and extending the .debug_pubtypes format to support type units.

New DWARF sections

New DWARF attributes

New DWARF form codes

New DWARF location expression opcodes

Extension to .debug_pubtypes

The .debug_pubtypes table will also contain the public types defined by type units generated with the compilation unit. Its header will point to the compilation unit in the .debug_info section, but the DW_AT_pubtypes attribute in the DW_TAG_type_unit DIE in the type unit will provide the link between the type unit and the pubtypes table.

None: DebugFission (last edited 2014-06-17 21:31:00 by CaryCoutant)