This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: [RFC] zstd as a compression algorithm for LTO
On June 19, 2019 6:03:21 PM GMT+02:00, Jeff Law <law@redhat.com> wrote:
>On 6/19/19 3:21 AM, Martin Liška wrote:
>> Hi.
>>
>> I've written a patch draft that replaces zlib with the zstd
>compression algorithm ([1])
>> in LTO. I'm also sending statistics that are collected for couple of
>quite big C++ source
>> files. Observation I did:
>>
>> - LTO stream compression takes 3-4% of LGEN compile time
>> - zstd in default compression level (3) generated slighly smaller LTO
>elf files
>> - zstd compression is 4-8x faster
>> - decompression is quite negligible, but for a bigger project (godot)
>I can
>> reduction from 1.37 to 0.53 seconds
>> - ZSTD API is much simpler to use
>>
>> Suggestion based on the observation:
>> - I would suggest to make zstd optional (--enable-zstd) and one would
>> use #include <zstd> + -lzstd
>> - I like the default level as we want to mainly speed up LTO
>compilation
>> - we can provide an option to control algorithm
>(-flto-compression-algorithm),
>> similarly to -flto-compression-level
>> - we can discuss possible compression of LTO bytecode that is
>distributed between WPA
>> stage and individual LTRANS phases.
>Presumably the reason we're not being more aggressive about switching
>is
>the build/run time dependency on zstd? I wonder if we could default to
>zstd and fallback to zlib when zstd isn't available?
Is zstd too big to include into the repository? But yes, we can properly encode the compression format in the LTO section header and use dlopen to 'find' the default to use.
Richard.
>jeff