This is the mail archive of the gcc@gcc.gnu.org mailing list for the GCC project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [RFC] zstd as a compression algorithm for LTO


On Wed, Jun 19, 2019 at 11:55 AM Richard Biener
<richard.guenther@gmail.com> wrote:
>
> On June 19, 2019 6:03:21 PM GMT+02:00, Jeff Law <law@redhat.com> wrote:
> >On 6/19/19 3:21 AM, Martin Liška wrote:
> >> Hi.
> >>
> >> I've written a patch draft that replaces zlib with the zstd
> >compression algorithm ([1])
> >> in LTO. I'm also sending statistics that are collected for couple of
> >quite big C++ source
> >> files. Observation I did:
> >>
> >> - LTO stream compression takes 3-4% of LGEN compile time
> >> - zstd in default compression level (3) generated slighly smaller LTO
> >elf files
> >> - zstd compression is 4-8x faster
> >> - decompression is quite negligible, but for a bigger project (godot)
> >I can
> >>   reduction from 1.37 to 0.53 seconds
> >> - ZSTD API is much simpler to use
> >>
> >> Suggestion based on the observation:
> >> - I would suggest to make zstd optional (--enable-zstd) and one would
> >>   use #include <zstd> + -lzstd
> >> - I like the default level as we want to mainly speed up LTO
> >compilation
> >> - we can provide an option to control algorithm
> >(-flto-compression-algorithm),
> >>   similarly to -flto-compression-level
> >> - we can discuss possible compression of LTO bytecode that is
> >distributed between WPA
> >>   stage and individual LTRANS phases.
> >Presumably the reason we're not being more aggressive about switching
> >is
> >the build/run time dependency on zstd?  I wonder if we could default to
> >zstd and fallback to zlib when zstd isn't available?
>
> Is zstd too big to include into the repository? But yes, we can properly encode the compression format in the LTO section header and use dlopen to 'find' the default to use.

At least allow it to be built as part of the normal build like GMP,
etc. are done.
And include it in downloading using contrib/download_prerequisites
like the libraries are done.

Thanks,
Andrew Pinski

>
> Richard.
>
> >jeff
>


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]