Bug 69429 - gcc create sparse exec/libs
Summary: gcc create sparse exec/libs
Status: RESOLVED INVALID
Alias: None
Product: gcc
Classification: Unclassified
Component: c (show other bugs)
Version: 4.9.3
: P3 major
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-01-22 11:10 UTC by Joakim Tjernlund
Modified: 2016-01-27 13:27 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Joakim Tjernlund 2016-01-22 11:10:18 UTC
When building small libs/exec on ppc32 I usally get sparse files like so:
ls -lsh | sort -n
....
 64K -rwxr-xr-x 1 jocke users  68K Jan 22 11:20 mgmt_alarmd*
 64K -rwxr-xr-x 1 jocke users  67K Jan 22 11:20 ne_memd*
 56K -rwxr-xr-x 1 jocke users  67K Jan 22 11:20 cp_dummy*
 56K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 mgmt_pmd*
 48K -rwxr-xr-x 1 jocke users  69K Jan 22 11:20 ntpdate*
 48K -rwxr-xr-x 1 jocke users  67K Jan 22 11:20 ne_rc*
 48K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 relayd*
 48K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 ntptimeset*
 44K -rwxr-xr-x 1 jocke users  67K Jan 22 11:20 mgmt_backup_tftpd*
 44K -rwxr-xr-x 1 jocke users  67K Jan 22 11:20 emxp2_cm_bl_shell*
 44K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 msntp*
 44K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 cp_hw_i2c*
 36K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 te_monitor*
 36K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 mgmt_invd*
 36K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 chat*
 36K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 cgi-fcgi*
 32K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 ne_db_mgr*
 32K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 dummy_hw_bl*
 32K -rwxr-xr-x 1 jocke users  29K Jan 22 11:19 net-snmp-config*
 28K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 tosv_server*
 28K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 te_log*
 28K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 ntptrace*
 28K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 hwdata*
 24K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 te*
 24K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 heat_test*
 24K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 alib_psupd*
 24K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 alib_memsupd*
 20K -rwxr-xr-x 1 jocke users  68K Jan 22 11:20 ntptime*
 20K -rwxr-xr-x 1 jocke users  68K Jan 22 11:20 dumpicn*
 20K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 te_server*
 20K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 mgmt_named*
 16K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 up_util_test_pmem*
 16K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 tosv_timer*
 16K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 tosv_test*
 16K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 tosv_supv*
 16K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 ne_rc_supv*
 16K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 ne_rc_memeater*
 16K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 ne_rc_load*
 16K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 ne_mem_ram_test*
 16K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 alib_test_psup*
 16K -rwxr-xr-x 1 jocke users  16K Jan 22 11:15 convert_backup*
 15M -rwxr-xr-x 1 jocke users  15M Jan 22 11:20 emxp2_hw_bl*
 12K -rwxr-xr-x 1 jocke users 9.0K Jan 22 11:18 swu_prepost_script.sh*
 12K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 tickadj*
 12K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 tclsh8.4*
 12K -rwxr-xr-x 1 jocke users  66K Jan 22 11:20 genkeys*

downgraded binutils from 2.25.1 to 2.24 but the problem remains.
Using earlier toolchain I can see that this extra sparse space
is just garbage because there the files are much smaller, some 12-16KB

This is a huge problem as these sparse file are packaged into a
tar file and when unpacked they loose the sparse and will expand to 66K on
disk and fill up the my small FS.
Comment 1 Jakub Jelinek 2016-01-22 12:20:18 UTC
Why do you file this against gcc?  If anything, it is related to binutils, which is separate project.  But, I'd say it is not a bug nevertheless, sparse files are an optimization, if you want to force no use of sparse files, you'd just always get the larger ones.  If using tar doesn't preserve holes in sparse files, use other means of copying them.
Comment 2 Joakim Tjernlund 2016-01-22 12:44:07 UTC
(In reply to Jakub Jelinek from comment #1)
> Why do you file this against gcc?  If anything, it is related to binutils,

You are right, I though it was a gcc problem because I downgraded binutils
to 2.24 but not I ses I only thought I did so.

> which is separate project.  But, I'd say it is not a bug nevertheless,
> sparse files are an optimization, if you want to force no use of sparse
> files, you'd just always get the larger ones.  If using tar doesn't preserve
> holes in sparse files, use other means of copying them.

Well, it breaks and tar cannot handle these sparse files and I think
a lot of packing tools uses tar.
I cannot change to something else on my targets as there is only tar installed

I found this binutils 2.15.x commit:
 https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commitdiff;h=9059a151e33f2f9b7b989a22e63d711a2c8a335b;hp=6e7e69e72dc1c53c8d5a8794c845026c48ff343a
(Set ppc COMMONPAGESIZE to 64k)

I do wonder why it now became necessary to increase page size in binutils?
Comment 3 Jakub Jelinek 2016-01-22 12:48:47 UTC
Because most of the powerpc distros now use 64KB page size instead of 4KB, so having only 4KB common page size means security protection of relro area usually does not work or covers smaller amount of data.
You can probably use -z common-page-size=4096 if you are sure you are never going to run your binaries on a kernel with 64KB pages.
Comment 4 Joakim Tjernlund 2016-01-27 13:24:29 UTC
(In reply to Jakub Jelinek from comment #3)
> Because most of the powerpc distros now use 64KB page size instead of 4KB,
> so having only 4KB common page size means security protection of relro area
> usually does not work or covers smaller amount of data.
> You can probably use -z common-page-size=4096 if you are sure you are never
> going to run your binaries on a kernel with 64KB pages.

Even for ppc32? It would be good with a configure option for ppc32 to
have 4KB pages by default as it can be cumbersome to pass -z common-page-size=4096
for all apps/libs built for ppc32
Comment 5 Jakub Jelinek 2016-01-27 13:27:36 UTC
Of course even for ppc32.  The ppc64 kernel supports ppc32 binaries.