Bug 100937 - configure: Add --enable-default-semantic-interposition
Summary: configure: Add --enable-default-semantic-interposition
Status: RESOLVED WONTFIX
Alias: None
Product: gcc
Classification: Unclassified
Component: driver (show other bugs)
Version: 11.0
: P3 normal
Target Milestone: ---
Assignee: Not yet assigned to anyone
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-06-06 23:10 UTC by Fangrui Song
Modified: 2021-11-22 22:54 UTC (History)
4 users (show)

See Also:
Host:
Target:
Build:
Known to work:
Known to fail:
Last reconfirmed:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Fangrui Song 2021-06-06 23:10:38 UTC
Add a configure option --enable-default-semantic-interposition to customize
-f(no-)semantic-interposition default.

The suppression of interprocedural optimizations and inlining for such
default visibility non-vague-linkage function definitions is the biggest difference between -fPIE/-fPIC.

Distributions may want to enable default -fno-semantic-interposition to
reclaim the lost performance from -fPIC (e.g. CPython is said to be 27% faster;
Clang is 3% faster).
Comment 1 Andrew Pinski 2021-06-06 23:12:55 UTC
NO. This is wrong for many reasons. First it makes portability a pain.
Comment 2 Fangrui Song 2021-06-06 23:15:26 UTC
How is it a portability problem?

clang -fpic has always been allowing interprocedural optimizations for non-vague-linkage function definitions. FreeBSD uses clang and software works with no problem.




For a vague-linkage function definition, a call site in the same
translation unit may inline the callee. Whether
-fno-semantic-interposition is enabled/disabled has no effect.

For a non-vague-linkage function definition, by default
(-fsemantic-interposition) the -fpic mode does not allow a call site
in the same translation unit to inline the callee or perform other
interprocedural optimizations.
-fno-semantic-interposition re-enables interprocedural optimizations.

If a caller inlines a callee, using LD_PRELOAD to interpose the callee
will not affect the caller. But many other LD_PRELOAD usage still
work.
We consider the small LD_PRELOAD limitation a good trade off for the speedup.
Comment 3 Andrew Pinski 2021-06-06 23:19:51 UTC
>clang -fpic has always been allowing interprocedural optimizations for non-vague-linkage function definitions. FreeBSD uses clang and software works with no problem.

That does not mean clang is correct here.
clang breaks ELF assumptions and that is all I am going to say.  If you want to break ELF fine, FreeBSD can break those.  But there is still a portability issue between distros using different options like this.
Comment 4 Andrew Pinski 2021-06-06 23:27:30 UTC
Also your patch did not change the documentation of the option.
Plus the documentation is clear that changing the default is most likely not wanted at all:
https://gcc.gnu.org/onlinedocs/gcc-11.1.0/gcc/Optimize-Options.html#index-fsemantic-interposition
Comment 5 Jakub Jelinek 2021-06-07 08:28:41 UTC
Agreed this is a very bad idea.  Users can always pick options they want explicitly, but changing such options' defaults will mean nothing will be able to rely on the default and so everything that will need semantic interposition will all of sudden need to start using -fsemantic-interposition.
Comment 6 Fangrui Song 2021-06-10 05:02:11 UTC
Then can you add a -fvisibility=protected variant which only applies to non-weak defined functions?

Two issues need to be fixed:

(1): https://sourceware.org/bugzilla/show_bug.cgi?id=27973

__attribute__((visibility("protected"))) void *foo () {
  return (void *)foo;
}
% gcc -fpic -shared -fuse-ld=bfd a.s
/usr/bin/ld.bfd: /tmp/ccWPJCLw.o: relocation R_X86_64_PC32 against protected symbol `foo' can not be used when making a shared object
/usr/bin/ld.bfd: final link failed: bad value
collect2: error: ld returned 1 exit status

(2): https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100593
[ELF] -fno-pic: Use GOT to take address of an external default visibility function 


Distributions want fast C++ non-vague-linkage functions can enable this option.
Comment 7 frankhb1989 2021-11-02 18:16:29 UTC
While I feel it is fair to keep the status quo, the decision here deserves some additional comments.

It is true that potability is generally important, but the current way here is actually not friendly to potability.

GCC and the GNU toolchain are not ELF-specific. Nor are they responsible to the authority of the specification. The "ELF assumptions" have no natural position to be the default at the very first glance from the users' view.

So the "portability" certainly include the ease of porting the programs to different targets with different image formats back and forth. Sticking on the ELF-centric defaults already fails (by reducing the platform-specific differences) if users want to really gain more portability, at least for PE/COFF targets (which do not support such interposition at all).

Although it is somewhat reasonable to distinguish platforms supporting symbol interposition as first-class ones (same to the current "primary platforms"), but this still seems technically weak.

Perhaps a more appropriate phrase for the reason is "backward compatibility".
Comment 8 Eric Gallager 2021-11-03 03:19:08 UTC
(In reply to frankhb1989 from comment #7) 
> GCC and the GNU toolchain are not ELF-specific. Nor are they responsible to
> the authority of the specification. The "ELF assumptions" have no natural
> position to be the default at the very first glance from the users' view.
> 
> So the "portability" certainly include the ease of porting the programs to
> different targets with different image formats back and forth. Sticking on
> the ELF-centric defaults already fails (by reducing the platform-specific
> differences) if users want to really gain more portability, at least for
> PE/COFF targets (which do not support such interposition at all).

This is related to bug 102665 btw
Comment 9 Jonathan Wakely 2021-11-03 12:25:28 UTC
(In reply to frankhb1989 from comment #7)
> GCC and the GNU toolchain are not ELF-specific. Nor are they responsible to
> the authority of the specification.

That seems like nonsense to me. The toolchain might not be ELF-specific, but on targets that *do* use ELF, of course the ELF specification is relevant, and deviating from it should not be done lightly.

>  The "ELF assumptions" have no natural
> position to be the default at the very first glance from the users' view.

More nonsense.

Just because portability between ELF and PE/COFF is difficult and requires care, doesn't mean we should also make it difficult to guarantee portability across different linux distros that are all using ELF. That's just silly. You don't make anything better by making everything equally bad.
Comment 10 frankhb1989 2021-11-22 13:26:41 UTC
(In reply to Jonathan Wakely from comment #9)
> (In reply to frankhb1989 from comment #7)
> The toolchain might not be ELF-specific, but
> on targets that *do* use ELF, of course the ELF specification is relevant,

True.

> and deviating from it should not be done lightly.

Deviating... well, are there any documented policies to clarify how much room to the non-conformance here?

> Just because portability between ELF and PE/COFF is difficult and requires
> care, doesn't mean we should also make it difficult to guarantee portability
> across different linux distros that are all using ELF. That's just silly.
> You don't make anything better by making everything equally bad.

I don't mean that one portability is just unconditionally more important than another. But there are certainly things beyond portability if you are really caring "everything".

Keeping the "portability", if any across ELF, makes sense to maintainers of the distros.

But it is not obvious to most users, which are not specialists of ELF. For those not relying on the sneaky feature, they silently pay the unneeded cost *by default*. This is probably against to the intuition to most C and C++ users.

Moreover, this also affects almost every *end users*, which should not care about such features at all.

So here is the "bad": the default configuration makes things unexpected (and a laborious way to prevent it) for most users, albeit only in QoI sense. Such default may even have effects on the adoption of free software.

Changing the default may likely reduce such bad things, at the cost of the compatibility. I am not arguing the change is absolutely necessary, but it is certainly NOT "making everything equally bad". (This would not remove interposition for the minority who need them indeed.)

BTW, I'm not aware of such a cross-distro guarantee has been widely adopted and relied on. It is well-known the Linux kernel has the policy to maintain the stability of syscalls, but in the userland... is it already distro-specific? Correct me if I am wrong here. (Any source?)
Comment 11 Fangrui Song 2021-11-22 19:52:18 UTC
To enable interposition on Mach-O, one needs a non-default configuration like: ld -interposable, DYLD_FORCE_FLAT_NAMESPACE or __attribute__((section("__DATA,__interpose"))).
On PE/COFF, such interposition just doesn't exist.

Having an option for -fno-semantic-interposition will actually improve portability.

(The -fno-semantic-interposition thing is probably the biggest performance gap between gcc -fpic and clang -fpic.)

As I said previously, -fvisibility=protected cannot be used because protected visibility is very broken in the GCC/GNU ld system and there is no signal it will be fixed anytime soon: https://maskray.me/blog/2021-01-09-copy-relocations-canonical-plt-entries-and-protected#summary
Comment 12 hubicka 2021-11-22 22:54:53 UTC
> (The -fno-semantic-interposition thing is probably the biggest performance gap
> between gcc -fpic and clang -fpic.)
Yep, it is often confusing to users (who do not understand what ELF
interposition is) that clang and gcc disagree on default flags here.
Recently -Ofast was extended to imply -fno-semantic-interposition that
will hopefully make more people notice this.

While doing that I have added per-symbol flag about interposition to the
symbol table, so we can also support 

__atttribute__ ((semantic_interposition))

and

__attribute__((no_semantic_interpoition))

if that would be useful for something.