Created attachment 36209 [details] minimal working example (C file) When compiling a 32 bit build of libx264 (on a 64 bit machine) it works in general, even with PIC. However the build breaks when I add -fstack-check to CFLAGS. The failing code is: int a, b, c, d, e; int fn1() { __asm__("" : "+r"(c), "+r"(e), "+r"(d), "+r"(a) : ""(b), "mg"(fn1), "mm"(c)); } (attached as me.c) This works: gcc -O3 -m32 -fPIC -c me.c This fails (and should work) gcc -O3 -m32 -fstack-check -fPIC -c me.c This also works (as a workaround): gcc -O3 -m32 -fstack-check -c me.c The error message is: me.c: In function ‘fn1’: me.c:3:3: error: ‘asm’ operand has impossible constraints __asm__("" : "+r"(c), "+r"(e), "+r"(d), "+r"(a) : ""(b), "mg"(fn1), "mm"(c)); ^ The related discussion for x264 is here: https://mailman.videolan.org/pipermail/x264-devel/2015-August/011246.html The flag -fstack-check is added by default on Arch Linux when building packages, but also additionally using hardened wrapper, where I also mentioned the problem: https://github.com/thestinger/hardening-wrapper/issues/6 I am the maintainer of lib32-libx264 on Arch Linux where the problem came up originally: https://aur.archlinux.org/packages/lib32-libx264/
Created attachment 36210 [details] Makefile with failing, working and workaround flags I found the bug using gcc 5.2.0 (Arch Linux), but I also tested gcc 5.1.0 and gcc 4.9.2. So this is not a brand new bug. Not sure if it did work as expected in versions before that.
X86 is very much under huge register pressure and that inline-asm uses 4 registers as input/output and two memory ones. -fastback-check forces one more register not to be available really.
It only temporarily needs a register to perform a write. The implementation might currently have to reserve it but it *shouldn't* have to. It should be free other than guaranteeing a one byte write per page.
The implementation currently reserves the frame pointer to be able to unwind the stack and recover from the stack overflow.
For the minimal testcase we can do: Index: ira.c =================================================================== --- ira.c (revision 229402) +++ ira.c (working copy) @@ -2273,7 +2273,9 @@ ira_setup_eliminable_regset (void) || (cfun->calls_alloca && EXIT_IGNORE_STACK) /* We need the frame pointer to catch stack overflow exceptions if the stack pointer is moving. */ - || (flag_stack_check && STACK_CHECK_MOVING_SP) + || (flag_stack_check + && STACK_CHECK_MOVING_SP + && (!crtl->is_leaf || cfun->calls_alloca)) || crtl->accesses_prior_frames || (SUPPORTS_STACK_ALIGNMENT && crtl->stack_realign_needed) /* We need a frame pointer for all Cilk Plus functions that use i.e. do not reserve the frame pointer if the function is leaf and doesn't call alloca, but this might not be sufficient for the original testcase.
Let's try to do something.
Author: ebotcazou Date: Wed Nov 11 14:22:43 2015 New Revision: 230168 URL: https://gcc.gnu.org/viewcvs?rev=230168&root=gcc&view=rev Log: PR target/67265 * ira.c (ira_setup_eliminable_regset): Do not necessarily create the frame pointer for stack checking if non-call exceptions aren't used. * config/i386/i386.c (ix86_finalize_stack_realign_flags): Likewise. Added: trunk/gcc/testsuite/gcc.target/i386/pr67265.c Modified: trunk/gcc/ChangeLog trunk/gcc/ira.c trunk/gcc/testsuite/ChangeLog
Author: ebotcazou Date: Wed Nov 11 14:24:39 2015 New Revision: 230170 URL: https://gcc.gnu.org/viewcvs?rev=230170&root=gcc&view=rev Log: PR target/67265 * config/i386/i386.c (ix86_finalize_stack_realign_flags): Likewise. Modified: trunk/gcc/config/i386/i386.c
Author: ebotcazou Date: Wed Nov 11 14:56:17 2015 New Revision: 230176 URL: https://gcc.gnu.org/viewcvs?rev=230176&root=gcc&view=rev Log: PR target/67265 * ira.c (ira_setup_eliminable_regset): Do not necessarily create the frame pointer for stack checking if non-call exceptions aren't used. * config/i386/i386.c (ix86_finalize_stack_realign_flags): Likewise. Added: branches/gcc-5-branch/gcc/testsuite/gcc.target/i386/pr67265.c - copied unchanged from r230168, trunk/gcc/testsuite/gcc.target/i386/pr67265.c Modified: branches/gcc-5-branch/gcc/ChangeLog branches/gcc-5-branch/gcc/config/i386/i386.c branches/gcc-5-branch/gcc/ira.c branches/gcc-5-branch/gcc/testsuite/ChangeLog
Author: ebotcazou Date: Wed Nov 11 16:04:34 2015 New Revision: 230179 URL: https://gcc.gnu.org/viewcvs?rev=230179&root=gcc&view=rev Log: PR target/67265 * ira.c (ira_setup_eliminable_regset): Do not necessarily create the frame pointer for stack checking if non-call exceptions aren't used. * config/i386/i386.c (ix86_finalize_stack_realign_flags): Likewise. Added: branches/gcc-4_9-branch/gcc/testsuite/gcc.target/i386/pr67265.c - copied unchanged from r230177, trunk/gcc/testsuite/gcc.target/i386/pr67265.c Modified: branches/gcc-4_9-branch/gcc/ChangeLog branches/gcc-4_9-branch/gcc/config/i386/i386.c branches/gcc-4_9-branch/gcc/ira.c branches/gcc-4_9-branch/gcc/testsuite/ChangeLog
Fixed on all active branches.
Author: ebotcazou Date: Thu Nov 12 11:55:11 2015 New Revision: 230245 URL: https://gcc.gnu.org/viewcvs?rev=230245&root=gcc&view=rev Log: PR target/67265 * config/i386/i386.c (ix86_adjust_stack_and_probe): Remove obsolete assertion on the CFA register. Added: trunk/gcc/testsuite/gcc.target/i386/pr67265-2.c Modified: trunk/gcc/ChangeLog trunk/gcc/config/i386/i386.c trunk/gcc/testsuite/ChangeLog
Author: ebotcazou Date: Thu Nov 12 11:59:23 2015 New Revision: 230247 URL: https://gcc.gnu.org/viewcvs?rev=230247&root=gcc&view=rev Log: PR target/67265 * config/i386/i386.c (ix86_adjust_stack_and_probe): Remove obsolete assertion on the CFA register. Added: branches/gcc-5-branch/gcc/testsuite/gcc.target/i386/pr67265-2.c - copied unchanged from r230245, trunk/gcc/testsuite/gcc.target/i386/pr67265-2.c Modified: branches/gcc-5-branch/gcc/ChangeLog branches/gcc-5-branch/gcc/config/i386/i386.c branches/gcc-5-branch/gcc/testsuite/ChangeLog
Author: ebotcazou Date: Thu Nov 12 12:01:40 2015 New Revision: 230249 URL: https://gcc.gnu.org/viewcvs?rev=230249&root=gcc&view=rev Log: PR target/67265 * config/i386/i386.c (ix86_adjust_stack_and_probe): Remove obsolete assertion on the CFA register. Added: branches/gcc-4_9-branch/gcc/testsuite/gcc.target/i386/pr67265-2.c - copied unchanged from r230247, trunk/gcc/testsuite/gcc.target/i386/pr67265-2.c Modified: branches/gcc-4_9-branch/gcc/ChangeLog branches/gcc-4_9-branch/gcc/config/i386/i386.c branches/gcc-4_9-branch/gcc/testsuite/ChangeLog