Build of GCC 5 is currently broken on x86_64-apple-darwin17.3.0 (macOS 10.13.2) because of a failure in building libsanitizer: In file included from ../../../../libsanitizer/asan/asan_malloc_mac.cc:17:0: /System/Library/Frameworks/CoreFoundation.framework/Headers/CFBase.h:663:61: error: expected initializer before 'API_AVAILABLE' CFTypeRef CFAutorelease(CFTypeRef CF_RELEASES_ARGUMENT arg) API_AVAILABLE(macos(10.9), ios(7.0), watchos(2.0), tvos(9.0)); The failure comes from the header file CoreFoundation/CFBase.h, which uses the API_AVAILABLE macro, which should be defined in /usr/include/os/availability.h but isn't: in that system header, the path followed by any non-clang compiler (such as GCC) ends up defining __API_AVAILABLE/__API_DEPRECATED/__API_DEPRECATED_WITH_REPLACEMENT/__API_UNAVAILABLE where it should define the corresponding API_AVAILABLE/etc macros (without leading underscores). I reported this bug to Apple (radar #36176941). Hopefully, they will fix their system headers. In the mean time, we can get bootstrap working again by fixinclud'ing the header in question. Patch to do so is available at https://github.com/Homebrew/formula-patches/blob/master/gcc%405/10.13_headers.patch
Let's note that this issue breaks bootstrap on GCC 5, but it can also be triggered on any other compiler, including GCC 6 and 7, although for some reason there it doesn't occur during bootstrap (the libsanitizer code is different there). $ cat a.c #include <CoreFoundation/CFBase.h> $ gcc-7 a.c In file included from a.c:1:0: /System/Library/Frameworks/CoreFoundation.framework/Headers/CFBase.h: In function 'CFAutorelease': /System/Library/Frameworks/CoreFoundation.framework/Headers/CFBase.h:663:61: error: expected declaration specifiers before 'API_AVAILABLE' CFTypeRef CFAutorelease(CFTypeRef CF_RELEASES_ARGUMENT arg) API_AVAILABLE(macos(10.9), ios(7.0), watchos(2.0), tvos(9.0)); ^~~~~~~~~~~~~
The issue is caused by `os/availability.h` or `AvailabilityInternal.h`. Instead of `API_AVAILABLE`, only `__API_AVAILABLE` is defined (Clang defines both) This is also the cause of a few failing tests (g++.dg/other/darwin-cfstring1.C for example)
On current 10.136, with Xcode 9.4 command line tools, and/or GCC 7.4 (with a new enough linker to support v2 tbd files), this seems to work for me: clang bootstrap (trunk) https://gcc.gnu.org/ml/gcc-testresults/2019-01/msg01458.html GCC: bootstrap: (trunk) https://gcc.gnu.org/ml/gcc-testresults/2019-01/msg00316.html (8x) https://gcc.gnu.org/ml/gcc-testresults/2019-01/msg00754.html (7x) https://gcc.gnu.org/ml/gcc-testresults/2019-01/msg00762.html AFAICT, Darwin 9 through 18 all bootstrap [with the proviso that Apple-4.2.1 no longer works on the earlier platforms, for trunk (it produces a compiler that fails its self-tests with a memory-management problem), I've been using GCC-5 as the bootstrap on Darwin9 and 10. Is this problem still reproducible to anyone? If not can we close it? Apropos comment #2, yes we have some header hassles; we need to work through the test cases fixing them up - patches welcome! Anyway - let's separate the test-case headers issue from this one.
Iain could you please test if this patch works for you too? If so, I'll send it as a patch tomorrow (For me, it even fixes g++.dg/other/darwin-cfstring1.C): Index: fixincludes/fixincl.x =================================================================== --- fixincludes/fixincl.x (revision 267969) +++ fixincludes/fixincl.x (working copy) @@ -2,11 +2,11 @@ * * DO NOT EDIT THIS FILE (fixincl.x) * - * It has been AutoGen-ed October 16, 2018 at 11:38:39 AM by AutoGen 5.18.7 + * It has been AutoGen-ed January 16, 2019 at 07:37:22 PM by AutoGen 5.18.12 * From the definitions inclhack.def * and the template file fixincl */ -/* DO NOT SVN-MERGE THIS FILE, EITHER Tue Oct 16 11:38:39 CEST 2018 +/* DO NOT SVN-MERGE THIS FILE, EITHER Wed Jan 16 19:37:22 CEST 2019 * * You must regenerate it. Use the ./genfixes script. * @@ -15,7 +15,7 @@ * certain ANSI-incompatible system header files which are fixed to work * correctly with ANSI C and placed in a directory that GNU C will search. * - * This file contains 251 fixup descriptions. + * This file contains 252 fixup descriptions. * * See README for more information. * @@ -2636,6 +2636,56 @@ static const char* apzDarwin_AvailabilityinternalP /* * * * * * * * * * * * * * * * * * * * * * * * * * * + * Description of Darwin_Api_Availability fix + */ +tSCC zDarwin_Api_AvailabilityName[] = + "darwin_api_availability"; + +/* + * File name selection pattern + */ +tSCC zDarwin_Api_AvailabilityList[] = + "os/availability.h\0"; +/* + * Machine/OS name selection pattern + */ +tSCC* apzDarwin_Api_AvailabilityMachs[] = { + "*-*-darwin*", + (const char*)NULL }; + +/* + * content selection pattern - do fix if pattern found + */ +tSCC zDarwin_Api_AvailabilitySelect0[] = + " *#define __API_AVAILABLE.*\n\ + *#define __API_DEPRECATED.*\n\ + *#define __API_DEPRECATED_WITH_REPLACEMENT.*\n\ + *#define __API_UNAVAILABLE.*\n"; + +/* + * content bypass pattern - skip fix if pattern found + */ +tSCC zDarwin_Api_AvailabilityBypass0[] = + "__IPHONE_OS_VERSION_MIN_REQUIRED"; + +#define DARWIN_API_AVAILABILITY_TEST_CT 2 +static tTestDesc aDarwin_Api_AvailabilityTests[] = { + { TT_NEGREP, zDarwin_Api_AvailabilityBypass0, (regex_t*)NULL }, + { TT_EGREP, zDarwin_Api_AvailabilitySelect0, (regex_t*)NULL }, }; + +/* + * Fix Command Arguments for Darwin_Api_Availability + */ +static const char* apzDarwin_Api_AvailabilityPatch[] = { + "format", + " #define API_AVAILABLE(...)\n\ + #define API_DEPRECATED(...)\n\ + #define API_DEPRECATED_WITH_REPLACEMENT(...)\n\ + #define API_UNAVAILABLE(...)\n", + (char*)NULL }; + +/* * * * * * * * * * * * * * * * * * * * * * * * * * + * * Description of Darwin_9_Long_Double_Funcs_2 fix */ tSCC zDarwin_9_Long_Double_Funcs_2Name[] = @@ -10188,9 +10238,9 @@ static const char* apzX11_SprintfPatch[] = { * * List of all fixes */ -#define REGEX_COUNT 289 +#define REGEX_COUNT 291 #define MACH_LIST_SIZE_LIMIT 187 -#define FIX_COUNT 251 +#define FIX_COUNT 252 /* * Enumerate the fixes @@ -10258,6 +10308,7 @@ typedef enum { CTRL_QUOTES_USE_FIXIDX, CXX_UNREADY_FIXIDX, DARWIN_AVAILABILITYINTERNAL_FIXIDX, + DARWIN_API_AVAILABILITY_FIXIDX, DARWIN_9_LONG_DOUBLE_FUNCS_2_FIXIDX, DARWIN_EXTERNC_FIXIDX, DARWIN_GCC4_BREAKAGE_FIXIDX, @@ -10760,6 +10811,11 @@ tFixDesc fixDescList[ FIX_COUNT ] = { DARWIN_AVAILABILITYINTERNAL_TEST_CT, FD_MACH_ONLY | FD_SUBROUTINE, aDarwin_AvailabilityinternalTests, apzDarwin_AvailabilityinternalPatch, 0 }, + { zDarwin_Api_AvailabilityName, zDarwin_Api_AvailabilityList, + apzDarwin_Api_AvailabilityMachs, + DARWIN_API_AVAILABILITY_TEST_CT, FD_MACH_ONLY | FD_SUBROUTINE, + aDarwin_Api_AvailabilityTests, apzDarwin_Api_AvailabilityPatch, 0 }, + { zDarwin_9_Long_Double_Funcs_2Name, zDarwin_9_Long_Double_Funcs_2List, apzDarwin_9_Long_Double_Funcs_2Machs, DARWIN_9_LONG_DOUBLE_FUNCS_2_TEST_CT, FD_MACH_ONLY | FD_SUBROUTINE, Index: fixincludes/inclhack.def =================================================================== --- fixincludes/inclhack.def (revision 267969) +++ fixincludes/inclhack.def (working copy) @@ -1298,6 +1298,33 @@ fix = { }; /* + * macOS 10.13 and 10.14 forget to define API_AVAILABLE if + * __attribute__((availability)) is not supported. + */ +fix = { + hackname = darwin_api_availability; + mach = "*-*-darwin*"; + files = os/availability.h; + bypass = "__IPHONE_OS_VERSION_MIN_REQUIRED"; + select = + " *#define __API_AVAILABLE.*\n" + " *#define __API_DEPRECATED.*\n" + " *#define __API_DEPRECATED_WITH_REPLACEMENT.*\n" + " *#define __API_UNAVAILABLE.*\n"; + c_fix = format; + c_fix_arg = + " #define API_AVAILABLE(...)\n" + " #define API_DEPRECATED(...)\n" + " #define API_DEPRECATED_WITH_REPLACEMENT(...)\n" + " #define API_UNAVAILABLE(...)\n"; + test_text = + "#define __API_AVAILABLE(...)\n" + "#define __API_DEPRECATED(...)\n" + "#define __API_DEPRECATED_WITH_REPLACEMENT(...)\n" + "#define __API_UNAVAILABLE(...)\n"; +}; + +/* * For the AAB_darwin7_9_long_double_funcs fix to be useful, * you have to not use "" includes. */ Index: fixincludes/tests/base/os/availability.h =================================================================== --- fixincludes/tests/base/os/availability.h (nonexistent) +++ fixincludes/tests/base/os/availability.h (working copy) @@ -0,0 +1,18 @@ +/* DO NOT EDIT THIS FILE. + + It has been auto-edited by fixincludes from: + + "fixinc/tests/inc/os/availability.h" + + This had to be done to correct non-standard usages in the + original, manufacturer supplied header file. */ + + + +#if defined( DARWIN_API_AVAILABILITY_CHECK ) + #define API_AVAILABLE(...) + #define API_DEPRECATED(...) + #define API_DEPRECATED_WITH_REPLACEMENT(...) + #define API_UNAVAILABLE(...) + +#endif /* DARWIN_API_AVAILABILITY_CHECK */
* thanks for looking at this! (my machines are all tied up right now, so any testing won't happen before 'tomorrow') * apologies if this comes across as negative - but TBH I'm a bit nervous about applying something like this without wider testing - certainly full reg-straps on affected systems. * A visual inspection of the patch, suggests it might be disabling more than is necessary - GCC has attributes "available" and "deprecated" - it's only "unavailable" that's currently missing [I have a patch for that somewhere, will have to dust it off for 10]. (deprecated_with_replacement could be implemented in terms of deprecated with message). * I did not need any patch to bootstrap open branches [7, 8, and trunk] on 10.13.6 (in fact, not on any Darwin that supports libsanitizer - bearing in mind that upstream doesn't support 10.6 now). * GCC5 is closed, so any patch we might have would be something applied "locally" to any private branches. * The underlying issue is that there are a number of problems with system headers [e.g. missing guards for __has_feature(), missing guards for block syntax, nullability and lightweight generics, the latter two especially an issue for objective c]. The problems tend to be dependent on the SDK version, which is unfortunate because typically one could target 10.13 from 10.14 using the 10.14 SDK, so the target OS version is not a 100% reliable test (TODO: investigate whether there's an SDK version that we can easily pick up). * for tests where there are a large number of problems (e.g. the objective c stuff in particular, where there are no easy work-arounds), it's my plan to cook up some proxy headers to allow us to continue to test what *does* work. * from my investigations, it seems that the underlying issue can be solved by including the relevant headers (but because of the points above "relevant headers" depends on the SDK version). So, it might be that there' a small number of tests for which a workaround will be better.. e.g at present, I have the following to fix the test diff --git a/gcc/testsuite/g++.dg/other/darwin-cfstring1.C b/gcc/testsuite/g++.dg/other/darwin-cfstring1.C index b2def1e858..7c08ef9c91 100644 --- a/gcc/testsuite/g++.dg/other/darwin-cfstring1.C +++ b/gcc/testsuite/g++.dg/other/darwin-cfstring1.C @@ -6,6 +6,12 @@ /* { dg-do compile { target *-*-darwin* } } */ /* { dg-options "-ftrack-macro-expansion=0 -mconstant-cfstrings" } */ +#if __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ >= 101300 +# include <CoreFoundation/CoreFoundation.h> +#elif __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ >= 101000 +# include <CoreFoundation/CFAvailability.h> +#endif + #include <CoreFoundation/CFString.h> #ifdef __CONSTANT_CFSTRINGS__
After reading your comment, I noticed that there were two things I forgot to mention: 1 - availability.h is the file where "API_AVAILABLE" is defined for Clang. 2 - the part of the file the patch changes is 1:1 copied from AvailabilityInternal.h, so someone must've forgotten to replace after pasting. But yes, there's no need to hurry to fix it. It's existed since October 2017; no one has noticed.
I don't think that people didn't notice. I rather think that they gave up building the sanitizer. See also https://github.com/spack/spack/tree/develop/var/spack/repos/builtin/packages/gcc and https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/gcc/darwin/headers-10.13-fix.patch , which includes this fix automatically when GCC is built via Spack.
(In reply to Erik Schnetter from comment #7) > I don't think that people didn't notice. I rather think that they gave up > building the sanitizer. See also > https://github.com/spack/spack/tree/develop/var/spack/repos/builtin/packages/ > gcc and > https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/ > gcc/darwin/headers-10.13-fix.patch , which includes this fix automatically > when GCC is built via Spack. see comment #3 For the record, I build the sanitisers on all systems > Darwin10/macOS 10.6 (where it's no longer supported upstream and is now pretty broken). However, I don't build all the older systems that often - but (for example) - see https://gcc.gnu.org/ml/gcc-testresults/2019-02/msg00055.html So .. I agree that we have test issues with headers [ comment #4 ] (and those need resolving in a general way, not piecemeal) - but AFAICT bootstrap is not broken on current 10.13. [maybe someone is using a different version of the SDK from me? if so, please be specific about the version of Xcode and the SDK in use, thanks.]
(In reply to MCCCS from comment #6) > After reading your comment, I noticed that > there were two things I forgot to mention: > > 1 - availability.h is the file where > "API_AVAILABLE" is defined for Clang. > > 2 - the part of the file the patch > changes is 1:1 copied from > AvailabilityInternal.h, so someone > must've forgotten to replace after > pasting. > > But yes, there's no need to hurry > to fix it. It's existed since > October 2017; no one has noticed. Well, it's still there in the Xcode 10.2 SDK, and I agree it looks like a pasto. I suppose you could cover both bases by augmenting the existing text with the versions without __ .. but perhaps that's OTT. I've done one reg-strap with it on Darwin18 - and it looks OK, so go ahead and post your patch. (a check on Darwin17 seems worthwhile too, so will put that into my queue)
(In reply to MCCCS from comment #6) > After reading your comment, I noticed that > there were two things I forgot to mention: > But yes, there's no need to hurry > to fix it. It's existed since > October 2017; no one has noticed. However, I do want to fix it - and I need a name and email address to put onto the patch credits (or you can post your patch independently - either works).
Author: iains Date: Sun Aug 18 18:54:13 2019 New Revision: 274624 URL: https://gcc.gnu.org/viewcvs?rev=274624&root=gcc&view=rev Log: [Darwin, fixincludes] Fix PR83531 There is no reasonable chance that the SDKs in question will be re- issued, so the only viable solution is a fixincludes. 2019-08-18 C.G. Dogan <gcc+cgdogan.00@gmail.com> Iain Sandoe <iain@sandoe.co.uk> PR target/83531 * inclhack.def (darwin_api_availability): New, strip leading underscores from API_XXXX defines. * fixincl.x: Regenerate. * tests/base/os/availability.h: New file. Added: trunk/fixincludes/tests/base/os/availability.h Modified: trunk/fixincludes/ChangeLog trunk/fixincludes/fixincl.x trunk/fixincludes/inclhack.def
so fixed for trunk so far.
Author: iains Date: Wed Aug 21 19:18:14 2019 New Revision: 274807 URL: https://gcc.gnu.org/viewcvs?rev=274807&root=gcc&view=rev Log: [Darwin, fixincludes] Backport fix for PR83531 There is no reasonable chance that the SDKs in question will be re- issued, so the only viable solution is a fixincludes. 2019-08-21 Iain Sandoe <iain@sandoe.co.uk> Backport from mainline. 2019-08-18 C.G. Dogan <gcc+cgdogan.00@gmail.com> Iain Sandoe <iain@sandoe.co.uk> PR target/83531 * inclhack.def (darwin_api_availability): New; strip leading underscores from API_XXXX defines. * fixincl.x: Regenerate. * tests/base/os/availability.h: New file. Added: branches/gcc-9-branch/fixincludes/tests/base/os/availability.h Modified: branches/gcc-9-branch/fixincludes/ChangeLog branches/gcc-9-branch/fixincludes/fixincl.x branches/gcc-9-branch/fixincludes/inclhack.def
fixed for 9.3
Author: iains Date: Sun Aug 25 19:31:47 2019 New Revision: 274913 URL: https://gcc.gnu.org/viewcvs?rev=274913&root=gcc&view=rev Log: [Darwin, fixincludes] Backport fix for PR83531 There is no reasonable chance that the SDKs in question will be re- issued, so the only viable solution is a fixincludes. 2019-08-24 Iain Sandoe <iain@sandoe.co.uk> Backport from mainline. 2019-08-18 C.G. Dogan <gcc+cgdogan.00@gmail.com> Iain Sandoe <iain@sandoe.co.uk> PR target/83531 * inclhack.def (darwin_api_availability): New; strip leading underscores from API_XXXX defines. * fixincl.x: Regenerate. * tests/base/os/availability.h: New file. Added: branches/gcc-8-branch/fixincludes/tests/base/os/availability.h Modified: branches/gcc-8-branch/fixincludes/ChangeLog branches/gcc-8-branch/fixincludes/fixincl.x branches/gcc-8-branch/fixincludes/inclhack.def
fixed for 8.4
Author: iains Date: Wed Sep 4 19:11:08 2019 New Revision: 275381 URL: https://gcc.gnu.org/viewcvs?rev=275381&root=gcc&view=rev Log: [Darwin, fixincludes] Backport fix for PR83531 There is no reasonable chance that the SDKs in question will be re- issued, so the only viable solution is a fixincludes. 2019-09-04 Iain Sandoe <iain@sandoe.co.uk> Backport from mainline. 2019-08-18 C.G. Dogan <gcc+cgdogan.00@gmail.com> Iain Sandoe <iain@sandoe.co.uk> PR target/83531 * inclhack.def (darwin_api_availability): New; strip leading underscores from API_XXXX defines. * fixincl.x: Regenerate. * tests/base/os/availability.h: New file. Added: branches/gcc-7-branch/fixincludes/tests/base/os/availability.h Modified: branches/gcc-7-branch/fixincludes/ChangeLog branches/gcc-7-branch/fixincludes/fixincl.x branches/gcc-7-branch/fixincludes/inclhack.def
fixed on open branches (needed for 6 and 5 for darwin-specific branches).