http://www.maillistarchive.com/threads/1657-gccgo-branch-and-darwin

This is going to require some work from somebody familiar with Darwin.

I was able to get gcc to compile, and compile and run a trivial Go program on OS X with the attached patch.  Since I know little about gcc, the patch is just a hack, but maybe it can help.

The issues it addresses are:

- some of the stuff that mksysinfo.sh generates doesn't compile.

- The frontend hardcodes ".go_export" both in the C++ code and in the argument to objcopy, but Darwin wants both a segment and section name.

- A lot of code relies on asm() for symbol names, but Darwin wants an underscore at the beginning (hardcode the underscore; obviously not a good plan).

- SIGRTMIN is not defined (use SIGUSR1/2).

- xnu doesn't support sem_init() (use pthreads).

- the compiler seems to want an explicit -o with -c, but compiler_c_o in libtool somehow gets defined as no.

- syscall.Sysctl isn't defined (I didn't bother to fix this but just commented out the only use).

- a typo in libgo/runtime/go-new-map.c which is probably unrelated and belongs in another bug.

GCC version: gcc-4.6-20110219 
System: Mac OS X 10.6.6 (with MacPorts installed)
Configuration: ./configure --prefix=/opt/gccgo --with-boot-ldflags="/opt/local/lib/libiconv.dylib /usr/lib/libiconv.dylib" --disable-bootstrap --with-ppl=/opt/local --with-cloog=/opt/local --with-mpfr=/opt/local --enable-gold=no --enable-ld=no --enable-languages=c,c++,go LDFLAGS="/opt/local/lib/libiconv.dylib /usr/lib/libiconv.dylib"

Created attachment 23482 [details]
hacky patch that should be split up

See also http://code.google.com/p/go/issues/detail?id=463 .

Created attachment 25168 [details]
updated patch for 4.6.1

hello world tests OK on Snow Leopard, with patch

Another problem is that the -static-libgo flag doesn't work, because the static library has the same name and location as the dynamic lib and therefore darwin will always prefer the dynamic library. It probably needs to be renamed (to something like libgo_static.a), since otherwise the -Wl,-search_paths_first flag is needed to avoid picking up libgo.dylib even if it comes later in the paths...

(In reply to comment #6)
> Created attachment 25168 [details]
> updated patch for 4.6.1
> 
> hello world tests OK on Snow Leopard, with patch

This patch fails on darwin11 when applied to gcc-4_6-branch...


f=`echo sort/sort.lo | sed -e 's/.lo$/.o/'`; missing-objcopy -j __GNU_GO.__go_export $f sort.gox.tmp && mv -f sort.gox.tmp sort.gox
/bin/sh: missing-objcopy: command not found
make[4]: *** [sort.gox] Error 127
make[4]: *** Waiting for unfinished jobs....

It seems to required binutils to be installed (which we really want to avoid).

(In reply to comment #8)
> > hello world tests OK on Snow Leopard, with patch
> 
> This patch fails on darwin11 when applied to gcc-4_6-branch...

Seems like the patch worked (= didn't fail to apply),
but that the build failed due to missing dependencies ?
 
> f=`echo sort/sort.lo | sed -e 's/.lo$/.o/'`; missing-objcopy -j
> __GNU_GO.__go_export $f sort.gox.tmp && mv -f sort.gox.tmp sort.gox
> /bin/sh: missing-objcopy: command not found
> make[4]: *** [sort.gox] Error 127
> make[4]: *** Waiting for unfinished jobs....
> 
> It seems to required binutils to be installed (which we really want to avoid).

Yes, it requires OBJCOPY=gobjcopy. Maybe this requirement
should be replaced with some custom "gox-extract" tool ?

Since the .go_export section name had to be changed, anyway:
http://golang.org/doc/gccgo_install.html#Imports

Here's my attempt at a native version.
(of that "gox-extract" shell script)

Instead of the default variant:
OBJCOPY=${OBJCOPY:-objcopy}
$OBJCOPY -j .go_export $1 $2

The Darwin version could be this:
OTOOL=${OTOOL:-otool}
$OTOOL -s __GNU_GO __go_export $1 |
grep -v "^$1:" | grep -v "Contents of (__GNU_GO,__go_export) section" |
cut -f 2- | tr -d ' ' | xxd -r -p - $2

It doesn't include the objcopy header,
but I believe that is skipped anyway ?

> OTOOL=${OTOOL:-otool}
> $OTOOL -s __GNU_GO __go_export $1 |
> grep -v "^$1:" | grep -v "Contents of (__GNU_GO,__go_export) section" |
> cut -f 2- | tr -d ' ' | xxd -r -p - $2

Apparently -X avoids that header text:
$OTOOL -s __GNU_GO __go_export -X $1 | cut -f 2- | tr -d ' ' | xxd -r -p - $2

> It doesn't include the objcopy header,
> but I believe that is skipped anyway ?

Or at least it was *supposed* to ignore it,
but the Stream_from_file was horribly buggy.
(apparently has a dyslectic problem with
comparisons, aggrevated by copy/paste ?)

It always returned "" instead of any data,
so failed to provide the required magic...
(or any other data beyond that, if asked)
Fixing that class, and it works just fine:

Index: gcc/go/gofrontend/import.cc
===================================================================
--- gcc/go/gofrontend/import.cc	(revision 178444)
+++ gcc/go/gofrontend/import.cc	(working copy)
@@ -836,7 +836,7 @@
 bool
 Stream_from_file::do_peek(size_t length, const char** bytes)
 {
-  if (this->data_.length() <= length)
+  if (this->data_.length() >= length)
     {
       *bytes = this->data_.data();
       return true;
@@ -854,7 +854,7 @@
       return false;
     }
 
-  if (lseek(this->fd_, - got, SEEK_CUR) != 0)
+  if (lseek(this->fd_, - got, SEEK_CUR) < 0)
     {
       if (!this->saw_error())
 	error("lseek failed: %m");
@@ -876,7 +876,7 @@
 void
 Stream_from_file::do_advance(size_t skip)
 {
-  if (lseek(this->fd_, skip, SEEK_CUR) != 0)
+  if (lseek(this->fd_, skip, SEEK_CUR) < 0)
     {
       if (!this->saw_error())
 	error("lseek failed: %m");

That bug should affect any other platform too,
if trying to use "naked" .gox (without object) ?

Created attachment 25177 [details]
import-export.diff

Just the import/export changes, i.e. outside "libgo" directory.

(In reply to comment #13)
> Created attachment 25177 [details]
> import-export.diff
> 
> Just the import/export changes, i.e. outside "libgo" directory.

This change is darwin-centric. Peter O'Gorman suggests something along the lines of...

>> just use a gnu make conditional, since gcc requires gnu make to build anyway, check if $OTOOL is != "" e.g. ifeq($(OTOOL),) ... use objcopy else .. use otool ..endif

The whole patch is Darwin-centric, in that it's only applied on Darwin.
You are right that it first needs to be conditionalized to go upstream.

afaik, $(OTOOL) would be something like "missing-otool" on e.g. Linux ?
(just as we now get $(OBJCOPY) set to "missing-objcopy" on Mac OS X...)

Created attachment 25196 [details]
import-export.diff

Here's an updated portable version. Using otool "if LIBGO_IS_DARWIN".

Used a shell function instead of a shell script, for "gengox" tool...

The most recent patch is against gcc 4.6, and mainline is different.

I have just moved the support for reading export data out of the Go frontend proper.  The purpose of this move is to permit the control for where the data lives to be entirely in gcc, including gcc/config/darwin.h.

I think that now the only thing which will need to change in the Go frontend proper is using otool instead of objcopy in libgo/Makefile.am.

Can you set up a new patch against mainline?

Thanks.

Somebody will have to work on this but I don't have the time.

(In reply to comment #17)
> I think that now the only thing which will need to change in the Go frontend
> proper is using otool instead of objcopy in libgo/Makefile.am.

The otool was simple enough, but the other go_export issues still remain.
And there are some new issues with ucontext being deprecated on Mac OS X.

Created attachment 26668 [details]
updated patch for trunk

Created attachment 26669 [details]
gcc-go_export.diff

Need to give both segment and section for export on Darwin:

/var/folders/SU/SU8yve9tFP4OXneVoY+KsU+++TI/-Tmp-//ccWicBIX.s:3:Expected comma after segment-name
/var/folders/SU/SU8yve9tFP4OXneVoY+KsU+++TI/-Tmp-//ccWicBIX.s:3:Rest of line ignored. 1st junk character valued 32 ( ).
/var/folders/SU/SU8yve9tFP4OXneVoY+KsU+++TI/-Tmp-//ccWicBIX.s:3:Junk character 92 (\).
/var/folders/SU/SU8yve9tFP4OXneVoY+KsU+++TI/-Tmp-//ccWicBIX.s:3:Rest of line ignored. 1st junk character valued 49 (1).

Created attachment 26670 [details]
gcc-ucontext.diff

ucontext is deprecated on Mac OS X 10.5-10.6, including ucontext.h gives error:

In file included from ../../../gcc/libgo/runtime/runtime.h:20:0,
                 from ../../../gcc/libgo/runtime/go-main.c:21:
/usr/include/ucontext.h:42:2: error: #error ucontext routines are deprecated, and require _XOPEN_SOURCE to be defined

ucontext is missing altogether from Mac OS X 10.7, but there is no TLS either:

error: thread-local storage not supported for this target

I'm fine with the last patch, though, I think it needs Ian to approve.

(In reply to comment #22)
> 
> ucontext is missing altogether from Mac OS X 10.7, but there is no TLS either:
> 
> error: thread-local storage not supported for this target

There is tis support within clang in Xcode 4.2 (see radar:10291355, "Xcode 4.2 miscompiles tls support in
MPFR 3.1.0" which was fixed in Xcode 4.2.1), however FSF gcc currently doesn't generate the required assembly for tis support.

(In reply to comment #23)
> I'm fine with the last patch, though, I think it needs Ian to approve.

Wouldn't it make more sense to file a joint PR with the OpenBSD folks in order to have the use of ucontext purged from libgo (considering that neither BSD should have it now and that feature has been deprecated from POSIX)?

I think it would be great if somebody would tell me something I can used instead of makecontext/getcontext/setcontext.  Unless somebody can come up with one, then I think the only alternative is to write our own processor-specific versions and distribute them with libgo.

The functions were removed from POSIX not because there was any adequate replacement, but because they suffer from some design deficiencies.

(In reply to comment #26)
> I think it would be great if somebody would tell me something I can used
> instead of makecontext/getcontext/setcontext.  Unless somebody can come up with
> one, then I think the only alternative is to write our own processor-specific
> versions and distribute them with libgo.
> 
> The functions were removed from POSIX not because there was any adequate
> replacement, but because they suffer from some design deficiencies.

Would implementing "cooperative threading" with pthreads, perhaps with conditional variables, as described in...

http://stackoverflow.com/questions/4298986/is-there-something-to-replace-the-ucontext-h-functions

be appropriate?

Using pthreads will be much less efficient than the current code using getcontext/setcontext.  Writing machine-specific replacement code would be a much better idea than that.

It still needs to generate the extra underscore for Darwin,
and it still needs to fix the bug mentioned in Comment #12.

Created attachment 26682 [details]
gcc-gox_import.diff

Created attachment 26683 [details]
gcc-darwin_goc2c.patch

Then it's more about fixing library issues, like the macros used
for TIOCNOTTY/TIOCSCTTY or st_atime/st_mtime/st_ctime and sysctl.

Created attachment 26699 [details]
gcc-darwin_libgo.patch

There is the other big change needed, which is to prefix all the asm names
with USER_LABEL_PREFIX (_) so that they will actually link properly later.

At least it compiles again, after adding asm prefix and ptrace/sysctl...
Hacked it in for now, but needs a better patch for runtime and syscall.

Still fails to actually run, but I guess that is due to something else:
"panic: runtime error: invalid memory address or nil pointer dereference"

Author: ian
Date: Thu Jan 24 18:12:23 2013
New Revision: 195436

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=195436
Log:
	PR go/46986
libgo/Makefile, libgo/go/os/stat_atimespec.go: fix typos

Modified:
    trunk/libgo/Makefile.am
    trunk/libgo/Makefile.in
    trunk/libgo/go/os/stat_atimespec.go

Author: ian
Date: Thu Jan 24 19:44:23 2013
New Revision: 195438

URL: http://gcc.gnu.org/viewcvs?root=gcc&view=rev&rev=195438
Log:
	PR go/46986
all: prepend #__USER_LABEL_PREFIX__ to mangled Go symbols
For old-fashioned Darwin.

Modified:
    trunk/libgo/go/bytes/indexbyte.c
    trunk/libgo/go/log/syslog/syslog_c.c
    trunk/libgo/go/sync/atomic/atomic.c
    trunk/libgo/go/sync/cas.c
    trunk/libgo/go/syscall/errno.c
    trunk/libgo/go/syscall/signame.c
    trunk/libgo/go/syscall/wait.c
    trunk/libgo/runtime/chan.c
    trunk/libgo/runtime/cpuprof.c
    trunk/libgo/runtime/env_posix.c
    trunk/libgo/runtime/go-breakpoint.c
    trunk/libgo/runtime/go-caller.c
    trunk/libgo/runtime/go-callers.c
    trunk/libgo/runtime/go-cgo.c
    trunk/libgo/runtime/go-fieldtrack.c
    trunk/libgo/runtime/go-getgoroot.c
    trunk/libgo/runtime/go-now.c
    trunk/libgo/runtime/go-reflect-map.c
    trunk/libgo/runtime/go-setenv.c
    trunk/libgo/runtime/go-signal.c
    trunk/libgo/runtime/go-typestring.c
    trunk/libgo/runtime/go-unsafe-new.c
    trunk/libgo/runtime/go-unsafe-newarray.c
    trunk/libgo/runtime/go-unsafe-pointer.c
    trunk/libgo/runtime/goc2c.c
    trunk/libgo/runtime/lfstack.c
    trunk/libgo/runtime/malloc.goc
    trunk/libgo/runtime/malloc.h
    trunk/libgo/runtime/mgc0.c
    trunk/libgo/runtime/mprof.goc
    trunk/libgo/runtime/panic.c
    trunk/libgo/runtime/parfor.c
    trunk/libgo/runtime/proc.c
    trunk/libgo/runtime/runtime.c
    trunk/libgo/runtime/runtime.h

*** Bug 55969 has been marked as a duplicate of this bug. ***

*** Bug 77861 has been marked as a duplicate of this bug. ***

(In reply to Jack Howarth from comment #24)
> (In reply to comment #22)
> > 
> > ucontext is missing altogether from Mac OS X 10.7, but there is no TLS either:
> > 
> > error: thread-local storage not supported for this target
> 
> There is tis support within clang in Xcode 4.2 (see radar:10291355, "Xcode
> 4.2 miscompiles tls support in
> MPFR 3.1.0" which was fixed in Xcode 4.2.1), however FSF gcc currently
> doesn't generate the required assembly for tis support.

This is bug 52268, for reference

Would this package help at all? https://github.com/blacktop/go-macho

No, the go-macho package is irrelevant here.

What is the current status of gccgo on Darwin?

gccgo still does not work on Darwin.

(In reply to Ian Lance Taylor from comment #44)
> gccgo still does not work on Darwin.

is work needed on the runtime, or on code-gen (or both)?
If there was someone who had some spare time to work on it...

A small bit of work is needed on the codegen, to read and write the export data.  And some work is required on the runtime, to clean it up to support Darwin.  It has to be done by someone with access to a modern Darwin system.

(In reply to Ian Lance Taylor from comment #46)
> A small bit of work is needed on the codegen, to read and write the export
> data.  And some work is required on the runtime, to clean it up to support
> Darwin.  It has to be done by someone with access to a modern Darwin system.

I could do something on this, perhaps, provided it is not overly demanding in terms of expertise. Or do those parts which are such :)

While I have more interest in Darwin PPC, since Go is broken there (ppc32 support exists in out-of-tree branch of Go 1.9, though not for Darwin, however the only Go bootstrappable from C, 1.4, is x86-only), access to modern x86_64 Darwin is there.

(In reply to Sergey Fedorov from comment #47)
> (In reply to Ian Lance Taylor from comment #46)
> > A small bit of work is needed on the codegen, to read and write the export
> > data.  And some work is required on the runtime, to clean it up to support
> > Darwin.  It has to be done by someone with access to a modern Darwin system.
> 
> I could do something on this, perhaps, provided it is not overly demanding
> in terms of expertise. Or do those parts which are such :)
> 
> While I have more interest in Darwin PPC, since Go is broken there (ppc32
> support exists in out-of-tree branch of Go 1.9, though not for Darwin,
> however the only Go bootstrappable from C, 1.4, is x86-only), access to
> modern x86_64 Darwin is there.

with care, most of the time, FE stuff tends to be quite neutral about the eventual Darwin target (generic Mach-O constraints are similar between the sub-targets***).  The runtime, of course, is likely to be more specific.   I'd recommend starting on x86_64 since that's pretty stable, and then we can look at i686 ... and when that works, ppc is likely to be quite easy.

Arm64 is still pretty experimental, and there are known TODOs for ABI compliance, so I'd avoid that sub-platform initially (despite that it's the most modern).

*** aarch64/arm64 Darwin has some much tighter constraints security-wise - in particular stack-based trampolines are not allowed - but we have a draft of a heap-based scheme in the devt. branch .. you'd just need to look at what has been done for C, Fortran and Ada.

If someone happens to have some WIP on this, more recent than 2012, please share, if possible.