This is the mail archive of the gcc@gcc.gnu.org mailing list for the GCC project.

Index Nav:	[Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav:	[Date Prev] [Date Next]	[Thread Prev] [Thread Next]
Other format:	[Raw text]

discussion of ObjC test results on Darwin(/x86) (lengthy)

From: Lars Sonchocky-Helldorf <lars dot sonchocky-helldorf at hamburg dot de>
To: gcc at gcc dot gnu dot org
Cc: objc-language at lists dot apple dot com
Date: Tue, 7 Dec 2004 03:29:13 +0100
Subject: discussion of ObjC test results on Darwin(/x86) (lengthy)

By analyzing the ObjC test results on (Open)Darwin(/x86) I've found out that most of the reported failures are indeed bogus failures caused by wrong parameters or erroneously run tests. I wish that could be corrected, either by correcting the parameters or by not running those tests where they don't apply.

At first I'll give an overview of all the results which I explain in detail later on.

There are two varieties of ObjC test results resulting from the two available ObjC runtimes on (Open)Darwin (you probably already new that).

Results for the next(or apple)-runtime:

=== objc tests ===

Running target unix FAIL: objc/execute/load-2.m execution, -O0 FAIL: objc/execute/load-2.m execution, -O1 FAIL: objc/execute/load-2.m execution, -O2 FAIL: objc/execute/load-2.m execution, -O3 -fomit-frame-pointer FAIL: objc/execute/load-2.m execution, -O3 -g FAIL: objc/execute/load-2.m execution, -Os FAIL: objc/execute/load.m execution, -O0 FAIL: objc/execute/load.m execution, -O1 FAIL: objc/execute/load.m execution, -O2 FAIL: objc/execute/load.m execution, -O3 -fomit-frame-pointer FAIL: objc/execute/load.m execution, -O3 -g FAIL: objc/execute/load.m execution, -Os FAIL: objc.dg/headers.m (test for excess errors) FAIL: objc.dg/stret-2.m scan-assembler objc_msgSend_stret FAIL: objc.dg/stret-2.m scan-assembler-not objc_msgSend[^_S] FAIL: objc.dg/stret-2.m scan-assembler-not objc_msgSendSuper[^_] FAIL: objc.dg/symtab-1.m scan-assembler L_OBJC_SYMBOLS.*:\\n\\t.long\\t0\\n\\t.long\\t0\\n\\t.short\\t2\\n\\t.sh ort\\t0\\n\\t.long\\tL_OBJC_CLASS_Derived.*\\n\\t.long\\tL_OBJC_CLASS_Ba se.*\\n

=== objc Summary ===

# of expected passes		1528
# of unexpected failures	17
# of unsupported tests		1

Results for the gnu(or gcc)-runtime:

=== objc tests ===

Running target unix/-fgnu-runtime FAIL: objc.dg/bitfield-2.m execution test FAIL: objc.dg/const-str-3.m (test for excess errors) WARNING: objc.dg/const-str-3.m compilation failed to produce executable FAIL: objc.dg/const-str-9.m (test for excess errors) ERROR: objc.dg/const-str-9.m: error executing dg-final: couldn't open "const-str-9.s": no such file or directory UNRESOLVED: objc.dg/const-str-9.m: error executing dg-final: couldn't open "const-str-9.s": no such file or directory FAIL: objc.dg/encode-1.m execution test FAIL: objc.dg/image-info.m (test for excess errors) ERROR: objc.dg/image-info.m: error executing dg-final: couldn't open "image-info.s": no such file or directory UNRESOLVED: objc.dg/image-info.m: error executing dg-final: couldn't open "image-info.s": no such file or directory FAIL: objc.dg/method-4.m scan-assembler _OBJC_CLASS_REFERENCES_0 FAIL: objc.dg/selector-1.m (test for warnings, line 22) FAIL: objc.dg/selector-1.m (test for excess errors) FAIL: objc.dg/stret-2.m scan-assembler objc_msgSend_stret FAIL: objc.dg/stret-2.m scan-assembler objc_msgSendSuper_stret FAIL: objc.dg/symtab-1.m scan-assembler L_OBJC_SYMBOLS.*:\\n\\t.long\\t0\\n\\t.long\\t0\\n\\t.short\\t2\\n\\t.sh ort\\t0\\n\\t.long\\tL_OBJC_CLASS_Derived.*\\n\\t.long\\tL_OBJC_CLASS_Ba se.*\\n FAIL: objc.dg/zero-link-1.m scan-assembler objc_getClass FAIL: objc.dg/zero-link-2.m scan-assembler _OBJC_CLASS_REFERENCES_0

=== objc Summary ===

# of expected passes		1528
# of unexpected failures	13
# of unresolved testcases	2
# of unsupported tests		1

the used compiler was (in both cases):

/Volumes/Data/Users/lars/GCC/FSF/gcc-build/gcc/xgcc version 4.0.0 20041205 (experimental)


Compiler version: 4.0.0 20041205 (experimental) objc
Platform: i686-apple-darwin7.2.1
configure flags: --prefix=/tmp/gcc --enable-languages=c,c++,java,objc

And now the details, starting with the next-runtime:

FAIL: objc/execute/load-2.m execution,  -O0
FAIL: objc/execute/load-2.m execution,  -O1
FAIL: objc/execute/load-2.m execution,  -O2
FAIL: objc/execute/load-2.m execution,  -O3 -fomit-frame-pointer
FAIL: objc/execute/load-2.m execution,  -O3 -g
FAIL: objc/execute/load-2.m execution,  -Os
FAIL: objc/execute/load.m execution,  -O0
FAIL: objc/execute/load.m execution,  -O1
FAIL: objc/execute/load.m execution,  -O2
FAIL: objc/execute/load.m execution,  -O3 -fomit-frame-pointer
FAIL: objc/execute/load.m execution,  -O3 -g
FAIL: objc/execute/load.m execution,  -Os

both are execution tests that check if +load is automatically called before main is run, once on a single class (load.m) once on two classes (load-2.m). This might be the right behaviour for the gnu-runtime but is different for the next-runtime, which loads classes on demand (and uses a caching system: OBJC_EXPORT IMP _class_lookupMethodAndLoadCache(Class, SEL); defined in objc4/runtime/objc-private.h).

To make load.m work I had to change it to:

#import <objc/Protocol.h>

static int static_variable = 0;

/*
 * +load is implemented in Protocol.m (objc4/runtime/Protocol.m)
 * (which subclasses Object) but not defined in Protocol.h
 * -> you'll get a warning:
 * 'Protocol' may not respond to '+load'
 */
@interface TestClass : Protocol
{

}
+ (void) load;
@end

@implementation TestClass
+ (void) load
{
  static_variable = 1;
  [super load];
}
@end

int main (void)
{
  id test = [[TestClass alloc] init];

  if (static_variable != 1)
    {
      abort ();
    }

  return 0;
}

FAIL: objc.dg/headers.m (test for excess errors)

headers.m tries to include <Foundation/NSString.h> which is not part of pure Darwin/OpenDarwin, but only distributed with Mac OS X (as part of Cocoa).

#ifdef __NEXT_RUNTIME__
#include <Foundation/NSString.h>
#else


FAIL: objc.dg/stret-2.m scan-assembler objc_msgSend_stret
FAIL: objc.dg/stret-2.m scan-assembler-not objc_msgSend[^_S]
FAIL: objc.dg/stret-2.m scan-assembler-not objc_msgSendSuper[^_]

Maybe that's a real failure. stret-2.s containts 'L_objc_msgSendSuper_stret$stub', '_objc_msgSendSuper_stret', 'L_objc_msgSend$stub' and '_objc_msgSend' but not just 'objc_msgSend_stret'. I have the .s file handy if needed. Drop me a line.

FAIL: objc.dg/symtab-1.m scan-assembler L_OBJC_SYMBOLS.*:\\n\\t.long\\t0\\n\\t.long\\t0\\n\\t.short\\t2\\n\\t.sh ort\\t0\\n\\t.long\\tL_OBJC_CLASS_Derived.*\\n\\t.long\\tL_OBJC_CLASS_Ba se.*\\n

the assembler in question looks like:

L_OBJC_SYMBOLS:
	.long	0
	.long	0
	.word	2
	.word	0
	.long	L_OBJC_CLASS_Derived
	.long	L_OBJC_CLASS_Base

I don't know if this is correct (words instead of shorts (imagine this in real life btw. ;-))).

and now the results for the gnu-runtime:

FAIL: objc.dg/bitfield-2.m execution test

contains (like others to follow):

/* { dg-options "-fnext-runtime -fsigned-char" } */

this clearly conflicts with -fgnu-runtime which was used to run those tests. the resulting command line was:

/Volumes/Data/Users/lars/GCC/FSF/gcc-build/gcc/xgcc -B/Volumes/Data/Users/lars/GCC/FSF/gcc-build/gcc/ /Users/lars/GCC/FSF/gcc-4.0-20041205/gcc/testsuite/objc.dg/bitfield-2.m -fnext-runtime -fsigned-char -I/Users/lars/GCC/FSF/gcc-4.0-20041205/gcc/testsuite/../../libobjc -L/Volumes/Data/Users/lars/GCC/FSF/gcc-build/i686-apple-darwin7.2.1/./ libobjc/.libs -lobjc -lm -fgnu-runtime -o ./bitfield-2.exe

btw. this test also fails if is omitted. but in the source code it says:

/* Check if bitfield ivars are correctly @encode'd when
   the NeXT runtime is used.  */

so maybe this test is obsolete for the gnu-runtime.


FAIL: objc.dg/const-str-3.m (test for excess errors)
WARNING: objc.dg/const-str-3.m compilation failed to produce executable

fails during linking with:

/usr/bin/ld: Undefined symbols:
_objc_getClass
collect2: ld returned 1 exit status

obviously objc_getClass() is not defined for the gnu-runtime which is used in line 41:

memcpy(&_FooClassReference, objc_getClass("Foo"), sizeof(_FooClassReference));

this could be rewritten for the gnu-runtime if that test makes sense there

furthermore '-fnext-runtime' and '-fgnu-runtime' are conflicting here too.

FAIL: objc.dg/const-str-9.m (test for excess errors) ERROR: objc.dg/const-str-9.m: error executing dg-final: couldn't open "const-str-9.s": no such file or directory UNRESOLVED: objc.dg/const-str-9.m: error executing dg-final: couldn't open "const-str-9.s": no such file or directory

Maybe a real bug. Compilation fails with:

/Users/lars/GCC/FSF/gcc-4.0-20041205/gcc/testsuite/objc.dg/const-str- 9.m:17: error: cannot find interface declaration for 'NXConstantString'

'-fnext-runtime' and '-fgnu-runtime' yadda yadda ...

FAIL: objc.dg/encode-1.m execution test

Fails to execute (but compiles). I don't know if this test is applicable to the gnu-runtime (some comment inside the source lets me assume something else)

FAIL: objc.dg/image-info.m (test for excess errors) ERROR: objc.dg/image-info.m: error executing dg-final: couldn't open "image-info.s": no such file or directory UNRESOLVED: objc.dg/image-info.m: error executing dg-final: couldn't open "image-info.s": no such file or directory

fails deliberately during compilation:

/Users/lars/GCC/FSF/gcc-4.0-20041205/gcc/testsuite/objc.dg/image- info.m:9:2: error: #error Feature not currently supported by the GNU runtime

compiler exited with status 1

because of lines 8 to 10:

#ifndef __NEXT_RUNTIME__
#error Feature not currently supported by the GNU runtime
#endif

not very gentleman like ;-)

FAIL: objc.dg/method-4.m scan-assembler _OBJC_CLASS_REFERENCES_0

containts for instance '___objc_class_ref_Object' but not '_OBJC_CLASS_REFERENCES_0'. Is this test applicable for the gnu-runtime?

FAIL: objc.dg/selector-1.m  (test for warnings, line 22)
FAIL: objc.dg/selector-1.m (test for excess errors)

see: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=18862

FAIL: objc.dg/stret-2.m scan-assembler objc_msgSend_stret
FAIL: objc.dg/stret-2.m scan-assembler objc_msgSendSuper_stret

I think not applicable. Every 'objc_msgSend' is replaced by 'objc_msg_lookup'. next and gnu runtime differ here in naming.

FAIL: objc.dg/symtab-1.m scan-assembler L_OBJC_SYMBOLS.*:\\n\\t.long\\t0\\n\\t.long\\t0\\n\\t.short\\t2\\n\\t.sh ort\\t0\\n\\t.long\\tL_OBJC_CLASS_Derived.*\\n\\t.long\\tL_OBJC_CLASS_Ba se.*\\n

the assembler in question looks like:

L_OBJC_SYMBOLS:
	.long	0
	.long	0
	.word	2
	.word	0
	.long	L_OBJC_CLASS_Derived
	.long	L_OBJC_CLASS_Base

btw: same as for the next-runtime


FAIL: objc.dg/zero-link-1.m scan-assembler objc_getClass
FAIL: objc.dg/zero-link-2.m scan-assembler _OBJC_CLASS_REFERENCES_0

the gnu-runtime works completely different for both cases (other names and so on). I could provide you with the .s files if you want to: drop me a line.

So, now somebody has any comments on that or is able to improve the situation?

This was a long one. hope I didn't tire anybody out. I am tired myself now (already 3:30 am here) ... :-)

regards, Lars

Index Nav:	[Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav:	[Date Prev] [Date Next]	[Thread Prev] [Thread Next]