Re: [Patch] Add option to reduce amount of reflection data to gcj...

[David Daney <ddaney at avtrex dot com>]

Andrew Haley wrote:
> David Daney writes:
>  > Here is my patch to optionally reduce the amount of reflection data 
>  > generated by gcj.  When applied to libgcj on i686 I see about 5% 
>  > reduction in memory image size.
>  > 
>  >     text    data     bss     dec     hex filename
>  > 18937435        3615244  260644 22813323        15c1a8b 
>  > /home/daney/gccsvn/native_clean/i686-pc-linux-gnu/libjava/.libs/libgcj.so.7.0.0
>  > 18284867        3254316  260644 21799827        14ca393 libgcj.so.7.0.0
>  > 
>  > On my GCC-3.4.3/mipsel-linux port of the patch when used in conjunction 
>  > with static linking I am getting about 12-15% savings on a fairly large 
>  > (1000 classes) application.
>  > 
>  > There are of course several caveats:  By eliminating a lot of reflection 
>  > meta-data, a lot of reflection operations will fail.  No big surprize 
>  > there.  JNI and the binary compatibility ABI depend on complete 
>  > reflection meta-data, so they cannot be used in conjunction with reduced 
>  > reflection meta-data.  The byte code interpereter, and the gij program 
>  > work on some, but not all, programs with reduced reflection meta-data.
>  > 
>  > Tested on i686-pc-linux-gnu (FC3).
>  > 
>  > When -freflection-data= is not specified, no regressions with full 
>  > bootstrap of all default languages.  I generated assembly language for a 
>  > 122 class package with and without the patch and the *only* differences 
>  > were the movement of the vtables from after to before the reflection 
>  > data.  This is to be expected from the patch.  Adding 
>  > -freflection-data=reduced to my libgcj.spec.in, resulted in many (about 
>  > 180) failures in the libgcj testsuite.  These were in some, but not all, 
>  > gij compilation tests as well as JNI, BC and some specific reflection 
>  > tests.  I am considering this a pass as some things are not expected to 
>  > work with the -freflection-data=reduced option enabled.
>  > 
>  > OK to commit?
>
> I don't think the -freflection-data=none option should be there in its
> current form, since it breaks so much.  Perhaps this option could be
> enhanced so that we don't silently generate incorrect code.  We should
> report an error instead.

OK, I changed it so it breaks slightly fewer things.

At compile time it is impossible to tell if a method implements part of 
an interface, so we have to assume that any public method implements 
part of an interface and emit reflection data for it.  With 
-freflection-data=none we are saying that no methods implement an 
interface, and no method reflection data is generated.

The whole idea behind this patch is that it breaks things in a 
controlled manner to save some space.  The fact that 
-freflection-data=none intentionally breaks things slightly more than 
-freflection-data=reduced can be useful.  Now it will not silently fail, 
a NoSuchMethodError should be thrown at runtime if it is abused.  The 
additional breakage in the first patch (removal of <clinit> data) was 
removed.

  I hope that allays your fears, but I can remove it if you strongly 
object.

> If it's incompatible with -findirect-dispatch, it should error if
> these options are combined on the command line.

Fixed.

How about this new patch:

Tested on i686-pc-linux-gnu (FC3).
When -freflection-data= is not specified, no regressions with full 
bootstrap of all default languages.  With 
%{fjni|findirect-dispatch:;:-freflection-data=reduced} added to 
libgcj.spec.in 125 failures in libjava, but as I indicated before I 
think they are all expected.


2006-03-12  David Daney  <ddaney@avtrex.com>

        * lang.opt (-freflection-data=): New option.
        * lang.c (handle_reflection_data): New function.
        (java_handle_option): Added case for -freflection-data=.
        (java_post_options): Generate an error if -freflection-data=
        used with -fjni or -findirect-dispatch.
        * java-tree.h (reflection_data_level) New enum.
        (flag_reflection_data): Declare new variable.
        * boehm.c (get_boehm_type_descriptor): Indicate all pointers
        if bitmap overflows and FLAG_REFLECTION_DATA_FULL.
        * class.c (flag_reflection_data): New variable.
        (uses_jv_markobj_p): New function.
        (make_class_data): Moved generation of vtable to before
        reflection data, generate less reflection data if not
        FLAG_REFLECTION_DATA_FULL.
        * gcj.texi: Document -freflection-data=

Index: gcc/java/class.c
===================================================================
--- gcc/java/class.c	(revision 111859)
+++ gcc/java/class.c	(working copy)
@@ -66,6 +66,10 @@
 static tree emit_assertion_table (tree);
 static void register_class (void);
 
+/* Generate full reflection data unless overridden by command line option. */
+
+enum reflection_data_level flag_reflection_data = FLAG_REFLECTION_DATA_FULL;
+
 struct obstack temporary_obstack;
 
 /* The compiler generates different code depending on whether or not
@@ -1548,6 +1552,16 @@
   return 1;
 }
 
+/* The forth (index of 3) element in the vtable is the GC descriptor.
+   A value of 2 indicates that the class uses _Jv_MarkObj. */
+static int
+uses_jv_markobj_p(tree dtable)
+{
+  tree v;
+  v = VEC_index (constructor_elt, CONSTRUCTOR_ELTS (dtable), 3)->value;
+  return (2 == TREE_INT_CST_LOW (v));
+}
+
 void
 make_class_data (tree type)
 {
@@ -1570,7 +1584,10 @@
   tree constant_pool_constructor;
   tree interfaces = null_pointer_node;
   int interface_len = 0;
+  int uses_jv_markobj = 0;
   tree type_decl = TYPE_NAME (type);
+  tree id_main = get_identifier("main");
+  tree id_class = get_identifier("java.lang.Class");
   /** Offset from start of virtual function table declaration
       to where objects actually point at, following new g++ ABI. */
   tree dtable_start_offset = build_int_cst (NULL_TREE,
@@ -1579,6 +1596,22 @@
   this_class_addr = build_class_ref (type);
   decl = TREE_OPERAND (this_class_addr, 0);
 
+  if (supers_all_compiled (type) && ! CLASS_INTERFACE (type_decl)
+      && !flag_indirect_dispatch)
+    {
+      tree dtable = get_dispatch_table (type, this_class_addr);
+      uses_jv_markobj = uses_jv_markobj_p(dtable);
+      dtable_decl = build_dtable_decl (type);
+      DECL_INITIAL (dtable_decl) = dtable;
+      TREE_STATIC (dtable_decl) = 1;
+      DECL_ARTIFICIAL (dtable_decl) = 1;
+      DECL_IGNORED_P (dtable_decl) = 1;
+      TREE_PUBLIC (dtable_decl) = 1;
+      rest_of_decl_compilation (dtable_decl, 1, 0);
+      if (type == class_type_node)
+	class_dtable_decl = dtable_decl;
+    }
+
   /* Build Field array. */
   field = TYPE_FIELDS (type);
   while (field && DECL_ARTIFICIAL (field))
@@ -1589,9 +1622,11 @@
     {
       if (! DECL_ARTIFICIAL (field))
 	{
-	  tree init = make_field_value (field);
 	  if (FIELD_STATIC (field))
 	    {
+              /* We must always create reflection data for static fields
+                 as it is used in the creation of the field itself. */
+              tree init = make_field_value (field);
 	      tree initial = DECL_INITIAL (field);
 	      static_field_count++;
 	      static_fields = tree_cons (NULL_TREE, init, static_fields);
@@ -1603,8 +1638,10 @@
 	      rest_of_decl_compilation (field, 1, 1);
 	      DECL_INITIAL (field) = initial;
 	    }
-	  else
+	  else if (uses_jv_markobj
+                   || (flag_reflection_data == FLAG_REFLECTION_DATA_FULL))
 	    {
+              tree init = make_field_value (field);
 	      instance_field_count++;
 	      instance_fields = tree_cons (NULL_TREE, init, instance_fields);
 	    }
@@ -1643,9 +1680,40 @@
 	 which we don't have a .class file.  */
       if (METHOD_DUMMY (method))
 	continue;
-      init = make_method_value (method);
-      method_count++;
-      methods = tree_cons (NULL_TREE, init, methods);
+
+      /* Generate method reflection data if:
+
+          - FLAG_REFLECTION_DATA_FULL.
+
+          - <clinit> -- The runtime uses reflection to initialize the
+            class.
+
+          - Any method in class java.lang.Class -- Class.forName() and
+            perhaps other things require it.
+
+          - class$ -- It does not work if reflection data missing.
+
+          If !FLAG_REFLECTION_DATA_NONE then also if:
+
+          - main -- Reflection is used to find main(String[]) methods.
+
+          - public -- It is potentially part of an interface.  The
+            runtime uses reflection data to build interface dispatch
+            tables.  */
+      if (flag_reflection_data == FLAG_REFLECTION_DATA_FULL
+          || DECL_CLINIT_P (method)
+          || DECL_NAME (type_decl) == id_class
+          || (DECL_NAME (method) == id_main
+              && flag_reflection_data != FLAG_REFLECTION_DATA_NONE)
+          || (METHOD_PUBLIC (method)
+              && !METHOD_STATIC (method)
+              && flag_reflection_data != FLAG_REFLECTION_DATA_NONE)
+          || TYPE_DOT_CLASS (type) == method)
+        {
+          init = make_method_value (method);
+          method_count++;
+          methods = tree_cons (NULL_TREE, init, methods);
+        }
     }
   method_array_type = build_prim_array_type (method_type_node, method_count);
   methods_decl = build_decl (VAR_DECL, mangled_classname ("_MT_", type),
@@ -1657,21 +1725,6 @@
   DECL_IGNORED_P (methods_decl) = 1;
   rest_of_decl_compilation (methods_decl, 1, 0);
 
-  if (supers_all_compiled (type) && ! CLASS_INTERFACE (type_decl)
-      && !flag_indirect_dispatch)
-    {
-      tree dtable = get_dispatch_table (type, this_class_addr);
-      dtable_decl = build_dtable_decl (type);
-      DECL_INITIAL (dtable_decl) = dtable;
-      TREE_STATIC (dtable_decl) = 1;
-      DECL_ARTIFICIAL (dtable_decl) = 1;
-      DECL_IGNORED_P (dtable_decl) = 1;
-      TREE_PUBLIC (dtable_decl) = 1;
-      rest_of_decl_compilation (dtable_decl, 1, 0);
-      if (type == class_type_node)
-	class_dtable_decl = dtable_decl;
-    }
-
   if (class_dtable_decl == NULL_TREE)
     {
       class_dtable_decl = build_dtable_decl (class_type_node);
@@ -1781,7 +1834,8 @@
 		    CLASS_INTERFACE (type_decl) ? null_pointer_node : super);
   PUSH_FIELD_VALUE (cons, "constants", constant_pool_constructor);
   PUSH_FIELD_VALUE (cons, "methods",
-		    build1 (ADDR_EXPR, method_ptr_type_node, methods_decl));
+                    methods_decl == NULL_TREE ? null_pointer_node
+		    : build1 (ADDR_EXPR, method_ptr_type_node, methods_decl));
   PUSH_FIELD_VALUE (cons, "method_count",
 		    build_int_cst (NULL_TREE, method_count));
 
Index: gcc/java/lang.opt
===================================================================
--- gcc/java/lang.opt	(revision 111859)
+++ gcc/java/lang.opt	(working copy)
@@ -164,6 +164,10 @@
 foutput-class-dir=
 Java Joined RejectNegative
 
+freflection-data=
+Java Joined RejectNegative
+Limit the amount of reflection data generated.  Valid values are: [full|reduced|none]
+
 fstore-check
 Java Var(flag_store_check) Init(1)
 Enable assignability checks for stores into object arrays
Index: gcc/java/lang.c
===================================================================
--- gcc/java/lang.c	(revision 111859)
+++ gcc/java/lang.c	(working copy)
@@ -216,6 +216,17 @@
 /* Each front end provides its own.  */
 const struct lang_hooks lang_hooks = LANG_HOOKS_INITIALIZER;
 
+static void
+handle_reflection_data (const char *arg)
+{
+  if (0 == strcmp (arg, "reduced"))
+    flag_reflection_data = FLAG_REFLECTION_DATA_REDUCED;
+  else if (0 == strcmp (arg, "none"))
+    flag_reflection_data = FLAG_REFLECTION_DATA_NONE;
+  else
+    flag_reflection_data = FLAG_REFLECTION_DATA_FULL;
+}
+
 /*
  * process java-specific compiler command-line options
  * return 0, but do not complain if the option is not recognized.
@@ -324,6 +335,10 @@
       jcf_path_extdirs_arg (arg);
       break;
 
+    case OPT_freflection_data_:
+      handle_reflection_data (arg);
+      break;
+
     case OPT_foutput_class_dir_:
       jcf_write_base_directory = arg;
       break;
@@ -608,6 +623,14 @@
   if (! flag_indirect_dispatch)
     flag_verify_invocations = true;
 
+  if (flag_reflection_data != FLAG_REFLECTION_DATA_FULL)
+    {
+      if (flag_indirect_dispatch)
+        error ("-findirect-dispatch is incompatible with -freflection-data=");
+      if (flag_jni)
+        error ("-fjni is incompatible with -freflection-data=");
+    }
+
   /* Open input file.  */
 
   if (filename == 0 || !strcmp (filename, "-"))
Index: gcc/java/java-tree.h
===================================================================
--- gcc/java/java-tree.h	(revision 111859)
+++ gcc/java/java-tree.h	(working copy)
@@ -157,6 +157,18 @@
 /* The virtual offset symbol table. Used by the runtime to fill out
    the otable. */
 
+enum reflection_data_level
+{
+  FLAG_REFLECTION_DATA_FULL = 0,    /* Generate full reflection data. */
+  FLAG_REFLECTION_DATA_REDUCED = 1, /* Generate only essential
+                                      reflection data. */
+  FLAG_REFLECTION_DATA_NONE = 2     /* Generate no reflection data. */
+};
+
+/* Limits the amount of reflection data generated.  */
+
+extern enum reflection_data_level flag_reflection_data;
+
 extern int flag_emit_class_files;
 
 extern int flag_filelist_file;
Index: gcc/java/gcj.texi
===================================================================
--- gcc/java/gcj.texi	(revision 111859)
+++ gcc/java/gcj.texi	(working copy)
@@ -546,6 +546,39 @@
 @code{Class.forName()} will search @samp{CLASSPATH} to find the
 desired class.
 
+@item -freflection-data=@var{full-or-reduced-or-none}
+This option causes the code generated by @command{gcj} to contain a
+reduced amount of the class meta-data used to support runtime
+reflection in @code{libgcj}. The cost of this savings is the loss of
+the ability to use certain reflection capabilities of the standard
+Java runtime environment. If set to @code{full} (the default if this
+option is not specified), full reflection meta-data is generated.  A
+setting of @code{reduced} eliminates all meta-data except for that
+which is needed to obtain correct runtime semantics.  The final option
+@code{none} eliminates even more meta-data, but it causes meta-data
+essential for proper dispatch of interface calls and class static
+initialization to be eliminated.
+
+For code that does not use reflection (i.e. the methods in the
+@var{java.lang.reflect} package), @code{-freflection-data=reduced}
+will result in proper operation with a savings in executable code size.
+
+If it is known that a class has no methods that are part of an
+interface implementation, greater code size savings may be obtained
+with @code{-freflection-data=none}
+
+JNI (@code{-fjni}) and the binary compatibility ABI
+(@code{-findirect-dispatch}) do not work properly without full
+reflection meta-data.  Because of this, it is an error to use these options
+with @code{-freflection-data=reduced} or @code{-freflection-data=none}.
+
+@strong{Caution:} If there is no reflection meta-data, code that uses
+a @code{SecurityManager} may not work properly.  Also calling
+@code{class.forName()} may fail if the calling method has no
+reflection meta-data.  Calls via an interface to an object of a class
+compiled with @code{-freflection-data=none} will cause an
+@code{NoSuchMethodError} to be thrown.
+
 @end table
 
 
Index: gcc/java/boehm.c
===================================================================
--- gcc/java/boehm.c	(revision 111859)
+++ gcc/java/boehm.c	(working copy)
@@ -1,5 +1,5 @@
 /* Functions related to the Boehm garbage collector.
-   Copyright (C) 2000, 2003, 2004 Free Software Foundation, Inc.
+   Copyright (C) 2000, 2003, 2004, 2006 Free Software Foundation, Inc.
 
 This file is part of GCC.
 
@@ -184,7 +184,9 @@
   /* If the object is all pointers, or if the part with pointers fits
      in our bitmap, then we are ok.  Otherwise we have to allocate it
      a different way.  */
-  if (all_bits_set != -1)
+  if (all_bits_set != -1
+      || (pointer_after_end
+          && (flag_reflection_data != FLAG_REFLECTION_DATA_FULL)))
     {
       /* In this case the initial part of the object is all reference
 	 fields, and the end of the object is all non-reference
@@ -193,7 +195,12 @@
 	 this:
 	 value = DS_LENGTH | WORDS_TO_BYTES (last_set_index + 1);
 	 DS_LENGTH is 0.
-	 WORDS_TO_BYTES shifts by log2(bytes-per-pointer).  */
+	 WORDS_TO_BYTES shifts by log2(bytes-per-pointer).
+
+         In the case of flag_reflection_data != FLAG_REFLECTION_DATA_FULL
+         and the bitmap would overflow, we tell the gc that the object
+         is all pointers so that we don't have to emit reflection data
+         for run time marking. */
       count = 0;
       low = 0;
       high = 0;