This is the mail archive of the gcc-patches@gcc.gnu.org mailing list for the GCC project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: PATCH: Improve Java build times

Right, these are JNI failures which cannot have anything to do with
Mark's patch. Mark, please go ahead and check in the patch!
I've checked in the attached combination of my patches and Bryce's
patches to the 3.1 branch; these together significantly speed up the
Java compiler.

Thanks to all for reviewing, testing, and being patient while we
built up enough confidence to get this checked in.

--
Mark Mitchell mark@codesourcery.com
CodeSourcery, LLC http://www.codesourcery.com
2002-06-03 Mark Mitchell <mark@codesourcery.com>

2002-06-03 Mark Mitchell <mark@codesourcery.com>

2002-05-18 Mark Mitchell <mark@codesourcery.com>
* java-tree.h (CLASS_BEING_LAIDOUT): Remove duplicate definition.
* jcf-io.c (dirent.h): Include it.
(fnmatch.h): Likewise.
(compare_path): New function.
(java_or_class_file): Likewise.
(memoized_dirlist_entry): New type.
(memoized_dirlist_lookup_eq): New function.
(memoized_dirlists): New variable.
(caching_stat): New function.
(memoized_class_lookup_eq): New function.
(memoized_class_lookups): Likewise.
(find_class): Use memoized_class_lookups and caching_stat.
* jcf.h (JCF_USE_SCANDIR): Define.
* parse.y (java_expand_classes): Write the class files in reverse
order.

2002-05-13 Mark Mitchell <mark@codesourcery.com>
* jcf-write.c (write_classfile): Unlink the temporary file if it
cannot be renamed. Use concat to build up the name of the
temporary file.

2002-05-13 Mark Mitchell <mark@codesourcery.com>
* jcf-write.c (write_classfile): Unlink the temporary file if it
cannot be renamed. Use concat to build up the name of the
temporary file.

2002-06-03 Mark Mitchell <mark@codesourcery.com>

2002-05-23 Bryce McKinlay <bryce@waitaki.otago.ac.nz>
* Makefile.am (all-recursive): Depend on $all_java_class_files so that
they build first.
* Makefile.in: Rebuilt.

2002-05-08 Mark Mitchell <mark@codesourcery.com>
* Makefile.am (all_java_source_files): New variable.
(all_java_class_files): Likewise.
.java.class: New rule.
(CLEANFILES): Remove tmp-list.
* Makefile.in: Regenerated.

Index: gcc/java/java-tree.h
===================================================================
RCS file: /cvs/gcc/gcc/gcc/java/java-tree.h,v
retrieving revision 1.136.2.2
diff -c -p -r1.136.2.2 java-tree.h
*** gcc/java/java-tree.h 24 Apr 2002 22:16:08 -0000 1.136.2.2
--- gcc/java/java-tree.h 4 Jun 2002 16:41:19 -0000
*************** extern tree *type_map;
*** 1424,1434 ****
layout of a class. */
#define CLASS_BEING_LAIDOUT(TYPE) TYPE_LANG_FLAG_6 (TYPE)

- /* True if class TYPE is currently being laid out. Helps in detection
- of inheritance cycle occurring as a side effect of performing the
- layout of a class. */
- #define CLASS_BEING_LAIDOUT(TYPE) TYPE_LANG_FLAG_6 (TYPE)
-
/* True if class TYPE has a field initializer finit$ function */
#define CLASS_HAS_FINIT_P(TYPE) TYPE_FINIT_STMT_LIST (TYPE)

--- 1424,1429 ----
Index: gcc/java/jcf-io.c
===================================================================
RCS file: /cvs/gcc/gcc/gcc/java/jcf-io.c,v
retrieving revision 1.33
diff -c -p -r1.33 jcf-io.c
*** gcc/java/jcf-io.c 3 Dec 2001 19:13:40 -0000 1.33
--- gcc/java/jcf-io.c 4 Jun 2002 16:41:19 -0000
***************
*** 1,5 ****
/* Utility routines for finding and reading Java(TM) .class files.
! Copyright (C) 1996, 1997, 1998, 1999, 2000 Free Software Foundation, Inc.

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
--- 1,5 ----
/* Utility routines for finding and reading Java(TM) .class files.
! Copyright (C) 1996, 1997, 1998, 1999, 2000, 2002 Free Software Foundation, Inc.

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
*************** The Free Software Foundation is independ
*** 29,34 ****
--- 29,39 ----
#include "tree.h"
#include "toplev.h"
#include "java-tree.h"
+ #include "hashtab.h"
+ #if JCF_USE_SCANDIR
+ #include <dirent.h>
+ #include <fnmatch.h>
+ #endif

#include "zlib.h"

*************** DEFUN(find_classfile, (filename, jcf, de
*** 304,313 ****
#endif
}

/* Returns a freshly malloc'd string with the fully qualified pathname
! of the .class file for the class CLASSNAME. Returns NULL on
! failure. If JCF != NULL, it is suitably initialized.
! SOURCE_OK is true if we should also look for .java file. */

const char *
DEFUN(find_class, (classname, classname_length, jcf, source_ok),
--- 309,460 ----
#endif
}

+ #if JCF_USE_SCANDIR
+
+ /* A comparison function (as for qsort) that compares KEY (a char *
+ giving the basename of a file) with the name stored in ENTRY (a
+ dirent **). */
+
+ static int
+ DEFUN(compare_path, (key, entry),
+ const void *key AND const void *entry)
+ {
+ return strcmp ((const char *) key,
+ (*((const struct dirent **) entry))->d_name);
+ }
+
+ /* Returns nonzero if ENTRY names a .java or .class file. */
+
+ static int
+ DEFUN(java_or_class_file, (entry),
+ const struct dirent *entry)
+ {
+ const char *base = basename (entry->d_name);
+ return (fnmatch ("*.java", base, 0) == 0 ||
+ fnmatch ("*.class", base, 0) == 0);
+ }
+
+ /* Information about the files present in a particular directory. */
+ typedef struct memoized_dirlist_entry
+ {
+ /* The name of the directory. */
+ const char *dir;
+ /* The number of .java and .class files present, or -1 if we could
+ not, for some reason, obtain the list. */
+ int num_files;
+ /* The .java and .class files in the directory, in alphabetical
+ order. */
+ struct dirent **files;
+ } memoized_dirlist_entry;
+
+ /* Returns true if ENTRY (a memoized_dirlist_entry *) correponds to
+ the directory given by KEY (a char *) giving the directory
+ name. */
+
+ static int
+ DEFUN(memoized_dirlist_lookup_eq, (entry, key),
+ const void *entry AND const void *key)
+ {
+ return strcmp ((const char *) key,
+ ((const memoized_dirlist_entry *) entry)->dir) == 0;
+ }
+
+ /* A hash table mapping directory names to the lists of .java and
+ .class files in that directory. */
+
+ static htab_t memoized_dirlists;
+
+ #endif
+
+ /* Like stat, but avoids actually making the stat system call if we
+ know that it cannot succeed. FILENAME and BUF are as for stat. */
+
+ static int
+ DEFUN(caching_stat, (filename, buf),
+ char *filename AND struct stat *buf)
+ {
+ #if JCF_USE_SCANDIR
+ char *sep;
+ char *base;
+ memoized_dirlist_entry *dent;
+ void **slot;
+
+ /* If the hashtable has not already been created, create it now. */
+ if (!memoized_dirlists)
+ memoized_dirlists = htab_create (37,
+ htab_hash_string,
+ memoized_dirlist_lookup_eq,
+ NULL);
+
+ /* Get the name of the directory. */
+ sep = strrchr (filename, DIR_SEPARATOR);
+ if (sep)
+ {
+ *sep = '\0';
+ base = sep + 1;
+ }
+ else
+ base = filename;
+
+ /* Obtain the entry for this directory form the hash table. */
+ slot = htab_find_slot (memoized_dirlists, filename, INSERT);
+ if (!*slot)
+ {
+ /* We have not already scanned this directory; scan it now. */
+ dent = ((memoized_dirlist_entry *)
+ ALLOC (sizeof (memoized_dirlist_entry)));
+ dent->dir = xstrdup (filename);
+ /* Unfortunately, scandir is not fully standardized. In
+ particular, the type of the function pointer passed as the
+ third argument sometimes takes a "const struct dirent *"
+ parameter, and sometimes just a "struct dirent *". We rely
+ on the ability to interchange these two types of function
+ pointers. */
+ dent->num_files = scandir (filename, &dent->files,
+ java_or_class_file,
+ alphasort);
+ *slot = dent;
+ }
+ else
+ dent = *((memoized_dirlist_entry **) slot);
+
+ /* Put the spearator back. */
+ if (sep)
+ *sep = DIR_SEPARATOR;
+
+ /* If the file is not in the list, there is no need to stat it; it
+ does not exist. */
+ if (dent->num_files != -1
+ && !bsearch (base, dent->files, dent->num_files,
+ sizeof (struct dirent *), compare_path))
+ return -1;
+ #endif
+
+ return stat (filename, buf);
+ }
+
+ /* Returns 1 if the CLASSNAME (really a char *) matches the name
+ stored in TABLE_ENTRY (also a char *). */
+
+ static int
+ DEFUN(memoized_class_lookup_eq, (table_entry, classname),
+ const void *table_entry AND const void *classname)
+ {
+ return strcmp ((const char *)classname, (const char *)table_entry) == 0;
+ }
+
+ /* A hash table keeping track of class names that were not found
+ during class lookup. (There is no need to cache the values
+ associated with names that were found; they are saved in
+ IDENTIFIER_CLASS_VALUE.) */
+ static htab_t memoized_class_lookups;
+
/* Returns a freshly malloc'd string with the fully qualified pathname
! of the .class file for the class CLASSNAME. CLASSNAME must be
! allocated in permanent storage; this function may retain a pointer
! to it. Returns NULL on failure. If JCF != NULL, it is suitably
! initialized. SOURCE_OK is true if we should also look for .java
! file. */

const char *
DEFUN(find_class, (classname, classname_length, jcf, source_ok),
*************** DEFUN(find_class, (classname, classname_
*** 324,334 ****
char *dep_file;
void *entry;
char *java_buffer;

/* Allocate and zero out the buffer, since we don't explicitly put a
null pointer when we're copying it below. */
! int buflen = jcf_path_max_len () + classname_length + 10;
! char *buffer = (char *) ALLOC (buflen);
memset (buffer, 0, buflen);

java_buffer = (char *) alloca (buflen);
--- 471,497 ----
char *dep_file;
void *entry;
char *java_buffer;
+ int buflen;
+ char *buffer;
+ hashval_t hash;
+
+ /* Create the hash table, if it does not already exist. */
+ if (!memoized_class_lookups)
+ memoized_class_lookups = htab_create (37,
+ htab_hash_string,
+ memoized_class_lookup_eq,
+ NULL);
+
+ /* Loop for this class in the hashtable. If it is present, we've
+ already looked for this class and failed to find it. */
+ hash = htab_hash_string (classname);
+ if (htab_find_with_hash (memoized_class_lookups, classname, hash))
+ return NULL;

/* Allocate and zero out the buffer, since we don't explicitly put a
null pointer when we're copying it below. */
! buflen = jcf_path_max_len () + classname_length + 10;
! buffer = (char *) ALLOC (buflen);
memset (buffer, 0, buflen);

java_buffer = (char *) alloca (buflen);
*************** DEFUN(find_class, (classname, classname_
*** 381,387 ****
else
continue;
}
! class = stat (buffer, &class_buf);
}

if (source_ok)
--- 544,550 ----
else
continue;
}
! class = caching_stat(buffer, &class_buf);
}

if (source_ok)
*************** DEFUN(find_class, (classname, classname_
*** 393,399 ****
for (m = 0; m < classname_length; ++m)
java_buffer[m + l] = (classname[m] == '.' ? '/' : classname[m]);
strcpy (java_buffer + m + l, ".java");
! java = stat (java_buffer, &java_buf);
if (java == 0)
break;
}
--- 556,562 ----
for (m = 0; m < classname_length; ++m)
java_buffer[m + l] = (classname[m] == '.' ? '/' : classname[m]);
strcpy (java_buffer + m + l, ".java");
! java = caching_stat (java_buffer, &java_buf);
if (java == 0)
break;
}
*************** DEFUN(find_class, (classname, classname_
*** 464,469 ****
--- 627,638 ----
#endif

free (buffer);
+
+ /* Remember that this class could not be found so that we do not
+ have to look again. */
+ *htab_find_slot_with_hash (memoized_class_lookups, classname, hash, INSERT)
+ = (void *) classname;
+
return NULL;
found:
#if JCF_USE_STDIO
Index: gcc/java/jcf-write.c
===================================================================
RCS file: /cvs/gcc/gcc/gcc/java/jcf-write.c,v
retrieving revision 1.99.2.3
diff -c -p -r1.99.2.3 jcf-write.c
*** gcc/java/jcf-write.c 22 Apr 2002 12:53:21 -0000 1.99.2.3
--- gcc/java/jcf-write.c 4 Jun 2002 16:41:20 -0000
*************** write_classfile (clas)
*** 3374,3389 ****

if (class_file_name != NULL)
{
! FILE *stream = fopen (class_file_name, "wb");
if (stream == NULL)
! fatal_io_error ("can't open %s for writing", class_file_name);

jcf_dependency_add_target (class_file_name);
init_jcf_state (state, work);
chunks = generate_classfile (clas, state);
write_chunks (stream, chunks);
if (fclose (stream))
! fatal_io_error ("error closing %s", class_file_name);
free (class_file_name);
}
release_jcf_state (state);
--- 3374,3402 ----

if (class_file_name != NULL)
{
! FILE *stream;
! char *temporary_file_name;
!
! /* The .class file is initially written to a ".tmp" file so that
! if multiple instances of the compiler are running at once
! they do not see partially formed class files. */
! temporary_file_name = concat (class_file_name, ".tmp", NULL);
! stream = fopen (temporary_file_name, "wb");
if (stream == NULL)
! fatal_io_error ("can't open %s for writing", temporary_file_name);

jcf_dependency_add_target (class_file_name);
init_jcf_state (state, work);
chunks = generate_classfile (clas, state);
write_chunks (stream, chunks);
if (fclose (stream))
! fatal_io_error ("error closing %s", temporary_file_name);
! if (rename (temporary_file_name, class_file_name) == -1)
! {
! remove (temporary_file_name);
! fatal_io_error ("can't create %s", class_file_name);
! }
! free (temporary_file_name);
free (class_file_name);
}
release_jcf_state (state);
Index: gcc/java/jcf.h
===================================================================
RCS file: /cvs/gcc/gcc/gcc/java/jcf.h,v
retrieving revision 1.28.2.2
diff -c -p -r1.28.2.2 jcf.h
*** gcc/java/jcf.h 10 Apr 2002 13:09:01 -0000 1.28.2.2
--- gcc/java/jcf.h 4 Jun 2002 16:41:20 -0000
*************** The Free Software Foundation is independ
*** 63,68 ****
--- 63,76 ----
#define JCF_word JCF_u4
#endif

+ /* If we have both "scandir" and "alphasort", we can cache directory
+ listings to reduce the time taken to search the classpath. */
+ #if defined(HAVE_SCANDIR) && defined(HAVE_ALPHASORT)
+ #define JCF_USE_SCANDIR 1
+ #else
+ #define JCF_USE_SCANDIR 0
+ #endif
+
struct JCF;
typedef int (*jcf_filbuf_t) PARAMS ((struct JCF*, int needed));

Index: gcc/java/parse.y
===================================================================
RCS file: /cvs/gcc/gcc/gcc/java/parse.y,v
retrieving revision 1.353.2.18
diff -c -p -r1.353.2.18 parse.y
*** gcc/java/parse.y 25 Apr 2002 01:08:29 -0000 1.353.2.18
--- gcc/java/parse.y 4 Jun 2002 16:41:22 -0000
*************** java_expand_classes ()
*** 9046,9055 ****
for (cur_ctxp = ctxp_for_generation; cur_ctxp; cur_ctxp = cur_ctxp->next)
{
tree current;
ctxp = cur_ctxp;
! for (current = ctxp->class_list; current; current = TREE_CHAIN (current))
{
! current_class = TREE_TYPE (current);
outgoing_cpool = TYPE_CPOOL (current_class);
if (flag_emit_class_files)
write_classfile (current_class);
--- 9046,9075 ----
for (cur_ctxp = ctxp_for_generation; cur_ctxp; cur_ctxp = cur_ctxp->next)
{
tree current;
+ tree reversed_class_list = NULL;
+
ctxp = cur_ctxp;
!
! /* We write out the classes in reverse order. This ensures that
! inner classes are written before their containing classes,
! which is important for parallel builds. Otherwise, the
! class file for the outer class may be found, but the class
! file for the inner class may not be present. In that
! situation, the compiler cannot fall back to the original
! source, having already read the outer class, so we must
! prevent that situation. */
! for (current = ctxp->class_list;
! current;
! current = TREE_CHAIN (current))
! reversed_class_list
! = tree_cons (NULL_TREE, current, reversed_class_list);
! ggc_add_tree_root (&reversed_class_list, 1);
!
! for (current = reversed_class_list;
! current;
! current = TREE_CHAIN (current))
{
! current_class = TREE_TYPE (TREE_VALUE (current));
outgoing_cpool = TYPE_CPOOL (current_class);
if (flag_emit_class_files)
write_classfile (current_class);
*************** java_expand_classes ()
*** 9061,9066 ****
--- 9081,9088 ----
finish_class ();
}
}
+
+ ggc_del_root (&reversed_class_list);
}
}

Index: libjava/Makefile.am
===================================================================
RCS file: /cvs/gcc/gcc/libjava/Makefile.am,v
retrieving revision 1.202.2.11
diff -c -p -r1.202.2.11 Makefile.am
*** libjava/Makefile.am 3 May 2002 18:22:15 -0000 1.202.2.11
--- libjava/Makefile.am 4 Jun 2002 16:41:23 -0000
*************** install-exec-hook:
*** 163,207 ****
$(LN_S) libgcjx.la gnu-awt-xlib.la; \
fi

! ## Make the .class files depend on the .zip file. This seems
! ## backwards, but is right. This doesn't catch all the .class files,
! ## but that is ok, because the ones it fails to pick up are defined in
! ## a .java file with some other class which is caught. Note that we
! ## only want to create headers for those files which do not have
! ## hand-maintained headers.
! $(built_java_source_files:.java=.class): libgcj-@gcc_version@.jar
! $(java_source_files:.java=.class): libgcj-@gcc_version@.jar
!
! ## The .class files for X will not be included in libgcj.jar, but the
! ## rule for libgcj.jar will cause all out-of-date .class files to be
! ## built. We need this to generate headers for the nat-files.
! $(x_java_source_files:.java=.class): libgcj-@gcc_version@.jar
!
! ## We have the zip file depend on the java sources and not the class
! ## files, because we don't know the names of all the class files.
! ## FIXME: this method fails in a peculiar case: if libgcj.jar is
! ## up-to-date, and foo.class is removed, and bar.java is touched, then
! ## `make libgcj.jar' will not rebuilt foo.class. That's because
! ## libgcj.jar is not out-of-date with respect to foo.java.
! libgcj-@gcc_version@.jar: $(built_java_source_files) $(java_source_files) $(x_java_source_files)
! ## Create a list of all Java sources, without exceeding any shell limits.
! @: $(shell echo Creating list of files to compile...) $(shell rm -f tmp-list || :) $(shell touch tmp-list) $(foreach source,$?,$(shell echo $(source) >> tmp-list))
! @set fnord $(MAKEFLAGS); amf=$$2; fail=no; \
! javac="$(JAVAC)"; \
! cat tmp-list | (while read f; do \
! echo $$javac $(JCFLAGS) -classpath \'\' -bootclasspath $(here):$(srcdir) -d $(here) $$f; \
! $$javac $(JCFLAGS) -classpath '' -bootclasspath $(here):$(srcdir) -d $(here) $$f \
! || case "$$amf" in *=*) exit 1;; *k*) fail=yes ;; *) exit 1;; esac; \
! done; \
! test "$$fail" = no)
! -@rm -f tmp-list libgcj-@gcc_version@.jar
## Note that we explicitly want to include directory information.
find java gnu javax org -type d -o -type f -name '*.class' | \
sed -e '/\/\./d' -e '/\/xlib/d' | \
$(ZIP) cfM0E@ $@

MOSTLYCLEANFILES = $(javao_files) $(nat_files) $(nat_headers) $(c_files) $(x_javao_files) $(x_nat_files) $(x_nat_headers)
! CLEANFILES = tmp-list libgcj-@gcc_version@.jar

clean-local:
## We just remove every .class file that was created.
--- 163,188 ----
$(LN_S) libgcjx.la gnu-awt-xlib.la; \
fi

! all_java_source_files = \
! $(java_source_files) \
! $(built_java_source_files) \
! $(x_java_source_files)
!
! all_java_class_files = $(all_java_source_files:.java=.class)
!
! .java.class:
! $(JAVAC) $(JCFLAGS) -classpath '' -bootclasspath $(here):$(srcdir) \
! -d $(here) $<
!
! libgcj-@gcc_version@.jar: $(all_java_class_files)
! -@rm -f libgcj-@gcc_version@.jar
## Note that we explicitly want to include directory information.
find java gnu javax org -type d -o -type f -name '*.class' | \
sed -e '/\/\./d' -e '/\/xlib/d' | \
$(ZIP) cfM0E@ $@

MOSTLYCLEANFILES = $(javao_files) $(nat_files) $(nat_headers) $(c_files) $(x_javao_files) $(x_nat_files) $(x_nat_headers)
! CLEANFILES = libgcj-@gcc_version@.jar

clean-local:
## We just remove every .class file that was created.
*************** texinfo: TexinfoDoclet.class
*** 1910,1916 ****
## internally by libgcj. We can't make the .o files depend on nat_headers,
## because in that case we'll force a complete rebuild of
## the C++ code whenever any .java file is touched.
! all-recursive: $(nat_headers) $(x_nat_headers)

## ################################################################

--- 1891,1899 ----
## internally by libgcj. We can't make the .o files depend on nat_headers,
## because in that case we'll force a complete rebuild of
## the C++ code whenever any .java file is touched.
! ## Also force all the class files to build first. This makes them build in
! ## the right order to improve performance.
! all-recursive: $(all_java_class_files) $(nat_headers) $(x_nat_headers)

## ################################################################


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]