JDK -> gcj: A Makefile and build system for java projects

Wed Apr 21 17:32:00 GMT 2004

All:

I've put together a Makefile and two bash scripts that I am using to move my java programs away
from Sun's JDK to gcj.  I believe I have implemented most of the flexibility on Sun's way of
doing things.

In the recent discussion of Richard Stallman's article "Free but shackled: The Java trap," there
were a few comments to the effect that some people would love to switch to gcj but are blocked by
the lack of a build system then can use, or unfamilarity with the GCC way of compiling and
linking, or some combination the two.

I know I have been blocked for the past year by those two issues and I suspect I am not alone.

Anyway, I have put together a tarball you can download from my website at 

  http://www.hopelesscase.com:81/gcj

with an example java project, a README with instructions, and the Makefile and bash scripts.

I have appeded the README to this post.

Chris Marshall

A Simple Makefile and Build System for Java Projects Using the GNU Java Compiler

by Chris Marshall
christopherlmarshall@yahoo.com
http://www.hopelesscase.com:81/gcj
2004-04

>>>1. INTRODUCTION

This set of files is a template I use to compile my java programs
with gcj.  I offer it in the hopes that someone else will find
it useful in making the switch from Sun's JDK to gcj, in the cases
where that makes sense for them. While Sun's JDK is ahead of gcj in 
some areas, gcj is ahead of Sun's JDK in other areas.  Two of those
areas are that gcj doesn't limit your redistribution rights, and 
it allows you to compile your programs to native executable binaries,
which can be very useful.

It took me several attempts (lasting over a year from first to last) 
to figure out how to use gcj in an equivalent fashion to how I had 
been using Sun's JDK.  Part of the reason it took so long is I was 
slow to ask for help on mailing lists.  The gcj mailing list has to 
be one of the most helpful out there.  Don't be afraid to subscribe 
and ask the simplest of getting-started questions.

In addition to handling java source files, the build system handles
C source files in a similar way.  C++ will be a straight forward 
extension when I get around to it.  I have placed some hooks for this 
already.

If you have any questions about how to get this working, please email me
at christopherlmarshall@yahoo.com.  I need your help in working out the
bugs I know are there but haven't discovered yet, both in the build 
scripts and in my attempt to write it up so people unfamiliar with 
GCC and related tools (like make) get start using it with a minimum of fuss.

>>>2. SUN'S BUILD SYSTEM

Let's start by reviewing how you use the JDK's javac and java commands to 
compile and load java programs.

Let's say we have our java code in our home directory, under a directory
called "project."  We have two packages, A, and B, with one source file each
(X.java, and Y.java).  Let's also say X refers to methods in Y and Y refers to
methods in X.  Here are the files:

~/project/A/X.java
~/project/B/Y.java

You could A/X.java everything like this:
   cd ~/project
   export CLASSPATH=~/project
   cd A
   javac X.java

As long as your CLASSPATH was set to ~/project, you could run A/X from
anywhere like this
   java A.X

For a larger project (with, say, a dozen package directories and hundreds of java files),
the compiling procedure is to cd to each package directory and run the command:
   javac *.java

Running any program is as simple as 
   java <package>.<mainclass>

Note how well the procedure scales.  Writing a bash script to do this straight forward.

>>>3. BUILDING PROGRAMS MANUALLY WITH GCJ

Here is what the above example looks like when using gcj (the GCC Java
compiler/linker front-end).

   cd ~/project
   gcj --classpath=. -c A/X.java -o A/X.o
   gcj --classpath=. -c B/Y.java -o B/Y.o
   gcj --main=A.X -o A/X A/X.o B/Y.o

That would take us through linking the A/X program.  You could then run the program
like you could any native binary:

   ~/project/A/X

Linking and running B/Y would be:

   gcj --main=B.Y -o B/Y B/Y.o A/X.o
   ~/project/B/Y

Since X has been compiled and linked into a native executable, you can also 
copy ~/project/A/X to any directory you want to and run it from there.  If 
you copy it to a place on your PATH, like /usr/loca/bin/X, then you can run
it from anywhere by just typing X.

Unlike the above method for Sun's JDK, this gcj manual procedure doesn't scale.
The tough part is the link step:

   gcj --main=A.X -o A/X A/X.o B/Y.o

We need some way of figuring out which java files A/X.java depends on, then we need
to convert those filenames to .o files, and list them after the "-o A/X" part of the
link command.  By the time you get to 12 packages and 150 java files, those lists
can get quite large.  Maintaining them by hand is out of the question.

Fortunately, gcj can make all of this information available to you with the -MD 
(make dependency) flag.  If you went through the compile step with the -MD flag
like this:

   cd ~/project
   gcj -MD --classpath=. -c A/X.java -o A/X.o

then, in addition to creating the object file A/X.o, gcj would also create a
make-style dependency file A/X.d that looked like this

A/X.o: A/X.java /usr/share/java/libgcj-3.2.3.jar ./B/Y.java

Which is saying that the object file A/X.o depends on A/X.java, the libgcj
standard library jar file, and the additional java file B/Y.java.

By processing A/X.d and A/Y.d, we could generate the list of object files we
needed in the link step.

That's the bulk of what I've done with my build system.  Written two
bash scripts and a Makefile (two, actually.  One that lives in ~/project and one that
lives in each package directory) to process the .d files that gcj is so kind as to 
generate for us.

When it is in place, you build your entire project like this:

   cd ~/project
   make depend
   make all

I've also written the standard clean and install targets.

>>>4. THIS FILES IN THIS PACKAGE AND HOW THEY WORK

Here are the files in this package:

./Makefile
./make_dep1
./make_dep2
./deps
./A/Z1.c
./A/Z1.h
./A/W1.c
./A/W1.h
./A/X1.c
./A/X1.h
./A/Makefile
./A/Z.java
./A/W.java
./A/X.java
./B/Makefile
./B/Y1.c
./B/Y1.h
./B/Y.java

Notice the mixture of Java and C files.  
Java and C files are not mixed in the sense that they link against
each other, but that C and Java files may exist in the same directories
and the build scripts I have written will handle them in the same way.
The heart of the system is in these files:

./Makefile
./make_dep1
./make_dep2
./deps
./A/Makefile
./B/Makefile

To apply these scripts to another java/c project, you would copy the first three
files to your top level project directory, and you would copy A/Makefile to 
each of your package directories and edit it (mostly filling in the cfiles and jfiles
variables to list the c and java files that you want linked into separate programs;
more on that shortly).  Before you try to do that, however, for heaven's sake, read
my warning about the role of the "deps" file.  You will be glad you did.

The .c, .h, and .java files are the simplest set of source files that illustrate the problems
this build system solves (mostly generating the list of object files you need to link into your
executable). 

The files ./A/X1.c and ./A/X.java both contain main functions and we want the build process to
figure out all of the dependencies required to build those two programs.  X.java
calls a static method in W.java which in turn calls a static method in Z.java.
However, X.java doesn't refer to any function in Z.java directly, which makes the
automatic generation of dependencies tricky.  The C files are similarly constructed,
but with 1's attached to their filenames.  You can't have C and Java files with the same
names in the same package directory with this build process.

Without further explanation, here is how you invoke make to build those two
programs:

make depend
make all

***IMPORTANT!*** The "deps" file, written to by make in the "make depend" step, 
has to exist or make won't let you do anything and you won't be able to figure 
out the error message.  In the event that nothing seems to be working and you are
presented with bizarre error messages, try to remember this warning and check 
that "deps" exists.  It is perfectly fine if it is empty as long as it exists
before you try to do a "make depend."  In fact, if you look at the "clean" target
in ./Makefile, you will notice that the last thing it does it delete deps, then 
create it as an empty file.

You can run A/X and A/X1 from to verify they work.  Both just print a simple message 
(the number 2, followed by the number 1, to be exact).

>>>5. THEORY OF OPERATION

My strategy is to use empty files with special names to pass information between successive
make invocations and the bash scripts.  Writing scripts in this style can be a very powerful
way to get work done.  When not documented, however, it can be almost impossible to figure out
how it works in a complex case.

I use empty files named *.e to mark which main source files should be built into 
stand-alone programs.  If you run

   make proglist

./Makefile goes to each subdirectory that has a Makefile in it and runs "make proglist"
in it.  

Let's examine ./A/Makefile

   # we list the main source files for each program here
   cfiles=X1.c
   jfiles=X.java

   efiles=${cfiles:=.e} ${jfiles:=.e}
   prgs=${cfiles:.c=} ${jfiles:.java=}

   proglist :
	touch ${efiles}

   clean :
	rm -f ${prgs} ${efiles}

When you invoke this Makefile with "make proglist", it takes the list of c files 
in the variable cfiles, and the list of java files in jfiles, adds the .e suffix
to each one, then creates empty files with those names.  In this case, it will create 
X1.c.e and X.java.e.  ./make_deps2 will look for files that end in .e and strip the .e
to arrive at the name of the source file we want to be linked into a stand alone program.
The program will be name will be determined by stripping the .java or .c suffix.

To edit A/Makefile for your use, you only have to change the definition of cfiles and jfiles.
For example:

   cfiles=W.c X.c Y.c
   jfiles=A.java B.java C.java

This is as far as I have gotten in writing this up.  I'll be back in a few days with more.

Chris Marshall
christopherlmarshall@yahoo.com

__________________________________
Do you Yahoo!?
Yahoo! Photos: High-quality 4x6 digital prints for 25Â¢
http://photos.yahoo.com/ph/print_splash