Re: IR files and Linkable Executables

[Erik Mouw <J dot A dot K dot Mouw at its dot tudelft dot nl>]

On Wed, 08 Mar 2000 18:10:47 -0800, Brian Chin wrote:
> I have two ideas for improvements for GCC. The first deals with machine
> independant code. I know that the standard for machine independant
> distribution is simply the source for the program. I would gather from
> GCC's ability to compile for several platforms and archetectures that
> there is a point where all of the code is in a format after the parser,
> and before it is translated into machine language. If you could save the
> code before it is fully translated, you would get several benifets
> concerning distribution:
> 
>     1. Size
> 
>     The distribution will be much smaller. It's a simple fact that
> compiled (or semi compiled) code is much smaller than the equivalent
> source code, since all of the syntactic components have been removed,
> leaving only the essentials. This size decrease would be great for those
> people want to download a platform independant version of a piece of
> software, simply for installing it.

I think the distribution size will be much larger. If you include parsed
files for each and every target (gcc supports over 100 targets), you'll
end up with much redundancy. The parsed files will only differ in minor
system header files. The current source files are the smallest possible
denominator.

>     2. Speed
> 
>     Because half of the compiling has already taken place (namely
> lexical analysis, parsing, and syntactic proccessing), the compiling
> time would be cut in half. The primary use of this advantage will become
> apparent in the next section

Normal gcc users compile gcc only once, and then install it, so I don't
think this is an issue.

>     3. Compatibility
> 
>     If arranged properly, it may be possible to create binaries that
> have both machine dependant and this machine independant code. If that
> is true, than transferring files from archetecture to archetecture would
> be no problem. If a program is of a forgien archetecture, then it can
> quickly recompile itself, then store it's binaries within itself.

I don't see how this would help gcc. At binary level there is no such
thing as "machine independent code": the binary is compiled for CPU A,
which is completely incompatible with CPU B.

> Although possibly not appropriate for such applications as GCC, things
> like kernel modules, which have to be recompiled often due to kernel
> changes, would be much easier to use (perhaps with a little work it
> could use the new headers as well...)

I suppose you're referring to Linux kernel modules. There are some very
sound technical reasons why modules have to be recompiled, have a look at
the linux-kernel mailing list archives for answers. Non-linux kernel
developers only compile the stable kernel releases, which are released
every two months (more or less). Two minutes less compile time is not
worth the extra work in gcc. The linux kernel might be an important 
application for gcc, it is not the _only_ application.

>     4. Ease
> 
>     Many distributed packages aren't very robust, relying heavily on
> system headers and environments which aren't garunteed to be similar to
> the developer's computer. Makefiles and the configure programs aid the
> process, but even they don't always work. With this semi-compiled
> format, all of the preproccessing has already taken place, and as long
> as local shared libraries are installed properly, the files should
> compile sucessfully.

The preprocessing step takes place _because_ of the differences in
systems. Providing semi-compiled binaries won't help, because they won't
fit for every target. BTW, gcc already provides some sort of semi-compiled
binaries, like the preprocessed Bison/YACC files.

>     Of course there would be a few problems to work through, concerning
> big-endian, little-endian incompatabilities, but those could be worked
> through. Is there any project concerning this sort of thing, and/or is
> there any documentation on GCC's intermediate representation?

Not to mention the other differences in targets, like floating point
format, number of registers, calling convention, etc. The number of
possible combinations to work out will explode very easy.

Gcc documentation can be found in the gcc info files.

>     The second idea I have combines the concepts of shared libraries and
> executable programs. Many of the command line programs in linux are the
> only way to do certian things. There isn't any interface to them but for
> the command line, and there isn't any output except for the stdout. Many
> of these programs have longs sections involving translating the
> human-readable text into a format the computer understands, then back
> again. However, there are some programs which would be much easier to
> use if there were a computer-only interface to them. Currently, the only
> way to use the data from programs like lsmod is to parse the output,
> which as you can guess, isn't particularly reliable or efficient. If we
> could define standard programming interfaces with these standard
> utilities, you could process the data computer-to-computer, not
> computer-to-human-to-computer, as is currently done. Basically, this
> means creating programs whose procedures could be exported to other
> programs. The easiest way to do this is probably a version of shared
> libraries. Is there currently any way to do this sort of thing, and if

There is no difference. The "human readable output" is just one binary
representation of the output. If you define a "computer readable output",
it's just another representation of the same output. In both cases you
need functions to translate the input/output to the internal
representation of the program. The plain text interface uses the well
known and portable functions in stdio.h.

"The Practice of Programming" by Brian Kernighan and Rob Pike has a
complete section (8.5) about the advantages of using plain text
interfaces for data exchange.

> not, are there any projects on it. Also, do you know where there is
> information on the executable format in linux?

IIRC, Intel has some information about the ELF format on its developers
web site (developer.intel.com).

> Thank you for bearing with me on this, and I thank you in advance for
> any reply you give me.

I hope I didn't sound too negative.


Just my $0.02,
Erik

-- 
"I'm just this guy you know?"  -- Zaphod Beeblebrox in
"The Hitchhikers Guide to the Galaxy" by Douglas Adams