Re: A sick idea - mmapped file output

[Linus Torvalds <torvalds at transmeta dot com>]

In article <20001102214053.G322@wolery.stanford.edu>,
Zack Weinberg <zackw@stanford.edu> wrote:
>The standalone preprocessor has a performance problem on its output
>end.  We do lots and lots of short fputs calls, and this can take up
>to 50% of total CPU time.  The sensible thing to do is to bypass
>stdio; we already do that on the input end, so there are no additional
>portability concerns.

I would suggest that you try using just plain "write()" instead of
fput(), which tends to be very reasonably optimized on most operating
systems simply because that's a _very_ common system call.

Sure, you'll avoid system calls with mmap(), but the problem with the
mmap approach is that 

 - a number of (especially older) systems will not guarantee 100%
   synchronization with mmap/read.  At the very least you should add a
   msync() call, but I suspect you might still find systems where there
   are separate caches for mmap and for "regular IO", and you might have
   aliasing and resulting non-reproducible behaviour.

 - page fault overhead. You'll probably get a page fault per page
   (usually every 4kB-32kB), and the overhead of this is likely to be
   higher than the overhead of a few write() calls.

 - portability. You'd still have to have the old approach, because there
   are too many systems where shared mappings simply won't work. Things
   like djgcc under DOS etc. In contrast, I suspect that any system that
   doesn't offer "write()" might as well be forgotten about anyway.

Comparing the mmap() with a stdio-based approach is unfair.  You should
at least try with direct write() calls.  If the common case is _really_
short writes, you could go half-way, with a really simple buffering
scheme rather than stdio (expose the buffer or similar).

			Linus