Re: RFC: Hint parameter for allocators.

[Dhruv Matani <dhruvbird at gmx dot net>]

On Sun, 2004-01-25 at 18:17, Gawain Bolton wrote:
> ----- Original Message ----- 
> From: "Dhruv Matani" <dhruvbird@gmx.net>
> To: "Loren James Rittle" <rittle@labs.mot.com>
> Cc: "libstdc++" <libstdc++@gcc.gnu.org>
> Sent: Sunday, January 25, 2004 8:00 AM
> Subject: Re: RFC: Hint parameter for allocators.
> 
> 
> > On Sat, 2004-01-24 at 00:49, Loren James Rittle wrote:
> > > In article <1074842538.1438.31.camel@home.free>,
> > > Dhruv Matani<dhruvbird@gmx.net> writes:
> [snip]
> > Just talking about real applications, do you know if any application's
> > behaviour is vaguely similar to the list_create_fill_sort test in th
> > testsuite? I'm asking this because I feel that most applications would
> > not exhibit such behaviour, since they would exhibit one of the two
> > types of beahviour given below:
> [snip]
> 
> As the original author of the list_create_fill_sort test program I can tell
> you that it was written as a simple benchmark for measuring the performance
> of list creation, filling (via push_back()) and sorting.

Ok.

> > Now, list_create_fill_sort does neither of the above. In fact, it lays
> > more emphasis on creation and destruction of the list rather that on the
> > data manipulation part, which IMHO is a very important and possibly the
> > more significant part of the program's life. Does this benchmark reflect
> > or give any useful results? More importantly, does anyone know of any
> > program that exhibits such behaviour?
> 
> Yes the benchmark gives useful results for looking at the performance of the
> 3 types of list operations it is designed to benchmark.  It is in no way
> meant to represent behaviour of a real world application.

Ok. The only reason I said that was because I would find it useful to
have benchmarks that tell me how this particular component would operate
(possibly) in a real application. Maybe we could have a new directory in
the testsuite named 'stress_tests'?

> > The primary reason for my interest in this is that I want to have
> > benchmarks that reflect actual program behaviour rather than have
> > benchmarks that favour certain conditions.
> 
> I do not believe any conditions are "favoured" by this benchmark.

In the context of what you just explained, yes, I agree with you.

> > The bitmap_allocator has it's
> > tradeoff's in the deallocate function, so it performs badly compared to
> > the default node allocator in this particular test. (38 sec for the
> > bitmap_allocator compared to 31 sec. for the default node allocator).
> > The main reason for bitmap_allocator's existence is that I wanted an
> > allocator to give memory that has good cache performance rather than
> > give memory in the smallest possible time. That's because I strongly
> > believe that data manipulation for a program is a bigger part than data
> > and resource acquisition. So, at the small price of (resource
> > relinquishing), if you can improve the data manipulation part, then it
> > would be a good tradeoff. So, to confirm, I put the bitmap allocator in
> > the containers_benchmark test, and it is better than the default node
> > allocator by 4.5s (11%~12%) decrease in the timing.
> 
> I think you just have to accept that your allocator is not the best in every
> single benchmark or application possible.

I agree with that fact, and that's exactly the reason I want to promote
benchmarks with real program behaviour. I as an allocator writer would
want the benchmarks that would give me a decent approximation of the
real applications that would use this allocator. I would say that it's
similar to the Big-Oh notation in analysis of algorithms. The stress
tests reflect that, while the test should be reflecting the Theta
notation (if you can draw a parallel between the testsuites and
algorithm analysis).

So, I want some figures that tell me what tradeoff's would be good for
real applications instead of stress applications.

> In fact, there may not be a "perfect allocator" which is superior to all
> others in every test/application.  This is probably why there are so many
> different kinds of allocators available.

Exactly. That's why there are so many allocators in the ext/ folder!

> Cheers,
> 
> 
> Gawain
-- 
	-Dhruv Matani.
http://www.geocities.com/dhruvbird/