gcc vectorizer/auto-parallelizer

[Thomas Veith]

On Thursday 18 March 2004 23:36, Scott Robert Ladd wrote:
> In my view, a GCC auto-parallelizer would likely be an extension of an
> explicit parallelization model, such as OpenMP.

Yes, as icc does :-)

> A group -- GOMP -- exists to work on OpenMP extensions to GCC:
> 	http://savannah.nongnu.org/projects/gomp/

Thank you, i will take a look..

> Their best use is on old code that isn't practical to modify; for new
> code, explicit parallelization is usually the best course.

Agreed 100%. On the other hand, writing hand-optimized assembler usually is 
also more effective than compiler-output; but only if you are high skilled 
and take great care - otherwise, the compiler will beat you. The same I think 
applies to auto-parallelizers. Aside of parallelizing loops they could also 
detect independend subtrees which could be run in parallel, so a compiler 
could do parallelization where a programmer wouldnt take care of. And for the 
programmer it is much more easy to write a program without taking care of 
parallelization - why should he do if the compiler could? 

I for one would love to have a compiler parallelize the evaluation of the 
fitness of my population in that wonderful libevocosm instead of have to do 
it myself :-))

> I've used auto-parallelizers, and have usually found them ineffective.

The question is: are they ineffective because the existing compilers do it not 
perfect, or are they ineffective by design?

I could imagine that a smart compiler and a smart runtime could be VERY 
effective; if the runtime knows the bandwidth and the latency of messages, 
the size of dynamic data (e.g. matrices, arrays) and the compiler the cost of 
loops you can decide at runtime when it pays off to migrate a loop or to keep 
it local.

OTOH I have no indepth-knowledge of gcc, so maybe this would be impossible to 
achieve.

> However, having recently acquired both clients and
> hardware for which OpenMP is applicable, I expect to be getting back to
> GOMP in a month or so.

Maybe I too :-).. The reason for my question is that I wanted to sit on top of 
such an auto-parallelizer and use a different runtime and switch the 
generator to use forks instead of threads so you could use openMosix for 
clustering. Imagine, a so parallelized application would fork n processes 
which get migrated to your n nodes sitting in a wait loop, and the main 
process calls them whenever there is something to be done in parallel. I know 
that this wouldn't be near as efficient as MPI or hand-coded parallelism, but 
a) efficient enough if the parallelization threshold is high enough; b) 
easier to implement as the same idea based on MPI/PVM. What do you think?

Best regards,
Thomas

-- 
Because we don't think about future generations, they will never forget us.
		-- Henrik Tikkanen