Parallelization of tokenizer in GCC!!!

[vivhari@gmail.com]

Dear Sirs/Madams,

I am doing a project on Cell Broadband Engine
to parallelize the tokenization process of a given C program.

As we are done with the tokenization part we want it to replace the
tokenization part of
an open source compiler.

I mean the compiler that can run in Cell BE environment(ppu-gcc).

If any of you would be able to help me locate the tokenization part of GCC and
the input / output format for tokenizer part of GCC, it would be very
useful to us.

Thanks in Advance

Regards
Vivek