Parallelization of tokenizer in GCC!!!

Ian Lance Taylor
Sat Feb 27 09:52:00 GMT 2010 writes:

> I am doing a project on Cell Broadband Engine
> to parallelize the tokenization process of a given C program.
> As we are done with the tokenization part we want it to replace the
> tokenization part of
> an open source compiler.
> I mean the compiler that can run in Cell BE environment(ppu-gcc).
> If any of you would be able to help me locate the tokenization part of GCC and
> the input / output format for tokenizer part of GCC, it would be very
> useful to us.

Please never send a message to both and  Thanks.

gcc does its tokenization in the libcpp library, mostly in
libcpp/lex.c and friends.


More information about the Gcc-help mailing list