This is the mail archive of the
gcc@gcc.gnu.org
mailing list for the GCC project.
Re: Git and GCC. Why not with fork, exec and pipes like in linux?
- From: "J.C. Pizarro" <jcpiza at gmail dot com>
- To: "Jon Smirl" <jonsmirl at gmail dot com>, "Linus Torvalds" <torvalds at linux-foundation dot org>
- Cc: "Jeff King" <peff at peff dot net>, "Nicolas Pitre" <nico at cam dot org>, "Daniel Berlin" <dberlin at dberlin dot org>, "Harvey Harrison" <harvey dot harrison at gmail dot com>, "David Miller" <davem at davemloft dot net>, ismail at pardus dot org dot tr, gcc at gcc dot gnu dot org, git at vger dot kernel dot org
- Date: Thu, 6 Dec 2007 21:37:32 +0100
- Subject: Re: Git and GCC. Why not with fork, exec and pipes like in linux?
- References: <998d0e4a0712061125h3d44139ctb7f5600bc8467292@mail.gmail.com>
On 2007/12/6, J.C. Pizarro <jcpiza@gmail.com>, i wrote:
> For multicores CPUs, don't divide the work in threads.
> To divide the work in processes!
>
> Tips, tricks and hacks: to use fork, exec, pipes and another IPC mechanisms like
> mutexes, shared memory's IPC, file locks, pipes, semaphores, RPCs, sockets, etc.
> to access concurrently and parallely to the filelocked database.
I'm sorry, we don't need exec. We need fork, pipes and another IPC mechanisms
because it so shares easy the C code for parallelism.
Thanks to Linus because GIT is implemented in C language to interact with
system calls of the kernel written in C.
> For Intel Quad Core e.g., x4 cores, it need a parent process and 4
> child processes linked to the parent with pipes.
For peak performance (e.g 99.9% usage), the minimum number of child
processes should be more than 4, normally between e.g. 6 and 10 processes
depending on the statistics of idle's stalls of the cores.
> The parent process can be
> * no-threaded using select/epoll/libevent
> * threaded using Pth (GNU Portable Threads), NPTL (from RedHat) or whatever.
Note: there is a little design's problem with slowdown of I/O bandwith when
the parent is multithreaded and the children MUST to be multithreaded that
we can't avoid them to be non-multithreaded for maximum I/O bandwith.
The "finding of the smallest spanning forest with deltas" consumes a lot of
CPU, so if it scales well in a CPU x4 cores then it can to reduce 4
hours to 1 hour.
J.C.Pizarro :)