Re: svn speed traversing slow filesystems

[Daniel Berlin <dberlin at dberlin dot org>]

On Fri, 2005-11-25 at 12:59 -0500, Kaveh R. Ghazi wrote:
>  > > >  > Actually, i just removed the need for most stat calls during
>  > > >  > update in 1.4.
>  > > >
>  > > > Thanks Dan, that's great, but for the remaining i/o calls, it
>  > > > really does matter if you use long/paths/with/lots/of/slashes
>  > > > rather than chdir and ./filenames instead.  I believe other
>  > > > recursive gnu utils besides gnufind like "rm -r" or "mkdir -p" etc
>  > > > were modified to use the chdir mechanism also because the benefit
>  > > > was so great.
>  > > >
>  > > Yes that's fine, but we can't do this in SVN. We do the real work in
>  > > libraries that are supposed to be thread-safe. The cwd is
>  > > per-process on POSIX systems, as far as I know.
>  > 
>  > Yes, for this problem there may exist openat and friends.  Of course
>  > you'd need to check for the availability of them and provide a
>  > fallback.
>  > Richard.
> 
> Certainly removing the need for some of these calls like Dan did is
> the best option.  But for the remaining open, lstat and/or unlink
> calls, a faster mechanism than long/directory/paths/in/every/syscall
> is preferred for speed.  As Richard mentioned the openat() et al.
> interfaces from Solaris serve this purpose in multi-threaded
> situations if that is a design requirement for SVN.
> 
> And AFAICT, these APIs are being considered (or have been implemented
> already) for linux as well.
> http://lkml.org/lkml/2005/11/9/290
> 
> But while I'd like to see openat etc adopted in SVN, unfortunately
> that doesn't help systems without those calls like older solaris2,
> e.g. solaris2.7.

We use APR (http://apr.apache.org) as the underlying portability layer
in SVN, so we don't directly make any of these kinds of calls.

So you really want APR to implement this stuff, and SVN will
automatically take advantage.