[prev in list] [next in list] [prev in thread] [next in thread] 

List:       git
Subject:    Re: [PATCH] Enable index-pack threading in msysgit.
From:       Karsten Blees <karsten.blees () gmail ! com>
Date:       2014-03-21 18:55:06
Message-ID: 532C8B0A.70202 () gmail ! com
[Download RAW message or body]

Am 21.03.2014 06:35, schrieb Stefan Zager:
> On Thu, Mar 20, 2014 at 10:21 PM, Duy Nguyen <pclouds@gmail.com> wrote:
> > On Fri, Mar 21, 2014 at 08:51:18AM +0700, Duy Nguyen wrote:
> > > On Thu, Mar 20, 2014 at 11:08 PM, Stefan Zager <szager@chromium.org> wrote:
> > > > Duy, would you like to re-post your patch without the new pread \
> > > > implementation?
> > > 
> > > I will but let me try out the sliding window idea first. My quick
> > > tests on git.git show me we may only need 21k mmap instead of 177k
> > > pread. That hints some potential performance improvement.
> > 
> > The patch at the bottom reuses (un)use_pack() instead of pread(). The
> > results on linux-2.6 do not look any different. I guess we can drop
> > the idea.
> > 
> > It makes me wonder, though, what's wrong a simple patch like this to
> > make pread in index-pack thread-safe? It does not look any different
> > either from the performance point of view, perhaps because
> > unpack_data() reads small deltas most of the time
> 
> When you serialize disk access in this way, the effect on performance
> is really dependent on the behavior of the OS, as well as the locality
> of the read offsets.  Assuming -- fairly, I think -- that the reads
> will be pretty randomly distributed (i.e., no locality to speak of),
> then your best bet is get as many read operations in flight as
> possible, and let the disk scheduler optimize the seek time.
> 

The read() implementation in MSVCRT.DLL is synchronized anyway, and I strongly \
suspect that this is also true for ReadFile() (at least for synchronous file handles, \
i.e. opened without FILE_FLAG_OVERLAPPED). So I guess separate file descriptors would \
help with parallel IO as well.

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[prev in list] [next in list] [prev in thread] [next in thread]