[prev in list] [next in list] [prev in thread] [next in thread] 

List:       gfs-devel
Subject:    Re: [gfs-devel] interaction between gfs and gnbd
From:       Ben Marzinski <marz0016 () sistina ! com>
Date:       2001-06-29 18:54:19
[Download RAW message or body]

> In short, the GFS driver use the linux block device layer CACHE.
> 
> But gnbd uses synchronous write, that means that it never uses write-cache.
> In fact this is worst than that. Each gnbd request is synchronously done
> even if the IO completion is expected by gfs for a bigger data aera. 

Let me clarify a little.  The gnbd client works much like any block device
driver.  It gets block requests, queues them, processes them, and when they
are completely processed, it marks them as uptodate.  The synchronous part is
this:  The gnbd server will not reply to the client that data has been written
to disk until the data is acutally on the server's disk.  The server could
simply notify the client that the data has been written to disk after the
server has issued the write request (but before that request has completed and
the data IS actually on the server's disk).  While this would cause a speed up,
if the the server crashed before it's write completed, the client would think
that the data had been successfully been written, when in fact, it hadn't.  I 
don't think that this is acceptable.

However, the client is under no obligation to wait for these writes to complete.
The data is simply marked as not uptodate until the server sends the reply.
So the only writes that gnbd waits for completion on are writes that are
supposed to be synchronous on the client.  So gnbd already does what you want
it to do, if I understand you correctly. 

gnbd is slow for a different reason. When it was originally written,
the functions controlling the exchange between the client and the server were
written with once concern. That they worked.  And after that, they never got 
messed with much because, until many other parts of gnbd, they worked.  The
biggest problem with gnbd as it currently stands is this: gnbd does not cluster
requests. The client does operations on each buffer seperately. If you want to
read 64 blocks worth of data.  The client contacts the server 64 times, asking
for one block.  For each block, the server reqeusts it, waits for it to
be available (if it is not already cached), and then sends it back to the
client (at least this is how reads work, writes are just as bad).  This is
a an unnecessarily slow design.

On the bright side.  It will definitely be fixed by the 4.2 release.  GNBD is
being rewritten to fix these problems. Up to 128 requests will be clustered and
sent to the server in one lump. The server will then issue one big read/write,
wait for the whole thing to complete, and send back a reply. This should speed
thing up to more respectable levels.

hope this helps
Ben Marzinski

marzinski@sistina.com

> 
> For read request, the only way to grab uptodate block is to have only one
> gnbd server (so only one read cache) or to, each time, mark blocks (all)
> not uptodate and ask for uptodate blocks (ie reload blocks in the cache).
> 
> This lead to poor performance. 
> To improve this situation, we only need IMHO in the gnbd client to know
> when and for which blocks ask for an IO completion and when and for which
> blocks remotely mark the buffer as "not uptodate".
> Then we had two new kinds of gnbd request and the gnbd server side should
> be trivial.
> 
> Christophe
> 
> -- 
> Christophe Barbé
> Software Engineer - christophe.barbe@lineo.fr
> Lineo France - Lineo High Availability Group
> 42-46, rue Médéric - 92110 Clichy - France
> phone (33).1.41.40.02.12 - fax (33).1.41.40.02.01
> http://www.lineo.com
> _______________________________________________
> gfs-devel mailing list
> gfs-devel@sistina.com
> http://lists.sistina.com/mailman/listinfo/gfs-devel
> Read the GFS Howto:  http://www.sistina.com/gfs/Pages/howto.html
_______________________________________________
gfs-devel mailing list
gfs-devel@sistina.com
http://lists.sistina.com/mailman/listinfo/gfs-devel
Read the GFS Howto:  http://www.sistina.com/gfs/Pages/howto.html

[prev in list] [next in list] [prev in thread] [next in thread]