[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lustre-announce
Subject:    Re: [Lustre-discuss] GigE-Performance II / 0-config]
From:       Phil Schwan <phil () clusterfs ! com>
Date:       2003-12-23 9:40:37
[Download RAW message or body]

Hi Christoph--

Christoph Biardzki wrote:
> 
> Observation #1:
> ---------------
> 
> Run "iozone -t1 -i0 -i1 -r1m -s10g -F /mnt/lustre/bigfile" on C.
> Run "ls -al" on /mnt/lustre on C at the same time.
> 
> => "ls" will return only after iozone finished writing (which can take 
> several minutes) Is this normal?

This is an exaggerated example of behaviour which is essentially normal
(although obviously not desirable).

In Lustre 1.0.x, programs like ls (anything which calls stat()) will
introduce a very hard barrier, to ensure that it reports a completely
accurate file size.  This barrier doesn't wait for iozone to finish
writing entirely, but it does wait for any cached dirty data to flush,
which by default can be quite a lot.

Try this, and see if this makes things better, even though it won't make
it perfect:

  for i in /proc/fs/lustre/osc/OSC_*MNT_*/max_dirty_mb; do
    echo 4 > $i
  done

This will reduce the amount of cached dirty data, which should improve
the response time.  This should not affect throughput for the non-ls
case, and will probably be the default setting in Lustre 1.0.3.  With
some experimentation, you may find that you can reduce it further.

We are in the process of removing this hard barrier, which is the
correct way to deal with this issue.

> Observation #2:
> ---------------
> Metadata performance from client ist still low, on the server its fine:

Interesting.  Let me see if someone here can reproduce that locally
first, because it will be much easier to debug.  We have seen effects
like this before when the TCP stack flips into a mode which causes a lot
of extra latency.

-Phil
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss@lists.clusterfs.com
https://lists.clusterfs.com/mailman/listinfo/lustre-discuss
[prev in list] [next in list] [prev in thread] [next in thread]