[prev in list] [next in list] [prev in thread] [next in thread] 

List:       redhat-linux-cluster
Subject:    Re: [Linux-cluster] GFS2 - monitoring the rate of Posix lock
From:       "Jankowski, Chris" <Chris.Jankowski () hp ! com>
Date:       2010-03-29 13:10:34
Message-ID: 036B68E61A28CA49AC2767596576CD596906B16255 () GVW1113EXC ! americas ! hpqcorp ! net
[Download RAW message or body]

Steve,

Thank you.

This looks quite promissing. I can have maximum performance and fast failover.

Regards,

Chris 

-----Original Message-----
From: linux-cluster-bounces@redhat.com [mailto:linux-cluster-bounces@redhat.com] On \
                Behalf Of Steven Whitehouse
Sent: Monday, 29 March 2010 23:36
To: linux clustering
Subject: Re: [Linux-cluster] GFS2 - monitoring the rate of Posix lock operations

Hi,

On Mon, 2010-03-29 at 12:15 +0000, Jankowski, Chris wrote:
> Steven,
> 
> > > > You can use localflocks on each node provided you never access any of the \
> > > > locked files from more than once node at once (which may be true depending on \
> > > > how the failover is designed). Then you will get local fcntl lock performance \
> > > > at the expense of cluster fcntl locks.
> 
> I could guarantee that only one node will use the filesystem by putting \
> mount/unmount into the start/stop script for the application service. This is the \
> easy part. 
> What I would like to understand is how the GFS2 recovery would look like after \
> failure of the node that had the filesystem mounted. I'd guess that the local locks \
> will be gone with the failed system and there is nothing to recover. The only thing \
> to do would be to replay to transaction log from the failed system.  Is this \
> correct? 
Yes, that is correct.

> This would work essentially like having a non-cluster filesystem such as ext3fs, \
> but in case of recovery from a node failure doing only transaction log replay \
> instaed of full fsck?  Or would fsck still be triggered on the attempt to mount the \
> filesystem on the other node.? 
> Thanks and regards,
> 
> Chris
> 
Fsck would not be triggered on the mount attempt. Its not possible to run fsck while \
the fs is mounted on any node,

Steve.

> 
> -----Original Message-----
> From: linux-cluster-bounces@redhat.com 
> [mailto:linux-cluster-bounces@redhat.com] On Behalf Of Steven 
> Whitehouse
> Sent: Monday, 29 March 2010 19:41
> To: linux clustering
> Subject: Re: [Linux-cluster] GFS2 - monitoring the rate of Posix lock 
> operations
> 
> On Sun, 2010-03-28 at 02:32 +0000, Jankowski, Chris wrote:
> > Steve,
> > 
> > Q2:
> > > > > Are you sure that the workload isn't causing too many cache invalidations \
> > > > > due to sharing files/directories between nodes? This is the most usual \
> > > > > cause of poor performance.
> > 
> > The other node is completely idle and kept that way by design. Users are \
> > connecting through an IP alias managed by the appplication service. Application \
> > administrators also log in through the alias to do their maintenance work. In the \
> > case of this particular test I manually listed what is running where. I am very \
> > concious of the fact that accesses from multiple nodes invalidate local in-memory \
> > caching. 
> > Q3:
> > > > > Have you used the noatime mount option? If you can use it, its highly \
> > > > > recommended. Also turn off selinux if that is running on the GFS2 \
> > > > > filesystem.
> > 
> > The filesystem is mounted with noatime and no nodiratime options. SELinux is \
> > disabled. 
> nodiratime isn't supported, noatime is enough.
> 
> > Q4:
> > > > > Potentially there might be. I don't know enough about the 
> > > > > application to say, but it depends on how the workload can be 
> > > > > arranged,
> > 
> > The application runs on one node at a time.  It has to, as it uses shared memory. \
> > The application uses a database of indexed files. There are thousands of them. \
> > Also, it uses standard UNIX flile locking and range locking. 
> > What else can I do to minimise the GFS2 locking overhead in this asymetrical \
> > configuration. 
> You can use localflocks on each node provided you never access any of the locked \
> files from more than once node at once (which may be true depending on how the \
> failover is designed). Then you will get local fcntl lock performance at the \
> expense of cluster fcntl locks. 
> > Q5:
> > Is this the case that when gfs_controld gets to 100% of one coe DPU usage then \
> > this is a hard limit on the number of Posix locks taken.  Is there only one \
> > gfs_lockd daemon servicng all GFS2 filesystems or are they run on a per \
> > filesystems basis?  In the latter case I would have thought that breaking the one \
> > filesystem that I have into several may help. Would it not? 
> > Thanks and regards,
> > 
> > Chris
> > 
> Assuming that you have a version in which gfs_controld takes care of 
> the locking (newer GFS2 send the locks via dlm_controld) then yes, 
> that will provide a hard limit on the rate at which locks can be 
> acquired/dropped,
> 
> Steve.
> 
> 
> --
> Linux-cluster mailing list
> Linux-cluster@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster
> 
> --
> Linux-cluster mailing list
> Linux-cluster@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-cluster

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster

--
Linux-cluster mailing list
Linux-cluster@redhat.com
https://www.redhat.com/mailman/listinfo/linux-cluster


[prev in list] [next in list] [prev in thread] [next in thread]