[prev in list] [next in list] [prev in thread] [next in thread] 

List:       ietf-nfsv4
Subject:    Re: [nfsv4] pNFS - bad devices
From:       Marc Eshel <eshel () almaden ! ibm ! com>
Date:       2005-11-28 20:59:44
Message-ID: OF3DBECBCA.BB4A0340-ON882570C7.007284E7-882570C7.00731916 () us ! ibm ! com
[Download RAW message or body]

nfsv4-bounces@ietf.org wrote on 11/28/2005 11:30:49 AM:

> I'll change that wording appropriately.  But, I would like the preferred 

> option to be that the client write out dirty data before returning the 
> layout (using the layout or through the MDS).

This is right if layoutchanged is false. If layoutchanged is true the 
client should first try to get a new layout and if the get new layout call 
fails than write the dirty data through the MDS.
Marc. 
> 
> -Garth
> 
> Marc Eshel wrote:
> > This will work too. The client will just have to find out if a new 
layout 
> > is available before deciding what to do next. 
> > What about the following text? it should be dropped or changed.
> > section 10.1: 
> > "The client should complete any in-flight I/O operations using the
> >    recalled layout(s) before returning it/them via LAYOUTRETURN.  If 
the
> >    client has buffered dirty data, it may chose to write it directly 
to
> >    storage before calling LAYOUTRETURN, or to write it later using
> >    normal NFSv4 WRITE operations to the metadata server."
> > 
> > 
> > Garth Goodson <Garth.Goodson@netapp.com> wrote on 11/22/2005 05:55:37 
PM:
> > 
> > 
> >>Ok, this is my suggestion.  I want to keep these changes simple and 
> >>limited to only the most necessary.  I added a 'layoutchanged' bool to 

> >>the recall structure.  Any comments?
> >>
> >>-Garth
> >>
> >>
> >>10.1  CB_LAYOUTRECALL
> >>
> >>    SYNOPSIS
> >>
> >>      layout_type, iomode, layoutchanged, layoutrecall -> -
> >>
> >>    ARGUMENT
> >>
> >>    ...
> >>
> >>      struct CB_LAYOUTRECALLargs {
> >>              pnfs_layouttype4        layout_type;
> >>              pnfs_layoutiomode4      iomode;
> >>              bool                    layoutchanged;
> >>              layoutrecall4           layoutrecall;
> >>      };
> >>
> >>    RESULT
> >>
> >>      struct CB_LAYOUTRECALLres {
> >>              nfsstat4        status;
> >>      };
> >>
> >>    DESCRIPTION
> >>
> >>    The CB_LAYOUTRECALL operation is used to begin the process of
> >>    recalling a layout, a portion thereof, or all layouts pertaining 
to 
> > 
> > a
> > 
> >>    particular file system (FSID).  If RECALL_FILE is specified, the
> >>    offset and length fields specify the portion of the layout to be
> >>    returned.  The iomode specifies the set of layouts to be returned.
> >>    An iomode of ANY specifies that all matching layouts, regardless 
of
> >>    iomode, must be returned; otherwise, only layouts that exactly 
match
> >>    the iomode must be returned.
> >>
> >>    If the layoutchanged field is TRUE, then the client SHOULD not 
flush
> >>    its dirty data to the devices specified by the layout being 
> > 
> > recalled.
> > 
> >>    Instead, it is preferable for the client to flush the dirty data
> >>    through the metadata server.  Alternatively, the client may 
attempt
> >>    to obtain a new layout.  Note: in order to obtain a new layout the
> >>    client must first return the old layout.  Since obtaining a new
> >>    layout is not guaranteed to succeed, the client must be ready to
> >>    flush its dirty data through the metadata server.
> >>
> >>...
> >>
> >>Halevy, Benny wrote:
> >>
> >>>In fact, there's no consistency related reason for the client to 
flush
> >>>any data in this scenario, and since layouts are orthogonal to cache
> >>>consistency a better approach for the client might be to return the
> >>>old layout and get a new one (can be in the same compound) while 
> > 
> > keeping
> > 
> >>>the dirty data in its cache and sync it later with the new layout
> >>>or via the MDS at the client's convenience.
> >>>
> >>>This can be achieved by adding a flag to CB_LAYOUTRECALL indicating
> >>>that the layout is just invalidated - i.e. there's a good chance for 
> > 
> > the
> > 
> >>>client to get a new layout immediately after returning the current 
> > 
> > one.
> > 
> >>>Benny
> >>>
> >>>
> >>>
> >>>>-----Original Message-----
> >>>>From: nfsv4-bounces@ietf.org 
> >>>>[mailto:nfsv4-bounces@ietf.org]On Behalf Of
> >>>>Marc Eshel
> >>>>Sent: Tuesday, November 01, 2005 9:22 PM
> >>>>To: Garth Goodson
> >>>>Cc: nfsv4@ietf.org
> >>>>Subject: [nfsv4] pNFS - bad devices
> >>>>
> >>>>
> >>>>When one of the data server crashes the metadata server uses 
> >>>>CB_LAYOUTRECALL to recall all layouts that were included the now 
dead 
> >>>>node. The client will than go and try to commit its data to the dead 

> >>>>node(device) before returning the layout which will cause at 
> >>>>the minimum 
> >>>>big delays. Maybe we can add more information to the layout recall 
to 
> >>>>include either the list of bad devices (don't even try to do 
> >>>>I/O to those 
> >>>>devices) or just a way to advice the client to flush/commit 
> >>>>the data using 
> >>>>the MDS  to complete the recall.
> >>>>Marc. 
> >>>>
> >>>>_______________________________________________
> >>>>nfsv4 mailing list
> >>>>nfsv4@ietf.org
> >>>>https://www1.ietf.org/mailman/listinfo/nfsv4
> >>>>
> 
> _______________________________________________
> nfsv4 mailing list
> nfsv4@ietf.org
> https://www1.ietf.org/mailman/listinfo/nfsv4


_______________________________________________
nfsv4 mailing list
nfsv4@ietf.org
https://www1.ietf.org/mailman/listinfo/nfsv4
[prev in list] [next in list] [prev in thread] [next in thread]