[prev in list] [next in list] [prev in thread] [next in thread]
List: lustre-discuss
Subject: [lustre-discuss] NFS Client Attributes caching - equivalent feature/config in Lustre
From: Pinkesh Valdria <pinkesh.valdria () oracle ! com>
Date: 2020-04-21 20:54:24
Message-ID: 996BD659-1ED1-44EB-8F47-8D647907B6C8 () oracle ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Does lustre have mount options to mimic NFS mount option behavior , listed below?
I know in most cases, Lustre would perform much better than NFS and can scale and \
support a lot of clients in parallel. I have a use case, where there are only \
few clients accessing the filesystem and the files are really small, but in millions \
and files are very infrequently updated. The files are stored on an NFS server \
and its mounted on the clients with the below mount options, which results in caching \
of file attributes/metadata on the client and thus reduces # of calls to metadata and \
delivers better performance.
NFS mount options
type nfs (rw,nolock,nocto,actimeo=900,nfsvers=3,proto=tcp)
A custom proprietary application which compile (make command) on some of these files \
takes 20-24 seconds to run. The same command when ran on the same files stored in \
BeeGFS parallel filesystem takes 80-90 seconds (4x times slow), mainly because \
there is no client caching in BeeGFS and client has to make a lot more metadata calls \
compared to NFS cache file attributes.
Question
I already tried BeeGFS and I am asking this question to determine, if Lustre \
performance would be better than NFS for very small file workloads (50 bytes, 200 \
bytes, 2KB files) with 5 millions files spread across nested directories. Does \
lustre have mount options to mimic NFS mount option behavior, listed below? Or is \
there some optional feature in Lustre to achieve this cache behavior?
https://linux.die.net/man/5/nfs
ac / noac
Selects whether the client may cache file attributes. If neither option is specified \
(or if ac is specified), the client caches file attributes.
For my custom applications, cache of file attributes is fine (no negative impact) \
and it helps to improve performance of NFS.
actimeo=n
Using actimeo sets all of acregmin, acregmax, acdirmin, and acdirmax to the same \
value. If this option is not specified, the NFS client uses the defaults for each of \
these options listed above.
For my applications, it's okay to use cache file attributes/metadata for few mins \
(eg: 5mins) by setting this value, it can reduce # of metadata calls been made to \
the server and especially with filesystems storing lot of small files, it's a huge \
performance penalty, which can be avoided.
nolock
When mounting servers that do not support the NLM protocol, or when mounting an NFS \
server through a firewall that blocks the NLM service port, specify the nolock mount \
option. Specifying the nolock option may also be advised to improve the performance \
of a proprietary application which runs on a single client and uses file locks \
extensively.
Appreciate any guidance.
Thanks,
pinkesh valdria
[Attachment #5 (text/html)]
<html xmlns:o="urn:schemas-microsoft-com:office:office" \
xmlns:w="urn:schemas-microsoft-com:office:word" \
xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" \
xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type \
content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 \
(filtered medium)"><style><!-- /* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang=EN-US link="#0563C1" vlink="#954F72"><div \
class=WordSection1><p class=MsoNormal>Does lustre have mount options to mimic NFS \
mount option behavior , listed below? <o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I know in most cases, \
Lustre would perform much better than NFS and can scale and support a lot of clients \
in parallel. I have a use case, where there are only few clients accessing the \
filesystem and the files are really small, but in millions and files are very \
infrequently updated. The files are stored on an NFS server and its mounted on \
the clients with the below mount options, which results in caching of file \
attributes/metadata on the client and thus reduces # of calls to metadata and \
delivers better performance. <o:p></o:p></p><p class=MsoNormal><b><u><o:p><span \
style='text-decoration:none'> </span></o:p></u></b></p><p \
class=MsoNormal><b><u>NFS mount options<o:p></o:p></u></b></p><p class=MsoNormal>type \
nfs (rw,<span style='color:red'>nolock,nocto,actimeo=900</span>,nfsvers=3,proto=tcp)<o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>A custom proprietary \
application which compile (make command) on some of these files takes 20-24 seconds \
to run. The same command when ran on the same files stored in BeeGFS parallel \
filesystem takes 80-90 seconds (4x times slow), mainly because there is no client \
caching in BeeGFS and client has to make a lot more metadata calls compared to NFS \
cache file attributes. <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal><b><u>Question<o:p></o:p></u></b></p><p class=MsoNormal>I already \
tried BeeGFS and I am asking this question to determine, if Lustre performance would \
be better than NFS for very small file workloads (50 bytes, 200 bytes, 2KB files) \
with 5 millions files spread across nested directories. Does lustre have mount \
options to mimic NFS mount option behavior, listed below? Or is there some optional \
feature in Lustre to achieve this cache behavior? <o:p></o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><a \
href="https://linux.die.net/man/5/nfs">https://linux.die.net/man/5/nfs</a><span \
style='font-size:12.0pt'><o:p></o:p></span></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><b><span \
style='font-family:"Verdana",sans-serif;color:#444444;background:white'>ac</span></b><span \
style='font-family:"Verdana",sans-serif;color:#444444;background:white'> / <b>noac</b></span><o:p></o:p></p><p \
class=MsoNormal><span \
style='font-family:"Verdana",sans-serif;color:#444444;background:white'>Selects \
whether the client may cache file attributes. If neither option is specified (or \
if <b>ac</b> is specified), the client caches file \
attributes.</span><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal><span style='color:red'>For my custom applications, cache of file \
attributes is fine (no negative impact) and it helps to improve performance of NFS. \
<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p \
style='background:white'><b><span \
style='font-size:12.0pt;font-family:"Verdana",sans-serif;color:#444444'>actimeo=</span></b><i><span \
style='font-size:12.0pt;font-family:"Verdana",sans-serif;color:#444444'>n</span></i><span \
style='font-size:12.0pt;font-family:"Verdana",sans-serif;color:#444444'><o:p></o:p></span></p><p \
style='background:white;font-variant-ligatures: normal;font-variant-caps: \
normal;orphans: 2;text-align:start;widows: 2;-webkit-text-stroke-width: \
0px;text-decoration-style: initial;text-decoration-color: \
initial;word-spacing:0px'><span \
style='font-size:12.0pt;font-family:"Verdana",sans-serif;color:#444444'>Using <b>actimeo</b> sets \
all of <b>acregmin</b>, <b>acregmax</b>, <b>acdirmin</b>, \
and <b>acdirmax</b> to the same value. If this option is not specified, the \
NFS client uses the defaults for each of these options listed \
above.<o:p></o:p></span></p><p class=MsoNormal><span style='color:red'>For my \
applications, it's okay to use cache file attributes/metadata for few mins (eg: \
5mins) by setting this value, it can reduce # of metadata calls been made to the \
server and especially with filesystems storing lot of small files, it's a huge \
performance penalty, which can be avoided. <o:p></o:p></span></p><p \
class=MsoNormal><span style='color:red'><o:p> </o:p></span></p><p \
class=MsoNormal><b><span \
style='font-family:"Verdana",sans-serif;color:#444444;background:white'>nolock<o:p></o:p></span></b></p><p \
class=MsoNormal><b><span \
style='font-family:"Verdana",sans-serif;color:#444444;background:white'><o:p> </o:p></span></b></p><p \
class=MsoNormal><span \
style='font-size:12.0pt;font-family:"Verdana",sans-serif;color:#444444;background:white'>When \
mounting servers that do not support the NLM protocol, or when mounting an NFS server \
through a firewall that blocks the NLM service port, specify \
the <b>nolock</b> mount option.</span> <span \
style='font-size:12.0pt;font-family:"Verdana",sans-serif;color:#444444;background:white'>Specifying \
the <b>nolock</b> option may also be advised to improve the performance of \
a proprietary application which runs on a single client and uses file locks \
extensively.</span><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Appreciate any guidance. \
<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p \
class=MsoNormal>Thanks,<o:p></o:p></p><p class=MsoNormal>pinkesh \
valdria<o:p></o:p></p></div></body></html>
_______________________________________________
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic