[prev in list] [next in list] [prev in thread] [next in thread]
List: lustre-devel
Subject: Re: [lustre-devel] Question about how online LFSCK works
From: Andreas Dilger via lustre-devel <lustre-devel () lists ! lustre ! org>
Date: 2022-11-23 17:25:05
Message-ID: 217785FC-1D57-4137-A354-A560F976F703 () whamcloud ! com
[Download RAW message or body]
[Attachment #2 (text/plain)]
The LFSCK checking is run in a multi-stage execution. At the lowest level, the OI \
Scrub scans the ldiskfs or ZFS inodes from the storage and performs local sanity \
checks (eg. FID from xattr vs. OI FID->inode mapping) and then LFSCK consumes these \
objects (semi-asynchronously) to perform distributed consistency checks (eg. \
namespace checks for remote/striped directories, link EA checks for child->parent \
directory lookup, layout checks for MDT inode to OST object references, etc). For \
certain types of checks, if inconsistencies are found (eg. unreferenced OST objects) \
they may be put into a list for supplementary checking after the scan has completed.
Since the scan is linear by the underlying filesystem inode table, it is of course \
possible to miss new entries added behind the current cursor, or see new entries \
added in front of the cursor. In most cases these objects can be checked \
independently and without issue since the intersection of the scanning and changes is \
very rare.
To avoid impacting performance, the checking is largely done without locking. \
However, in the rare cases that an inconsistency is found, LFSCK will lock the \
object(s) involved and re-check the consistency, to avoid such transient errors.
Due to the dynamic nature of the filesystem, and the scale at which Lustre operates, \
we can't keep the system static during a full check. In some cases transient errors \
on modified objects may be deferred for a later scan.
All object updates are marked with a transaction number that is monotonically \
increasing (per target), so in some cases LFSCK can ignore errors that are found on \
objects modified since the start of the scan, and their consistency could be \
re-verified on a subsequent LFSCK run.
Cheers, Andreas
On Nov 23, 2022, at 06:38, Saisha Kamat via lustre-devel \
<lustre-devel@lists.lustre.org> wrote:
Hi all,
I am a Ph.D student at UNC-Charlotte, working on Lustre File System Checker.
I had a question related to how online LFSCK works. Specifically, how does LFSCK \
differentiate old metadata and new metadata created during its execution? Will the \
new metadata be checked? and if yes, when will they be checked by LFSCK? Can you \
please point me to the source code to understand more details?
Thanks very much!
Saisha
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
[Attachment #3 (text/html)]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
The LFSCK checking is run in a multi-stage execution. At the lowest level, the OI \
Scrub scans the ldiskfs or ZFS inodes from the storage and performs local sanity \
checks (eg. FID from xattr vs. OI FID->inode mapping) and then LFSCK consumes \
these objects (semi-asynchronously) to perform distributed consistency checks (eg. \
namespace checks for remote/striped directories, link EA checks for child->parent \
directory lookup, layout checks for MDT inode to OST object references, etc). \
For certain types of checks, if inconsistencies are found (eg. unreferenced \
OST objects) they may be put into a list for supplementary checking after the scan \
has completed. <div><br>
</div>
<div>Since the scan is linear by the underlying filesystem inode table, it is of \
course possible to miss new entries added behind the current cursor, or see new \
entries added in front of the cursor. In most cases these objects can be \
checked independently and without issue since the intersection of the scanning and \
changes is very rare.</div> <div><br>
</div>
<div>To avoid impacting performance, the checking is largely done without locking. \
However, in the rare cases that an inconsistency is found, LFSCK will lock the \
object(s) involved and re-check the consistency, to avoid such transient \
errors.</div> <div><br>
</div>
<div>Due to the dynamic nature of the filesystem, and the scale at which Lustre \
operates, we can't keep the system static during a full check. In some cases \
transient errors on modified objects may be deferred for a later scan. </div> \
<div><br> </div>
<div>All object updates are marked with a transaction number that is monotonically \
increasing (per target), so in some cases LFSCK can ignore errors that are found on \
objects modified since the start of the scan, and their consistency could be \
re-verified on a subsequent LFSCK run. </div>
<div><br>
<div dir="ltr">Cheers, Andreas</div>
<div dir="ltr"><br>
<blockquote type="cite">On Nov 23, 2022, at 06:38, Saisha Kamat via lustre-devel \
<lustre-devel@lists.lustre.org> wrote:<br> <br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<table cellpadding="0" class="gmail-ajC" \
style="border-spacing:0px;line-height:20px;font-family:"Google \
Sans",Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px"> <tbody>
<tr class="gmail-ajv">
<td colspan="2" tabindex="0" class="gmail-gL" \
style="vertical-align:top;width:auto;padding:2px 0px"> <span class="gmail-gI" \
style="vertical-align:top">Hi all,<br> I am a Ph.D student at UNC-Charlotte, working \
on Lustre File System Checker.<br> <br>
I had a question related to how online LFSCK works. Specifically, how does LFSCK \
differentiate old metadata and new metadata created during its execution? \
Will the new metadata be checked? and if yes, when will they be checked by LFSCK? Can \
you please point me to the source code to understand more details?<br>
<br>
Thanks very much!<br>
<span aria-label="" class="gmail-c-mrkdwn__br" \
style="box-sizing:inherit;display:block;height:8px;font-family:Slack-Lato,Slack-Fracti \
ons,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures"></span>Saisha<br>
<br>
</span></td>
</tr>
</tbody>
</table>
</div>
<span>_______________________________________________</span><br>
<span>lustre-devel mailing list</span><br>
<span>lustre-devel@lists.lustre.org</span><br>
<span>http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org</span><br>
</div>
</blockquote>
</div>
</body>
</html>
_______________________________________________
lustre-devel mailing list
lustre-devel@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org
--===============2499426323955620875==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic