[prev in list] [next in list] [prev in thread] [next in thread]
List: gluster-users
Subject: Re: [Gluster-users] Gluster 3.8.13 data corruption
From: Mahdi Adnan <mahdi.adnan () outlook ! com>
Date: 2017-10-09 13:25:35
Message-ID: DM5PR01MB2506F9767697B90344DC24DFFF740 () DM5PR01MB2506 ! prod ! exchangelabs ! com
[Download RAW message or body]
I only got this issue with this one template (CentOS7), i checked other tem=
plates and it worked fine.
I didn't quite understand what you need, i have one UUID and three files, o=
ne for the img and .meta and the other one is .lease.
I got the output of all three files which represent the VM i created from t=
he template, is this correct ?
https://paste.fedoraproject.org/paste/qzHmK8t-eJHM3hcZBVs5Yw
--
Respectfully
Mahdi A. Mahdi
________________________________
From: Krutika Dhananjay <kdhananj@redhat.com>
Sent: Monday, October 9, 2017 1:59 PM
To: Mahdi Adnan
Cc: Lindsay Mathieson; gluster-users@gluster.org
Subject: Re: [Gluster-users] Gluster 3.8.13 data corruption
OK.
Is this problem unique to templates for a particular guest OS type? Or is t=
his something you see for all guest OS?
Also, can you get the output of `getfattr -d -m . -e hex <path>` for the fo=
llowing two "paths" from all of the bricks:
path to the file representing the vm created off this template wrt the bric=
k. It will usually be $BRICKPATH/xxxx....xx/images/$UUID where $UUID repres=
ents the uuid of the vm created from the template. If im not wrong, there =
would be two sets of a certain UUID2, UUID2.lease, UUID2.meta. Please get =
me the output of the command above for both the uuid files. The one that ha=
s more than 2 hard links is the template. When you attach this output, plea=
se make that distinction clear, so it will be easier to debug.
-Krutika
On Fri, Oct 6, 2017 at 6:38 PM, Mahdi Adnan <mahdi.adnan@outlook.com<mailto=
:mahdi.adnan@outlook.com>> wrote:
Hi,
Thank you for your reply.
Lindsay,
Uunfortunately i do not have backup for this template.
Krutika,
The stat-prefetch is already disabled on the volume.
--
Respectfully
Mahdi A. Mahdi
________________________________
From: Krutika Dhananjay <kdhananj@redhat.com<mailto:kdhananj@redhat.com>>
Sent: Friday, October 6, 2017 7:39 AM
To: Lindsay Mathieson
Cc: Mahdi Adnan; gluster-users@gluster.org<mailto:gluster-users@gluster.org=
>
Subject: Re: [Gluster-users] Gluster 3.8.13 data corruption
Could you disable stat-prefetch on the volume and create another vm off tha=
t template and see if it works?
-Krutika
On Fri, Oct 6, 2017 at 8:28 AM, Lindsay Mathieson <lindsay.mathieson@gmail.=
com<mailto:lindsay.mathieson@gmail.com>> wrote:
Any chance of a backup you could do bit compare with?
Sent from my Windows 10 phone
From: Mahdi Adnan<mailto:mahdi.adnan@outlook.com>
Sent: Friday, 6 October 2017 12:26 PM
To: gluster-users@gluster.org<mailto:gluster-users@gluster.org>
Subject: [Gluster-users] Gluster 3.8.13 data corruption
Hi,
We're running Gluster 3.8.13 replica 2 (SSDs), it's used as storage domain =
for oVirt.
Today, we found an issue with one of the VMs template, after deploying a VM=
from this template it will not boot, it stuck at mount the root partition.
We've been using this templates for months now and we did not had any issue=
s with it.
Both oVirt and Gluster logs is not showing any errors or warnings.
I exported the template and tried running on a standalone machine and it di=
d not work.
Do you think this might be a silent data corruption ? Bitrot is not enabled=
.
Volume info can be found at:
https://paste.fedoraproject.org/paste/f~1jNIObDa2zoG7ATdCdZg
--
Respectfully
Mahdi A. Mahdi
_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org<mailto:Gluster-users@gluster.org>
http://lists.gluster.org/mailman/listinfo/gluster-users
[Attachment #3 (text/html)]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body>
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} \
</style> <div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; \
color: rgb(0, 0, 0);"> I only got this issue with this one template (CentOS7), i \
checked other templates and it worked fine.</div> <div style="font-family: Calibri, \
Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> I didn't quite \
understand what you need, i have one UUID and three files, one for the img and .meta \
and the other one is .lease.</div> <div style="font-family: Calibri, Helvetica, \
sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"> I got the output of all three \
files which represent the VM i created from the template, is this correct ?</div> \
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> https://paste.fedoraproject.org/paste/qzHmK8t-eJHM3hcZBVs5Yw<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: \
rgb(0, 0, 0);"> <br>
</div>
<div id="signature"><br>
<div class="ecxmoz-signature">-- <br>
<br>
<font color="#3366ff"><font color="#000000">Respectfully<b><br>
</b><b>Mahdi A. Mahdi</b></font></font><font color="#3366ff"><br>
<br>
</font><font color="#3366ff"></font></div>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" \
style="font-size:11pt" color="#000000"><b>From:</b> Krutika Dhananjay \
<kdhananj@redhat.com><br> <b>Sent:</b> Monday, October 9, 2017 1:59 PM<br>
<b>To:</b> Mahdi Adnan<br>
<b>Cc:</b> Lindsay Mathieson; gluster-users@gluster.org<br>
<b>Subject:</b> Re: [Gluster-users] Gluster 3.8.13 data corruption</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>OK.<br>
<br>
</div>
Is this problem unique to templates for a particular guest OS type? Or is this \
something you see for all guest OS?<br> <br>
</div>
Also, can you get the output of `getfattr -d -m . -e hex <path>` for the \
following two "paths" from all of the bricks:<br> </div>
path to the file representing the vm created off this template wrt the brick. It will \
usually be $BRICKPATH/xxxx....xx/images/$UUID where $UUID represents the uuid of the \
vm created from the template. If im not wrong, there would be two sets of a \
certain UUID2, UUID2.lease, UUID2.meta. Please get me the output of the \
command above for both the uuid files. The one that has more than 2 hard links is the \
template. When you attach this output, please make that distinction clear, so it will \
be easier to debug.<br> <br>
<br>
</div>
-Krutika<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Oct 6, 2017 at 6:38 PM, Mahdi Adnan <span dir="ltr">
<<a href="mailto:mahdi.adnan@outlook.com" \
target="_blank">mahdi.adnan@outlook.com</a>></span> wrote:<br> <blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> <div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Hi,</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Thank you for your reply.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Lindsay, </div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Uunfortunately i do not have backup for this template.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Krutika,</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
The stat-prefetch is already disabled on the volume.</div>
<span class="">
<div id="m_9009249656460538578signature"><br>
<div class="m_9009249656460538578ecxmoz-signature">-- <br>
<br>
<font color="#3366ff"><font color="#000000">Respectfully<b><br>
</b><b>Mahdi A. Mahdi</b></font></font><font color="#3366ff"><br>
<br>
</font><font color="#3366ff"></font></div>
</div>
<hr style="display:inline-block;width:98%">
</span>
<div id="m_9009249656460538578divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" \
color="#000000" face="Calibri, sans-serif"><b>From:</b> Krutika Dhananjay <<a \
href="mailto:kdhananj@redhat.com" target="_blank">kdhananj@redhat.com</a>><br> \
<b>Sent:</b> Friday, October 6, 2017 7:39 AM<br> <b>To:</b> Lindsay Mathieson<br>
<b>Cc:</b> Mahdi Adnan; <a href="mailto:gluster-users@gluster.org" target="_blank">
gluster-users@gluster.org</a><br>
<b>Subject:</b> Re: [Gluster-users] Gluster 3.8.13 data corruption</font>
<div> </div>
</div>
<div>
<div class="h5">
<div>
<div dir="ltr">
<div>Could you disable stat-prefetch on the volume and create another vm off that \
template and see if it works?<br> <br>
</div>
-Krutika<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Oct 6, 2017 at 8:28 AM, Lindsay Mathieson <span \
dir="ltr"> <<a href="mailto:lindsay.mathieson@gmail.com" \
target="_blank">lindsay.mathieson@gmail.com</a>></span> wrote:<br> <blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"> <div link="blue" vlink="#954F72" lang="EN-AU">
<div class="m_9009249656460538578m_5841451042050301745WordSection1">
<p class="MsoNormal">Any chance of a backup you could do bit compare with?</p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Sent from my Windows 10 phone</p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="border:none;padding:0cm"><b>From: </b><a \
href="mailto:mahdi.adnan@outlook.com" target="_blank">Mahdi Adnan</a><br> <b>Sent: \
</b>Friday, 6 October 2017 12:26 PM<br> <b>To: </b><a \
href="mailto:gluster-users@gluster.org" \
target="_blank">gluster-users@gluster.org</a><br> <b>Subject: </b>[Gluster-users] \
Gluster 3.8.13 data corruption</p> </div>
<div>
<div class="m_9009249656460538578h5">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span \
style="font-size:12.0pt;color:black">Hi,<u></u><u></u></span></p> <div>
<p class="MsoNormal"><span \
style="font-size:12.0pt;color:black"><u></u> <u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">We're running Gluster \
3.8.13 replica 2 (SSDs), it's used as storage domain for \
oVirt.<u></u><u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Today, we found an \
issue with one of the VMs template, after deploying a VM from this template it will \
not boot, it stuck at mount the root partition.<u></u><u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">We've been using this \
templates for months now and we did not had any issues with \
it.<u></u><u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Both oVirt and \
Gluster logs is not showing any errors or warnings.<u></u><u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">I exported the \
template and tried running on a standalone machine and it did not \
work.<u></u><u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Do you think this \
might be a silent data corruption ? Bitrot is not enabled.<u></u><u></u></span></p> \
</div> <div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Volume info can be \
found at:<u></u><u></u></span></p> </div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><a \
href="https://paste.fedoraproject.org/paste/f~1jNIObDa2zoG7ATdCdZg" \
target="_blank">https://paste.fedoraproject.or<wbr>g/paste/f~1jNIObDa2zoG7ATdCdZg</a><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span \
style="font-size:12.0pt;color:black"><u></u> <u></u></span></p> </div>
<p class="MsoNormal" style="margin-bottom:12.0pt">-- <br>
<br>
<span style="color:black">Respectfully<b><br>
Mahdi A. Mahdi</b></span><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" \
target="_blank">Gluster-users@gluster.org</a><br> <a \
href="http://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" \
target="_blank">http://lists.gluster.org/mailm<wbr>an/listinfo/gluster-users</a><br> \
</blockquote> </div>
<br>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</body>
</html>
_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users
--===============7985364620646017894==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic