[prev in list] [next in list] [prev in thread] [next in thread]
List: gluster-users
Subject: Re: [Gluster-users] Brick offline after upgrade
From: David Cunningham <dcunningham () voisonics ! com>
Date: 2021-04-01 21:59:03
Message-ID: CAHGbP-8_PX3H02ZuX8v4nrtiDQ5pqABiz92EQv=CkM7JFSLCeg () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Thanks Strahil, the oldest we have is 5.x.
On Tue, 30 Mar 2021 at 17:50, Strahil Nikolov <hunter86_bg@yahoo.com> wrote:
> Yes. Versions 3.11 (and older) require going to latest 3.12 ( I think it
> was 3.12.15) before you could go up.
>
> Don't forget to backup and check for pending heals.
>
> Best Regards,
> Strahil Nikolov
>
> On Tue, Mar 30, 2021 at 4:13, David Cunningham
> <dcunningham@voisonics.com> wrote:
> Thank you Strahil. So if we take into account the depreciated options from
> all release notes then the direct upgrade should be okay.
>
>
> On Fri, 26 Mar 2021 at 02:01, Strahil Nikolov <hunter86_bg@yahoo.com>
> wrote:
>
> Hey David,
>
> usually it should work directly, but please take into consideration all
> release notes (usually '.0) as some options are deprecated like tiering.
>
> Best Regards,
> Strahil Nikolov
>
> On Tue, Mar 23, 2021 at 2:47, David Cunningham
> <dcunningham@voisonics.com> wrote:
> Hello,
>
> We ended up restoring the backup since it was easy on a test system.
>
> Does anyone know if you need to upgrade multiple major versions
> sequentially, or can you jump to the highest version? For example, to go
> from GlusterFS 5 to 8 can you upgrade to 8 directly, or must you do 6 and 7
> in between?
>
> Thanks in advance.
>
>
> On Sat, 20 Mar 2021 at 09:58, David Cunningham <dcunningham@voisonics.com>
> wrote:
>
> Hi Strahil,
>
> It's as follows. Do you see anything unusual? Thanks.
>
> root@caes8:~# ls -al /var/lib/glusterd/vols/gvol0/
> total 52
> drwxr-xr-x 3 root root 4096 Mar 18 17:06 .
> drwxr-xr-x 3 root root 4096 Jul 17 2018 ..
> drwxr-xr-x 2 root root 4096 Mar 18 17:06 bricks
> -rw------- 1 root root 16 Mar 18 17:06 cksum
> -rw------- 1 root root 3848 Mar 18 16:52
> gvol0.caes8.nodirectwritedata-gluster-gvol0.vol
> -rw------- 1 root root 2270 Feb 14 2020 gvol0.gfproxyd.vol
> -rw------- 1 root root 1715 Mar 18 16:52 gvol0.tcp-fuse.vol
> -rw------- 1 root root 729 Mar 18 17:06 info
> -rw------- 1 root root 0 Feb 14 2020 marker.tstamp
> -rw------- 1 root root 168 Mar 18 17:06 node_state.info
> -rw------- 1 root root 18 Mar 18 17:06 quota.cksum
> -rw------- 1 root root 0 Jul 17 2018 quota.conf
> -rw------- 1 root root 13 Mar 18 17:06 snapd.info
> -rw------- 1 root root 1829 Mar 18 16:52 trusted-gvol0.tcp-fuse.vol
> -rw------- 1 root root 896 Feb 14 2020 trusted-gvol0.tcp-gfproxy-fuse.vol
>
>
> On Fri, 19 Mar 2021 at 17:51, Strahil Nikolov <hunter86_bg@yahoo.com>
> wrote:
>
> [2021-03-18 23:52:52.084754] E [MSGID: 101019] [xlator.c:715:xlator_init] \
> 0-gvol0-server: Initialization of volume 'gvol0-server' failed, review your volfile \
> again
> What is the content of :
>
> /var/lib/glusterd/vols/gvol0 ?
>
>
> Best Regards,
>
> Strahil Nikolov
>
> On Fri, Mar 19, 2021 at 3:02, David Cunningham
> <dcunningham@voisonics.com> wrote:
> Hello,
>
> We have a single node/brick GlusterFS test system which unfortunately had
> GlusterFS upgraded from version 5 to 6 while the GlusterFS processes were
> still running. I know this is not what the "Generic Upgrade procedure"
> recommends.
>
> Following a restart the brick is not online, and we can't see any error
> message explaining exactly why. Would anyone have an idea of where to look?
>
> Since the logs from the time of the upgrade and reboot are a bit lengthy
> I've attached them in a text file.
>
> Thank you in advance for any advice!
>
> --
> David Cunningham, Voisonics Limited
> http://voisonics.com/
> USA: +1 213 221 1092
> New Zealand: +64 (0)28 2558 3782
> ________
>
>
>
> Community Meeting Calendar:
>
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://meet.google.com/cpu-eiue-hvk
> Gluster-users mailing list
> Gluster-users@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
>
>
>
> --
> David Cunningham, Voisonics Limited
> http://voisonics.com/
> USA: +1 213 221 1092
> New Zealand: +64 (0)28 2558 3782
>
>
>
> --
> David Cunningham, Voisonics Limited
> http://voisonics.com/
> USA: +1 213 221 1092
> New Zealand: +64 (0)28 2558 3782
>
>
>
> --
> David Cunningham, Voisonics Limited
> http://voisonics.com/
> USA: +1 213 221 1092
> New Zealand: +64 (0)28 2558 3782
>
>
--
David Cunningham, Voisonics Limited
http://voisonics.com/
USA: +1 213 221 1092
New Zealand: +64 (0)28 2558 3782
[Attachment #5 (text/html)]
<div dir="ltr"><div>Thanks Strahil, the oldest we have is \
5.x.</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Tue, 30 Mar 2021 at 17:50, Strahil Nikolov <<a \
href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Yes. Versions 3.11 \
(and older) require going to latest 3.12 ( I think it was 3.12.15) before you could \
go up.<div id="gmail-m_-7089103123205029351yMail_cursorElementTracker_1617078459196"><br></div><div \
id="gmail-m_-7089103123205029351yMail_cursorElementTracker_1617078503986">Don't \
forget to backup and check for pending heals.</div><div \
id="gmail-m_-7089103123205029351yMail_cursorElementTracker_1617078532597"><br></div><div \
id="gmail-m_-7089103123205029351yMail_cursorElementTracker_1617078532782">Best \
Regards,</div><div id="gmail-m_-7089103123205029351yMail_cursorElementTracker_1617078539357">Strahil \
Nikolov</div><div id="gmail-m_-7089103123205029351yMail_cursorElementTracker_1617078459331"> \
<br> <blockquote style="margin:0px 0px 20px"> <div \
style="font-family:Roboto,sans-serif;color:rgb(109,0,246)"> <div>On Tue, Mar 30, 2021 \
at 4:13, David Cunningham</div><div><<a href="mailto:dcunningham@voisonics.com" \
target="_blank">dcunningham@voisonics.com</a>> wrote:</div> </div> <div \
style="padding:10px 0px 0px 20px;margin:10px 0px 0px;border-left:1px solid \
rgb(109,0,246)"> <div id="gmail-m_-7089103123205029351yiv2575628617"><div><div \
dir="ltr"><div>Thank you Strahil. So if we take into account the depreciated options \
from all release notes then the direct upgrade should be okay.</div><div><br \
clear="none"></div></div><br clear="none"><div \
id="gmail-m_-7089103123205029351yiv2575628617yqt11143"><div><div dir="ltr">On Fri, 26 \
Mar 2021 at 02:01, Strahil Nikolov <<a rel="nofollow noopener noreferrer" \
shape="rect" href="mailto:hunter86_bg@yahoo.com" \
target="_blank">hunter86_bg@yahoo.com</a>> wrote:<br \
clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex">Hey David,<div \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yMail_cursorElementTracker_1616677205106"><br \
clear="none"></div><div \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yMail_cursorElementTracker_1616677205260">usually \
it should work directly, but please take into consideration all release notes \
(usually '.0) as some options are deprecated like tiering.</div><div \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yMail_cursorElementTracker_1616677277001"><br \
clear="none"></div><div \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yMail_cursorElementTracker_1616677277158">Best \
Regards,</div><div id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yMail_cursorElementTracker_1616677281391">Strahil \
Nikolov<br clear="none"> <br clear="none"> <blockquote style="margin:0px 0px 20px"> \
<div style="font-family:Roboto,sans-serif;color:rgb(109,0,246)"> <div>On Tue, Mar 23, \
2021 at 2:47, David Cunningham</div><div><<a rel="nofollow noopener noreferrer" \
shape="rect" href="mailto:dcunningham@voisonics.com" \
target="_blank">dcunningham@voisonics.com</a>> wrote:</div> </div> <div \
style="padding:10px 0px 0px 20px;margin:10px 0px 0px;border-left:1px solid \
rgb(109,0,246)"> <div \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yiv4273975035"><div><div \
dir="ltr"><div>Hello,</div><div><br clear="none"></div><div>We ended up restoring the \
backup since it was easy on a test system.</div><div><br clear="none"></div><div>Does \
anyone know if you need to upgrade multiple major versions sequentially, or can you \
jump to the highest version? For example, to go from GlusterFS 5 to 8 can you upgrade \
to 8 directly, or must you do 6 and 7 in between?</div><div><br \
clear="none"></div><div>Thanks in advance.</div><div><br clear="none"></div></div><br \
clear="none"><div id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yiv4273975035yqt70408"><div><div \
dir="ltr">On Sat, 20 Mar 2021 at 09:58, David Cunningham <<a rel="nofollow \
noopener noreferrer" shape="rect" href="mailto:dcunningham@voisonics.com" \
target="_blank">dcunningham@voisonics.com</a>> wrote:<br \
clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Strahil,</div><div><br \
clear="none"></div><div>It's as follows. Do you see anything unusual? Thanks.<br \
clear="none"></div><div><br clear="none"></div><div>root@caes8:~# ls -al \
/var/lib/glusterd/vols/gvol0/<br clear="none">total 52<br clear="none">drwxr-xr-x 3 \
root root 4096 Mar 18 17:06 .<br clear="none">drwxr-xr-x 3 root root 4096 Jul 17 \
2018 ..<br clear="none">drwxr-xr-x 2 root root 4096 Mar 18 17:06 bricks<br \
clear="none">-rw------- 1 root root 16 Mar 18 17:06 cksum<br \
clear="none">-rw------- 1 root root 3848 Mar 18 16:52 \
gvol0.caes8.nodirectwritedata-gluster-gvol0.vol<br clear="none">-rw------- 1 root \
root 2270 Feb 14 2020 gvol0.gfproxyd.vol<br clear="none">-rw------- 1 root root \
1715 Mar 18 16:52 gvol0.tcp-fuse.vol<br clear="none">-rw------- 1 root root 729 Mar \
18 17:06 info<br clear="none">-rw------- 1 root root 0 Feb 14 2020 \
marker.tstamp<br clear="none">-rw------- 1 root root 168 Mar 18 17:06 <a \
rel="nofollow noopener noreferrer" shape="rect" href="http://node_state.info" \
target="_blank">node_state.info</a><br clear="none">-rw------- 1 root root 18 Mar \
18 17:06 quota.cksum<br clear="none">-rw------- 1 root root 0 Jul 17 2018 \
quota.conf<br clear="none">-rw------- 1 root root 13 Mar 18 17:06 <a rel="nofollow \
noopener noreferrer" shape="rect" href="http://snapd.info" \
target="_blank">snapd.info</a><br clear="none">-rw------- 1 root root 1829 Mar 18 \
16:52 trusted-gvol0.tcp-fuse.vol<br clear="none">-rw------- 1 root root 896 Feb 14 \
2020 trusted-gvol0.tcp-gfproxy-fuse.vol<br clear="none"> </div></div><br \
clear="none"><div><div dir="ltr">On Fri, 19 Mar 2021 at 17:51, Strahil Nikolov <<a \
rel="nofollow noopener noreferrer" shape="rect" href="mailto:hunter86_bg@yahoo.com" \
target="_blank">hunter86_bg@yahoo.com</a>> wrote:<br \
clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><pre \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yiv4273975035 \
gmail-m_5719197461300425235gmail-m_1661177139554706364yMail_cursorElementTracker_1616129229406">[2021-03-18 \
23:52:52.084754] E [MSGID: 101019] [xlator.c:715:xlator_init] 0-gvol0-server: \
Initialization of volume 'gvol0-server' failed, review your volfile \
again</pre><pre id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162 \
044yiv4273975035gmail-m_5719197461300425235gmail-m_1661177139554706364yMail_cursorElementTracker_1616129231016">What \
is the content of :</pre><pre \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yiv4273975035 \
gmail-m_5719197461300425235gmail-m_1661177139554706364yMail_cursorElementTracker_1616129231253"><span \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yiv4273975035 \
gmail-m_5719197461300425235gmail-m_1661177139554706364yMail_cursorElementTracker_1616129365977" \
style="color:rgb(37,37,37);font-family:Consolas,Monaco,monospace;font-size:14.4px;white-space:normal;background-color:rgb(245,245,245)">/var/lib/glusterd/vols/gvol0 \
?</span><br clear="none"></pre><pre \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yiv4273975035 \
gmail-m_5719197461300425235gmail-m_1661177139554706364yMail_cursorElementTracker_1616129421463"><br \
clear="none"></pre> <pre \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yiv4273975035 \
gmail-m_5719197461300425235gmail-m_1661177139554706364yMail_cursorElementTracker_1616129428124">Best \
Regards,</pre><pre id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911 \
162044yiv4273975035gmail-m_5719197461300425235gmail-m_1661177139554706364yMail_cursorElementTracker_1616129437894">Strahil \
Nikolov</pre> <blockquote style="margin:0px 0px 20px"> <div \
style="font-family:Roboto,sans-serif;color:rgb(109,0,246)"> <div>On Fri, Mar 19, 2021 \
at 3:02, David Cunningham</div><div><<a rel="nofollow noopener noreferrer" \
shape="rect" href="mailto:dcunningham@voisonics.com" \
target="_blank">dcunningham@voisonics.com</a>> wrote:</div> </div> <div \
style="padding:10px 0px 0px 20px;margin:10px 0px 0px;border-left:1px solid \
rgb(109,0,246)"> <div \
id="gmail-m_-7089103123205029351yiv2575628617gmail-m_-2900376454911162044yiv4273975035gmail-m_5719197461300425235gmail-m_1661177139554706364yiv9088882711"><div \
dir="ltr"><div>Hello,</div><div><br clear="none"></div><div>We have a single \
node/brick GlusterFS test system which unfortunately had GlusterFS upgraded from \
version 5 to 6 while the GlusterFS processes were still running. I know this is not \
what the "Generic Upgrade procedure" recommends.</div><div><br \
clear="none"></div><div>Following a restart the brick is not online, and we can't \
see any error message explaining exactly why. Would anyone have an idea of where to \
look?</div><div><br clear="none"></div><div>Since the logs from the time of the \
upgrade and reboot are a bit lengthy I've attached them in a text \
file.</div><div><br clear="none"></div><div>Thank you in advance for any advice!<br \
clear="none"></div><div><br clear="none">-- <br clear="none"><div dir="ltr"><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div \
dir="ltr"><div>David Cunningham, Voisonics Limited<br clear="none"><a rel="nofollow \
noopener noreferrer" shape="rect" href="http://voisonics.com/" \
target="_blank">http://voisonics.com/</a><br clear="none">USA: +1 213 221 1092<br \
clear="none">New Zealand: +64 (0)28 2558 \
3782</div></div></div></div></div></div></div></div></div></div></div></div></div> \
</div>________<br clear="none"><br clear="none"><br clear="none"><br \
clear="none">Community Meeting Calendar:<br clear="none"><br clear="none">Schedule \
-<br clear="none">Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br \
clear="none">Bridge: <a rel="nofollow noopener noreferrer" shape="rect" \
href="https://meet.google.com/cpu-eiue-hvk" \
target="_blank">https://meet.google.com/cpu-eiue-hvk</a><br \
clear="none">Gluster-users mailing list<br clear="none"><a rel="nofollow noopener \
noreferrer" shape="rect" href="mailto:Gluster-users@gluster.org" \
target="_blank">Gluster-users@gluster.org</a><br clear="none"><a rel="nofollow \
noopener noreferrer" shape="rect" \
href="https://lists.gluster.org/mailman/listinfo/gluster-users" \
target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br \
clear="none"> </div> </blockquote></blockquote></div><br clear="all"><br \
clear="none">-- <br clear="none"><div dir="ltr"><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>David \
Cunningham, Voisonics Limited<br clear="none"><a rel="nofollow noopener noreferrer" \
shape="rect" href="http://voisonics.com/" \
target="_blank">http://voisonics.com/</a><br clear="none">USA: +1 213 221 1092<br \
clear="none">New Zealand: +64 (0)28 2558 \
3782</div></div></div></div></div></div></div></div></div></div></div> \
</blockquote></div></div><br clear="all"><br clear="none">-- <br clear="none"><div \
dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><div>David Cunningham, Voisonics Limited<br \
clear="none"><a rel="nofollow noopener noreferrer" shape="rect" \
href="http://voisonics.com/" target="_blank">http://voisonics.com/</a><br \
clear="none">USA: +1 213 221 1092<br clear="none">New Zealand: +64 (0)28 2558 \
3782</div></div></div></div></div></div></div></div></div></div></div> </div></div> \
</div> </blockquote></div></blockquote></div></div><br clear="all"><br \
clear="none">-- <br clear="none"><div dir="ltr"><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>David \
Cunningham, Voisonics Limited<br clear="none"><a rel="nofollow noopener noreferrer" \
shape="rect" href="http://voisonics.com/" \
target="_blank">http://voisonics.com/</a><br clear="none">USA: +1 213 221 1092<br \
clear="none">New Zealand: +64 (0)28 2558 \
3782</div></div></div></div></div></div></div></div></div></div></div> </div></div> \
</div> </blockquote></div></blockquote></div><br clear="all"><br>-- <br><div \
dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>David Cunningham, Voisonics \
Limited<br><a href="http://voisonics.com/" \
target="_blank">http://voisonics.com/</a><br>USA: +1 213 221 1092<br>New Zealand: +64 \
(0)28 2558 3782</div></div></div></div></div></div></div></div></div></div></div>
________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic