[prev in list] [next in list] [prev in thread] [next in thread]
List: drbd-user
Subject: Re: [DRBD-user] DRBD Can't Mount Under Pacemaker
From: JA E <eshmam2 () gmail ! com>
Date: 2015-06-27 6:30:38
Message-ID: CAD+O+OPBj-KG9MH6G-4upzQfiAM3jc2c2rFvLQp9PYpenu8Z6A () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi Dennis,
Thanks for the reply.
As far my understanding I need a ordering constraint which will sense the
drbd primary and then will allow webserver_fs to start mounting and a
colocation constraint which make sure all resource
(webserver,virtual_ip, webserver_data_sync
,webserver_fs ) to run in master node.
But it pity after hours of head scratch and memorizing documentation I am
unable to phrase those lines. May be it too much too ask, but would you
please write those constraints for me.
Regards,
JAE
On Fri, Jun 26, 2015 at 8:56 PM, Dennis Jacobfeuerborn <
dennisml@conversis.de> wrote:
> On 26.06.2015 12:42, JA E wrote:
> > Hi,
> >
> > I am newbie to clustering. I managed to setup cluster with
> > pacemaker,corosysnc, drbd and pcs in two nodes. But after test restart of
> > both node it seems drbd can't mount the desired folder when controlled by
> > pacemaker, manually it's fine.
> >
> >
> >> *[root@master Desktop]# pcs config*
> >> *Cluster Name: cluster_web*
> >> *Corosync Nodes:*
> >> * master slave *
> >> *Pacemaker Nodes:** master slave *
> >>
> >> *Resources: *
> >> * Resource: virtual_ip (class=ocf provider=heartbeat type=IPaddr2)*
> >> * Attributes: ip2.168.73.133 cidr_netmask2 *
> >> * Operations: start interval=0s timeout s
> >> (virtual_ip-start-timeout-20s)*
> >> * stop interval=0s timeout s
> (virtual_ip-stop-timeout-20s)*
> >> * monitor interval0s (virtual_ip-monitor-interval-30s)*
> >> * Resource: webserver (class=ocf provider=heartbeat type=apache)*
> >> * Attributes: configfile=/etc/httpd/conf/httpd.conf
> >> statusurl=http://localhost/server-status <
> http://localhost/server-status> *
> >> * Operations: start interval=0s timeout@s
> (webserver-start-timeout-40s)*
> >> * stop interval=0s timeout`s
> (webserver-stop-timeout-60s)*
> >> * monitor interval=1min (webserver-monitor-interval-1min)*
> >> * Master: webserver_data_sync*
> >> * Meta Attrs: master-max=1 master-node-max=1 clone-max=2
> clone-node-max=1
> >> notify=true *
> >> * Resource: webserver_data (class=ocf provider=linbit type=drbd)*
> >> * Attributes: drbd_resource=drbd0 *
> >> * Operations: start interval=0s timeout$0
> >> (webserver_data-start-timeout-240)*
> >> * promote interval=0s timeout
> >> (webserver_data-promote-timeout-90)*
> >> * demote interval=0s timeout
> >> (webserver_data-demote-timeout-90)*
> >> * stop interval=0s timeout0
> >> (webserver_data-stop-timeout-100)*
> >> * monitor interval`s
> (webserver_data-monitor-interval-60s)*
> >> * Resource: webserver_fs (class=ocf provider=heartbeat type=Filesystem)*
> >> * Attributes: device=/dev/drbd0 directory=/var/www/html fstype=ext3 *
> >> * Operations: start interval=0s timeout`
> >> (webserver_fs-start-timeout-60)*
> >> * stop interval=0s timeout`
> (webserver_fs-stop-timeout-60)**
> >> monitor interval timeout@
> >> (webserver_fs-monitor-interval-20)*
> >>
> >> *Stonith Devices: **Fencing Levels: *
> >>
> >> *Location Constraints:*
> >> * Resource: webserver*
> >> * Enabled on: node01 (score:50) (id:location-webserver-node01-50)*
> >> * Enabled on: master (score:50) (id:location-webserver-master-50)*
> >> *Ordering Constraints:*
> >> * start virtual_ip then start webserver (kind:Mandatory)
> >> (id:order-virtual_ip-webserver-mandatory)*
> >> * start webserver_fs then start webserver (kind:Mandatory)
> >> (id:order-webserver_fs-webserver-mandatory)*
> >> *Colocation Constraints:*
> >> * webserver with virtual_ip (score:INFINITY)
> >> (id:colocation-webserver-virtual_ip-INFINITY)** webserver_fs with
> >> webserver_data_sync (score:INFINITY) (with-rsc-role:Master)
> >> (id:colocation-webserver_fs-webserver_data_sync-INFINITY)*
> >>
> >> *Cluster Properties:*
> >> * cluster-infrastructure: corosync*
> >> * cluster-name: cluster_web*
> >> * dc-version: 1.1.12-a14efad*
> >> * have-watchdog: false*
> >> * no-quorum-policy: ignore** stonith-enabled: false*
> >>
> >>
> >
> >>
> >> *[root@master Desktop]# drbdadm dump*
> >> *# /etc/drbd.conf*
> >> *global {*
> >> * usage-count yes;*
> >> * cmd-timeout-medium 600;*
> >> * cmd-timeout-long 0;**}*
> >>
> >> *common {*
> >> * net {*
> >> * protocol C;*
> >> * }**}*
> >>
> >> *# resource drbd0 on master: not ignored, not stacked*
> >> *# defined at /etc/drbd.d/drbd0.res:1*
> >> *resource drbd0 {*
> >> * on master {*
> >> * volume 0 {*
> >> * device /dev/drbd0 minor 0;*
> >> * disk /dev/vg_drbd0/lv_drbd0;*
> >> * meta-disk internal;*
> >> * }*
> >> * address ipv4 192.168.73.131:7789
> >> <http://192.168.73.131:7789/>;*
> >> * }*
> >> * on slave {*
> >> * volume 0 {*
> >> * device /dev/drbd0 minor 0;*
> >> * disk /dev/vg_drbd0/lv_drbd0;*
> >> * meta-disk internal;*
> >> * }*
> >> * address ipv4 192.168.73.132:7789
> >> <http://192.168.73.132:7789/>;*
> >> * }**}*
> >>
> >>
> >
> >>
> >> *[root@master Desktop]# pcs status*
> >> *Cluster name: cluster_web*
> >> *Last updated: Fri Jun 26 03:04:18 2015*
> >> *Last change: Fri Jun 26 02:13:11 2015*
> >> *Stack: corosync*
> >> *Current DC: master (1) - partition with quorum*
> >> *Version: 1.1.12-a14efad*
> >> *2 Nodes configured**5 Resources configured*
> >>
> >> *Online: [ master slave ]*
> >> *Full list of resources:*
> >>
> >> * virtual_ip (ocf::heartbeat:IPaddr2): Started master *
> >> * webserver (ocf::heartbeat:apache): Stopped *
> >> * Master/Slave Set: webserver_data_sync [webserver_data]*
> >> * Masters: [ master ]*
> >> * Slaves: [ slave ]** webserver_fs (ocf::heartbeat:Filesystem):
> >> Stopped *
> >>
> >> *Failed actions:*
> >> * webserver_fs_start_0 on master 'unknown error' (1): call#,
> >> status=complete, exit-reason='Couldn't mount filesystem /dev/drbd0 on
> >> /var/www/html', last-rc-change='Fri Jun 26 02:20:45 2015', queued=0ms,
> >> execms** webserver_fs_start_0 on slave 'unknown error' (1):
> >> call#, status=complete, exit-reason='Couldn't mount filesystem
> /dev/drbd0
> >> on /var/www/html', last-rc-change='Fri Jun 26 02:20:45 2015',
> queued=0ms,
> >> execyms*
> >>
> >>
> >> *PCSD Status:*
> >> * master: Online** slave: Online*
> >>
> >> *Daemon Status:*
> >> * corosync: active/enabled*
> >> * pacemaker: active/enabled** pcsd: active/enabled*
> >
> >
> > cat /proc/drbd
> >> version: 8.4.6 (api:1/proto:86-101)
> >> GIT-hash: 833d830e0152d1e457fa7856e71e11248ccf3f70 build by
> phil@Build64R7,
> >> 2015-04-10 05:13:52
> >> 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
> >> ns:0 nr:0 dw:0 dr:216 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
> >
> >
> > I know it might be a silly configuration issue but would be a great help.
>
> Hi,
> I've only skimmed the configuration but what I noticed is that there
> seems to be no order constraint that says that the filesystem should be
> mounted after one of the drbd peers has transitioned to the Master role.
> I suspect that Pacemaker tries to mount the filesystem before the
> corresponding drbd resource has been set to primary.
>
> Also I only see two colocation rules that tie the webserver with the
> virtual_ip and the webserver_fs with the webserver_data_sync but none
> that tells pacemaker to run the ip or webserver on the same system where
> webserver_fs is running. So according to this configuration it would be
> valid to run ip and webserver on one node and drbd primary and
> webserver_fs on the other.
>
> Regards,
> Dennis
>
>
> _______________________________________________
> drbd-user mailing list
> drbd-user@lists.linbit.com
> http://lists.linbit.com/mailman/listinfo/drbd-user
>
[Attachment #5 (text/html)]
<div dir="ltr">Hi Dennis,<div>Thanks for the reply.</div><div><br></div><div>As far \
my understanding I need a ordering constraint which will sense the drbd primary and \
then will allow <span style="font-size:12.8000001907349px">webserver_fs to start \
mounting and a colocation constraint which make sure all resource \
(webserver,virtual_ip, </span><span \
style="font-size:12.8000001907349px">webserver_data_sync ,</span><span \
style="font-size:12.8000001907349px">webserver_fs ) to run in master \
node.</span></div><div><span \
style="font-size:12.8000001907349px"><br></span></div><div><span \
style="font-size:12.8000001907349px">But it pity after hours of head scratch and \
memorizing documentation I am unable to phrase those lines. May be it too much too \
ask, but would you please write those constraints for me.</span></div><div><span \
style="font-size:12.8000001907349px"><br></span></div><div><span \
style="font-size:12.8000001907349px">Regards,</span></div><div><span \
style="font-size:12.8000001907349px">JAE</span></div></div><div \
class="gmail_extra"><br><div class="gmail_quote">On Fri, Jun 26, 2015 at 8:56 PM, \
Dennis Jacobfeuerborn <span dir="ltr"><<a href="mailto:dennisml@conversis.de" \
target="_blank">dennisml@conversis.de</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><span class="">On <a href="tel:26.06.2015%2012" \
value="+12606201512">26.06.2015 12</a>:42, JA E wrote:<br> > Hi,<br>
><br>
> I am newbie to clustering. I managed to setup cluster with<br>
> pacemaker,corosysnc, drbd and pcs in two nodes. But after test restart of<br>
> both node it seems drbd can't mount the desired folder when controlled \
by<br> > pacemaker, manually it's fine.<br>
><br>
><br>
</span>>> *[root@master Desktop]# pcs config*<br>
>> *Cluster Name: cluster_web*<br>
>> *Corosync Nodes:*<br>
>> * master slave *<br>
>> *Pacemaker Nodes:** master slave *<br>
>><br>
>> *Resources: *<br>
>> * Resource: virtual_ip (class=ocf provider=heartbeat type=IPaddr2)*<br>
>> * Attributes: ip=192.168.73.133 cidr_netmask=32 *<br>
>> * Operations: start interval=0s timeout=20s<br>
>> (virtual_ip-start-timeout-20s)*<br>
>> * stop interval=0s timeout=20s \
(virtual_ip-stop-timeout-20s)*<br> >> * monitor \
interval=30s (virtual_ip-monitor-interval-30s)*<br> >> * Resource: webserver \
(class=ocf provider=heartbeat type=apache)*<br> >> * Attributes: \
configfile=/etc/httpd/conf/httpd.conf<br> >> statusurl=<a \
href="http://localhost/server-status" rel="noreferrer" \
target="_blank">http://localhost/server-status</a> <<a \
href="http://localhost/server-status" rel="noreferrer" \
target="_blank">http://localhost/server-status</a>> *<br> >> * Operations: \
start interval=0s timeout=40s (webserver-start-timeout-40s)*<br> >> * \
stop interval=0s timeout=60s (webserver-stop-timeout-60s)*<br> >> * \
monitor interval=1min (webserver-monitor-interval-1min)*<br> >> * Master: \
webserver_data_sync*<br> >> * Meta Attrs: master-max=1 master-node-max=1 \
clone-max=2 clone-node-max=1<br> >> notify=true *<br>
>> * Resource: webserver_data (class=ocf provider=linbit type=drbd)*<br>
>> * Attributes: drbd_resource=drbd0 *<br>
>> * Operations: start interval=0s timeout=240<br>
>> (webserver_data-start-timeout-240)*<br>
>> * promote interval=0s timeout=90<br>
>> (webserver_data-promote-timeout-90)*<br>
>> * demote interval=0s timeout=90<br>
>> (webserver_data-demote-timeout-90)*<br>
>> * stop interval=0s timeout=100<br>
>> (webserver_data-stop-timeout-100)*<br>
>> * monitor interval=60s \
(webserver_data-monitor-interval-60s)*<br> >> * Resource: webserver_fs \
(class=ocf provider=heartbeat type=Filesystem)*<br> >> * Attributes: \
device=/dev/drbd0 directory=/var/www/html fstype=ext3 *<br> >> * Operations: \
start interval=0s timeout=60<br> >> (webserver_fs-start-timeout-60)*<br>
>> * stop interval=0s timeout=60 \
(webserver_fs-stop-timeout-60)**<br> >> monitor interval=20 \
timeout=40<br> >> (webserver_fs-monitor-interval-20)*<br>
>><br>
>> *Stonith Devices: **Fencing Levels: *<br>
>><br>
>> *Location Constraints:*<br>
>> * Resource: webserver*<br>
>> * Enabled on: node01 (score:50) (id:location-webserver-node01-50)*<br>
>> * Enabled on: master (score:50) (id:location-webserver-master-50)*<br>
>> *Ordering Constraints:*<br>
>> * start virtual_ip then start webserver (kind:Mandatory)<br>
>> (id:order-virtual_ip-webserver-mandatory)*<br>
>> * start webserver_fs then start webserver (kind:Mandatory)<br>
>> (id:order-webserver_fs-webserver-mandatory)*<br>
>> *Colocation Constraints:*<br>
>> * webserver with virtual_ip (score:INFINITY)<br>
>> (id:colocation-webserver-virtual_ip-INFINITY)** webserver_fs with<br>
<span class="">>> webserver_data_sync (score:INFINITY) \
(with-rsc-role:Master)<br> </span>>> \
(id:colocation-webserver_fs-webserver_data_sync-INFINITY)*<br> >><br>
>> *Cluster Properties:*<br>
>> * cluster-infrastructure: corosync*<br>
>> * cluster-name: cluster_web*<br>
>> * dc-version: 1.1.12-a14efad*<br>
>> * have-watchdog: false*<br>
>> * no-quorum-policy: ignore** stonith-enabled: false*<br>
>><br>
>><br>
><br>
>><br>
>> *[root@master Desktop]# drbdadm dump*<br>
>> *# /etc/drbd.conf*<br>
>> *global {*<br>
>> * usage-count yes;*<br>
>> * cmd-timeout-medium 600;*<br>
>> * cmd-timeout-long 0;**}*<br>
>><br>
>> *common {*<br>
>> * net {*<br>
>> * protocol C;*<br>
>> * }**}*<br>
>><br>
>> *# resource drbd0 on master: not ignored, not stacked*<br>
>> *# defined at /etc/drbd.d/drbd0.res:1*<br>
>> *resource drbd0 {*<br>
>> * on master {*<br>
>> * volume 0 {*<br>
>> * device /dev/drbd0 minor 0;*<br>
>> * disk /dev/vg_drbd0/lv_drbd0;*<br>
>> * meta-disk internal;*<br>
>> * }*<br>
>> * address ipv4 <a href="http://192.168.73.131:7789" \
rel="noreferrer" target="_blank">192.168.73.131:7789</a><br> >> <<a \
href="http://192.168.73.131:7789/" rel="noreferrer" \
target="_blank">http://192.168.73.131:7789/</a>>;*<br> >> * }*<br>
>> * on slave {*<br>
>> * volume 0 {*<br>
>> * device /dev/drbd0 minor 0;*<br>
>> * disk /dev/vg_drbd0/lv_drbd0;*<br>
>> * meta-disk internal;*<br>
>> * }*<br>
>> * address ipv4 <a href="http://192.168.73.132:7789" \
rel="noreferrer" target="_blank">192.168.73.132:7789</a><br> >> <<a \
href="http://192.168.73.132:7789/" rel="noreferrer" \
target="_blank">http://192.168.73.132:7789/</a>>;*<br> >> * }**}*<br>
>><br>
>><br>
><br>
>><br>
>> *[root@master Desktop]# pcs status*<br>
>> *Cluster name: cluster_web*<br>
>> *Last updated: Fri Jun 26 03:04:18 2015*<br>
>> *Last change: Fri Jun 26 02:13:11 2015*<br>
>> *Stack: corosync*<br>
>> *Current DC: master (1) - partition with quorum*<br>
>> *Version: 1.1.12-a14efad*<br>
>> *2 Nodes configured**5 Resources configured*<br>
>><br>
>> *Online: [ master slave ]*<br>
>> *Full list of resources:*<br>
>><br>
>> * virtual_ip (ocf::heartbeat:IPaddr2): Started master *<br>
>> * webserver (ocf::heartbeat:apache): Stopped *<br>
>> * Master/Slave Set: webserver_data_sync [webserver_data]*<br>
>> * Masters: [ master ]*<br>
>> * Slaves: [ slave ]** webserver_fs (ocf::heartbeat:Filesystem):<br>
>> Stopped *<br>
>><br>
>> *Failed actions:*<br>
>> * webserver_fs_start_0 on master 'unknown error' (1): \
call=23,<br> <span class="">>> status=complete, exit-reason='Couldn't \
mount filesystem /dev/drbd0 on<br> >> /var/www/html', \
last-rc-change='Fri Jun 26 02:20:45 2015', queued=0ms,<br> </span>>> \
exec=87ms** webserver_fs_start_0 on slave 'unknown error' (1):<br> <span \
class="">>> call=23, status=complete, exit-reason='Couldn't mount \
filesystem /dev/drbd0<br> >> on /var/www/html', last-rc-change='Fri Jun \
26 02:20:45 2015', queued=0ms,<br> </span>>> exec=79ms*<br>
>><br>
>><br>
>> *PCSD Status:*<br>
>> * master: Online** slave: Online*<br>
>><br>
>> *Daemon Status:*<br>
>> * corosync: active/enabled*<br>
>> * pacemaker: active/enabled** pcsd: active/enabled*<br>
<span class="">><br>
><br>
> cat /proc/drbd<br>
>> version: 8.4.6 (api:1/proto:86-101)<br>
>> GIT-hash: 833d830e0152d1e457fa7856e71e11248ccf3f70 build by \
phil@Build64R7,<br> >> 2015-04-10 05:13:52<br>
>> 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----<br>
>> ns:0 nr:0 dw:0 dr:216 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f \
oos:0<br> ><br>
><br>
> I know it might be a silly configuration issue but would be a great help.<br>
<br>
</span>Hi,<br>
I've only skimmed the configuration but what I noticed is that there<br>
seems to be no order constraint that says that the filesystem should be<br>
mounted after one of the drbd peers has transitioned to the Master role.<br>
I suspect that Pacemaker tries to mount the filesystem before the<br>
corresponding drbd resource has been set to primary.<br>
<br>
Also I only see two colocation rules that tie the webserver with the<br>
virtual_ip and the webserver_fs with the webserver_data_sync but none<br>
that tells pacemaker to run the ip or webserver on the same system where<br>
webserver_fs is running. So according to this configuration it would be<br>
valid to run ip and webserver on one node and drbd primary and<br>
webserver_fs on the other.<br>
<br>
Regards,<br>
Dennis<br>
<br>
<br>
_______________________________________________<br>
drbd-user mailing list<br>
<a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>
<a href="http://lists.linbit.com/mailman/listinfo/drbd-user" rel="noreferrer" \
target="_blank">http://lists.linbit.com/mailman/listinfo/drbd-user</a><br> \
</blockquote></div><br></div>
_______________________________________________
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic