[prev in list] [next in list] [prev in thread] [next in thread]
List: gluster-devel
Subject: Fwd: [Gluster-devel] gluster working, but error appearing every two seconds in logs - NEW INFO
From: jordi () cdmon ! com (Jordi Moles Blanco)
Date: 2009-02-20 9:16:53
Message-ID: 499E7505.7030805 () cdmon ! com
[Download RAW message or body]
En/na Basavanagowda Kanur ha escrit:
> Forwarding to list
>
> ---------- Forwarded message ----------
> From: *Basavanagowda Kanur* <gowda at gluster.com <mailto:gowda at gluster.com>>
> Date: Thu, Feb 19, 2009 at 11:56 PM
> Subject: Re: [Gluster-devel] gluster working, but error appearing
> every two seconds in logs - NEW INFO
> To: jordi at cdmon.com <mailto:jordi at cdmon.com>
> Cc: Anand Avati <avati at zresearch.com <mailto:avati at zresearch.com>>,
> Gluster List <gluster-devel at nongnu.org <mailto:gluster-devel at nongnu.org>>
>
>
> Jordi,
> Please find replies inline.
>
>
> Hi,
>
> since the last time we were in contact I've been trying to track
> down where the problem is. I've been monitoring almost every
> possible thing related to network traffic, and... eventually.... i
> found out what the problem is by chance!!
>
> Did you try 'telnet <server> <port>' at the time when you were getting
> 'Connection refused' errors?
>
>
> It turns out that when in a client-server mounting gluster i run
> "df -h", i get this:
>
> ***********
> 2009-02-19 17:15:43 E [tcp-client.c:190:tcp_connect] espai1:
> non-blocking connect() returned: 111 (Connection refused)
> 2009-02-19 17:15:43 W [client-protocol.c:332:client_protocol_xfer]
> espai1: not connected at the moment to submit frame type(1) op(15)
> 2009-02-19 17:15:43 E [client-protocol.c:3891:client_statfs_cbk]
> espai1: no proper reply from server, returning ENOTCONN
> 2009-02-19 17:15:43 E [tcp-client.c:190:tcp_connect] espai5:
> non-blocking connect() returned: 111 (Connection refused)
> 2009-02-19 17:15:43 W [client-protocol.c:332:client_protocol_xfer]
> espai5: not connected at the moment to submit frame type(1) op(15)
> 2009-02-19 17:15:43 E [client-protocol.c:3891:client_statfs_cbk]
> espai5: no proper reply from server, returning ENOTCONN
> 2009-02-19 17:15:43 E [tcp-client.c:190:tcp_connect] espai2:
> non-blocking connect() returned: 111 (Connection refused)
> 2009-02-19 17:15:43 W [client-protocol.c:332:client_protocol_xfer]
> espai2: not connected at the moment to submit frame type(1) op(15)
> 2009-02-19 17:15:43 E [client-protocol.c:3891:client_statfs_cbk]
> espai2: no proper reply from server, returning ENOTCONN
> 2009-02-19 17:15:43 E [tcp-client.c:190:tcp_connect] espai6:
> non-blocking connect() returned: 111 (Connection refused)
> 2009-02-19 17:15:43 W [client-protocol.c:332:client_protocol_xfer]
> espai6: not connected at the moment to submit frame type(1) op(15)
> 2009-02-19 17:15:43 E [client-protocol.c:3891:client_statfs_cbk]
> espai6: no proper reply from server, returning ENOTCONN
>
> ************
>
> so... the reason why it is appearing so often is that i've got
> munin monitoring this gluster environment, and it performs a "df"
> command to check the disk space of all the servers, including, of
> course, the gluster mount point. When this happens... the error
> log shown above these lines is reported and eventually.... the
> mount point in that server fails. No data is lost, but i have to
> remount glusterfs as it becomes stale and data is not accessible.
>
> is this a normal behaviour?
>
> No. This is not normal behaviour. 'Connection refused' error comes
> when either the server is not running or there is a packet filtering
> between server and client.
>
>
>
>
> i could stop munin from running "df" every 5 minutes... but
> still... is there any problem in my setup or is this what gluster
> is supposed to do?
>
> The 'Connection refused' problem is not caused by any application
> accessing the mount-point. 'Connection refused' can come only if
> server is dead or packet is being filtered out.
>
>
>
> Thanks.
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org <mailto:Gluster-devel at nongnu.org>
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>
>
>
>
> --
> gowda
>
>
>
> --
> hard work often pays off after time, but laziness always pays off now
> ------------------------------------------------------------------------
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
>
hi,
********
Did you try 'telnet <server> <port>' at the time when you were getting
'Connection refused' errors?
*******
no, i hadn't tried that, i tried this morning. I ran a infinite loop in
shellscript with "df -h" while i was doing some telnet and ssh attempts
to the node servers, and it worked fine, to all of them. It doesn't seem
there's any network problem between clients and nodes, even when i'm
running "df" constantly on the gluster mount point.
I know you told me that...
" 'Connection refused' error comes when either the server is not running
or there is a packet filtering between server and client."
and it makes sense, but i've even tried to replace the switch device
which links clients and nodes, and this error message keeps appearing.
The nodes, however, don't log anything, they "don't see" any kind of
problem, it's only the client server which reports de problem. What's
more, this problem keeps appearing through the whole day, but the mount
point keeps beeing accessible. It's just after some time of this problem
occurring that it eventually fails and the mount point gets stale. After
that, i simply umount/mount and everything works fine again for some
more time.
I can always reproduce this problem, i only have to run a simple "df -h"
command.
This error messages have been reported using WARNING log level. This
morning i also tried with DEBUG log level. Nodes don't report anything
special, but clients do, this is the full log file when it loads the
gluster mount point
****************************************
2009-02-20 09:50:50 D [glusterfs.c:167:get_spec_fp] glusterfs: loading spec from \
/etc/glusterfs/glusterfs-client.vol 2009-02-20 09:50:50 D [spec.y:107:new_section] \
parser: New node for 'espai1' 2009-02-20 09:50:50 D [xlator.c:115:xlator_set_type] \
xlator: attempt to load file //lib/glusterfs/1.3.12a/xlator/protocol/client.so \
2009-02-20 09:50:50 D [spec.y:127:section_type] parser: Type:espai1:protocol/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:espai1:transport-type:tcp/client 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:espai1:remote-host:10.0.0.3 2009-02-20 \
09:50:50 D [spec.y:141:section_option] parser: Option:espai1:remote-subvolume:espai \
2009-02-20 09:50:50 D [spec.y:209:section_end] parser: end:espai1 2009-02-20 09:50:50 \
D [spec.y:107:new_section] parser: New node for 'espai2' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:espai2:protocol/client 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:espai2:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:espai2:remote-host:10.0.0.4 2009-02-20 09:50:50 D [spec.y:141:section_option] \
parser: Option:espai2:remote-subvolume:espai 2009-02-20 09:50:50 D \
[spec.y:209:section_end] parser: end:espai2 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'espai3' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:espai3:protocol/client 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:espai3:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:espai3:remote-host:10.0.0.5 2009-02-20 09:50:50 D [spec.y:141:section_option] \
parser: Option:espai3:remote-subvolume:espai 2009-02-20 09:50:50 D \
[spec.y:209:section_end] parser: end:espai3 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'espai4' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:espai4:protocol/client 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:espai4:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:espai4:remote-host:10.0.0.6 2009-02-20 09:50:50 D [spec.y:141:section_option] \
parser: Option:espai4:remote-subvolume:espai 2009-02-20 09:50:50 D \
[spec.y:209:section_end] parser: end:espai4 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'espai5' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:espai5:protocol/client 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:espai5:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:espai5:remote-host:10.0.0.7 2009-02-20 09:50:50 D [spec.y:141:section_option] \
parser: Option:espai5:remote-subvolume:espai 2009-02-20 09:50:50 D \
[spec.y:209:section_end] parser: end:espai5 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'espai6' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:espai6:protocol/client 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:espai6:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:espai6:remote-host:10.0.0.8 2009-02-20 09:50:50 D [spec.y:141:section_option] \
parser: Option:espai6:remote-subvolume:espai 2009-02-20 09:50:50 D \
[spec.y:209:section_end] parser: end:espai6 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'namespace1' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:namespace1:protocol/client 2009-02-20 09:50:50 \
D [spec.y:141:section_option] parser: Option:namespace1:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:namespace1:remote-host:10.0.0.3 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:namespace1:remote-subvolume:nm 2009-02-20 \
09:50:50 D [spec.y:209:section_end] parser: end:namespace1 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'namespace2' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:namespace2:protocol/client 2009-02-20 09:50:50 \
D [spec.y:141:section_option] parser: Option:namespace2:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:namespace2:remote-host:10.0.0.4 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:namespace2:remote-subvolume:nm 2009-02-20 \
09:50:50 D [spec.y:209:section_end] parser: end:namespace2 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'namespace3' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:namespace3:protocol/client 2009-02-20 09:50:50 \
D [spec.y:141:section_option] parser: Option:namespace3:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:namespace3:remote-host:10.0.0.5 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:namespace3:remote-subvolume:nm 2009-02-20 \
09:50:50 D [spec.y:209:section_end] parser: end:namespace3 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'namespace4' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:namespace4:protocol/client 2009-02-20 09:50:50 \
D [spec.y:141:section_option] parser: Option:namespace4:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:namespace4:remote-host:10.0.0.6 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:namespace4:remote-subvolume:nm 2009-02-20 \
09:50:50 D [spec.y:209:section_end] parser: end:namespace4 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'namespace5' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:namespace5:protocol/client 2009-02-20 09:50:50 \
D [spec.y:141:section_option] parser: Option:namespace5:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:namespace5:remote-host:10.0.0.7 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:namespace5:remote-subvolume:nm 2009-02-20 \
09:50:50 D [spec.y:209:section_end] parser: end:namespace5 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'namespace6' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/protocol/client.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:namespace6:protocol/client 2009-02-20 09:50:50 \
D [spec.y:141:section_option] parser: Option:namespace6:transport-type:tcp/client \
2009-02-20 09:50:50 D [spec.y:141:section_option] parser: \
Option:namespace6:remote-host:10.0.0.8 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:namespace6:remote-subvolume:nm 2009-02-20 \
09:50:50 D [spec.y:209:section_end] parser: end:namespace6 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'grup1' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/cluster/afr.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:grup1:cluster/afr 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:grup1->espai1 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:grup1->espai3 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:grup1->espai5 2009-02-20 09:50:50 D \
[spec.y:209:section_end] parser: end:grup1 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'grup2' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/cluster/afr.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:grup2:cluster/afr 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:grup2->espai2 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:grup2->espai4 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:grup2->espai6 2009-02-20 09:50:50 D \
[spec.y:209:section_end] parser: end:grup2 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'nm' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/cluster/afr.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:nm:cluster/afr 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:nm->namespace1 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:nm->namespace2 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:nm->namespace3 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:nm->namespace4 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:nm->namespace5 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:nm->namespace6 2009-02-20 09:50:50 D \
[spec.y:209:section_end] parser: end:nm 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'g01' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/cluster/unify.so 2009-02-20 09:50:50 D \
[spec.y:127:section_type] parser: Type:g01:cluster/unify 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:g01->grup1 2009-02-20 09:50:50 D \
[spec.y:196:section_sub] parser: child:g01->grup2 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:g01:scheduler:rr 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:g01:namespace:nm 2009-02-20 09:50:50 D \
[spec.y:209:section_end] parser: end:g01 2009-02-20 09:50:50 D \
[spec.y:107:new_section] parser: New node for 'io-cache' 2009-02-20 09:50:50 D \
[xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/performance/io-cache.so 2009-02-20 09:50:50 D \
[xlator.c:145:xlator_set_type] xlator: dlsym(notify) on \
//lib/glusterfs/1.3.12a/xlator/performance/io-cache.so: undefined symbol: notify -- \
neglecting 2009-02-20 09:50:50 D [spec.y:127:section_type] parser: \
Type:io-cache:performance/io-cache 2009-02-20 09:50:50 D [spec.y:141:section_option] \
parser: Option:io-cache:cache-size:512MB 2009-02-20 09:50:50 D \
[spec.y:141:section_option] parser: Option:io-cache:page-size:1MB 2009-02-20 09:50:50 \
D [spec.y:141:section_option] parser: Option:io-cache:force-revalidate-timeout:2 \
2009-02-20 09:50:50 D [spec.y:196:section_sub] parser: child:io-cache->g01 2009-02-20 \
09:50:50 D [spec.y:209:section_end] parser: end:io-cache 2009-02-20 09:50:50 D \
[glusterfs.c:128:fuse_graph] glusterfs: setting option mount-point to /mnt/glusterfs \
2009-02-20 09:50:50 D [xlator.c:115:xlator_set_type] xlator: attempt to load file \
//lib/glusterfs/1.3.12a/xlator/mount/fuse.so 2009-02-20 09:50:50 D \
[client-protocol.c:5013:init] espai1: defaulting transport-timeout to 42 2009-02-20 \
09:50:50 D [transport.c:80:transport_load] transport: attempt to load file \
//lib/glusterfs/1.3.12a/transport/tcp/client.so 2009-02-20 09:50:50 D \
[client-protocol.c:5040:init] espai1: defaulting limits.transaction-size to 268435456 \
2009-02-20 09:50:50 D [client-protocol.c:5013:init] espai3: defaulting \
transport-timeout to 42 2009-02-20 09:50:50 D [transport.c:80:transport_load] \
transport: attempt to load file //lib/glusterfs/1.3.12a/transport/tcp/client.so \
2009-02-20 09:50:50 D [client-protocol.c:5040:init] espai3: defaulting \
limits.transaction-size to 268435456 2009-02-20 09:50:50 D \
[client-protocol.c:5013:init] espai5: defaulting transport-timeout to 42 2009-02-20 \
09:50:50 D [transport.c:80:transport_load] transport: attempt to load file \
//lib/glusterfs/1.3.12a/transport/tcp/client.so 2009-02-20 09:50:50 D \
[client-protocol.c:5040:init] espai5: defaulting limits.transaction-size to 268435456 \
2009-02-20 09:50:50 D [afr.c:6084:init] grup1: self-heal is enabled (default) \
2009-02-20 09:50:50 D [afr.c:6112:init] grup1: (default) reads will be scheduled \
between the children 2009-02-20 09:50:50 D [client-protocol.c:5340:notify] espai1: \
got GF_EVENT_PARENT_UP, attempting connect on transport 2009-02-20 09:50:50 D \
[client-protocol.c:5340:notify] espai3: got GF_EVENT_PARENT_UP, attempting connect on \
transport 2009-02-20 09:50:50 D [client-protocol.c:5340:notify] espai5: got \
GF_EVENT_PARENT_UP, attempting connect on transport 2009-02-20 09:50:50 D \
[client-protocol.c:5013:init] espai2: defaulting transport-timeout to 42 2009-02-20 \
09:50:50 D [transport.c:80:transport_load] transport: attempt to load file \
//lib/glusterfs/1.3.12a/transport/tcp/client.so 2009-02-20 09:50:50 D \
[client-protocol.c:5040:init] espai2: defaulting limits.transaction-size to 268435456 \
2009-02-20 09:50:50 D [client-protocol.c:5013:init] espai4: defaulting \
transport-timeout to 42 2009-02-20 09:50:50 D [transport.c:80:transport_load] \
transport: attempt to load file //lib/glusterfs/1.3.12a/transport/tcp/client.so \
2009-02-20 09:50:50 D [client-protocol.c:5040:init] espai4: defaulting \
limits.transaction-size to 268435456 2009-02-20 09:50:50 D \
[client-protocol.c:5013:init] espai6: defaulting transport-timeout to 42 2009-02-20 \
09:50:50 D [transport.c:80:transport_load] transport: attempt to load file \
//lib/glusterfs/1.3.12a/transport/tcp/client.so 2009-02-20 09:50:50 D \
[client-protocol.c:5040:init] espai6: defaulting limits.transaction-size to 268435456 \
2009-02-20 09:50:50 D [afr.c:6084:init] grup2: self-heal is enabled (default) \
2009-02-20 09:50:50 D [afr.c:6112:init] grup2: (default) reads will be scheduled \
between the children 2009-02-20 09:50:50 D [client-protocol.c:5340:notify] espai2: \
got GF_EVENT_PARENT_UP, attempting connect on transport 2009-02-20 09:50:50 D \
[client-protocol.c:5340:notify] espai4: got GF_EVENT_PARENT_UP, attempting connect on \
transport 2009-02-20 09:50:50 D [client-protocol.c:5340:notify] espai6: got \
GF_EVENT_PARENT_UP, attempting connect on transport 2009-02-20 09:50:50 D \
[unify.c:4018:init] g01: namespace node specified as nm 2009-02-20 09:50:50 D \
[scheduler.c:42:get_scheduler] scheduler: attempt to load file rr.so
2009-02-20 09:50:50 D [unify.c:4048:init] g01: Child node count is 2
2009-02-20 09:50:50 D [rr.c:48:rr_init] g01: 'option rr.limits.min-free-disk' not \
specified, defaulting to 5 2009-02-20 09:50:50 D [client-protocol.c:5013:init] \
namespace1: defaulting transport-timeout to 42 2009-02-20 09:50:50 D \
[transport.c:80:transport_load] transport: attempt to load file \
//lib/glusterfs/1.3.12a/transport/tcp/client.so 2009-02-20 09:50:50 D \
[client-protocol.c:5040:init] namespace1: defaulting limits.transaction-size to \
268435456 2009-02-20 09:50:50 D [client-protocol.c:5013:init] namespace2: defaulting \
transport-timeout to 42 2009-02-20 09:50:50 D [transport.c:80:transport_load] \
transport: attempt to load file //lib/glusterfs/1.3.12a/transport/tcp/client.so \
2009-02-20 09:50:50 D [client-protocol.c:5040:init] namespace2: defaulting \
limits.transaction-size to 268435456 2009-02-20 09:50:50 D \
[client-protocol.c:5013:init] namespace3: defaulting transport-timeout to 42 \
2009-02-20 09:50:50 D [transport.c:80:transport_load] transport: attempt to load file \
//lib/glusterfs/1.3.12a/transport/tcp/client.so 2009-02-20 09:50:50 D \
[client-protocol.c:5040:init] namespace3: defaulting limits.transaction-size to \
268435456 2009-02-20 09:50:50 D [client-protocol.c:5013:init] namespace4: defaulting \
transport-timeout to 42 2009-02-20 09:50:50 D [transport.c:80:transport_load] \
transport: attempt to load file //lib/glusterfs/1.3.12a/transport/tcp/client.so \
2009-02-20 09:50:50 D [client-protocol.c:5040:init] namespace4: defaulting \
limits.transaction-size to 268435456 2009-02-20 09:50:50 D \
[client-protocol.c:5013:init] namespace5: defaulting transport-timeout to 42 \
2009-02-20 09:50:50 D [transport.c:80:transport_load] transport: attempt to load file \
//lib/glusterfs/1.3.12a/transport/tcp/client.so 2009-02-20 09:50:50 D \
[client-protocol.c:5040:init] namespace5: defaulting limits.transaction-size to \
268435456 2009-02-20 09:50:50 D [client-protocol.c:5013:init] namespace6: defaulting \
transport-timeout to 42 2009-02-20 09:50:50 D [transport.c:80:transport_load] \
transport: attempt to load file //lib/glusterfs/1.3.12a/transport/tcp/client.so \
2009-02-20 09:50:50 D [client-protocol.c:5040:init] namespace6: defaulting \
limits.transaction-size to 268435456 2009-02-20 09:50:50 D [afr.c:6084:init] nm: \
self-heal is enabled (default) 2009-02-20 09:50:50 D [afr.c:6112:init] nm: (default) \
reads will be scheduled between the children
****************************************
and this is when it starts to fail (the same i showed you previously but
with DEBUG level):
****************************************
009-02-20 09:50:50 D [tcp-client.c:77:tcp_connect] espai1: socket fd = 6
2009-02-20 09:50:50 D [tcp-client.c:107:tcp_connect] espai1: finalized on port `1019'
2009-02-20 09:50:50 D [tcp-client.c:128:tcp_connect] espai1: defaulting remote-port \
to 6996 2009-02-20 09:50:50 D [common-utils.c:179:gf_resolve_ip] resolver: DNS cache \
not present, freshly probing hostname: 10.0.0.3 2009-02-20 09:50:50 D \
[common-utils.c:204:gf_resolve_ip] resolver: returning IP:10.0.0.3[0] for hostname: \
10.0.0.3 2009-02-20 09:50:50 D [common-utils.c:212:gf_resolve_ip] resolver: flushing \
DNS cache 2009-02-20 09:50:50 D [tcp-client.c:161:tcp_connect] espai1: connect on 6 \
in progress (non-blocking) 2009-02-20 09:50:50 E [tcp-client.c:190:tcp_connect] \
espai1: non-blocking connect() returned: 111 (Connection refused) 2009-02-20 09:50:50 \
D [client-protocol.c:4757:client_protocol_reconnect] espai3: attempting reconnect \
2009-02-20 09:50:50 D [tcp-client.c:77:tcp_connect] espai3: socket fd = 6 2009-02-20 \
09:50:50 D [tcp-client.c:107:tcp_connect] espai3: finalized on port `1019' 2009-02-20 \
09:50:50 D [tcp-client.c:128:tcp_connect] espai3: defaulting remote-port to 6996 \
2009-02-20 09:50:50 D [common-utils.c:179:gf_resolve_ip] resolver: DNS cache not \
present, freshly probing hostname: 10.0.0.5 2009-02-20 09:50:50 D \
[common-utils.c:204:gf_resolve_ip] resolver: returning IP:10.0.0.5[0] for hostname: \
10.0.0.5 2009-02-20 09:50:50 D [common-utils.c:212:gf_resolve_ip] resolver: flushing \
DNS cache
****************************************
Just let me know if i can provide any extra information.
Thanks.
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic