[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    one minute delay in running a simple ls command on hadoop (maybe near security groups..): hadoop 0.2
From:       Gopi Krishna M <mgopik () gmail ! com>
Date:       2013-03-31 6:29:02
Message-ID: CAEy65xr6KGtmTzvjNHwDZpmnWZM9-OFSmhFb=b0CpMAcxbK3nQ () mail ! gmail ! com
[Download RAW message or body]

Hi..

I have installed hadoop 0.23.5 and is working fine on two of my
installations.  In a new installation on Windows Azure VMs, I am seeing an
inordinate delay (of 1 minute) when doing a simple dfs ls command.

This is a 3 node cluster, running 64 bit Ubuntu 12.04.1 LTS, Sun jre

command : bin/hadoop dfs -ls /

With log set to DEBUG, following is the output on the console.  The logs in
bold show the sleep/timeout/delay of *1 minute*.  Any pointers on what to
check for / what is the reason for this delay?

13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess
with annotation @org.apache.
hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of
successful kerberos logins
and latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
13/03/31 05:48:55 DEBUG lib.MutableMetricsFactory: field
org.apache.hadoop.metrics2.lib.MutableRate
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure
with annotation @org.apache.
hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of
failed kerberos logins and
latency (milliseconds)], always=false, type=DEFAULT, sampleName=Ops)
13/03/31 05:48:55 DEBUG impl.MetricsSystemImpl: UgiMetrics, User and group
related metrics
13/03/31 05:49:55 DEBUG security.Groups:  Creating new Groups object
*13/03/31 05:49:55 DEBUG security.Groups: Group mapping
impl=org.apache.hadoop.security.ShellBasedUni*
*xGroupsMapping; cacheTimeout=300000*
*13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login*
13/03/31 05:50:55 DEBUG security.UserGroupInformation: hadoop login commit
13/03/31 05:50:55 DEBUG security.UserGroupInformation: using local
user:UnixPrincipal: hduser
13/03/31 05:50:55 DEBUG security.UserGroupInformation: UGI loginUser:hduser
(auth:SIMPLE)
13/03/31 05:50:56 DEBUG hdfs.DFSClient: Short circuit read is false
13/03/31 05:50:56 DEBUG ipc.Client: The ping interval is 60000 ms.
13/03/31 05:50:56 DEBUG ipc.Client: Use SIMPLE authentication for protocol
ClientProtocol
13/03/31 05:50:56 DEBUG ipc.Client: Connecting to XXXXXX:9000


Thanks
Gopi

[Attachment #3 (text/html)]

<div dir="ltr">Hi..<div><br></div><div style>I have installed hadoop 0.23.5 and is \
working fine on two of my installations.  In a new installation on Windows Azure VMs, \
I am seeing an inordinate delay (of 1 minute) when doing a simple dfs ls \
command.</div> <div style><br></div><div style>This is a 3 node cluster, running 64 \
bit Ubuntu 12.04.1 LTS, Sun jre</div><div style><br></div><div style>command : \
bin/hadoop dfs -ls /</div><div style><br></div><div style>With log set to DEBUG, \
following is the output on the console.  The logs in bold show the \
sleep/timeout/delay of <b>1 minute</b>.  Any pointers on what to check for / what is \
the reason for this delay?</div> <div style><br></div><div style><div>13/03/31 \
05:48:55 DEBUG lib.MutableMetricsFactory: field \
org.apache.hadoop.metrics2.lib.MutableRate</div><div>org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginSuccess \
with annotation @org.apache.</div> \
<div>hadoop.metrics2.annotation.Metric(valueName=Time, about=, value=[Rate of \
successful kerberos logins</div><div>and latency (milliseconds)], always=false, \
type=DEFAULT, sampleName=Ops)</div><div>13/03/31 05:48:55 DEBUG \
lib.MutableMetricsFactory: field org.apache.hadoop.metrics2.lib.MutableRate</div> \
<div>org.apache.hadoop.security.UserGroupInformation$UgiMetrics.loginFailure with \
annotation @org.apache.</div><div>hadoop.metrics2.annotation.Metric(valueName=Time, \
about=, value=[Rate of failed kerberos logins and</div> <div>latency (milliseconds)], \
always=false, type=DEFAULT, sampleName=Ops)</div><div>13/03/31 05:48:55 DEBUG \
impl.MetricsSystemImpl: UgiMetrics, User and group related metrics</div><div>13/03/31 \
05:49:55 DEBUG security.Groups:  Creating new Groups object</div> <div><b>13/03/31 \
05:49:55 DEBUG security.Groups: Group mapping \
impl=org.apache.hadoop.security.ShellBasedUni</b></div><div><b>xGroupsMapping; \
cacheTimeout=300000</b></div><div><b>13/03/31 05:50:55 DEBUG \
security.UserGroupInformation: hadoop login</b></div> <div>13/03/31 05:50:55 DEBUG \
security.UserGroupInformation: hadoop login commit</div><div>13/03/31 05:50:55 DEBUG \
security.UserGroupInformation: using local user:UnixPrincipal: \
hduser</div><div>13/03/31 05:50:55 DEBUG security.UserGroupInformation: UGI \
loginUser:hduser (auth:SIMPLE)</div> <div>13/03/31 05:50:56 DEBUG hdfs.DFSClient: \
Short circuit read is false</div><div>13/03/31 05:50:56 DEBUG ipc.Client: The ping \
interval is 60000 ms.</div><div>13/03/31 05:50:56 DEBUG ipc.Client: Use SIMPLE \
authentication for protocol ClientProtocol</div> <div>13/03/31 05:50:56 DEBUG \
ipc.Client: Connecting to XXXXXX:9000</div></div><div style><br></div><div \
style><br></div><div style>Thanks</div><div style>Gopi</div><div \
style><br></div></div>



[prev in list] [next in list] [prev in thread] [next in thread]