[prev in list] [next in list] [prev in thread] [next in thread]
List: hadoop-user
Subject: Re: Two map reduce jobs running at once creates port conflict.
From: sandeep das <yarnhadoop () gmail ! com>
Date: 2015-10-23 4:59:16
Message-ID: CAD1=LDr7t3qY2R6EyrSNRSUjMDpCudiC0uw5uekNqpWLtBBV+w () mail ! gmail ! com
[Download RAW message or body]
I may not be correct here but as far as I can see it's regarding your HDFS
data node port. Have you created your "UNIX Domain Socket path" and not
provided the read/write permissions for the user from which you are running
your map reduce jobs.
If you are using cloudera as your setup then can find this configuration
from your cloudera manager's web page. All you have to do is go to hdfs ->
configuration -> search for "unix" you might this configuration entry.
If there is any value configured then make sure that you have set proper
permissions for this folder.
I hope this helps.
On Thu, Oct 22, 2015 at 11:34 PM, Edward Capriolo <edlinuxguru@gmail.com>
wrote:
> I have just updated to CDH 5.4.2.
>
>
> When multiple map reduce jobs run at once a port bind conflict sometimes =
happens. It seems like from the message that binding to 0.0.0.0:0 will pick=
a random port which should not cause a conflict but that does not seem to =
happen.
>
> at sun.nio.ch.Net.bind(Net.java:436)
> at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:=
214)
> at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
> at org.apache.hadoop.ipc.Server.bind(Server.java:407)
> ... 19 more
> 2015-10-04 19:31:10,567 INFO [main] org.apache.hadoop.service.AbstractSer=
vice: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in stat=
e STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: j=
ava.net.BindException: Problem binding to [0.0.0.0:0] java.net.BindExceptio=
n: Address already in use; For more details see: http://wiki.apache.org/ha=
doop/BindException
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindExce=
ption: Problem binding to [0.0.0.0:0] java.net.BindException: Address alrea=
dy in use; For more details see: http://wiki.apache.org/hadoop/BindExcepti=
on
> at org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl.getSe=
rver(RpcServerFactoryPBImpl.java:139)
> at org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC.getServer(HadoopYarnPro=
toRPC.java:65)
> at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStar=
t(MRClientService.java:119)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:=
193)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMast=
er.java:1084)
> at org.apache.hadoop.service.AbstractService.start(AbstractService.java:=
193)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java=
:1500)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma=
tion.java:1671)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(=
MRAppMaster.java:1496)
> at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:=
1429)
> Caused by: java.net.BindException: Problem binding to [0.0.0.0:0] java.ne=
t.BindException: Address already in use; For more details see: http://wiki=
.apache.org/hadoop/BindException
>
> Does anyone know why this happens? Also a work around that does not invol=
ve an upgrade?
>
>
> TX
>
>
[Attachment #3 (text/html)]
<div dir="ltr"><div><div><div>I may not be correct here but as far as I can see \
it's regarding your HDFS data node port. Have you created your "UNIX Domain \
Socket path" and not provided the read/write permissions for the user from which \
you are running your map reduce jobs.<br><br></div>If you are using cloudera as your \
setup then can find this configuration from your cloudera manager's web page. All \
you have to do is go to hdfs -> configuration -> search for "unix" \
you might this configuration entry. <br><br></div>If there is any value configured \
then make sure that you have set proper permissions for this folder.<br><br></div>I \
hope this helps.<br><br><br></div><div class="gmail_extra"><br><div \
class="gmail_quote">On Thu, Oct 22, 2015 at 11:34 PM, Edward Capriolo <span \
dir="ltr"><<a href="mailto:edlinuxguru@gmail.com" \
target="_blank">edlinuxguru@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><pre \
style="margin-top:0px;margin-bottom:0px;padding:0px;border:0px;font-size:13px">I have \
just updated to <span \
style="font-family:Arial,Helvetica,sans-serif;font-size:18px;white-space:normal">CDH \
5.4.2. </span></pre><pre \
style="margin-top:0px;margin-bottom:0px;padding:0px;border:0px"><br></pre><pre \
style="margin-top:0px;margin-bottom:0px;padding:0px;border:0px">When multiple map \
reduce jobs run at once a port bind conflict sometimes happens. It seems like from \
the message that binding to <a href="http://0.0.0.0:0" target="_blank">0.0.0.0:0</a> \
will pick a random port which should not cause a conflict but that does not seem to \
happen.</pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;border:0px"><span \
style="font-size:13px">at sun.nio.ch.Net.bind(Net.java:</span><span \
style="font-size:13px">436) at <a href="http://sun.nio.ch" \
target="_blank">sun.nio.ch</a>.</span><span \
style="font-size:13px">ServerSocketChannelImpl.bind(</span><span \
style="font-size:13px">ServerSocketChannelImpl.java:</span><span \
style="font-size:13px">214) at <a href="http://sun.nio.ch" \
target="_blank">sun.nio.ch</a>.</span><span \
style="font-size:13px">ServerSocketAdaptor.bind(</span><span \
style="font-size:13px">ServerSocketAdaptor.java:74) at \
org.apache.hadoop.ipc.Server.</span><span \
style="font-size:13px">bind(Server.java:407)
... 19 more
2015-10-04 19:31:10,567 INFO [main] org.apache.hadoop.service.</span><span \
style="font-size:13px">AbstractService: Service \
org.apache.hadoop.mapreduce.</span><span style="font-size:13px">v2.app.MRAppMaster \
failed in state STARTED; cause: org.apache.hadoop.yarn.</span><span \
style="font-size:13px">exceptions.</span><span \
style="font-size:13px">YarnRuntimeException: java.net.BindException: Problem binding \
to [</span><a href="http://0.0.0.0:0/" rel="nofollow" \
style="font-size:13px;margin:0px;padding:0px;border:0px;text-decoration:none;color:rgb(102,17,204)" \
target="_blank">0.0.0.0:0</a><span style="font-size:13px">] java.net.BindException: \
Address already in use; For more details see: </span><a \
href="http://wiki.apache.org/hadoop/BindException" rel="nofollow" \
style="font-size:13px;margin:0px;padding:0px;border:0px;text-decoration:none;color:rgb(102,17,204)" \
target="_blank">http://wiki.apache.org/hadoop/BindException</a><span \
style="font-size:13px"> org.apache.hadoop.yarn.</span><span \
style="font-size:13px">exceptions.</span><span \
style="font-size:13px">YarnRuntimeException: java.net.BindException: Problem binding \
to [</span><a href="http://0.0.0.0:0/" rel="nofollow" \
style="font-size:13px;margin:0px;padding:0px;border:0px;text-decoration:none;color:rgb(102,17,204)" \
target="_blank">0.0.0.0:0</a><span style="font-size:13px">] java.net.BindException: \
Address already in use; For more details see: </span><a \
href="http://wiki.apache.org/hadoop/BindException" rel="nofollow" \
style="font-size:13px;margin:0px;padding:0px;border:0px;text-decoration:none;color:rgb(102,17,204)" \
target="_blank">http://wiki.apache.org/hadoop/BindException</a><span \
style="font-size:13px"> at org.apache.hadoop.yarn.</span><span \
style="font-size:13px">factories.impl.pb.</span><span \
style="font-size:13px">RpcServerFactoryPBImpl.</span><span \
style="font-size:13px">getServer(</span><span \
style="font-size:13px">RpcServerFactoryPBImpl.java:</span><span \
style="font-size:13px">139) at org.apache.hadoop.yarn.ipc.</span><span \
style="font-size:13px">HadoopYarnProtoRPC.getServer(</span><span \
style="font-size:13px">HadoopYarnProtoRPC.java:65) at \
org.apache.hadoop.mapreduce.</span><span \
style="font-size:13px">v2.app.client.MRClientService.</span><span \
style="font-size:13px">serviceStart(MRClientService.</span><span \
style="font-size:13px">java:119) at org.apache.hadoop.service.</span><span \
style="font-size:13px">AbstractService.start(</span><span \
style="font-size:13px">AbstractService.java:193) at \
org.apache.hadoop.mapreduce.</span><span \
style="font-size:13px">v2.app.MRAppMaster.</span><span \
style="font-size:13px">serviceStart(MRAppMaster.java:</span><span \
style="font-size:13px">1084) at org.apache.hadoop.service.</span><span \
style="font-size:13px">AbstractService.start(</span><span \
style="font-size:13px">AbstractService.java:193) at \
org.apache.hadoop.mapreduce.</span><span \
style="font-size:13px">v2.app.MRAppMaster$4.run(</span><span \
style="font-size:13px">MRAppMaster.java:1500) at java.security.</span><span \
style="font-size:13px">AccessController.doPrivileged(</span><span \
style="font-size:13px">Native Method) at javax.security.auth.Subject.</span><span \
style="font-size:13px">doAs(Subject.java:415) at \
org.apache.hadoop.security.</span><span \
style="font-size:13px">UserGroupInformation.doAs(</span><span \
style="font-size:13px">UserGroupInformation.java:</span><span \
style="font-size:13px">1671) at org.apache.hadoop.mapreduce.</span><span \
style="font-size:13px">v2.app.MRAppMaster.</span><span \
style="font-size:13px">initAndStartAppMaster(</span><span \
style="font-size:13px">MRAppMaster.java:1496) at \
org.apache.hadoop.mapreduce.</span><span \
style="font-size:13px">v2.app.MRAppMaster.main(</span><span \
style="font-size:13px">MRAppMaster.java:1429) Caused by: java.net.BindException: \
Problem binding to [</span><a href="http://0.0.0.0:0/" rel="nofollow" \
style="font-size:13px;margin:0px;padding:0px;border:0px;text-decoration:none;color:rgb(102,17,204)" \
target="_blank">0.0.0.0:0</a><span style="font-size:13px">] java.net.BindException: \
Address already in use; For more details see: </span><a \
href="http://wiki.apache.org/hadoop/BindException" rel="nofollow" \
style="font-size:13px;margin:0px;padding:0px;border:0px;text-decoration:none;color:rgb(102,17,204)" \
target="_blank">http://wiki.apache.org/hadoop/BindException</a>
Does anyone know why this happens? Also a work around that does not involve an \
upgrade?</pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;border:0px"><br></pre><pre \
style="margin-top:0px;margin-bottom:0px;padding:0px;border:0px">TX</pre></div> \
</blockquote></div><br></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic