[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Re: YARN Pi example job stuck at 0%(No MR tasks are started by ResourceManager)
From:       anil gupta <anilgupta84 () gmail ! com>
Date:       2012-07-31 18:26:42
Message-ID: CAF1+Vs-yKrrGwSKErztu9Eocmu-4Eb2unp=CkpxtG9NNnoiyMA () mail ! gmail ! com
[Download RAW message or body]


Hi Harsh and Others,

I was able to run the job when I login as user "hdfs". However, it fails if
i run it as "root". I was suspecting this as a problem before also and it
came out to be true.

Thanks,
Anil gupta

On Mon, Jul 30, 2012 at 9:21 PM, abhiTowson cal
<abhishek.dodda1@gmail.com>wrote:

> Hi anil,
> 
> Was trying several things.I didn't hadoop-env.sh, so i created it.
> 
> Regards
> Abhishek
> 
> On Mon, Jul 30, 2012 at 11:51 PM, anil gupta <anilgupta84@gmail.com>
> wrote:
> > That's pretty interesting!! From where you figured out that you need to
> add
> > that property? Just trying to understand how adding that property fixed
> the
> > issue.
> > 
> > On Mon, Jul 30, 2012 at 8:12 PM, abhiTowson cal
> > <abhishek.dodda1@gmail.com>wrote:
> > 
> > > Hi anil,
> > > 
> > > Adding property resolved issue for me, and i also made this change
> > > 
> > > vim hadoop-env.sh
> > > 
> > > export JAVA_HOME=/usr/lib/java-1.6.0/jdk1.6.0_33
> > > if [ "$JAVA_HOME" != "" ]; then
> > > #echo "run java in $JAVA_HOME"
> > > JAVA_HOME=$JAVA_HOME
> > > fi
> > > 
> > > if [ "$JAVA_HOME" = "" ]; then
> > > echo "Error: JAVA_HOME is not set."
> > > exit 1
> > > fi
> > > 
> > > JAVA=$JAVA_HOME/bin/java
> > > JAVA_HEAP_MAX=-Xmx1000m
> > > 
> > > Regards
> > > Abhishek
> > > 
> > > 
> > > On Mon, Jul 30, 2012 at 10:47 PM, anil gupta <anilgupta84@gmail.com>
> > > wrote:
> > > > Hi Abhishek,
> > > > 
> > > > Did you mean that adding yarn.resourcemanager.resource-tracker.address
> > > > along with yarn.log-aggregation-enable in my configuration will
> resolve
> > > the
> > > > problem in which map-reduce job fails at 0% with the following error:
> In
> > > > the web page of
> > > > 
> > > 
> http://data-node:8042/node/containerlogs/container_1343687008058_0003_01_000001/rootthe
> 
> > > > page says:
> > > > Failed redirect for container_1343687008058_0003_01_000001  Failed
> while
> > > > trying to construct the redirect url to the log server. Log Server url
> > > may
> > > > not be configured. Unknown container. Container either has not
> started or
> > > > has already completed or doesn't belong to this node at all.
> > > > Please let me know.
> > > > 
> > > > Thanks,
> > > > Anil Gupta
> > > > 
> > > > On Mon, Jul 30, 2012 at 7:30 PM, abhiTowson cal
> > > > <abhishek.dodda1@gmail.com>wrote:
> > > > 
> > > > > hi anil,
> > > > > 
> > > > > Adding these help me resolve the issue for me
> > > > > yarn.resourcemanager.resource-tracker.address
> > > > > 
> > > > > Regards
> > > > > Abhishek
> > > > > 
> > > > > On Mon, Jul 30, 2012 at 7:56 PM, anil gupta <anilgupta84@gmail.com>
> > > wrote:
> > > > > > Hi Rahul,
> > > > > > 
> > > > > > Thanks for your response. I can certainly enable the
> > > > > > yarn.log-aggregation-enable to true. But after enabling this what
> > > manual
> > > > > > steps i will have to take to run jobs. Could you please elaborate.
> > > > > > 
> > > > > > Thanks,
> > > > > > Anil
> > > > > > 
> > > > > > On Mon, Jul 30, 2012 at 4:26 PM, Rahul Jain <rjain7@gmail.com>
> wrote:
> > > > > > 
> > > > > > > The inability to look at map-reduce logs for failed logs is due to
> > > > > number
> > > > > > > of open issues in yarn; see my recent comment here:
> > > > > > > 
> > > > > > > 
> > > > > 
> > > 
> https://issues.apache.org/jira/browse/MAPREDUCE-4428?focusedCommentId=13412995&page= \
> com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13412995
> > > > > > > 
> > > > > > > You can workaround this by enabling log aggregation and manually
> > > copying
> > > > > > > job logs from HDFS log location. Of course that is a painful way
> till
> > > > > the
> > > > > > > yarn log collection and history bugs are resolved in an upcoming
> > > > > release.
> > > > > > > 
> > > > > > > -Rahul
> > > > > > > 
> > > > > > > 
> > > > > > > > 12/07/27 09:38:27 INFO mapred.ResourceMgrDelegate: Submitted
> > > > > application
> > > > > > > > application_1343365114818_0002 to ResourceManager at ihub-an-l1/
> > > > > > > > 172.31.192.151:8040
> > > > > > > > 12/07/27 09:38:27 INFO mapreduce.Job: The url to track the job:
> > > > > > > > http://ihub-an-l1:9999/proxy/application_1343365114818_0002/
> > > > > > > > 12/07/27 09:38:27 INFO mapreduce.Job: Running job:
> > > > > job_1343365114818_0002
> > > > > > > > 
> > > > > > > > No Map-Reduce task are started by the cluster. I dont see any
> > > errors
> > > > > > > > anywhere in the application. Please help me in resolving this
> > > problem.
> > > > > > > > 
> > > > > > > > Thanks,
> > > > > > > > Anil Gupta
> > > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > --
> > > > > > Thanks & Regards,
> > > > > > Anil Gupta
> > > > > 
> > > > 
> > > > 
> > > > 
> > > > --
> > > > Thanks & Regards,
> > > > Anil Gupta
> > > 
> > 
> > 
> > 
> > --
> > Thanks & Regards,
> > Anil Gupta
> 



-- 
Thanks & Regards,
Anil Gupta



[prev in list] [next in list] [prev in thread] [next in thread]