[prev in list] [next in list] [prev in thread] [next in thread]
List: hadoop-user
Subject: Re: Job object toString() is throwing an exception
From: Corey Nolet <cjnolet () gmail ! com>
Date: 2014-11-26 3:55:21
Message-ID: CAOHP_tHkVE8qyW8ax7ueQu68f6hzVtWJjZQ-wcG-Z8o80Lgx2w () mail ! gmail ! com
[Download RAW message or body]
Here's the stack trace. I was going to file a ticket for this but wanted to
check on the user list first to make sure there wasn't already a fix in the
works. It has to do with the Scala shell doing a toString() each time a
command is typed in. The stack trace stops the instance of Job from ever
being assigned.
scala> val job = new org.apache.hadoop.mapreduce.Job
warning: there were 1 deprecation warning(s); re-run with -deprecation for
details
java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
at org.apache.hadoop.mapreduce.Job.ensureState(Job.java:283)
at org.apache.hadoop.mapreduce.Job.toString(Job.java:452)
at
scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:324)
at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:329)
at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:337)
at .<init>(<console>:10)
at .<clinit>(<console>)
at $print(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:789)
at
org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1062)
at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:615)
at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:646)
at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:610)
at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:814)
at
org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:859)
at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:771)
at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:616)
at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:624)
at org.apache.spark.repl.SparkILoop.loop(SparkILoop.scala:629)
at
org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:954)
at
org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:902)
at
org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:902)
at
scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:902)
at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:997)
at org.apache.spark.repl.Main$.main(Main.scala:31)
at org.apache.spark.repl.Main.main(Main.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
On Tue, Nov 25, 2014 at 9:39 PM, Rohith Sharma K S <
rohithsharmaks@huawei.com> wrote:
> Could you give error message or stack trace?
>
>
>
> *From:* Corey Nolet [mailto:cjnolet@gmail.com]
> *Sent:* 26 November 2014 07:54
> *To:* user@hadoop.apache.org
> *Subject:* Job object toString() is throwing an exception
>
>
>
> I was playing around in the Spark shell and newing up an instance of Job
> that I could use to configure the inputformat for a job. By default, the
> Scala shell println's the result of every command typed. It throws an
> exception when it printlns the newly created instance of Job because it
> looks like it's setting a state upon allocation and it's not happy with the
> state that it's in when toString() is called before the job is submitted.
>
>
>
> I'm using Hadoop 2.5.1. I don't see any tickets for this for 2.6. Has
> anyone else ran into this?
>
[Attachment #3 (text/html)]
<div dir="ltr">
<p class="">Here's the stack trace. I was going to file a ticket for this but \
wanted to check on the user list first to make sure there wasn't already a fix in \
the works. It has to do with the Scala shell doing a toString() each time a command \
is typed in. The stack trace stops the instance of Job from ever being \
assigned.</p><p class=""><br></p><p class="">scala> val job = new \
org.apache.hadoop.mapreduce.Job</p> <p class="">warning: there were 1 deprecation \
warning(s); re-run with -deprecation for details</p> <p \
class="">java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING</p> \
<p class=""><span class=""> </span>at \
org.apache.hadoop.mapreduce.Job.ensureState(Job.java:283)</p> <p class=""><span \
class=""> </span>at org.apache.hadoop.mapreduce.Job.toString(Job.java:452)</p> <p \
class=""><span class=""> </span>at \
scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:324)</p>
<p class=""><span class=""> </span>at \
scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:329)</p> <p class=""><span \
class=""> </span>at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:337)</p>
<p class=""><span class=""> </span>at .<init>(<console>:10)</p>
<p class=""><span class=""> </span>at .<clinit>(<console>)</p>
<p class=""><span class=""> </span>at $print(<console>)</p>
<p class=""><span class=""> </span>at \
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)</p> <p class=""><span \
class=""> </span>at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)</p>
<p class=""><span class=""> </span>at \
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</p>
<p class=""><span class=""> </span>at \
java.lang.reflect.Method.invoke(Method.java:606)</p> <p class=""><span \
class=""> </span>at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:789)</p>
<p class=""><span class=""> </span>at \
org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1062)</p> <p \
class=""><span class=""> </span>at \
org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:615)</p> <p \
class=""><span class=""> </span>at \
org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:646)</p> <p \
class=""><span class=""> </span>at \
org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:610)</p> <p \
class=""><span class=""> </span>at \
org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:814)</p> <p \
class=""><span class=""> </span>at \
org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:859)</p> <p \
class=""><span class=""> </span>at \
org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:771)</p> <p class=""><span \
class=""> </span>at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:616)</p>
<p class=""><span class=""> </span>at \
org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:624)</p> <p \
class=""><span class=""> </span>at \
org.apache.spark.repl.SparkILoop.loop(SparkILoop.scala:629)</p> <p class=""><span \
class=""> </span>at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:954)</p>
<p class=""><span class=""> </span>at \
org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:902)</p> \
<p class=""><span class=""> </span>at \
org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:902)</p> \
<p class=""><span class=""> </span>at \
scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)</p>
<p class=""><span class=""> </span>at \
org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:902)</p> <p class=""><span \
class=""> </span>at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:997)</p>
<p class=""><span class=""> </span>at \
org.apache.spark.repl.Main$.main(Main.scala:31)</p> <p class=""><span \
class=""> </span>at org.apache.spark.repl.Main.main(Main.scala)</p> <p class=""><span \
class=""> </span>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)</p> \
<p class=""><span class=""> </span>at \
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)</p> <p \
class=""><span class=""> </span>at \
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)</p>
<p class=""><span class=""> </span>at \
java.lang.reflect.Method.invoke(Method.java:606)</p> <p class=""><span \
class=""> </span>at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328)</p>
<p class=""><span class=""> </span>at \
org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)</p> <p class=""><span \
class=""> </span>at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)</p> \
<p class=""><br></p></div><div class="gmail_extra"><br><div class="gmail_quote">On \
Tue, Nov 25, 2014 at 9:39 PM, Rohith Sharma K S <span dir="ltr"><<a \
href="mailto:rohithsharmaks@huawei.com" \
target="_blank">rohithsharmaks@huawei.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Could \
you give error message or stack trace?<u></u><u></u></span></p> <p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <div style="border:none;border-top:solid #b5c4df \
1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal"><b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Corey \
Nolet [mailto:<a href="mailto:cjnolet@gmail.com" \
target="_blank">cjnolet@gmail.com</a>] <br>
<b>Sent:</b> 26 November 2014 07:54<br>
<b>To:</b> <a href="mailto:user@hadoop.apache.org" \
target="_blank">user@hadoop.apache.org</a><br> <b>Subject:</b> Job object toString() \
is throwing an exception<u></u><u></u></span></p> </div><span class="">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">I was playing around in the Spark shell and newing up an \
instance of Job that I could use to configure the inputformat for a job. By default, \
the Scala shell println's the result of every command typed. It throws an \
exception when it printlns the newly created instance of Job because it looks like \
it's setting a state upon allocation and it's not happy with the state that \
it's in when toString() is called before the job is submitted.<u></u><u></u></p> \
<div> <p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I'm using Hadoop 2.5.1. I don't see any tickets for this \
for 2.6. Has anyone else ran into this? <u></u><u></u></p> </div>
</div>
</span></div>
</div>
</blockquote></div><br></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic