[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Re: Running multiple copies of each task
From:       Namikaze Minato <lloydsensei () gmail ! com>
Date:       2015-12-18 11:34:09
Message-ID: CACmJb3yQVSdVurd0=pmMJnaoSAKa_mA1c-xYeUjta4P-dMQEjw () mail ! gmail ! com
[Download RAW message or body]

Talking about priority without telling us which scheduler and which
scheduling method you are using is like forgetting to say in which unit you
are measuring somehting :)

Regards,
LLoyd

On 17 December 2015 at 20:09, Sultan Alamro <sultan.alamro@gmail.com> wrote:

> Thanks Namikaze!
>
> Another question:
>
> New tasks in hadoop always have higher priority than speculative tasks.
> Does anyone know how and where I can change this priority?
>
>
> Thanks,
> Sultan
>
>
> On Thu, Dec 3, 2015 at 7:46 AM, Namikaze Minato <lloydsensei@gmail.com>
> wrote:
>
>> I think you are looking for mapreduce.reduce.speculative
>> Be careful, for some reason, this fell into my spam folder.
>>
>> Regards,
>> LLoyd
>>
>> On 3 December 2015 at 01:05, Sultan Alamro <sultan.alamro@gmail.com>
>> wrote:
>> > Hi there,
>> >
>> > I have been looking at the hadoop source code 2.6.0 trying to
>> understand the
>> > low level details and how the framework is actually working.
>> >
>> > I have a simple idea and I am trying to figure out where and how the
>> idea
>> > can be implemented. The idea can be described in one sentence: "Running
>> > multiple copies of each task". However, implementing the idea is not as
>> > simple as I think.
>> >
>> > What I am aware of is that I only need to modify a few classes. But,
>> which
>> > classes?
>> >
>> > I just need someone to guide me to the right direction.
>> >
>> >
>> > Best,
>> > Sultan
>>
>
>

[Attachment #3 (text/html)]

<div dir="ltr">Talking about priority without telling us which scheduler and which \
scheduling method you are using is like forgetting to say in which unit you are \
measuring somehting :)<div><br></div><div>Regards,</div><div>LLoyd</div></div><div \
class="gmail_extra"><br><div class="gmail_quote">On 17 December 2015 at 20:09, Sultan \
Alamro <span dir="ltr">&lt;<a href="mailto:sultan.alamro@gmail.com" \
target="_blank">sultan.alamro@gmail.com</a>&gt;</span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div><div><div>Thanks \
Namikaze!<br><br></div>Another question:<br><br>New tasks in hadoop always have \
higher priority than speculative tasks. <br>Does anyone know how and where I can \
change this priority? <br><br><br></div>Thanks, \
<br></div>Sultan<br><div><div><br></div></div></div><div class="HOEnZb"><div \
class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Dec 3, 2015 \
at 7:46 AM, Namikaze Minato <span dir="ltr">&lt;<a \
href="mailto:lloydsensei@gmail.com" \
target="_blank">lloydsensei@gmail.com</a>&gt;</span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex">I think you are looking for mapreduce.reduce.speculative<br> \
Be careful, for some reason, this fell into my spam folder.<br> <br>
Regards,<br>
LLoyd<br>
<div><div><br>
On 3 December 2015 at 01:05, Sultan Alamro &lt;<a \
href="mailto:sultan.alamro@gmail.com" target="_blank">sultan.alamro@gmail.com</a>&gt; \
wrote:<br> &gt; Hi there,<br>
&gt;<br>
&gt; I have been looking at the hadoop source code 2.6.0 trying to understand the<br>
&gt; low level details and how the framework is actually working.<br>
&gt;<br>
&gt; I have a simple idea and I am trying to figure out where and how the idea<br>
&gt; can be implemented. The idea can be described in one sentence: &quot;Running<br>
&gt; multiple copies of each task&quot;. However, implementing the idea is not as<br>
&gt; simple as I think.<br>
&gt;<br>
&gt; What I am aware of is that I only need to modify a few classes. But, which<br>
&gt; classes?<br>
&gt;<br>
&gt; I just need someone to guide me to the right direction.<br>
&gt;<br>
&gt;<br>
&gt; Best,<br>
&gt; Sultan<br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>



[prev in list] [next in list] [prev in thread] [next in thread]