[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Re: Scheduler Algorithm in Hadoop
From:       Arun C Murthy <acm () hortonworks ! com>
Date:       2011-09-27 23:26:05
Message-ID: 505DD28D-39E6-4F15-A856-D87A6DF70CC4 () hortonworks ! com
[Download RAW message or body]

Amal,
 
 Welcome to Hadoop!

 Currently we have two, very different, versions of MapReduce: MRv1 and MRv2.
 
 Most of the active developers on MapReduce are working on MRv2. If you want to take \
a look please see the sources under trunk/branch-0.23:  \
hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler


 For MRv1 you need to look at schedulers in branch-0.20-security:
 src/contrib/capacity-scheduler
 src/contrib/fairscheduler

Arun

On Sep 27, 2011, at 4:12 PM, Amal De Silva wrote:

> Hi
> 
> I am a scheduling/optimization algorithm expert with many years of
> experience (in applying scheduling to manufacturing and transportation
> industry). I want to see if I can improve the scheduling algorithms in
> Hadoop like the Fair Scheduler. However, I am struggling to get a
> basic hadoop version running so that I can write a new Scheduler
> class.
> 
> Can someone give me some pointers on how I need to go about doing
> this. All I want to do is write a new Scheduler class and Test it.  I
> would like to get to a stage that I can do this with the least effort.
> 
> Thanks
> Amal de Silva


[prev in list] [next in list] [prev in thread] [next in thread]