[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Re: Running hadoop dfs cluster on a single machine
From:       Dali Kilani <dali.kilani () gmail ! com>
Date:       2009-07-31 3:45:58
Message-ID: 77d4e5730907302045y4a88682cgcc4d955102bd8b99 () mail ! gmail ! com
[Download RAW message or body]


Ryan,
any reason, why you wouldn't simply have a RAID array with the drives : it
will be faster, less wasteful, and easier to manage ...

Dali

On Thu, Jul 30, 2009 at 6:54 PM, Amandeep Khurana <amansk@gmail.com> wrote:

> Ryan,
>
> You can surely do this by having 4 virtual machines (I havent tried that
> myself though). However, make sure you understand the principles behind
> hadoop. The fundamental reason to go towards hadoop is to distribute the
> load across multiple machines. Everything that hadoop provides is with that
> basic idea in mind. Replication basically provides tolerance towards
> failing
> hard drives. With 2 hard drives, a replication factor of >2 wont add value.
> Secondly, HDFS is slower than the filesystem you would run it on... So,
> throughputs would be low.
>
> So, make sure you understand the design philosophy and then move forward on
> it.
>
> -ak
>
>
>
> On Thu, Jul 30, 2009 at 6:15 PM, Ryan Smith <ryan.justin.smith@gmail.com
> >wrote:
>
> > Sorry if this has been asked already, I didnt find anything on google.
> >
> > I want to run a dfs cluster on one machine.  8 core processor, 8 GB RAM,
> 3
> > x
> > 2TB drives.  I want triple replication on the dfs.
> > If i make 4 IP aliases on the machine, (10.1.2.30,10.1.2.31, 10.1.2.32,
> > 10.1.2.33) can i run 1 master (NN & SNN) and 3 slaves (3 DN)?
> > Or what is the best approach?  I want to make a private hadoop storage
> for
> > myself, but i dont want to buy a bunch of servers.
> > Thanks, any advice is appreciated.
> > -Ryan
> >
>



-- 
Dali Kilani
===========
Phone :  (650) 492-5921 (Google Voice)
E-Fax  :  (775) 552-2982


[prev in list] [next in list] [prev in thread] [next in thread]