[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Re: how to process small fraction of input?
From:       David_ca <davidsuperca () gmail ! com>
Date:       2009-07-31 16:53:35
Message-ID: 4f0b93d60907310953y7cdc80c7r7a2e27c58a91e18b () mail ! gmail ! com
[Download RAW message or body]


I was thinking of  creating a custom RecordReader.
The RecordReader would keep track of the number of records it has
processed and when it is over the limit,
make the RecordReader.next method  return false, and so signal
a premature end of file.

On Fri, Jul 31, 2009 at 7:32 AM, David_ca <davidsuperca@gmail.com> wrote:

> Hi,
>
> For input of a large size, I would like to run my program using the whole
> input
> but only process a fraction of the input. The idea is to see if everything
> is working
> correctly but don't take too long.
>
> Is the InputSampler the way to go? If it is can someone give me more info
> how to use it.
>
>
> thanks,
> David
>


[prev in list] [next in list] [prev in thread] [next in thread]