[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Collecting text in output
From:       Anurup Pavuluri <apavulur () andrew ! cmu ! edu>
Date:       2006-12-08 0:57:43
Message-ID: 4578B887.4010700 () andrew ! cmu ! edu
[Download RAW message or body]


Hi,

In my Map function I am collecting Text as the output. When I run the 
job the job gives a java.io.IOException. The following code is exactly 
what I am doing. Did someone else have a similar problem with Text? It 
works for IntWritable instead of Text.

    private Text word = new Text();

    public void map(WritableComparable key, Writable value,
        OutputCollector output,
        Reporter reporter) throws IOException {
	
        String line = ((Text)value).toString();
        StringTokenizer itr = new StringTokenizer(line);
        for(int i=0; i<3; i++)
        {
                itr.nextToken();
        }
        String ipAddress = itr.nextToken();
        ipAddress = ipAddress.trim();
        word.set(ipAddress);
        output.collect(word, word);
    }
  }

  public static class Reduce extends MapReduceBase implements Reducer {

    public void reduce(WritableComparable key, Iterator values,
        OutputCollector output,
        Reporter reporter) throws IOException {
        int sum = 0;
        Text word = new Text();
      while (values.hasNext()) {
        sum += ((IntWritable) values.next()).get();
      }
        word.set("word");
        output.collect(key, word);
    }
  }



-- Anurup Pavuluri Graduate Student Information Networking Institute, 
Carnegie Mellon University. E-Mail: apavulur@andrew.cmu.edu Phone: 412 
726 7193(M)



[prev in list] [next in list] [prev in thread] [next in thread]