[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Re: ReflectionUtils.setConf would configure anything Configurable twice?
From:       steven zhuang <zhuangxin8448 () gmail ! com>
Date:       2011-06-29 7:05:06
Message-ID: BANLkTimEfe8CFf8NtUhbc=5Q5Nezf4aP_w () mail ! gmail ! com
[Download RAW message or body]


Anyone have the same partition problem using Streaming in 0.21.0?



On Fri, Jun 24, 2011 at 10:22 AM, steven zhuang <zhuangxin8448@gmail.com>wrote:

> hello, list,
>
>               Recently I have upgraded our Hadoop cluster from 0.20.2 to
> 0.21.0, and I found there is something wrong introduced by the upgrade.
>
>               In the setConf method of
> org.apache.hadoop.util.ReflectionUtils, any instance of Configurable would
> be configured twice, and this may cause trouble.
>               For example, in 0.21.0, KeyFieldBasedPartitioner implements
> the Configurable interface. When configured twice, it get two KeyDescription
> and gives out wrong partition number.
>
> I have created a ticket for this:
> https://issues.apache.org/jira/browse/HADOOP-7425
>
>
> Paste the source code in 0.21.0 (and 0.20.2 too) below:
>
> public static void setConf(Object theObject, Configuration conf) {
>     if (conf != null) {
>        if (theObject instanceof Configurable) {
>            ((Configurable) theObject).setConf(conf);
>        }
>       setJobConf(theObject, conf);
>    }
> }
>
>
>
> --
>         best wishes.
>                              steven
>
>


-- 
        best wishes.
                             steven


[prev in list] [next in list] [prev in thread] [next in thread]