[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Problem with field separator in FieldSelectionHelper
From:       é <kai.wei.cn () gmail ! com>
Date:       2013-03-31 16:57:37
Message-ID: CAByXZ9XyTvBX1TqtyBLcO8CirXmnOjHfb3oxGDF4uwpUZ8i0Rg () mail ! gmail ! com
[Download RAW message or body]

I found that org.apache.hadoop.mapreduce.lib.fieldsel.FieldSelectionHelper
and the corresponding old
api org.apache.hadoop.mapred.lib.FieldSelectionMapReduce take user
specified separator string as a regular expression in String.split(), but
also use it as a normal string in StringBuffer.append(). It will be a
problem if the separator string contains meta character. I suggest take
separator literally by calling Pattern.quote(separator). Or just use
another property to specify the separator which should be added in the
output.

If this is not a known problem, I will file a jira issue and submit the fix.

[Attachment #3 (text/html)]

<div dir="ltr">I found that  \
org.apache.hadoop.mapreduce.lib.fieldsel.FieldSelectionHelper and the  corresponding  \
old api  org.apache.hadoop.mapred.lib.FieldSelectionMapReduce take user specified \
separator string as a regular expression in String.split(), but also use it as a \
normal string in StringBuffer.append(). It will be a problem if the separator string \
contains meta character. I suggest take separator literally by calling \
Pattern.quote(separator). Or just use another property to specify the separator which \
should be added in the output.<div> <br></div><div style>If this is not a known \
problem, I will file a jira issue and submit the fix.</div></div>



[prev in list] [next in list] [prev in thread] [next in thread]