[prev in list] [next in list] [prev in thread] [next in thread] 

List:       flume-commits
Subject:    git commit: FLUME-2124. Upgrade Morphline Solr Sink to CDK 1.4.1
From:       hshreedharan () apache ! org
Date:       2013-07-11 1:00:37
Message-ID: 5368dfcb8b7e4adbab9b0535770c0df3 () git ! apache ! org
[Download RAW message or body]

Updated Branches:
  refs/heads/flume-1.5 2d506844a -> f8358f22a


FLUME-2124. Upgrade Morphline Solr Sink to CDK 1.4.1

(Wolfgang Hoschek via Hari Shreedharan)


Project: http://git-wip-us.apache.org/repos/asf/flume/repo
Commit: http://git-wip-us.apache.org/repos/asf/flume/commit/f8358f22
Tree: http://git-wip-us.apache.org/repos/asf/flume/tree/f8358f22
Diff: http://git-wip-us.apache.org/repos/asf/flume/diff/f8358f22

Branch: refs/heads/flume-1.5
Commit: f8358f22ad090bfe2f2be4799a517c38f56f34da
Parents: 2d50684
Author: Hari Shreedharan <hshreedharan@apache.org>
Authored: Wed Jul 10 17:58:54 2013 -0700
Committer: Hari Shreedharan <hshreedharan@apache.org>
Committed: Wed Jul 10 18:00:20 2013 -0700

----------------------------------------------------------------------
 flume-ng-doc/sphinx/FlumeUserGuide.rst              | 4 ++--
 flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml | 8 +++++++-
 2 files changed, 9 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/flume/blob/f8358f22/flume-ng-doc/sphinx/FlumeUserGuide.rst
                
----------------------------------------------------------------------
diff --git a/flume-ng-doc/sphinx/FlumeUserGuide.rst \
b/flume-ng-doc/sphinx/FlumeUserGuide.rst index 63cad21..fb42528 100644
--- a/flume-ng-doc/sphinx/FlumeUserGuide.rst
+++ b/flume-ng-doc/sphinx/FlumeUserGuide.rst
@@ -1835,7 +1835,7 @@ This sink extracts data from Flume events, transforms it, and \
loads it in near-r  
 This sink is well suited for use cases that stream raw data into HDFS (via the \
HdfsSink) and simultaneously extract, transform and load the same data into Solr (via \
MorphlineSolrSink). In particular, this sink can process arbitrary heterogeneous raw \
data from disparate data sources and turn it into a data model that is useful to \
Search applications.  
-The ETL functionality is customizable using a `morphline configuration file \
<http://cloudera.github.io/cdk/docs/0.4.0/cdk-morphlines/index.html>`_ that defines a \
chain of transformation commands that pipe event records from one command to another. \
 +The ETL functionality is customizable using a `morphline configuration file \
<http://cloudera.github.io/cdk/docs/0.4.1/cdk-morphlines/index.html>`_ that defines a \
chain of transformation commands that pipe event records from one command to another. \
  
 Morphlines can be seen as an evolution of Unix pipelines where the data model is \
generalized to work with streams of generic records, including arbitrary binary \
payloads. A morphline command is a bit like a Flume Interceptor. Morphlines can be \
embedded into Hadoop components such as Flume.  
@@ -2595,7 +2595,7 @@ prefix            ""       The prefix string constant to \
prepend to each generat  Morphline Interceptor
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-This interceptor filters the events through a `morphline configuration file \
<http://cloudera.github.io/cdk/docs/0.4.0/cdk-morphlines/index.html>`_ that defines a \
chain of transformation commands that pipe records from one command to another. +This \
interceptor filters the events through a `morphline configuration file \
<http://cloudera.github.io/cdk/docs/0.4.1/cdk-morphlines/index.html>`_ that defines a \
chain of transformation commands that pipe records from one command to another.  For \
example the morphline can ignore certain events or alter or insert certain event \
headers via regular expression based pattern matching, or it can auto-detect and set \
a MIME type via Apache Tika on events that are intercepted. For example, this kind of \
packet sniffing can be used for content based dynamic routing in a Flume topology.  \
MorphlineInterceptor can also help to implement dynamic routing to multiple Apache \
Solr collections (e.g. for multi-tenancy).  

http://git-wip-us.apache.org/repos/asf/flume/blob/f8358f22/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
                
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml \
b/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml index 5115090..5452aa9 100644
--- a/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
@@ -34,7 +34,7 @@ limitations under the License.
     <solr.version>4.3.0</solr.version>
     <solr.expected.version>4.3.0</solr.expected.version> <!-- sanity check to verify \
we actually run against the expected version rather than some outdated version -->  \
                <tika.version>1.3</tika.version>
-    <cdk.version>0.4.0</cdk.version>
+    <cdk.version>0.4.1</cdk.version>
     <slf4j.version>1.6.1</slf4j.version>
     <surefire.version>2.12.4</surefire.version>
   </properties>
@@ -102,6 +102,12 @@ limitations under the License.
 
     <dependency>
       <groupId>com.cloudera.cdk</groupId>
+      <artifactId>cdk-morphlines-json</artifactId>
+      <version>${cdk.version}</version>
+    </dependency>
+
+    <dependency>
+      <groupId>com.cloudera.cdk</groupId>
       <artifactId>cdk-morphlines-twitter</artifactId>
       <version>${cdk.version}</version>
     </dependency>


[prev in list] [next in list] [prev in thread] [next in thread]