[prev in list] [next in list] [prev in thread] [next in thread] 

List:       flume-user
Subject:    Pass parameters to flume script
From:       IT CTO <goi.cto () gmail ! com>
Date:       2015-10-20 15:45:03
Message-ID: CAN-e8+Y-OyZgZgCDQNRSFFZoX1vB8B_oV2ZRK5xzBu0KyNxFDw () mail ! gmail ! com
[Download RAW message or body]

Hi,

I need to read 5 directories with different log data and store them in 5
different directories in hdfs.
I have flume script that uses spooldir and hdfs sink to do this for a
single directory.
For performance reasons I want to run this script using 5 agents each for a
separate direcory.
BUT I don;t want to duplication the script but rather pass a parameter from
the command line that will be used in the script to read from the right
spoolDir.

Does flume support passing command line parameters to script?
Any other solution?

Eran
-- 
Eran | "You don't need eyes to see, you need vision" (Faithless)

[Attachment #3 (text/html)]

<div dir="ltr">Hi,<div><br></div><div>I need to read 5 directories with different log \
data and store them in 5 different directories in hdfs.</div><div>I have flume script \
that uses spooldir and hdfs sink to do this for a single directory.</div><div>For \
performance reasons I want to run this script using 5 agents each for a separate \
direcory.</div><div>BUT I don;t want to duplication the script but rather pass a \
parameter from the command line that will be used in the script to read from the \
right spoolDir.</div><div><br></div><div>Does flume support passing command line \
parameters to script?</div><div>Any other solution?</div><div><br></div><div>Eran  \
</div></div><div dir="ltr">-- <br></div><div dir="ltr">Eran | &quot;You don&#39;t \
need eyes to see, you need vision&quot; (Faithless)</div>



[prev in list] [next in list] [prev in thread] [next in thread]