[prev in list] [next in list] [prev in thread] [next in thread] 

List:       flume-user
Subject:    Re: Multiple HDFS HA sinks
From:       "Hari Shreedharan" <hshreedharan () cloudera ! com>
Date:       2015-03-22 21:46:44
Message-ID: 1427060803926.4d680e13 () Nodemailer
[Download RAW message or body]

You'd specify the path as: hdfs://<nameservice>/path/to/write/to (Any substitutions work in the path)




Thanks,  Hari

On Sun, Mar 22, 2015 at 1:23 PM, Balkrishna Sharma
<balkrishna.sharma@gmail.com> wrote:

> Hi
> My flume agent writes to couple of HDFS sinks, each of which is in HA mode.
> How do I use the logical name for each of the cluster. i.e. how do I use
> dfs.nameservices for specifying path for each sink ?
> Flume seems to pick the first hdfs-site.xml it finds in its path and that
> has only one dfs.nameservices in it. The other other hdfs-site.xml in the
> other following config directory in the flumepath is ignored.
> Anyway I can club both the nameservices in the same hdfs-site.xml ?
> Thanks.
[Attachment #3 (text/html)]

<span id="mailbox-conversation"><div>You'd specify the path as: \
hdfs://&lt;nameservice&gt;/path/to/write/to (Any substitutions work in the \
path)</div></span><div class="mailbox_signature"> <br>

Thanks,  <div>Hari</div>
</div>
<br><br><div class="gmail_quote"><p>On Sun, Mar 22, 2015 at 1:23 PM, Balkrishna \
Sharma <span dir="ltr">&lt;<a href="mailto:balkrishna.sharma@gmail.com" \
target="_blank">balkrishna.sharma@gmail.com</a>&gt;</span> wrote:<br></p><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex;"><div><div dir="ltr">Hi  <div>My flume agent writes to couple \
of HDFS sinks, each of which is in HA mode.</div> <div><br></div>
<div>How do I use the logical name for each of the cluster. i.e. how do I use \
dfs.nameservices for specifying path for each sink ?</div> <div>Flume seems to pick \
the first hdfs-site.xml it finds in its path and that has only one dfs.nameservices \
in it. The other other hdfs-site.xml in the other following config directory in the \
flumepath is ignored.</div> <div>Anyway I can club both the nameservices in the same \
hdfs-site.xml ?</div> <div><br></div>
<div>Thanks.</div>
</div></div></blockquote></div><br>



[prev in list] [next in list] [prev in thread] [next in thread]