[prev in list] [next in list] [prev in thread] [next in thread]
List: hadoop-user
Subject: Re: HDFS Shell tool
From: Ravi Prakash <ravihadoop () gmail ! com>
Date: 2017-02-10 15:58:06
Message-ID: CAMs9kVj0TOkCYZ69Lpn15chyMpoKJLnbDTzbeyLmk7g-w1Bn2Q () mail ! gmail ! com
[Download RAW message or body]
Hi Vity!
Please let me reiterate that I think its great work and I'm glad you
thought of sharing it with the community. Thanks a lot.
I can think of a few reasons for using WebHDFS, although, if these are not
important to you, it may not be worth the effort:
1. You can point to an HttpFS gateway in case you do not have network
access to the datanodes.
2. WebHDFS is a lot more likely to be compatible with different versions of
Hadoop (https://github.com/avast/hdfs-shell/blob/master/build.gradle#L80)
Although, the community is trying really hard to maintain compatibility
going forward for FileSystem too.
3. You may be able to eliminate linking a lot of jars that hadoop-client
would pull in.
Having said that there may well be reasons why you don't want to use
WebHDFS.
Thanks again!
Ravi
On Fri, Feb 10, 2017 at 12:38 AM, Vit=C3=A1sek, Ladislav <vitasek@avast.com=
>
wrote:
> Hello Ravi,
> I am glad you like it.
> Why should I use WebHDFS? Our cluster sysops, include me, prefer command
> line. :-)
>
> -Vity
>
> 2017-02-09 22:21 GMT+01:00 Ravi Prakash <ravihadoop@gmail.com>:
>
>> Great job Vity!
>>
>> Thanks a lot for sharing. Have you thought about using WebHDFS?
>>
>> Thanks
>> Ravi
>>
>> On Thu, Feb 9, 2017 at 7:12 AM, Vit=C3=A1sek, Ladislav <vitasek@avast.co=
m>
>> wrote:
>>
>>> Hello Hadoop fans,
>>> I would like to inform you about our tool we want to share.
>>>
>>> We created a new utility - HDFS Shell to work with HDFS more faster.
>>>
>>> https://github.com/avast/hdfs-shell
>>>
>>> *Feature highlights*
>>> - HDFS DFS command initiates JVM for each command call, HDFS Shell does
>>> it only once - which means great speed enhancement when you need to wor=
k
>>> with HDFS more often
>>> - Commands can be used in a short way - eg. *hdfs dfs -ls /*, *ls /* -
>>> both will work
>>> - *HDFS path completion using TAB key*
>>> - you can easily add any other HDFS manipulation function
>>> - there is a command history persisting in history log
>>> (~/.hdfs-shell/hdfs-shell.log)
>>> - support for relative directory + commands *cd* and *pwd*
>>> - it can be also launched as a daemon (using UNIX domain sockets)
>>> - 100% Java, it's open source
>>>
>>> You suggestions are welcome.
>>>
>>> -L. Vitasek aka Vity
>>>
>>>
>>
>
[Attachment #3 (text/html)]
<div dir="ltr"><div>Hi Vity!<br><br></div><div>Please let me reiterate that I think \
its great work and I'm glad you thought of sharing it with the community. Thanks \
a lot.<br><br>I can think of a few reasons for using WebHDFS, although, if these are \
not important to you, it may not be worth the effort:<br></div><div>1. You can point \
to an HttpFS gateway in case you do not have network access to the datanodes.<br>2. \
WebHDFS is a lot more likely to be compatible with different versions of Hadoop (<a \
href="https://github.com/avast/hdfs-shell/blob/master/build.gradle#L80">https://github.com/avast/hdfs-shell/blob/master/build.gradle#L80</a>) \
Although, the community is trying really hard to maintain compatibility going forward \
for FileSystem too.<br></div><div>3. You may be able to eliminate linking a lot of \
jars that hadoop-client would pull in.<br><br></div><div>Having said that there may \
well be reasons why you don't want to use WebHDFS.<br><br></div><div>Thanks \
again!<br></div><div>Ravi<br></div><div><br></div></div><div \
class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 10, 2017 at 12:38 AM, \
Vitásek, Ladislav <span dir="ltr"><<a href="mailto:vitasek@avast.com" \
target="_blank">vitasek@avast.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hello Ravi,<br></div>I am glad \
you like it. <br></div>Why should I use WebHDFS? Our cluster sysops, include me, \
prefer command line. :-)<br><br></div>-Vity<br></div><div class="HOEnZb"><div \
class="h5"><div class="gmail_extra"><br><div class="gmail_quote">2017-02-09 22:21 \
GMT+01:00 Ravi Prakash <span dir="ltr"><<a href="mailto:ravihadoop@gmail.com" \
target="_blank">ravihadoop@gmail.com</a>></span>:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div><div><div>Great job \
Vity!<br><br></div>Thanks a lot for sharing. Have you thought about using \
WebHDFS?<br><br></div>Thanks<span class="m_-8644041981422321691HOEnZb"><font \
color="#888888"><br></font></span></div><span \
class="m_-8644041981422321691HOEnZb"><font \
color="#888888">Ravi<br></font></span></div><div \
class="m_-8644041981422321691HOEnZb"><div class="m_-8644041981422321691h5"><div \
class="gmail_extra"><br><div class="gmail_quote">On Thu, Feb 9, 2017 at 7:12 AM, \
Vitásek, Ladislav <span dir="ltr"><<a href="mailto:vitasek@avast.com" \
target="_blank">vitasek@avast.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hello Hadoop fans,<br>I would \
like to inform you about our tool we want to share. <br><br>We created a new utility \
- HDFS Shell to work with HDFS more faster.<br><br><a \
href="https://github.com/avast/hdfs-shell" \
target="_blank">https://github.com/avast/hdfs-<wbr>shell</a><br><br><b>Feature \
highlights</b><br>- HDFS DFS command initiates JVM for each command call, HDFS Shell \
does it only once - which means great speed enhancement when you need to work with \
HDFS more often<br>- Commands can be used in a short way - eg. <b>hdfs dfs -ls /</b>, \
<b>ls /</b> - both will work<br>- <b>HDFS path completion using TAB key</b><br>- you \
can easily add any other HDFS manipulation function<br>- there is a command history \
persisting in history log (~/.hdfs-shell/hdfs-shell.log)<br>- support for relative \
directory + commands <b>cd</b> and <b>pwd</b><br></div><div>- it can be also launched \
as a daemon (using UNIX domain sockets)<br></div>- 100% Java, it's open \
source<br><br></div>You suggestions are welcome.<br><br></div>-L. Vitasek aka \
Vity<br><div><div><br></div></div></div> </blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic