[prev in list] [next in list] [prev in thread] [next in thread]
List: hadoop-user
Subject: Re: HDFS Federation with HA
From: Rams Venkatesh <bigcherish () gmail ! com>
Date: 2018-10-10 14:18:08
Message-ID: 0C2FF767-08D2-4756-80B5-201531CDE619 () gmail ! com
[Download RAW message or body]
Please email to user-unsubscribe@hadoop.apache.org to get yourself removed from the \
distribution list
> On 10 Oct 2018, at 3:14 pm, Pankaj Kumar <pankaj.kumar@idctechnologies.com> wrote:
>
> Remove Me.
>
> ---
> Thanks & Regards
> Pankaj Kumar
> IDC Technologies Inc.
> Work: 408-457-9381- Ext-4102
> Mailto: pankaj.kumar@idctechnologies.com
> ---------------------------------------------------------
> Empowering Technologies Services
> Remote Services | IT Services | BPO |
> IT Consulting | Staffing Solutions |
> ---------------------------------------------------------
>
> From: rahul patwari [mailto:rahulpatwari8383@gmail.com]
> Sent: Wednesday, October 10, 2018 3:56 AM
> To: tasanuma@yahoo-corp.jp
> Cc: user@hadoop.apache.org
> Subject: Re: HDFS Federation with HA
>
> Hi Takanobu,
>
> Thanks for the reply. So, does that mean that 3 dedicated journal nodes are \
> required for each nameservice? or can we have the same journal nodes for all the \
> nameservices?
> Thanks,
> Rahul
>
> On Wed, Oct 10, 2018 at 1:00 PM Takanobu Asanuma <tasanuma@yahoo-corp.jp> wrote:
> Hi Rahul,
>
> Yes, it is possible. Here are the sample configurations.
>
> ===========================================
> dfs.nameservices=ns1,ns2
> dfs.ha.namenodes.ns1=nn1,nn2
> dfs.ha.namenodes.ns2=nn1,nn2
>
> dfs.namenode.rpc-address.ns1.nn1=ns1nn1:8020
> dfs.namenode.rpc-address.ns1.nn2=ns1nn2:8020
> dfs.namenode.http-address.ns1.nn1=ns1nn1:9870
> dfs.namenode.http-address.ns1.nn2=ns1nn2:9870
>
> dfs.namenode.rpc-address.ns2.nn1=ns2nn1:8020
> dfs.namenode.rpc-address.ns2.nn2=ns2nn2:8020
> dfs.namenode.http-address.ns2.nn1=ns2nn1:9870
> dfs.namenode.http-address.ns2.nn2=ns2nn2:9870
>
> dfs.client.failover.proxy.provider.ns1=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
> dfs.client.failover.proxy.provider.ns2=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
>
> dfs.namenode.shared.edits.dir.ns1=qjournal://ns1jn1:8485;ns1jn2:8485;ns1jn3:8485/ns1
> dfs.namenode.shared.edits.dir.ns2=qjournal://ns2jn1:8485;ns2jn2:8485;ns2jn3:8485/ns2
> ===========================================
>
> Regards,
> - Takanobu
>
> on 2018/10/10 3:07, "rahul patwari"<rahulpatwari8383@gmail.com> wrote:
>
> Hi,
>
>
> Is it possible in the current architecture of HDFS to have multiple \
> nameservices(federation), with Active and Backup Namenode(HA) for each of the \
> nameservices?
>
> Thanks,
> Rahul
>
>
[Attachment #3 (text/html)]
<html><head><meta http-equiv="content-type" content="text/html; \
charset=utf-8"></head><body dir="auto"><span style="background-color: rgba(255, 255, \
255, 0);">Please email to <a href="mailto:user-unsubscribe@hadoop.apache.org" \
dir="ltr" x-apple-data-detectors="true" x-apple-data-detectors-type="link" \
x-apple-data-detectors-result="0">user-unsubscribe@hadoop.apache.org</a> to get \
yourself removed from the distribution list</span><br><br><div dir="ltr"><br>On 10 \
Oct 2018, at 3:14 pm, Pankaj Kumar <<a \
href="mailto:pankaj.kumar@idctechnologies.com">pankaj.kumar@idctechnologies.com</a>> \
wrote:<br><br></div><blockquote type="cite"><div dir="ltr"><meta \
http-equiv="Content-Type" content="text/html; charset=utf-8"><meta name="Generator" \
content="Microsoft Word 14 (filtered medium)"><style><!-- /* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Arial","sans-serif";
color:windowtext;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--><div class="WordSection1"><p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Arial","sans-serif"">Remove \
Me.<o:p></o:p></span></p><p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Arial","sans-serif""><o:p> </o:p></span></p><p \
class="MsoNormal"><b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">---<o:p></o:p></span></b></p><p \
class="MsoNormal"><b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Thanks \
& Regards</span></b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><br><b>Pankaj \
Kumar<o:p></o:p></b></span></p><p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">IDC \
</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Technologies</span><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif""> \
Inc.<o:p></o:p></span></p><p class="MsoNormal"><b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Work: \
408-457-9381- Ext-4102</span></b><b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></b></p><p \
class="MsoNormal"><b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Mailto:</span></b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif""> \
<b><a href="mailto:pankaj.kumar@idctechnologies.com">pankaj.kumar@idctechnologies.com</a></b><o:p></o:p></span></p><p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">---------------------------------------------------------<o:p></o:p></span></p><p \
class="MsoNormal"><b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Empowering \
Technologies Services</span></b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p><p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Remote \
Services | IT Services | BPO |<o:p></o:p></span></p><p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">IT \
Consulting | Staffing Solutions |<o:p></o:p></span></p><p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif"">---------------------------------------------------------</span><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif""><o:p></o:p></span></p><p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Arial","sans-serif""><o:p> </o:p></span></p><p \
class="MsoNormal"><b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> rahul \
patwari [<a href="mailto:rahulpatwari8383@gmail.com">mailto:rahulpatwari8383@gmail.com</a>] \
<br><b>Sent:</b> Wednesday, October 10, 2018 3:56 AM<br><b>To:</b> <a \
href="mailto:tasanuma@yahoo-corp.jp">tasanuma@yahoo-corp.jp</a><br><b>Cc:</b> <a \
href="mailto:user@hadoop.apache.org">user@hadoop.apache.org</a><br><b>Subject:</b> \
Re: HDFS Federation with HA<o:p></o:p></span></p><p \
class="MsoNormal"><o:p> </o:p></p><div><div><p class="MsoNormal"><span \
style="font-family:"Arial","sans-serif"">Hi \
Takanobu,<o:p></o:p></span></p></div><div><p class="MsoNormal"><span \
style="font-family:"Arial","sans-serif""><o:p> </o:p></span></p></div><div><p \
class="MsoNormal"><span \
style="font-family:"Arial","sans-serif"">Thanks for the reply. \
So, does that mean that 3 dedicated journal nodes are required for \
each nameservice? or can we have the same journal nodes for all the \
nameservices?<o:p></o:p></span></p></div><div><p class="MsoNormal"><span \
style="font-family:"Arial","sans-serif""><o:p> </o:p></span></p></div><div><p \
class="MsoNormal"><span \
style="font-family:"Arial","sans-serif"">Thanks,<o:p></o:p></span></p></div><div><p \
class="MsoNormal"><span \
style="font-family:"Arial","sans-serif"">Rahul <o:p></o:p></span></p></div></div><p \
class="MsoNormal"><o:p> </o:p></p><div><div><p class="MsoNormal">On Wed, Oct 10, \
2018 at 1:00 PM Takanobu Asanuma <<a \
href="mailto:tasanuma@yahoo-corp.jp">tasanuma@yahoo-corp.jp</a>> \
wrote:<o:p></o:p></p></div><blockquote style="border:none;border-left:solid #CCCCCC \
1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in"><p \
class="MsoNormal" style="margin-bottom:12.0pt">Hi Rahul,<br><br>Yes, it is possible. \
Here are the sample configurations.<br><br>=========================================== \
<br>dfs.nameservices=ns1,ns2<br>dfs.ha.namenodes.ns1=nn1,nn2<br>dfs.ha.namenodes.ns2=n \
n1,nn2<br><br>dfs.namenode.rpc-address.ns1.nn1=ns1nn1:8020<br>dfs.namenode.rpc-address \
.ns1.nn2=ns1nn2:8020<br>dfs.namenode.http-address.ns1.nn1=ns1nn1:9870<br>dfs.namenode. \
http-address.ns1.nn2=ns1nn2:9870<br><br>dfs.namenode.rpc-address.ns2.nn1=ns2nn1:8020<b \
r>dfs.namenode.rpc-address.ns2.nn2=ns2nn2:8020<br>dfs.namenode.http-address.ns2.nn1=ns \
2nn1:9870<br>dfs.namenode.http-address.ns2.nn2=ns2nn2:9870<br><br>dfs.client.failover. \
proxy.provider.ns1=org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyPr \
ovider<br>dfs.client.failover.proxy.provider.ns2=org.apache.hadoop.hdfs.server.namenod \
e.ha.ConfiguredFailoverProxyProvider<br><br>dfs.namenode.shared.edits.dir.ns1=qjournal \
://ns1jn1:8485;ns1jn2:8485;ns1jn3:8485/ns1<br>dfs.namenode.shared.edits.dir.ns2=qjourn \
al://ns2jn1:8485;ns2jn2:8485;ns2jn3:8485/ns2<br>===========================================<br><br>Regards,<br>- \
Takanobu<br><br>on 2018/10/10 3:07, "rahul patwari"<<a \
href="mailto:rahulpatwari8383@gmail.com" \
target="_blank">rahulpatwari8383@gmail.com</a>> wrote:<br><br> \
Hi,<br><br><br> Is it possible in the current architecture of HDFS to \
have multiple nameservices(federation), with Active and Backup Namenode(HA) for each \
of the nameservices?<br><br><br> Thanks,<br> Rahul \
<br><br><br><o:p></o:p></p></blockquote></div></div></div></blockquote></body></html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic