[prev in list] [next in list] [prev in thread] [next in thread]
List: hadoop-user
Subject: RE: Does Hadoop 2.6.0 have job level blacklisting?
From: Chris Mawata <chris.mawata () gmail ! com>
Date: 2015-03-30 13:22:14
Message-ID: CAEpEg_CHNvj8RasLjEeMtrPnOU_2=+4K==dGT1HUCZbQiyJP3w () mail ! gmail ! com
[Download RAW message or body]
Thanks very much Rohith.
Chris
On Mar 30, 2015 12:11 AM, "Rohith Sharma K S" <rohithsharmaks@huawei.com>
wrote:
> Hi Chris
>
>
>
> Is there still job level blacklisting as there was in earlier versions?
>
> > > yes, job level blacklisting support is there. Application Master has
> to identify the nodes which it wants to blacklists and send those nodes
> details to ResourceManager via ApplicationMasterProtocol#allocate request.
> On blacklisted nodes, containers will not be assigned thereafter.
>
>
>
> * Java Doc*
>
>
> https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/ApplicationMast \
> erProtocol.html#allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest)
>
>
> https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/protocolrecords/AllocateRequest.html
>
>
>
> Thanks & Regards
>
> Rohith Sharma K S
>
> *From:* Chris Mawata [mailto:chris.mawata@gmail.com]
> *Sent:* 29 March 2015 01:10
> *To:* user@hadoop.apache.org
> *Subject:* Does Hadoop 2.6.0 have job level blacklisting?
>
>
>
> At
> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Monitoring_Health_of_NodeManagers
>
> is a description of how you can have a script check the health of a node
> and indicate to the ResourceManager that it is unhealthy. This seems to be
> at the cluster level. Is there still job level blacklisting as there was in
> earlier versions?
>
>
>
> Chris Mawata
>
[Attachment #3 (text/html)]
<p dir="ltr">Thanks very much Rohith.<br>
Chris</p>
<div class="gmail_quote">On Mar 30, 2015 12:11 AM, "Rohith Sharma K S" \
<<a href="mailto:rohithsharmaks@huawei.com">rohithsharmaks@huawei.com</a>> \
wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 \
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi \
Chris<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal">Is there still job level blacklisting as \
there was in earlier versions?<u></u><u></u></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">>> \
yes, job level blacklisting support is there. Application Master has to identify \
the nodes which it wants to blacklists and send those nodes details to \
ResourceManager via ApplicationMasterProtocol#allocate request. On blacklisted nodes, \
containers will not be assigned thereafter.<u></u><u></u></span></p> <p \
class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><b><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> \
Java Doc<u></u><u></u></span></b></p> <p class="MsoNormal"><a \
href="https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/Application \
MasterProtocol.html#allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest)" \
target="_blank">https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/A \
pplicationMasterProtocol.html#allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest)</a><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><a \
href="https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/protocolrecords/AllocateRequest.html" \
target="_blank">https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/protocolrecords/AllocateRequest.html</a><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> \
<u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks \
& Regards<u></u><u></u></span></p> <p class="MsoNormal"><span \
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Rohith \
Sharma K S<u></u><u></u></span></p> <div style="border:none;border-top:solid #b5c4df \
1.0pt;padding:3.0pt 0cm 0cm 0cm"> <p class="MsoNormal"><b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span \
style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Chris \
Mawata [mailto:<a href="mailto:chris.mawata@gmail.com" \
target="_blank">chris.mawata@gmail.com</a>] <br>
<b>Sent:</b> 29 March 2015 01:10<br>
<b>To:</b> <a href="mailto:user@hadoop.apache.org" \
target="_blank">user@hadoop.apache.org</a><br> <b>Subject:</b> Does Hadoop 2.6.0 have \
job level blacklisting?<u></u><u></u></span></p> </div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">At <a \
href="http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Monitoring_Health_of_NodeManagers" \
target="_blank"> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Monitoring_Health_of_NodeManagers</a><u></u><u></u></p>
<div>
<p class="MsoNormal">is a description of how you can have a script check the health \
of a node and indicate to the ResourceManager that it is unhealthy. This seems to be \
at the cluster level. Is there still job level blacklisting as there was in earlier \
versions?<u></u><u></u></p> </div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Chris Mawata<u></u><u></u></p>
</div>
</div>
</div>
</div>
</blockquote></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic