[prev in list] [next in list] [prev in thread] [next in thread]
List: hadoop-user
Subject: Re: ContainerId starts with 1 ?
From: 李响 <waterlx () gmail ! com>
Date: 2024-03-20 14:28:24
Message-ID: CAPcmo=4iEGEHC1bDz+R_i891nfXr+0H4VZdVMnRSirnvjTJdDA () mail ! gmail ! com
[Download RAW message or body]
Dear Hadoop/Yarn community,
I still beg your help for the question above.
Additionally, I might have other questions.
The target is to get the driver container id of a Spark app, from Yarn
Aggregation Log. I would like to call
LogAggregationIndexedFileController#readAggregatedLogsMeta()
<https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-ya \
rn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifile/LogAggregationIndexedFileController.java#L656C1-L656C56>
then get the first ContainLogMeta from the list returned, then call
getContainerId() from it.
The questions are:
1. Is the first ContainerLogMeta always the driver container?
2. If the driver failed to get up for the first time somehow, but
succeed in its second try. The container id will be added by 1 if I
understand it correctly. Under this case, will the first ContainLogMeta
returned by that function above be the first failed container, or the
second successful container? Or the container id gets not changed after a
failure?
Thanks!
On Fri, Feb 23, 2024 at 4:21 PM 李响 <waterlx@gmail.com> wrote:
> Dear Hadoop/Yarn community,
>
> In Yarn, a container is represented as
> container_e*epoch*_*clusterTimestamp*_*appId*_*attemptId*_*containerId*
>
> Regarding the last section, "containerId", as the sequential number of
> containers, I notice it does not start with 0, but 1.
>
> My question is:
> 1. Is that observation correct?
> 2. Sorry I do not find the code to support that. I read ContainerId.java
> and ContainerIdPBImpl.java but does not find the answer. Could you please
> show me the code path to support it staring with 1?
> 3. It seems counter-intuitive for me, as a programmer ^_^, who thinks the
> index should start with 0, rather than 1. If it is designed to start with
> 1, any background / thought / discussion to share?
>
> Thanks !!!
>
>
>
> --
>
> 李响 Xiang Li
>
>
>
--
李响 Xiang Li
手机 cellphone ：+86-136-8113-8972
邮件 e-mail ：waterlx@gmail.com
[Attachment #3 (text/html)]
<div dir="ltr"><div>Dear Hadoop/Yarn community,</div><div><br></div><div>I still beg \
your help for the question above.</div><div><br></div><div>Additionally, I might have \
other questions.</div><div>The target is to get the driver container id of a Spark \
app, from Yarn Aggregation Log. I would like to call <a \
href="https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hado \
op-yarn-common/src/main/java/org/apache/hadoop/yarn/logaggregation/filecontroller/ifil \
e/LogAggregationIndexedFileController.java#L656C1-L656C56">LogAggregationIndexedFileController#readAggregatedLogsMeta()</a> \
then get the first ContainLogMeta from the list returned, then call getContainerId() \
from it.<br></div><div>The questions are:</div><div><ol><li>Is the first \
ContainerLogMeta always the driver container?</li><li>If the driver failed to get up \
for the first time somehow, but succeed in its second try. The container id will be \
added by 1 if I understand it correctly. Under this case, will the first \
ContainLogMeta returned by that function above be the first failed container, or the \
second successful container? Or the container id gets not changed after a \
failure?</li></ol><div>Thanks!<br></div></div><div><br></div></div><br><div \
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 23, 2024 at \
4:21 PM 李响 <<a href="mailto:waterlx@gmail.com" \
target="_blank">waterlx@gmail.com</a>> wrote:<br></div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div dir="ltr">Dear Hadoop/Yarn \
community,<div><br></div><div>In Yarn, a container is represented as <span \
style="background-color:rgb(204,204,204)">container_e*epoch*_*clusterTimestamp*_*appId*_*attemptId*_<font \
color="#ff0000">*containerId*</font></span></div><div><br></div><div>Regarding the \
last section, "containerId", as the sequential number of containers, I \
notice it does not start with 0, but 1. </div><div><br></div><div>My question \
is:</div><div>1. Is that observation correct?</div><div>2. Sorry I do not find the \
code to support that. I read ContainerId.java and ContainerIdPBImpl.java but does not \
find the answer. Could you please show me the code path to support it staring with \
1?</div><div>3. It seems counter-intuitive for me, as a programmer ^_^, who thinks \
the index should start with 0, rather than 1. If it is designed to start with 1, any \
background / thought / discussion to share?</div><div><br></div><div>Thanks \
!!!</div><div> </div><div><br clear="all"><div><br></div><span \
class="gmail_signature_prefix">-- </span><br><div dir="ltr" \
class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div \
dir="ltr"><div><br><font face="verdana, sans-serif"> \
李响 Xiang Li<br><br></font><br></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><br><span class="gmail_signature_prefix">-- \
</span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div \
dir="ltr"><div><div dir="ltr"><div><br><font face="verdana, sans-serif"> \
李响 Xiang Li<br><br>手机 cellphone ：+86-136-8113-8972<br>邮件 e-mail \
：<a href="mailto:waterlx@gmail.com" target="_blank">waterlx@gmail.com</a> \
</font><br></div></div></div></div></div></div></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic