[prev in list] [next in list] [prev in thread] [next in thread]
List: mesos-user
Subject: Re: Questions about secret handling in Mesos
From: Zhitao Li <zhitaoli.cs () gmail ! com>
Date: 2018-05-11 23:57:47
Message-ID: CANMwFmUUXQDXV9Jh_LFibfdoxPwi0mExh-vKk_yrbeMcCwoq3g () mail ! gmail ! com
[Download RAW message or body]
Hi Vinod,
I filed a task https://issues.apache.org/jira/browse/MESOS-8909 for this.
If we can agree that this is something worth pursing, I'll try to post some
ideas on whether there is an efficient way to do it.
On Thu, Apr 26, 2018 at 3:32 PM, Vinod Kone <vinodkone@apache.org> wrote:
> We do direct protobuf to JSON conversion for our API endpoints and I don't
> think we do any special case logic for `Secret` type in that conversion. So
> `value` based secrets will have their value show up in v1 (and likely v0)
> API endpoints.
>
> On Mon, Apr 23, 2018 at 9:25 AM, Zhitao Li <zhitaoli.cs@gmail.com> wrote:
>
>> Hi Alexander,
>>
>> We discovered that in our own testing thus do not plan to use the
>> environment variable. For the `volume/secret` case, I believe it's possible
>> to be careful enough so we do not log that, so it's more about whether we
>> want to promise that.
>>
>> What do you think?
>>
>> On Mon, Apr 23, 2018 at 5:13 AM, Alexander Rojas <alexander@mesosphere.io
>> > wrote:
>>
>>>
>>> Hey Zhitao,
>>>
>>> I sadly have to tell you that the first assumption is not correct. If
>>> you use environment based secrets, docker and verbose mode, they will get
>>> printed (see this patch https://reviews.apache.org/r/57846/). The
>>> reason is that the docker command will get logged and it might contain your
>>> secrets. You may end up with some logging line like:
>>>
>>> ```
>>> I0129 14:09:22.444318 docker.cpp:1139] Running docker -H
>>> unix:///var/run/docker.suck run --cpu-shares 25 --memory 278435456 -e
>>> ADMIN_PASSWORD=test_password …
>>> ```
>>>
>>>
>>> On 19. Apr 2018, at 19:57, Zhitao Li <zhitaoli.cs@gmail.com> wrote:
>>>
>>> Hello,
>>>
>>> We at Uber plan to use volume/secret isolator to send secrets from Uber
>>> framework to Mesos agent.
>>>
>>> For this purpose, we are referring to these documents:
>>>
>>> - File based secrets design doc
>>> <https://docs.google.com/document/d/18raiiUfxTh-JBvjd6RyHe_TOScY87G_bMi5zBzMZmpc/edit#>
>>> and slides
>>> <http://schd.ws/hosted_files/mesosconasia2017/70/Secrets%20Management%20in%20Mesos.pdf>
>>> .
>>> - Apache Mesos secrets documentation
>>> <http://mesos.apache.org/documentation/latest/secrets/>
>>>
>>> Could you please confirm that the following assumptions are correct?
>>>
>>> - Mesos agent and master will never log the secret data at any
>>> logging level;
>>> - Mesos agent and master will never expose the secret data as part
>>> of any API response;
>>> - Mesos agent and master will never store the secret in any
>>> persistent storage, but only on tmpfs or ramfs;
>>> - When the secret is first downloaded on the mesos agent, it will be
>>> stored as "root" on the tmpfs/ramfs before being mounted in the container
>>> ramfs.
>>>
>>> If above assumptions are true, then I would like to see them documented
>>> in this as part of the Apache Mesos secrets documentation
>>> <http://mesos.apache.org/documentation/latest/secrets/>. Otherwise,
>>> we'd like to have a design discussion with maintainer of the isolator.
>>>
>>> We appreciate your help regarding this. Thanks!
>>>
>>> Regards,
>>> Aditya And Zhitao
>>>
>>>
>>>
>>
>>
>> --
>> Cheers,
>>
>> Zhitao Li
>>
>
>
--
Cheers,
Zhitao Li
[Attachment #3 (text/html)]
<div dir="ltr">Hi Vinod,<div><br></div><div>I filed a task <a \
href="https://issues.apache.org/jira/browse/MESOS-8909">https://issues.apache.org/jira/browse/MESOS-8909</a> \
for this. If we can agree that this is something worth pursing, I'll try to post \
some ideas on whether there is an efficient way to do it.</div></div><div \
class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 26, 2018 at 3:32 PM, \
Vinod Kone <span dir="ltr"><<a href="mailto:vinodkone@apache.org" \
target="_blank">vinodkone@apache.org</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">We do direct protobuf to JSON conversion for \
our API endpoints and I don't think we do any special case logic for `Secret` \
type in that conversion. So `value` based secrets will have their value show up in v1 \
(and likely v0) API endpoints.<div><div class="h5"><div class="gmail_extra"><br><div \
class="gmail_quote">On Mon, Apr 23, 2018 at 9:25 AM, Zhitao Li <span dir="ltr"><<a \
href="mailto:zhitaoli.cs@gmail.com" \
target="_blank">zhitaoli.cs@gmail.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div dir="ltr">Hi Alexander,<div><br></div><div>We discovered \
that in our own testing thus do not plan to use the environment variable. For the \
`volume/secret` case, I believe it's possible to be careful enough so we do not \
log that, so it's more about whether we want to promise \
that.</div><div><br></div><div>What do you think?</div></div><div \
class="gmail_extra"><div><div \
class="m_4376266507473712726m_-6268861154670962478h5"><br><div class="gmail_quote">On \
Mon, Apr 23, 2018 at 5:13 AM, Alexander Rojas <span dir="ltr"><<a \
href="mailto:alexander@mesosphere.io" \
target="_blank">alexander@mesosphere.io</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div \
style="word-wrap:break-word;line-break:after-white-space"><br><div> <div \
style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style:normal;font-va \
riant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">Hey \
Zhitao,</div><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-st \
yle:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-alig \
n:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><br></div><div \
style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style:normal;font-va \
riant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">I \
sadly have to tell you that the first assumption is not correct. If you use \
environment based secrets, docker and verbose mode, they will get printed (see this \
patch <a href="https://reviews.apache.org/r/57846/" \
target="_blank">https://reviews.apache.o<wbr>rg/r/57846/</a>). The reason is that the \
docker command will get logged and it might contain your secrets. You may end up with \
some logging line like:</div><div \
style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style:normal;font-va \
riant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><br></div><div \
style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style:normal;font-va \
riant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">```</div><div \
style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style:normal;font-va \
riant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">I0129 \
14:09:22.444318 docker.cpp:1139] Running docker -H <a>unix:///var/run/docker.suck</a> \
ru<wbr>n --cpu-shares 25 --memory 278435456 -e ADMIN_PASSWORD=test_password \
…</div><div style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px;font-style: \
normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:st \
art;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">```</div><br \
class="m_4376266507473712726m_-6268861154670962478m_-2315690748434557669m_-4618816202974833672Apple-interchange-newline">
</div><div><div class="m_4376266507473712726m_-6268861154670962478m_-2315690748434557669h5">
<div><br><blockquote type="cite"><div>On 19. Apr 2018, at 19:57, Zhitao Li <<a \
href="mailto:zhitaoli.cs@gmail.com" target="_blank">zhitaoli.cs@gmail.com</a>> \
wrote:</div><br class="m_4376266507473712726m_-6268861154670962478m_-2315690748434557669m_-4618816202974833672Apple-interchange-newline"><div><div \
dir="ltr"><div style="font-family:Helvetica;font-size:12px"><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-variant-ligatures:normal">Hello,</div><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-variant-ligatures:normal"><br></div><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-variant-ligatures:normal">We \
at Uber plan to use volume/secret isolator to send secrets from Uber framework to \
Mesos agent.</div><div><br></div>For this purpose, we are referring to these \
documents:<br class="m_4376266507473712726m_-6268861154670962478m_-2315690748434557669m_-4618816202974833672gmail-Apple-interchange-newline"></div><div \
style="font-family:Helvetica;font-size:12px"><ul><li>File based secrets <a \
href="https://docs.google.com/document/d/18raiiUfxTh-JBvjd6RyHe_TOScY87G_bMi5zBzMZmpc/edit#" \
target="_blank">design doc </a>and <a \
href="http://schd.ws/hosted_files/mesosconasia2017/70/Secrets%20Management%20in%20Mesos.pdf" \
target="_blank">slides</a>.<br></li><li><a \
href="http://mesos.apache.org/documentation/latest/secrets/" target="_blank">Apache \
Mesos secrets documentation</a></li></ul></div><div \
style="font-family:Helvetica;font-size:12px">Could you please confirm that the \
following assumptions are correct?<br></div><div \
style="font-family:Helvetica;font-size:12px"><ul><li>Mesos agent and master will \
never log the secret data at any logging level;</li><li>Mesos agent and master will \
never expose the secret data as part of any API response;</li><li>Mesos agent and \
master will never store the secret in any persistent storage, but only on tmpfs or \
ramfs;</li><li>When the secret is first downloaded on the mesos agent, it will be \
stored as "root" on the tmpfs/ramfs before being mounted in the container \
ramfs.</li></ul></div><div style="font-family:Helvetica;font-size:12px">If above \
assumptions are true, then I would like to see them documented in this as part of the \
<a href="http://mesos.apache.org/documentation/latest/secrets/" \
target="_blank">Apache Mesos secrets documentation</a>. Otherwise, we'd like to \
have a design discussion with maintainer of the isolator.</div><div \
style="font-family:Helvetica;font-size:12px"><br><div \
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:small;font-variant-ligatures:normal">We \
appreciate your help regarding this. Thanks!<br></div></div><div \
style="font-family:Helvetica;font-size:12px"><br></div><div \
style="font-family:Helvetica;font-size:12px">Regards,</div><div \
style="font-family:Helvetica;font-size:12px">Aditya And Zhitao</div> </div>
</div></blockquote></div><br></div></div></div></blockquote></div><br><br \
clear="all"><div><br></div></div></div><span \
class="m_4376266507473712726m_-6268861154670962478HOEnZb"><font color="#888888">-- \
<br><div class="m_4376266507473712726m_-6268861154670962478m_-2315690748434557669gmail_signature" \
data-smartmail="gmail_signature">Cheers,<br><br>Zhitao Li</div> </font></span></div>
</blockquote></div><br></div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div \
class="gmail_signature" data-smartmail="gmail_signature">Cheers,<br><br>Zhitao \
Li</div> </div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic