[prev in list] [next in list] [prev in thread] [next in thread]
List: ceph-dev
Subject: Re: ceph-volume questions / enhancements
From: John Zachary Dover <zac.dover () gmail ! com>
Date: 2023-01-16 10:48:11
Message-ID: CACJ_=XqMr38sQ47Z9K_cTcMPPsohhPZhMjs++xsVDeLRxx1y6g () mail ! gmail ! com
[Download RAW message or body]
Stefan,
The material in the documentation that stated that Ceph works only with
LUKS version 1 has been removed.
https://tracker.ceph.com/issues/58354
Zac Dover
Upstream Docs
Ceph Foundation
On Wed, Dec 28, 2022 at 8:04 PM John Zachary Dover <zac.dover(a)gmail.com>
wrote:
> Stefan,
>
> The documentation portion of this complaint is tracked here:
>
> https://tracker.ceph.com/issues/58354
>
> I will make this change when everyone is back from Christmas break and I
> can get technical verification from the ceph-volume team for the changes
> that I'll make.
>
> Zac Dover
> Upstream Docs
> Ceph Foundation
>
> On Tue, Dec 20, 2022 at 9:00 PM Stefan Kooman <stefan(a)bit.nl> wrote:
>
>> Hi,
>>
>> I'm working on some patches to get ceph-volume encryption support some
>> additional options (related to encryption performance, see: [1]).
>> While looking into the current code base of ceph-volume, and Ceph
>> encryption docs [2], it looks like some basic (internal) functionality
>> is missing, some code is not working (anymore), and the docs are partly
>> incorrect.
>>
>> functionality missing: ceph-volume has no code to look up CEPH_MON
>> config settings, i.e "ceph config get osd param". Instead, at least in
>> encryption.py, it only checks the ceph config file on disk. I'm pretty
>> sure this is not sufficient anymore, as operators have started to rely
>> on the central (mon) config database. Moreover, one of the parameters
>> you can configure, "osd_dmcrypt_key_size", cannot be set:
>>
>> ceph config set osd dmcrypt_key_size 256
>> Error EINVAL: unrecognized config option 'dmcrypt_key_size'
>>
>> Broken code: One of the current parameters cryptsetup can be fed is
>> "osd_dmcrypt_key_size". When set however, ceph-volume will throw the
>> following error:
>>
>> "TypeError: get_safe() got an unexpected keyword argument 'check_valid'".
>>
>> See [3] for the LOC. If I remove that check it works and ceph-volume
>> uses the value from the config file. I can (should?) log a tracker
>> ticket for this. Apparently nobody is using a non-default key size ...
>> or using crypto ;-).
>>
>> Wrong docs: The docs claim "only LUKS (version 1) is used". But this
>> seems no longer to be true in all cases. Modern distros use LUKS version
>> 2 by default. And ceph-volume encryption.py is not explicitly forcing
>> the LUKS format version, so the default (2) gets used. That does seem to
>> work fine however, as I've got a test cluster fully encrypted with LUKS
>> 2 OSDs, see:
>>
>> cryptsetup luksDump
>>
>> ceph--e3cf57cd--27dc--4cf2--9784--b2b5198dfcbb-osd--block--5a743d7f--2f60-=
-47da--b9f3--46aa6f5df284
>> LUKS header information
>> Version: 2
>> ...
>>
>> All in all it looks like the ceph-volume code base needs some love :-).
>> Some functionality, like checking CEPH_MON config variables seems to be
>> present in cephadm. I'm not sure if cephadm / ceph-volume code can /
>> should be refactored to have both make use of it? My changes will be
>> minor, but with ceph-volume in the current state I wonder how to best
>> proceed.
>>
>> I would appreciate any comments,
>>
>> Gr. Stefan
>>
>> [1]:
>>
>> https://lists.ceph.io/hyperkitty/list/ceph-users(a)ceph.io/thread/D5URLVPV=
GX52O6WYV474T6IVTHGHCASB/
>> [2]: https://docs.ceph.com/en/latest/ceph-volume/lvm/encryption/#
>> [3]:
>>
>> https://github.com/ceph/ceph/blob/main/src/ceph-volume/ceph_volume/util/en=
cryption.py#L22
>> _______________________________________________
>> Dev mailing list -- dev(a)ceph.io
>> To unsubscribe send an email to dev-leave(a)ceph.io
>>
>
["attachment.html" (text/html)]
<div dir="ltr"><div>Stefan,</div><div><br></div><div>The material in the \
documentation that stated that Ceph works only with LUKS version 1 has been \
removed.</div><div><br></div><div><a \
href="https://tracker.ceph.com/issues/58354">https://tracker.ceph.com/issues/58354</a></div><div><br></div><div>Zac \
Dover</div><div>Upstream Docs</div><div>Ceph Foundation<br></div></div><br><div \
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Dec 28, 2022 at 8:04 PM \
John Zachary Dover <<a \
href="mailto:zac.dover@gmail.com">zac.dover@gmail.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div \
dir="ltr"><div>Stefan,</div><div><br></div><div>The documentation portion of this \
complaint is tracked here:</div><div><br></div><div><a \
href="https://tracker.ceph.com/issues/58354" \
target="_blank">https://tracker.ceph.com/issues/58354</a></div><div><br></div><div>I \
will make this change when everyone is back from Christmas break and I can get \
technical verification from the ceph-volume team for the changes that I'll \
make.</div><div><br></div><div>Zac Dover</div><div>Upstream Docs</div><div>Ceph \
Foundation<br></div></div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Tue, Dec 20, 2022 at 9:00 PM Stefan Kooman <<a \
href="mailto:stefan@bit.nl" target="_blank">stefan@bit.nl</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br> <br>
I'm working on some patches to get ceph-volume encryption support some <br>
additional options (related to encryption performance, see: [1]).<br>
While looking into the current code base of ceph-volume, and Ceph <br>
encryption docs [2], it looks like some basic (internal) functionality <br>
is missing, some code is not working (anymore), and the docs are partly <br>
incorrect.<br>
<br>
functionality missing: ceph-volume has no code to look up CEPH_MON <br>
config settings, i.e "ceph config get osd param". Instead, at least in <br>
encryption.py, it only checks the ceph config file on disk. I'm pretty <br>
sure this is not sufficient anymore, as operators have started to rely <br>
on the central (mon) config database. Moreover, one of the parameters <br>
you can configure, "osd_dmcrypt_key_size", cannot be set:<br>
<br>
ceph config set osd dmcrypt_key_size 256<br>
Error EINVAL: unrecognized config option 'dmcrypt_key_size'<br>
<br>
Broken code: One of the current parameters cryptsetup can be fed is <br>
"osd_dmcrypt_key_size". When set however, ceph-volume will throw the <br>
following error:<br>
<br>
"TypeError: get_safe() got an unexpected keyword argument \
'check_valid'".<br> <br>
See [3] for the LOC. If I remove that check it works and ceph-volume <br>
uses the value from the config file. I can (should?) log a tracker <br>
ticket for this. Apparently nobody is using a non-default key size ... <br>
or using crypto ;-).<br>
<br>
Wrong docs: The docs claim "only LUKS (version 1) is used". But this <br>
seems no longer to be true in all cases. Modern distros use LUKS version <br>
2 by default. And ceph-volume encryption.py is not explicitly forcing <br>
the LUKS format version, so the default (2) gets used. That does seem to <br>
work fine however, as I've got a test cluster fully encrypted with LUKS <br>
2 OSDs, see:<br>
<br>
cryptsetup luksDump <br>
ceph--e3cf57cd--27dc--4cf2--9784--b2b5198dfcbb-osd--block--5a743d7f--2f60--47da--b9f3--46aa6f5df284<br>
LUKS header information<br>
Version: 2<br>
...<br>
<br>
All in all it looks like the ceph-volume code base needs some love :-). <br>
Some functionality, like checking CEPH_MON config variables seems to be <br>
present in cephadm. I'm not sure if cephadm / ceph-volume code can / <br>
should be refactored to have both make use of it? My changes will be <br>
minor, but with ceph-volume in the current state I wonder how to best <br>
proceed.<br>
<br>
I would appreciate any comments,<br>
<br>
Gr. Stefan<br>
<br>
[1]: <br>
<a href="https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/D5URLVPVGX52O6WYV474T6IVTHGHCASB/" \
rel="noreferrer" target="_blank">https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/thread/D5URLVPVGX52O6WYV474T6IVTHGHCASB/</a><br>
[2]: <a href="https://docs.ceph.com/en/latest/ceph-volume/lvm/encryption/#" \
rel="noreferrer" target="_blank">https://docs.ceph.com/en/latest/ceph-volume/lvm/encryption/#</a><br>
[3]: <br>
<a href="https://github.com/ceph/ceph/blob/main/src/ceph-volume/ceph_volume/util/encryption.py#L22" \
rel="noreferrer" target="_blank">https://github.com/ceph/ceph/blob/main/src/ceph-volume/ceph_volume/util/encryption.py#L22</a><br>
_______________________________________________<br>
Dev mailing list -- <a href="mailto:dev@ceph.io" target="_blank">dev@ceph.io</a><br>
To unsubscribe send an email to <a href="mailto:dev-leave@ceph.io" \
target="_blank">dev-leave@ceph.io</a><br> </blockquote></div>
</blockquote></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic