[prev in list] [next in list] [prev in thread] [next in thread] 

List:       ceph-devel
Subject:    Re: ceph-volume: --block.[db|wal] params
From:       Igor Fedotov <ifedotov () suse ! de>
Date:       2018-12-28 10:02:17
Message-ID: bb29b023-cb23-6f43-a2fa-e40cf91119fb () suse ! de
[Download RAW message or body]


On 12/21/2018 5:50 PM, Alfredo Deza wrote:
> On Tue, Dec 18, 2018 at 9:11 AM Igor Fedotov <ifedotov@suse.de> wrote:
>>
>> On 12/18/2018 5:07 PM, Sage Weil wrote:
>>> On Tue, 18 Dec 2018, Igor Fedotov wrote:
>>>> On 12/17/2018 6:23 PM, Alfredo Deza wrote:
>>>>> On Mon, Dec 17, 2018 at 9:20 AM Igor Fedotov <ifedotov@suse.de> wrote:
>>>>>> Hi All,
>>>>>>
>>>>>> Got a question while playing with ceph-volume
>>>>>>
>>>>>> Are --block.db and --block.wal options mandatory for 'ceph-volume lvm
>>>>>> prepare --bluestore' command?
>>>>> These are not mandatory (per the bluestore spec). Bluestore has 4 ways
>>>>> of deploying an OSD:
>>>>>
>>>>> 1) with a data (block) device only
>>>>> 2) with a data device and a block.db
>>>>> 3) with a data device and a wal.db
>>>>> 4) with a data device, block.db, and wal.db
>>>>>
>>>>> There is no enforcement for a specific one, any of the above is valid.
>>>>>> No formal syntax error is provided if omitted but OSD isn't prepared
>>>>>> with the following output:
>>>> ...
>>>>> Would help here to see what the actual ceph-volume call was (can't
>>>>> tell by looking at ceph-osd output)
>>>>>
>>>>> Looks like you tried several times to deploy for the same OSD? Do you
>>>>> think you can provide a way to replicate how you got here?
>>>> This is 100% reproducible for fresh "vstart" cluster:
>>>>
>>>> ../src/vstart -n
>>>>
>>>> chmod 777 /tmp/<asok-file>
>>>>
>>>> chmod 777 ./dev
>>>>
>>>> chmod 777 ./out
>>>>
>>>> ceph auth add client.bootstrap-osd mon 'allow profile bootstrap-osd'
>>>>
>>>> ceph auth export -i ./keyring > /var/lib/ceph/bootstrap-osd/ceph.keyring
>>>>
>>>> CEPH_CONF=ceph.conf /usr/local/bin/ceph-volume lvm prepare --data nvme/data
>>>> --bluestore
>>> vstart's ceph.conf debugs a bunch of bluestore_* options that normal users
>>> generally don't need/use.. maybe that is affecting things?
>>>
>>>           bluestore fsck on mount = true
>>>           bluestore block create = true
>>>           bluestore block db path = /home/sage/src/ceph/build/dev/osd$id/block.db.file
>>>           bluestore block db size = 67108864
>>>           bluestore block db create = true
>>>           bluestore block wal path = /home/sage/src/ceph/build/dev/osd$id/block.wal.file
>>>           bluestore block wal size = 1048576000
>>>           bluestore block wal create = true
>>>
>>> sage
>> Yeah, that's the point. Works fine if these lines are disabled.
> Maybe you can try and narrow down what options specifically from that
> group are problematic? Without actually trying anything, I can think
> that the
> paths might be an issue, for example:
>
>            bluestore block db path =
> /home/sage/src/ceph/build/dev/osd$id/block.db.file
>
> ceph-volume doesn't support any non-standard path like that

Any subset (including just a single line) of the following parameters 
prevents from OSD preparation:

         bluestore block db path = 
/home/if/ceph/build/dev/osd$id/block.db.file

         bluestore block db create = true

         bluestore block wal path = 
/home/if/ceph/build/dev/osd$id/block.wal.file

         bluestore block wal create = true


>>
[prev in list] [next in list] [prev in thread] [next in thread]