[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Deploy HDFS (on openstack VM) over Ceph (bare metal) on same hardware
From:       Julien Laurenceau <julien.laurenceau () pepitedata ! com>
Date:       2019-05-21 7:47:54
Message-ID: CA+ZM0DHeEL4g87rU3MkxY1DZFAWdXZfpUroHsXSq80G8tXsd=g () mail ! gmail ! com
[Download RAW message or body]

Hi,

I am designing an elastic rack formed of compute nodes, storage nodes and
openstack control nodes.

I plan to deploy:

   - Ceph on bare metal on storage nodes for ceph-osd and control nodes for
   monitor and metadata.
   - Openstack to orchestrate resources spanning the compute and storage
   nodes.
   - Hadoop worker on VMs instanciated on the storage nodes leveraging ceph
   for storage through cinder volumes.
   - Hadoop master on VMs instanciated on the storage nodes leveraging ceph
   for storage through cinder volumes.
   - Spark, k8s and a bunch of tools on VMs instantiated on the compute
   nodes leveraging ceph for storage through cinder volumes. Workload is
   multi-tenant.

I am really concerned about layering Ceph bare metal under HDFS on the same
physical nodes. This design being mainly driven by lack of resources, it
seems to me that it just cannot reliably withstand load.

I am not yet at the stage of stress tests, but it seems to me that
segregating HDFS bare metal side by side to Ceph-osd bare metal on
different nodes is mandatory.

What would be your pro/cons regarding these designs ?

Regards


PS:
https://superuser.com/questions/1437971/deploy-hdfs-on-openstack-vm-over-ceph-bare-metal-on-same-hardware

[Attachment #3 (text/html)]

<div dir="ltr"><p style="margin:0px 0px \
1em;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica \
Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;box-sizing:inherit;clear:both;color:rgb(36,39,41)">Hi,</p><p \
style="margin:0px 0px \
1em;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica \
Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;box-sizing:inherit;clear:both;color:rgb(36,39,41)">I \
am designing an elastic rack formed of compute nodes, storage nodes and openstack \
control nodes.</p><p style="margin:0px 0px \
1em;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica \
Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;box-sizing:inherit;clear:both;color:rgb(36,39,41)">I \
plan to deploy:</p><ul style="margin:0px 0px 1em \
30px;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica \
Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;list-style-position:initial;box-sizing:inherit;color:rgb(36,39,41)"><li \
style="margin:0px 0px \
0.5em;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inher \
it;font-stretch:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;box-sizing:inherit">Ceph \
on bare metal on storage nodes for ceph-osd and control nodes for monitor and \
metadata.</li><li style="margin:0px 0px \
0.5em;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inher \
it;font-stretch:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;box-sizing:inherit">Openstack \
to orchestrate resources spanning the compute and storage nodes.</li><li \
style="margin:0px 0px \
0.5em;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inher \
it;font-stretch:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;box-sizing:inherit">Hadoop \
worker on VMs instanciated on the storage nodes leveraging ceph for storage through \
cinder volumes.</li><li style="margin:0px 0px \
0.5em;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font-weight:inher \
it;font-stretch:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;box-sizing:inherit">Hadoop \
master on VMs instanciated on the storage nodes leveraging ceph for storage through \
cinder volumes.</li><li \
style="margin:0px;padding:0px;border:0px;font-style:inherit;font-variant:inherit;font- \
weight:inherit;font-stretch:inherit;line-height:inherit;font-family:inherit;vertical-align:baseline;box-sizing:inherit">Spark, \
k8s and a bunch of tools on VMs instantiated on the compute nodes leveraging ceph for \
storage through cinder volumes. Workload is multi-tenant.</li></ul><p \
style="margin:0px 0px \
1em;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica \
Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;box-sizing:inherit;clear:both;color:rgb(36,39,41)">I \
am really concerned about layering Ceph bare metal under HDFS on the same physical \
nodes. This design being mainly driven by lack of resources, it seems to me that it \
just cannot reliably withstand load.</p><p style="margin:0px 0px \
1em;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica \
Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;box-sizing:inherit;clear:both;color:rgb(36,39,41)">I \
am not yet at the stage of stress tests, but it seems to me that segregating HDFS \
bare metal side by side to Ceph-osd bare metal on different nodes is mandatory.</p><p \
style="margin:0px 0px \
1em;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica \
Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;box-sizing:inherit;clear:both;color:rgb(36,39,41)">What \
would be your pro/cons regarding these designs ?<br><br></p><p style="margin:0px 0px \
1em;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica \
Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;box-sizing:inherit;clear:both;color:rgb(36,39,41)">Regards</p><p \
style="margin:0px 0px \
1em;padding:0px;border:0px;font-variant-numeric:inherit;font-variant-east-asian:inherit;font-stretch:inherit;line-height:inherit;font-family:Arial,&quot;Helvetica \
Neue&quot;,Helvetica,sans-serif;font-size:15px;vertical-align:baseline;box-sizing:inherit;clear:both;color:rgb(36,39,41)"><br>PS: \
<a href="https://superuser.com/questions/1437971/deploy-hdfs-on-openstack-vm-over-ceph-bare-metal-on-same-hardware" \
style="font-family:Arial,Helvetica,sans-serif;font-size:small">https://superuser.com/q \
uestions/1437971/deploy-hdfs-on-openstack-vm-over-ceph-bare-metal-on-same-hardware</a></p></div>




[prev in list] [next in list] [prev in thread] [next in thread]