[prev in list] [next in list] [prev in thread] [next in thread]
List: ceph-ansible
Subject: Re: [Ceph-ansible] Cluster network config
From: Alfredo De Luca <alfredo.deluca () gmail ! com>
Date: 2020-01-29 13:19:28
Message-ID: CAAWpFTFzQxxoo-8NZAeTsCvq2zskPhBf+ay0R3oJT8=O142h3g () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
thanks. that's what I thought.
Cheers
On Wed, Jan 29, 2020 at 12:35 PM Stolte, Felix <f.stolte@fz-juelich.de>
wrote:
> You have to configure your network interfaces yourself before running
> ceph-ansible. But there are several ansible modules you can use to
> accomplish that task.
>
> Regards Felix
>
> IT-Services
> Telefon 02461 61-9243
> E-Mail: mailto:f.stolte@fz-juelich.de
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
>
>
> Von: Alfredo De Luca <alfredo.deluca@gmail.com>
> Datum: Mittwoch, 29. Januar 2020 um 12:07
> An: "Stolte, Felix" <f.stolte@fz-juelich.de>
> Cc: "ceph-ansible@lists.ceph.com" <ceph-ansible@lists.ceph.com>
> Betreff: Re: [Ceph-ansible] Cluster network config
>
> Thanks Felix.
> I only configured the cluster_network (and also of course the public
> network) but no IP.
> But I think you need the cluster_network IF/IP already up before you
> deploy
>
> right?
>
>
> On Wed, Jan 29, 2020 at 11:53 AM Stolte, Felix <mailto:
> f.stolte@fz-juelich.de> wrote:
> If you provide the cluster_network in the ceph configuration, the
> osd-daemons will automatically bind to the interface which is connected to
> that network. No need for you to specify any IPs (this is true also for the
> cluster network). If I do recall correctly, you can specify host,public and
> cluster ip for each osd in your ceph config, but there is no need to do
> that.
>
> Regards Felix
>
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
>
>
> Von: Alfredo De Luca <mailto:alfredo.deluca@gmail.com>
> Datum: Mittwoch, 29. Januar 2020 um 11:28
> An: "Stolte, Felix" <mailto:f.stolte@fz-juelich.de>
> Cc: "mailto:ceph-ansible@lists.ceph.com" <mailto:
> ceph-ansible@lists.ceph.com>
> Betreff: Re: [Ceph-ansible] Cluster network config
>
> lk to Hi Felix. thanks for that.
> I assumed you need the cluster rnetwork on the OSD as they mainly talk to
> each other
> I will give a try now...
> but w questions
>
> 1. in the conf do you need to specify the interface of the cluster
> network on top of the cluster network itself? I can\t find any parameter
> for that
> 2. if there are no variable for that.... those IP in the cluster network
> must be configured before deployment?
>
> Cheers and thanks again
>
>
> On Wed, Jan 29, 2020 at 9:56 AM Stolte, Felix <mailto:mailto:
> f.stolte@fz-juelich.de> wrote:
> Hi Alfredo,
>
> why do you use the cluster IPs in your ansible inventory? Put in the IPs
> from the public_network and you should be fine. As long as you specify the
> "cluster_network" variable, ceph-ansible takes care of the rest.
>
> Regards Felix
>
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
>
>
> Von: Alfredo De Luca <mailto:mailto:alfredo.deluca@gmail.com>
> Datum: Mittwoch, 29. Januar 2020 um 09:42
> An: "Stolte, Felix" <mailto:mailto:f.stolte@fz-juelich.de>
> Cc: "mailto:mailto:ceph-ansible@lists.ceph.com" <mailto:mailto:
> ceph-ansible@lists.ceph.com>
> Betreff: Re: [Ceph-ansible] Cluster network config
>
> Hi Felix.
> thanks for your reply.
> So...
> ceph version 14.2.6 (f0aa067ac7a02ee46ea48aa26c6e298b5ea272e9) nautilus
> (stable)
> 3 nodes with 5 SSD / 2 Tb each
> public network = http://10.31.10.0/24 - 1Gb link
> cluster network = http://10.31.20.0/24 - private network between the 3
> nodes
> Ceph-ansible branch is stable-4.0
>
> The all 3 nodes can ping and ssh each other.
> The first problem we had that deploying the cluster from a management
> server it says it can't reach the internal server address.... it is
> right....but how to bypass this issue if I nees to have in my inventory the
> OSD ip addresses with an internal network? that's my question.
>
> In my inventory file I have
> <SNIP>
> [osds]
> 10.31.20.7
> 10.31.20.8
> 10.31.20.9
> <SNIP>
>
> Then we deployed ceph-ansible from one of the node in the cluster and
> went ok.
> But how to fix this if we have an external deployment management that
> doesn't have access to the cluster network?
>
> Thanks. Appreciate your help
>
>
> Alfredo
>
>
>
>
>
>
>
>
>
> On Wed, Jan 29, 2020 at 8:26 AM Stolte, Felix <mailto:mailto:mailto:mailto
> :f.stolte@fz-juelich.de> wrote:
> Hi Alfredo,
>
> it is not only possible, but it is in fact recommended to define separate
> networks for public and cluster networks. To resolve your issue we would
> need more information about your configuration. Which ceph version are you
> using? Which ceph-ansible branch? Have you defined both cluster_network and
> public_network variables? Are the osd servers able to communicate over the
> cluster network?
>
> Best regards
> Felix
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
>
> -------------------------------------------------------------------------------------
>
> -------------------------------------------------------------------------------------
>
>
> Von: Ceph-ansible <mailto:mailto:mailto:mailto:
> ceph-ansible-bounces@lists.ceph.com> im Auftrag von Alfredo De Luca
> <mailto:mailto:mailto:mailto:alfredo.deluca@gmail.com>
> Datum: Dienstag, 28. Januar 2020 um 15:19
> An: "mailto:mailto:mailto:mailto:ceph-ansible@lists.ceph.com" <mailto:
> mailto:mailto:mailto:ceph-ansible@lists.ceph.com>
> Betreff: [Ceph-ansible] Cluster network config
>
> Hi all.
> We ve been deploying ceph with ceph-ansible for a while now. Not yet in
> production but all it's working well.
> Til now we collapsed public and cluster network on the same network/NIC
> but since we have now another port of 10G I d like to configure
> ceph-ansible cluster network with the 10g and so I created a private
> network on the 10Gb without external access.
> Is it possible to deploy it in a such configuration? I quickly tried and
> ceph-ansible is complaining about not reaching the private network.
>
> Any ideas? Maybe it's not possible.....?
>
> Cheers
>
>
> --
> Alfredo
>
>
>
>
> --
> Alfredo
>
>
>
>
> --
> Alfredo
>
>
>
>
> --
> Alfredo
>
>
>
--
*Alfredo*
[Attachment #5 (text/html)]
<div dir="ltr"><div class="gmail_default" \
style="font-family:verdana,sans-serif">thanks. that's what I thought. </div><div \
class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div \
class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div \
class="gmail_default" style="font-family:verdana,sans-serif">Cheers</div><div \
class="gmail_default" style="font-family:verdana,sans-serif"><br></div></div><br><div \
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jan 29, 2020 at 12:35 \
PM Stolte, Felix <<a \
href="mailto:f.stolte@fz-juelich.de">f.stolte@fz-juelich.de</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">You have to configure \
your network interfaces yourself before running ceph-ansible. But there are several \
ansible modules you can use to accomplish that task.<br> <br>
Regards Felix<br>
<br>
IT-Services<br>
Telefon 02461 61-9243<br>
E-Mail: mailto:<a href="mailto:f.stolte@fz-juelich.de" \
target="_blank">f.stolte@fz-juelich.de</a><br>
-------------------------------------------------------------------------------------<br>
-------------------------------------------------------------------------------------<br>
Forschungszentrum Juelich GmbH<br>
52425 Juelich<br>
Sitz der Gesellschaft: Juelich<br>
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498<br>
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher<br>
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),<br>
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,<br>
Prof. Dr. Sebastian M. Schmidt<br>
-------------------------------------------------------------------------------------<br>
-------------------------------------------------------------------------------------<br>
<br>
<br>
Von: Alfredo De Luca <<a href="mailto:alfredo.deluca@gmail.com" \
target="_blank">alfredo.deluca@gmail.com</a>><br>
Datum: Mittwoch, 29. Januar 2020 um 12:07<br>
An: "Stolte, Felix" <<a href="mailto:f.stolte@fz-juelich.de" \
target="_blank">f.stolte@fz-juelich.de</a>><br>
Cc: "<a href="mailto:ceph-ansible@lists.ceph.com" \
target="_blank">ceph-ansible@lists.ceph.com</a>" <<a \
href="mailto:ceph-ansible@lists.ceph.com" \
target="_blank">ceph-ansible@lists.ceph.com</a>><br>
Betreff: Re: [Ceph-ansible] Cluster network config<br>
<br>
Thanks Felix.<br>
I only configured the cluster_network (and also of course the public network) but no \
IP. <br> But I think you need the cluster_network IF/IP already up before you deploy \
<br> <br>
right?<br>
<br>
<br>
On Wed, Jan 29, 2020 at 11:53 AM Stolte, Felix <mailto:<a \
href="mailto:f.stolte@fz-juelich.de" target="_blank">f.stolte@fz-juelich.de</a>> \
wrote:<br> If you provide the cluster_network in the ceph configuration, the \
osd-daemons will automatically bind to the interface which is connected to that \
network. No need for you to specify any IPs (this is true also for the cluster \
network). If I do recall correctly, you can specify host,public and cluster ip for \
each osd in your ceph config, but there is no need to do that.<br> <br>
Regards Felix<br>
<br>
-------------------------------------------------------------------------------------<br>
-------------------------------------------------------------------------------------<br>
Forschungszentrum Juelich GmbH<br>
52425 Juelich<br>
Sitz der Gesellschaft: Juelich<br>
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498<br>
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher<br>
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),<br>
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,<br>
Prof. Dr. Sebastian M. Schmidt<br>
-------------------------------------------------------------------------------------<br>
-------------------------------------------------------------------------------------<br>
<br>
<br>
Von: Alfredo De Luca <mailto:<a href="mailto:alfredo.deluca@gmail.com" \
target="_blank">alfredo.deluca@gmail.com</a>><br>
Datum: Mittwoch, 29. Januar 2020 um 11:28<br>
An: "Stolte, Felix" <mailto:<a href="mailto:f.stolte@fz-juelich.de" \
target="_blank">f.stolte@fz-juelich.de</a>><br>
Cc: "mailto:<a href="mailto:ceph-ansible@lists.ceph.com" \
target="_blank">ceph-ansible@lists.ceph.com</a>" <mailto:<a \
href="mailto:ceph-ansible@lists.ceph.com" \
target="_blank">ceph-ansible@lists.ceph.com</a>><br>
Betreff: Re: [Ceph-ansible] Cluster network config<br>
<br>
lk to Hi Felix. thanks for that. <br>
I assumed you need the cluster rnetwork on the OSD as they mainly talk to each \
other<br> I will give a try now...<br>
but w questions<br>
<br>
1. in the conf do you need to specify the interface of the cluster network on top \
of the cluster network itself? I can\t find any parameter for that<br> 2. if there \
are no variable for that.... those IP in the cluster network must be configured \
before deployment?<br> <br>
Cheers and thanks again<br>
<br>
<br>
On Wed, Jan 29, 2020 at 9:56 AM Stolte, Felix <mailto:<a href="mailto:mailto" \
target="_blank">mailto</a>:<a href="mailto:f.stolte@fz-juelich.de" \
target="_blank">f.stolte@fz-juelich.de</a>> wrote:<br> Hi Alfredo,<br>
<br>
why do you use the cluster IPs in your ansible inventory? Put in the IPs from the \
public_network and you should be fine. As long as you specify the "cluster_network" \
variable, ceph-ansible takes care of the rest.<br> <br>
Regards Felix<br>
<br>
-------------------------------------------------------------------------------------<br>
-------------------------------------------------------------------------------------<br>
Forschungszentrum Juelich GmbH<br>
52425 Juelich<br>
Sitz der Gesellschaft: Juelich<br>
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498<br>
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher<br>
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),<br>
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,<br>
Prof. Dr. Sebastian M. Schmidt<br>
-------------------------------------------------------------------------------------<br>
-------------------------------------------------------------------------------------<br>
<br>
<br>
Von: Alfredo De Luca <mailto:<a href="mailto:mailto" target="_blank">mailto</a>:<a \
href="mailto:alfredo.deluca@gmail.com" \
target="_blank">alfredo.deluca@gmail.com</a>><br>
Datum: Mittwoch, 29. Januar 2020 um 09:42<br>
An: "Stolte, Felix" <mailto:<a href="mailto:mailto" \
target="_blank">mailto</a>:<a href="mailto:f.stolte@fz-juelich.de" \
target="_blank">f.stolte@fz-juelich.de</a>><br>
Cc: "mailto:<a href="mailto:mailto" target="_blank">mailto</a>:<a \
href="mailto:ceph-ansible@lists.ceph.com" \
target="_blank">ceph-ansible@lists.ceph.com</a>" <mailto:<a \
href="mailto:mailto" target="_blank">mailto</a>:<a \
href="mailto:ceph-ansible@lists.ceph.com" \
target="_blank">ceph-ansible@lists.ceph.com</a>><br>
Betreff: Re: [Ceph-ansible] Cluster network config<br>
<br>
Hi Felix.<br>
thanks for your reply. <br>
So... <br>
ceph version 14.2.6 (f0aa067ac7a02ee46ea48aa26c6e298b5ea272e9) nautilus (stable)<br>
3 nodes with 5 SSD / 2 Tb each<br>
public network = <a href="http://10.31.10.0/24" rel="noreferrer" \
target="_blank">http://10.31.10.0/24</a> - 1Gb link<br> cluster network = <a \
href="http://10.31.20.0/24" rel="noreferrer" target="_blank">http://10.31.20.0/24</a> \
- private network between the 3 nodes<br> Ceph-ansible branch is stable-4.0<br>
<br>
The all 3 nodes can ping and ssh each other. <br>
The first problem we had that deploying the cluster from a management server it says \
it can't reach the internal server address.... it is right....but how to bypass \
this issue if I nees to have in my inventory the OSD ip addresses with an internal \
network? that's my question.<br> <br>
In my inventory file I have <br>
<SNIP><br>
[osds]<br>
10.31.20.7<br>
10.31.20.8<br>
10.31.20.9<br>
<SNIP><br>
<br>
Then we deployed ceph-ansible from one of the node in the cluster and went ok. \
<br> But how to fix this if we have an external deployment management that \
doesn't have access to the cluster network? <br> <br>
Thanks. Appreciate your help<br>
<br>
<br>
Alfredo<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
On Wed, Jan 29, 2020 at 8:26 AM Stolte, Felix <mailto:<a href="mailto:mailto" \
target="_blank">mailto</a>:mailto:<a href="mailto:mailto" \
target="_blank">mailto</a>:<a href="mailto:f.stolte@fz-juelich.de" \
target="_blank">f.stolte@fz-juelich.de</a>> wrote:<br> Hi Alfredo,<br>
<br>
it is not only possible, but it is in fact recommended to define separate networks \
for public and cluster networks. To resolve your issue we would need more information \
about your configuration. Which ceph version are you using? Which ceph-ansible \
branch? Have you defined both cluster_network and public_network variables? Are the \
osd servers able to communicate over the cluster network?<br> <br>
Best regards<br>
Felix<br>
-------------------------------------------------------------------------------------<br>
-------------------------------------------------------------------------------------<br>
Forschungszentrum Juelich GmbH<br>
52425 Juelich<br>
Sitz der Gesellschaft: Juelich<br>
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498<br>
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher<br>
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),<br>
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,<br>
Prof. Dr. Sebastian M. Schmidt<br>
-------------------------------------------------------------------------------------<br>
-------------------------------------------------------------------------------------<br>
<br>
<br>
Von: Ceph-ansible <mailto:<a href="mailto:mailto" \
target="_blank">mailto</a>:mailto:<a href="mailto:mailto" \
target="_blank">mailto</a>:<a href="mailto:ceph-ansible-bounces@lists.ceph.com" \
target="_blank">ceph-ansible-bounces@lists.ceph.com</a>> im Auftrag von Alfredo De \
Luca <mailto:<a href="mailto:mailto" target="_blank">mailto</a>:mailto:<a \
href="mailto:mailto" target="_blank">mailto</a>:<a \
href="mailto:alfredo.deluca@gmail.com" \
target="_blank">alfredo.deluca@gmail.com</a>><br>
Datum: Dienstag, 28. Januar 2020 um 15:19<br>
An: "mailto:<a href="mailto:mailto" target="_blank">mailto</a>:mailto:<a \
href="mailto:mailto" target="_blank">mailto</a>:<a \
href="mailto:ceph-ansible@lists.ceph.com" \
target="_blank">ceph-ansible@lists.ceph.com</a>" <mailto:<a \
href="mailto:mailto" target="_blank">mailto</a>:mailto:<a href="mailto:mailto" \
target="_blank">mailto</a>:<a href="mailto:ceph-ansible@lists.ceph.com" \
target="_blank">ceph-ansible@lists.ceph.com</a>><br>
Betreff: [Ceph-ansible] Cluster network config<br>
<br>
Hi all.<br>
We ve been deploying ceph with ceph-ansible for a while now. Not yet in production \
but all it's working well. <br> Til now we collapsed public and cluster network \
on the same network/NIC but since we have now another port of 10G I d like to \
configure ceph-ansible cluster network with the 10g and so I created a private \
network on the 10Gb without external access. <br> Is it possible to deploy it in a \
such configuration? I quickly tried and ceph-ansible is complaining about not \
reaching the private network. <br> <br>
Any ideas? Maybe it's not possible.....?<br>
<br>
Cheers<br>
<br>
<br>
-- <br>
Alfredo<br>
<br>
<br>
<br>
<br>
-- <br>
Alfredo<br>
<br>
<br>
<br>
<br>
-- <br>
Alfredo<br>
<br>
<br>
<br>
<br>
-- <br>
Alfredo<br>
<br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" \
class="gmail_signature"><div \
dir="ltr"><div><i><b>Alfredo</b></i><br><div><br></div></div></div></div>
_______________________________________________
Ceph-ansible mailing list
Ceph-ansible@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-ansible-ceph.com
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic