[prev in list] [next in list] [prev in thread] [next in thread]
List: linux-ha
Subject: Re: [Linux-HA] ldirector - jboss port
From: mike <mgbutler () nbnet ! nb ! ca>
Date: 2011-09-26 23:20:21
Message-ID: 4E8108B5.3040505 () nbnet ! nb ! ca
[Download RAW message or body]
On 11-09-25 08:04 AM, mike wrote:
> On 11-09-25 05:13 AM, Vladislav Bogdanov wrote:
>> 25.09.2011 11:09, Vladislav Bogdanov wrote:
>>> 25.09.2011 02:29, mike wrote:
>>>> On 11-09-24 02:43 PM, Vladislav Bogdanov wrote:
>>>>> 24.09.2011 16:21, mike wrote:
>>>>>> On 11-09-24 05:02 AM, Vladislav Bogdanov wrote:
>>>>>>> 23.09.2011 21:15, mike wrote:
>>>>>>>> Last year I set up an HA cluster with ldirector pointing to 2 load
>>>>>>>> balanced "real" servers. We had jboss on the backend listening to the
>>>>>>>> Real IP on port 8080. Initially, we could not get the backend to reply -
>>>>>>>> we kept getting refused connections when we tried going through the VIP.
>>>>>>>> Thanks to Horms and a few others, I had the app guy start up jboss
>>>>>>>> listening on all interfaces (0.0.0.0) and all was well - worked perfectly.
>>>>>>>>
>>>>>>>> Well now I'm told they want several more "instances" of jboss all
>>>>>>>> listening on the same port, 8080 and so starting up listening on 0.0.0.0
>>>>>>>> is not an option. My initial response was "guys, start each instance
>>>>>>>> listening to a different port - problem solved". They're not too happy
>>>>>>>> with this solution so I'm here asking - is there any way possible to
>>>>>>>> load balance to several jboss instances running on the same backend
>>>>>>>> servers and on the same port?
>>>>>>> (assuming you run linux)
>>>>>>> You can look at /proc/sys/net/ipv4/ip_nonlocal_bind tunable.
>>>>>>> Setting it to 1 allows you to configure applications to listen on
>>>>>>> non-existent IP address. And that address could be virtual, migrating
>>>>>>> over nodes. So, you have two clone instances of application, running on
>>>>>>> different nodes, and bound to the same non-wildcard address. Repeat for
>>>>>>> different application instances and addresses.
>>>>>>>
>>>>>>> This should help.
>>>>>>>
>>>>>>> Best,
>>>>>>> Vladislav
>>>>>>> _______________________________________________
>>>>>>> Linux-HA mailing list
>>>>>>> Linux-HA@lists.linux-ha.org
>>>>>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>>>>>>> See also: http://linux-ha.org/ReportingProblems
>>>>>>>
>>>>>> Thanks Vladislav,
>>>>>>
>>>>>> If I understand you correctly then I should set ip_nonlocal_bind_tunable
>>>>>> to 1 on both backend servers (not the director servers) and then bind
>>>>>> the applications to the VIP on the same backend servers. I already have
>>>>>> the VIP added to the loopback on the backend servers.
>>>>> This way you can bind app to specific VIP, not to wildcard address. Just
>>>>> specify that VIP in their configuration.
>>>>> And you can remove VIP from loopbacks and leave it to pacemaker to place
>>>>> it on a real interface. From my experience, having address on loopback
>>>>> and then adding it to real interface does not always work fine. At least
>>>>> I had problems with IPSEC on such setups.
>>>>> Sure, You can bind several app instances to different specific VIPs and
>>>>> have them running in standby mode until VIP is migrated to that node.
>>>>>
>>>>> Vladislav
>>>>>
>>>>> _______________________________________________
>>>>> Linux-HA mailing list
>>>>> Linux-HA@lists.linux-ha.org
>>>>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>>>>> See also: http://linux-ha.org/ReportingProblems
>>>>>
>>>> Thanks again Vladislav,
>>>>
>>>> I must be be missing a small piece here. I have a test cluster running
>>>> load balancing to an Apache backend and so I'm trying to incorporate
>>>> your suggestions but cant seem to get it to work.
>>>>
>>>> Here is what I did (btw the cluster currently works in a LVS-DR
>>>> configuration)
>>>>
>>>> Since both backend servers are running Debian I changed
>>>> /proc/sys/net/ipv4/ip_nonlocal_bind from 0 to 1
>>>> I removed the VIP from the loopbacks
>>>> Changed Apache to start up listening on the VIP in
>>>> /etc/apache2/ports.conf and restarted it
>>>>
>>>> When I go back to the directors ipvsadm does not show the backend
>>>> servers and I am unable to contact the webserver at the VIP.
>>>>
>>>> Did I miss a step?
>>> Hmmm... I think that this is just a very different task ;)
>>> Originally you asked about different cloned instances of jboss listening
>>> on a same port. Don't you think that LVS+apache load-balancing task
>>> differs a bit from that? :)
>>>
>>> Ok, if I understand your original setup correctly, you have added
>>> different ip addresses to loopback interfaces of your nodes. Am I correct?
>>> And then you use either proxy_arp (which could lead to undesired effects
>>> in complex setups btw) or add static routes to that addresses on your
>>> directors via nodes ethernet addresses.
>>>
>>> Now you just removed (FIXME!) that addresses from loopbacks and didn't
>>> add them anywhere. Surely, you broke you setup because addresses a re
>>> not reachable anymore.
>>>
>>> What you probably need is to use ClusterIP resource agent (I didn't use
>>> it myself, so I may be wrong here) to dynamically add bunch of addresses
>>> to ethernet interfaces of your nodes, and make *all* your apache
>>> instances listen on *all* of that addresses (just with many Listen
>>> directives in apache config). That would succeed if you have
>>> ip_nonlocal_bind set to 1, and you may bind different apache instances
>>> to another addresses on the same port. This way apache will silently
>>> ignore all "unconfigured" addresses until they appear to be added to a
>>> system. Once that happen, it will accept requests directed to that
>>> addresses.
>>>
>>> Again, I probably didn't fully understand your current setup and what
>>> you want to achieve. Anyways, I hope that information above helps.
>>>
>> Reading again your original post I understand that task is no so
>> different as I thought. Sorry for confusion.
>>
>> Anyways, the same technique I described (just adopted to jboss)
>> ultimately may help.
>>
>> Vladislav
>> _______________________________________________
>> Linux-HA mailing list
>> Linux-HA@lists.linux-ha.org
>> http://lists.linux-ha.org/mailman/listinfo/linux-ha
>> See also: http://linux-ha.org/ReportingProblems
>>
> Thanks very much Vladisalv - I'll try your suggestions.
> _______________________________________________
> Linux-HA mailing list
> Linux-HA@lists.linux-ha.org
> http://lists.linux-ha.org/mailman/listinfo/linux-ha
> See also: http://linux-ha.org/ReportingProblems
>
Hi Vladislav,
Well after much frustration I decided to use iptables.
I'll start up each instance of JBOSS listening on a VIP. When ldirectord
sends traffic to the RIP via the VIP, I re-direct at the backend server.
Something like this:
iptables -t nat -A PREROUTING -p tcp -d 192.168.2.17 --dport 8080 -j
DNAT --to-destination 192.168.2.5:8080
iptables -t nat -A PREROUTING -p tcp -d 192.168.2.16 --dport 8080 -j
DNAT --to-destination 192.168.2.3:8080
So anything than the director sends to the Real IP will be re-directed
to the VIP on the backend server. This only works if I have a loopback
alias set to the VIP as well as JBOSS listening on the VIP.
Took me a while to realize what is really going on. ldirectord is
assessing the availability of the realserver by sending it a request on
its RIP (not its VIP). Since JBoss is listening on the VIP only,
ldirectord cannot reach it. I therefore need a redirect from iptables to
send the request internally to the VIP on the loopback of the backend
server.
Thanks for your help!
-mike
_______________________________________________
Linux-HA mailing list
Linux-HA@lists.linux-ha.org
http://lists.linux-ha.org/mailman/listinfo/linux-ha
See also: http://linux-ha.org/ReportingProblems
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic