[prev in list] [next in list] [prev in thread] [next in thread]
List: activemq-dev
Subject: Re: Having some trouble making a simple network...
From: "Hiram Chirino" <hiram () hiramchirino ! com>
Date: 2006-10-30 3:07:22
Message-ID: af2843cd0610291907o7f16b5a8tc9d6e0970ebb5304 () mail ! gmail ! com
[Download RAW message or body]
Hey Jason,
Well if you want to take stab at it, the code that needs to get
modified is in the o.a.activemq.network package. The
DiscoveryNetworkConnector creates new DemandForwardingBridge to
connect the local broker to a remotebroker. Notice that the
DemandForwardingBridge's constructor takes a local and remote
Transport as constructor arguments. This sets up the bridge to sends
message from the local transport to the remote one. In theory you
should be able to transpose those two and then the local broker should
receive messages from the remote broker.
So perhaps we just need to add a option to the
DiscoveryNetworkConnector so that in addition to creating the normal
bridge that forwards from local -> remote, we create a second bridge
to forward from remote -> local. I Imagine that we could create the
extra set of connections in the DiscoveryNetworkConnector
.onServiceAdd() call.
But this could prove to be risky since network bridges can just about
fail at any point in the process and handling all the failures has
proven tricky and inverting the local/remote nature of the transport
could introduce failures at points that had not been considered
previously.
I'm sure there may be other approaches.. perhaps Rob had something else I mind.
On 10/29/06, Jason Dillon <jason@planet57.com> wrote:
> While waiting for bi-directional network connections... is there
> anything I can do do get this working? So that the master broker
> does not need to have explicit knowledge of all slave brokers? I can
> open a port on the slaves... though I don't like that, but as a short-
> term fix until bi-directional network connections are working I am
> willing to do it.
>
> Maybe multicast from the master broker to slaves?
>
> --jason
>
>
> On Oct 16, 2006, at 3:45 AM, Rob Davies wrote:
>
> > It'll take a week - or so - simply because I've a lot of other
> > stuff to do. It's not a major piece of work - but there's some
> > fiddly things to change in the broker to support it - so we've just
> > got to be careful not to break anything else.
> > On 16 Oct 2006, at 08:11, Jason Dillon wrote:
> >
> >> Any idea how easy this is going to be to implement? Should it be
> >> fairly simple, or a major design change?
> >>
> >> --jason
> >>
> >>
> >> On Oct 15, 2006, at 11:38 PM, Rob Davies wrote:
> >>
> >>> Hi Jason,
> >>>
> >>> I just created a jira for this: http://issues.apache.org/activemq/
> >>> browse/AMQ-979
> >>>
> >>> On 16 Oct 2006, at 05:40, Jason Dillon wrote:
> >>>
> >>>> How then do I setup a hub/spoke network where the hub does not
> >>>> know about all of the spokes, and where the spokes may be behind
> >>>> firewalls, only allowing outgoing connections? I had thought
> >>>> that having the remote broker define a network connection to the
> >>>> central broker would have been enough to connect them or bi-
> >>>> directional message flow. But it sounds like that is not the
> >>>> case based on what you've said.
> >>>>
> >>>> How should I configure the brokers assuming that the remote
> >>>> brokers maybe behind a firewall then?
> >>>>
> >>>> --jason
> >>>>
> >>>>
> >>>> On Oct 15, 2006, at 9:30 PM, Hiram Chirino wrote:
> >>>>
> >>>>> Hey Jason,
> >>>>>
> >>>>> Not sure what those errors are about, but first off... if network
> >>>>> connections are only defined from the remote brokers to the
> >>>>> central broker,
> >>>>> then only messages can sent to the central broker. The central
> >>>>> broker will
> >>>>> not be able to send message back to the remote broker.
> >>>>>
> >>>>> Regards,
> >>>>> Hiram
> >>>>>
> >>>>> On 10/15/06, Jason Dillon <jason@planet57.com> wrote:
> >>>>>>
> >>>>>> Hiya... I'm having some trouble making a simple broker network
> >>>>>> for
> >>>>>> GBuild. The idea was to embed a broker in each node, and then
> >>>>>> have
> >>>>>> the slave nodes connect to the master node, so that all client
> >>>>>> code
> >>>>>> will always be connected, and let activemq handle broker to
> >>>>>> broker
> >>>>>> connectivity.
> >>>>>>
> >>>>>> But, I can not seems to get it to work.
> >>>>>>
> >>>>>> NOTE: This is not master/slave in terms of broker fail-over...
> >>>>>> its
> >>>>>> just hub/spoke where the hub is the master and slave a
> >>>>>> spoke... just
> >>>>>> for clarity on the bits below.
> >>>>>>
> >>>>>> My central manager (which is what slave nodes connect to) has:
> >>>>>>
> >>>>>> <beans xmlns="http://activemq.org/config/1.0">
> >>>>>> <bean
> >>>>>> class="org.springframework.beans.factory.config.PropertyPlacehold
> >>>>>> erConfi
> >>>>>> gurer"/>
> >>>>>> <broker useJmx="true">
> >>>>>> <managementContext>
> >>>>>> <managementContext connectorPort="5003"
> >>>>>> jmxDomainName="org.apache.activemq"/>
> >>>>>> </managementContext>
> >>>>>> <persistenceAdapter>
> >>>>>> <journaledJDBC journalLogFiles="5" dataDirectory="$
> >>>>>> {gshell.home}/activemq"/>
> >>>>>> </persistenceAdapter>
> >>>>>> <transportConnectors>
> >>>>>> <transportConnector uri="vm://localhost"/>
> >>>>>> <transportConnector uri="tcp://localhost:16161"/>
> >>>>>> </transportConnectors>
> >>>>>> </broker>
> >>>>>> </beans>
> >>>>>>
> >>>>>> And my slave nodes have:
> >>>>>>
> >>>>>> <beans xmlns="http://activemq.org/config/1.0">
> >>>>>> <bean
> >>>>>> class="org.springframework.beans.factory.config.PropertyPlacehold
> >>>>>> erConfi
> >>>>>> gurer"/>
> >>>>>> <broker useJmx="true">
> >>>>>> <managementContext>
> >>>>>> <managementContext connectorPort="5004"
> >>>>>> jmxDomainName="org.apache.activemq"/>
> >>>>>> </managementContext>
> >>>>>> <persistenceAdapter>
> >>>>>> <journaledJDBC journalLogFiles="5" dataDirectory="$
> >>>>>> {gshell.home}/activemq"/>
> >>>>>> </persistenceAdapter>
> >>>>>> <transportConnectors>
> >>>>>> <transportConnector uri="vm://localhost"/>
> >>>>>> </transportConnectors>
> >>>>>> <networkConnectors>
> >>>>>> <networkConnector uri="static:(tcp://gbuild.org:
> >>>>>> 16161)"
> >>>>>> failover="true"/>
> >>>>>> </networkConnectors>
> >>>>>> </broker>
> >>>>>> </beans>
> >>>>>>
> >>>>>> But... for some reason this is not working... and I don't know
> >>>>>> why.
> >>>>>> Connectivity is good, as when I change the slave client's to
> >>>>>> connect
> >>>>>> with "tcp://gbuild.org:16161" instead of "vm://localhost"
> >>>>>> everything
> >>>>>> works fine.
> >>>>>>
> >>>>>> But when the slaves use "vm://localhost" then they never see any
> >>>>>> messages, and the master node complains with "No subscriptions
> >>>>>> registered, will not dispatch message at this time" when new
> >>>>>> messages
> >>>>>> are queued... though when the slave starts I do see it
> >>>>>> connecting to
> >>>>>> gbuild.org:16161 and I see the master node create a consumer
> >>>>>> for the
> >>>>>> client (and remove it when I stop the slave).
> >>>>>>
> >>>>>> Not sure if this matters, but I also see these logs on the
> >>>>>> master node:
> >>>>>>
> >>>>>> <snip>
> >>>>>> 19:54:45,283 DEBUG [Service] Async error occurred:
> >>>>>> java.lang.NullPointerException
> >>>>>> java.lang.NullPointerException
> >>>>>> at
> >>>>>> edu.emory.mathcs.backport.java.util.concurrent.ConcurrentHashMap.
> >>>>>> hash
> >>>>>> (ConcurrentHashMap.java:154)
> >>>>>> at
> >>>>>> edu.emory.mathcs.backport.java.util.concurrent.ConcurrentHashMap.
> >>>>>> get
> >>>>>> (ConcurrentHashMap.java:759)
> >>>>>> at
> >>>>>> org.apache.activemq.broker.AbstractConnection.processAddConnectio
> >>>>>> n
> >>>>>> (AbstractConnection.java:616)
> >>>>>> at
> >>>>>> org.apache.activemq.broker.jmx.ManagedTransportConnection.process
> >>>>>> AddConn
> >>>>>> ection(ManagedTransportConnection.java:87)
> >>>>>> at org.apache.activemq.command.ConnectionInfo.visit
> >>>>>> (ConnectionInfo.java:121)
> >>>>>> at org.apache.activemq.broker.AbstractConnection.service
> >>>>>> (AbstractConnection.java:238)
> >>>>>> at org.apache.activemq.broker.TransportConnection
> >>>>>> $1.onCommand
> >>>>>> (TransportConnection.java:63)
> >>>>>> at
> >>>>>> org.apache.activemq.transport.ResponseCorrelator.onCommand
> >>>>>> (ResponseCorrelator.java:95)
> >>>>>> at
> >>>>>> org.apache.activemq.transport.TransportFilter.onCommand
> >>>>>> (TransportFilter.java:65)
> >>>>>> at
> >>>>>> org.apache.activemq.transport.WireFormatNegotiator.onCommand
> >>>>>> (WireFormatNegotiator.java:133)
> >>>>>> at
> >>>>>> org.apache.activemq.transport.InactivityMonitor.onCommand
> >>>>>> (InactivityMonitor.java:122)
> >>>>>> at
> >>>>>> org.apache.activemq.transport.TransportSupport.doConsume
> >>>>>> (TransportSupport.java:84)
> >>>>>> at org.apache.activemq.transport.tcp.TcpTransport.run
> >>>>>> (TcpTransport.java:136)
> >>>>>> at java.lang.Thread.run(Thread.java:595)
> >>>>>> 19:54:45,284 DEBUG [Transport] Transport failed:
> >>>>>> java.io.IOException:
> >>>>>> Unknown data type: 68
> >>>>>> java.io.IOException: Unknown data type: 68
> >>>>>> at
> >>>>>> org.apache.activemq.openwire.OpenWireFormat.doUnmarshal
> >>>>>> (OpenWireFormat.java:344)
> >>>>>> at org.apache.activemq.openwire.OpenWireFormat.unmarshal
> >>>>>> (OpenWireFormat.java:275)
> >>>>>> at org.apache.activemq.transport.tcp.TcpTransport.run
> >>>>>> (TcpTransport.java:135)
> >>>>>> at java.lang.Thread.run(Thread.java:595)
> >>>>>> </snip>
> >>>>>>
> >>>>>> I'm using the latest 4.1-incubator-SNAPSHOT in m2:
> >>>>>>
> >>>>>> <dependency>
> >>>>>> <groupId>org.apache.activemq</groupId>
> >>>>>> <artifactId>activeio-core</artifactId>
> >>>>>> <version>3.0-SNAPSHOT</version>
> >>>>>> </dependency>
> >>>>>>
> >>>>>> <dependency>
> >>>>>> <groupId>org.apache.activemq</groupId>
> >>>>>> <artifactId>activemq-core</artifactId>
> >>>>>> <version>4.1-incubator-SNAPSHOT</version>
> >>>>>> </dependency>
> >>>>>>
> >>>>>> * * *
> >>>>>>
> >>>>>> I really want to get this broker network working so that the
> >>>>>> client
> >>>>>> code can be free from reconnect fluff... and just assume that the
> >>>>>> local embedded broker will handle all inter-broker
> >>>>>> connectivity issues.
> >>>>>>
> >>>>>> Anyone have any idea what's wrong?
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> --jason
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Regards,
> >>>>> Hiram
> >>>>>
> >>>>> Blog: http://hiramchirino.com
> >>>>
> >>>
> >>
> >
>
>
--
Regards,
Hiram
Blog: http://hiramchirino.com
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic