[prev in list] [next in list] [prev in thread] [next in thread] 

List:       activemq-dev
Subject:    Re: [DISCUSS] Artemis addressing improvements, JMS component removal and potential 2.0.0
From:       Clebert Suconic <clebert.suconic () gmail ! com>
Date:       2016-11-28 19:36:37
Message-ID: CAKF+bsrC3NrCAJXuqCUY8FcrCo0EVLx_FYO4pwzh821+9bnU5w () mail ! gmail ! com
[Download RAW message or body]

If / when we do the 2.0 bump, I would like to move a few classes.
Mainly under server.impl... I would like to move activations under a
package for activation, replicationendpoints for a package for
replications...    some small stuff like that just to reorganize
little things like this a bit.

We can't do that now as that would break API and compatibility, but if
we do the bump, I would like to make that simple move.

On Thu, Nov 24, 2016 at 4:41 AM, Martyn Taylor <mtaylor@redhat.com> wrote:
> Hi Matt,
>
> Comments inline.
>
> On Mon, Nov 21, 2016 at 7:02 PM, Matt Pavlovich <mattrpav@gmail.com> wrote:
>
>> Martyn-
>>
>> I think you nailed it here-- well done =)
>>
>> My notes in-line--
>>
>> On 11/21/16 10:45 AM, Martyn Taylor wrote:
>>
>>> 1. Ability to route messages to queues with the same address, but
>>> different
>>> routing semantics.
>>>
>>> The proposal outlined in ARTEMIS-780 outlines a new model that introduces
>>> an address object at the configuration and management layer. In the
>>> proposal it is not possible to create 2 addresses with different routing
>>> types. This causes a problem with existing clients (JMS, STOMP and for
>>> compatability with other vendors).
>>>
>>> Potential Modification: Addresses can have multiple routing type
>>> "endpoints", either "multicast" only, "anycast" only or both. The example
>>> below would be used to represent a JMS Topic called "foo", with a single
>>> subscription queue and a JMS Queue called "foo". N.B. The actual XML is
>>> just an example, there are multiple ways this could be represented that we
>>> can define later.
>>>
>>> <*addresses*>   <*address* *name**="foo"*>      <*anycast*>
>>> <*queues*>            <*queue* *name**="**foo"* />         </*queues*>
>>>       </*anycast*>      <*mulcast*>         <*queues*>
>>> <*queue* *name**="my.topic.subscription" */>         </*queues*>
>>> </*multicast*>   </*address*></*addresses*>
>>>
>> I think this solves it. The crux of the issues (for me) boils down to
>> auto-creation of destinations across protocols. Having this show up in the
>> configs would give developers and admins more information to troubleshoot
>> the mixed address type+protocol scenario.
>>
>> 2. Sending to "multicast", "anycast" or "all"
>>>
>>> As mentioned earlier JMS (and other clients such as STOMP via prefixing)
>>> allow the producer to identify the type of end point it would like to send
>>> to.
>>>
>>> If a JMS client creates a producer and passes in a topic with address
>>> "foo". Then only the queues associated with the "multicast" section of the
>>> address. A similar thing happens when the JMS producer sends to a "queue"
>>> messages should be distributed amongst the queues associated with the
>>> "anycast" section of the address.
>>>
>>> There may also be a case when a producer does not identify the endpoint
>>> type, and simply sends to "foo". AMQP or MQTT may want to do this. In this
>>> scenario both should happen. All the queues under the multicast section
>>> get
>>> a copy of the message, and one queue under the anycast section gets the
>>> message.
>>>
>>> Modification: None Needed. Internal APIs would need to be updated to allow
>>> this functionality.
>>>
>> I think the "deliver to all" scenario should be fine. This seems analogous
>> to a CompositeDestination in ActiveMQ 5.x. I'll map through some scenarios
>> and report back any gotchas.
>>
>> 3. Support for prefixes to identify endpoint types
>>>
>>> Many clients, ActiveMQ 5.x, STOMP and potential clients from alternate
>>> vendors, identify the endpoint type (in producer and consumer) using a
>>> prefix notation.
>>>
>>> e.g. queue:///foo
>>>
>>> Which would identify:
>>>
>>> <*addresses*>   <*address* *name**="foo"*>      <*anycast*>
>>> <*queues*>            <*queue* *name**="my.foo.queue" */>
>>> </*queues*>      </*anycast*>   </*address*></*addresses*>
>>>
>>> Modifications Needed: None to the model. An additional parameter to the
>>> acceptors should be added to identify the prefix.
>>>
>> Just as a check point in the syntax+naming convention in your provided
>> example... would the name actually be:
>>
>> <*queue* *name**="foo" .. vs "my.foo.queue" ?
>>
> The queue name can be anything.  It's the address that is used by
> consumer/producer.  The protocol handler / broker will decided which queue
> to connect to.
>
>>
>> 4. Multiple endpoints are defined, but client does not specify "endpoint
>>> routing type" when consuming
>>>
>>> Handling cases where consumers does not pass enough information in their
>>> address or via protocol specific mechanisms to identify an endpoint. Let's
>>> say an AMQP client, requests to subscribe to the address "foo", but passes
>>> no extra information. In the cases where there are only a single endpoint
>>> type defined, the consumer would associated with that endpoint type.
>>> However, when both endpoint types are defined, the protocol handler does
>>> not know whether to associate this consumer with a queue under the
>>> "anycast" section, or whether to create a new queue under the "multicast"
>>> section. e.g.
>>>
>>> Consume: "foo"
>>>
>>> <*addresses*>
>>>
>>>     <*address* *name**="foo"*>      <*anycast*>         <*queues*>
>>>        <*queue* *name**="**foo"* />         </*queues*>
>>> </*anycast*>      <*multicast*>         <*queues*>            <*queue*
>>> *name**="my.topic.subscription" */>         </*queues*>
>>> </*multicast*>   </*address*></*addresses*>
>>>
>>> In this scenario, we can make the default configurable on the
>>> protocol/acceptor. Possible options for this could be:
>>>
>>> "multicast": Defaults multicast
>>>
>>> "anycast": Defaults to anycast
>>>
>>> "error": Returns an error to the client
>>>
>>> Alternatively each protocol handler could handle this in the most sensible
>>> way for that protocol. MQTT might default to "multicast", STOMP "anycast",
>>> and AMQP to "error".
>>>
>>
>> Yep, this works great. I think there are two flags on the acceptors.. one
>> for auto-create and one for default handling of name collision. The
>> defaults would most likely be the same.
>>
>> Something along the lines of:
>> auto-create-default = "multicast | anycast"
>> no-prefix-default = "multicast | anycast | error"
>>
>> 5. Fully qualified address names
>>>
>>> This feature allows a client to identify a particular address on a
>>> specific
>>> broker in a cluster. This could be achieved by the client using some form
>>> of address as:
>>>
>>> queue:///host/broker/address/
>>>
>>> Matt could you elaborate on the drivers behind this requirement please.
>>>
>>> I am of the opinion that this is out of the scope of the addressing
>>> changes, and is more to do with redirecting in cluster scenarios. The
>>> current model will support this address syntax if we want to use it in the
>>> future.
>>>
>> I agree that tackling the impl of this should be out-of-scope. My
>> recommendation is to consider it in addressing now, so we can hopefully
>> avoid any breakage down the road.
>>
>> A widely used feature in other EMS brokers (IBM MQ, Tibco EMS, etc) is the
>> ability to fully address a destination using a format similar to this:
>>
>> queue://brokerB/myQueue
>>
> The advantage of this is to allow for scaling of the number of destinations
>> and allows for more dynamic broker networks to be created without
>> applications having to have connection information for all brokers in a
>> broker network. Think simple delivery+routing, and not horizontal scaling.
>> It is very analogous to SMTP mail routing.
>>
>> Producer behavior:
>>
>> 1. Client X connects to brokerA and sends it a message addressed:
>> queue://brokerB/myQueue
>> 2. brokerA accepts the message on behalf of brokerB and handles all
>> acknowledgement and persistence accordingly
>> 3. brokerA would then store the message in a "queue" for brokerB. Note:
>> All messages for brokerB are generally stored in one queue-- this is how it
>> helps with destination scaling
>>
>> Broker to broker behavior:
>>
>> There are generally two scenarios: always-on or periodic-check
>>
>> In "always-on"
>> 1. brokerA looks for a brokerB in its list of cluster connections and then
>> sends all messages for all queues for brokerB (or brokerB pulls all
>> messages, depending on cluster connection config)
>>
>> In "periodic-check"
>> 1. brokerB connects to brokerA (or vice-versa) on a given time interval
>> and then receives any messages that have arrived since last check
>>
>> TL;DR;
>>
>> It would be cool to consider remote broker delivery for messages while
>> refactoring the address handling code. This would bring Artemis inline with
>> the rest of the commercial EMS brokers. The impact now, hopefully, is minor
>> and just thinking about default prefixes.
>>
> Understood, from our conversations on IRC I can see why this might be
> useful.
>
>>
>> Thanks,
>> -Matt
>>
>>
>>



-- 
Clebert Suconic
[prev in list] [next in list] [prev in thread] [next in thread]