[prev in list] [next in list] [prev in thread] [next in thread] 

List:       activemq-users
Subject:    Re: consuming artemis with a reactive stream
From:       Matthieu Baechler <matthieu () apache ! org>
Date:       2020-06-12 6:55:46
Message-ID: f073e03774e486877af4209867429e652e8def48.camel () apache ! org
[Download RAW message or body]

Hi Howard,

Thank you for your answer.

On Fri, 2020-06-12 at 10:47 +0800, Howard Gao wrote:
> If I understand it right, you are using concurrent consumers that
> share the
> session. The session is not thread safe.
> You should use one consumer per session.

We are aware of that, we built the reactive stream in a way that
respect that requirement:

* an Akka actor is running in a fixed thread and receive the messages
from the message queue

* it then read the payload (the actor is practically the only consumer)
and put in the stream alongside a correlation id that the worker can
use to ask the actor the ack or nack a given message

* the message sending is also handled by the same Akka actor


Does it make sense?

I understand that it's not the way the driver is meant to be used to it
should be a decent design for adapting it to a reactive stream.

We just lack with individual nack thing.

Cheers,

-- Matthieu Baechler


> 
> On Thu, Jun 11, 2020 at 3:14 PM Matthieu Baechler <
> matthieu@apache.org>
> wrote:
> 
> > Hi folks,
> > 
> > I work on Apache James, the JVM mail server, for some years.
> > 
> > We implemented our Mail Queue using ActiveMQ a long time ago.
> > 
> > The code is not really nice and the performances are not great (if
> > you
> > are curious you can look at it here
> > 
> > https://github.com/apache/james-project/tree/master/server/queue/queue-activemq
> > )
> > 
> > James is using reactive streams more and more to enable good
> > performances, so I decided to rewrite our ActiveMQ Mail Queue using
> > Akka Stream.
> > 
> > We start to have a working implementation using artemis core
> > protocol
> > but we fall today on an unexpected problem.
> > 
> > For our streaming architecture, we take advantage of async handling
> > of
> > messages using a `MessageHandler`. We have a single thread source
> > that
> > receive messages from the driver and push them in a stream.
> > 
> > We then have many subscribers (workers) to the stream because
> > handling
> > email is a heavy process.
> > 
> > Finally, we ack each message individually when a worker succeed at
> > handling the mail.
> > 
> > This is the happy path and we found what we want in the driver API
> > for
> > this.
> > 
> > However, we didn't found a way to handle the failure path: when a
> > worker fails, we are supposed to "nack" the message individually to
> > allow another worker to take it from the queue.
> > 
> > The only thing we found is that we can rollback the entire session.
> > As
> > there's, by design, a single session open for the stream source,
> > doing
> > a rollback would nack some messages that are being process, right?
> > 
> > We looked at the wire package to understand the protocol and didn't
> > find any solution.
> > 
> > Is there any solution to this specific issue? What would you advise
> > us
> > to do?
> > 
> > Cheers,
> > 
> > -- Matthieu Baechler
> > 
> > 

[prev in list] [next in list] [prev in thread] [next in thread]