[prev in list] [next in list] [prev in thread] [next in thread] 

List:       activemq-dev
Subject:    Re: Asynchronous Replication plugin
From:       Nikita Shupletsov <nshuplet () amazon ! com ! INVALID>
Date:       2022-09-20 0:10:04
Message-ID: 634fafde-99ed-bff9-8244-db3679c17d40 () amazon ! com
[Download RAW message or body]

I remeasured with and without the added transactions to double check.

The setup: the broker was hosted on a separate machine. I was sending 
and receiving messages from my local machine. 100 connections for 
reading, 100 connections for writing.

The results:
without the transactions: 515 TPS for writing, 480 TPS for reading

with the transactions: 530 TPS for writing, 430 TPS for reading.

To be clear, we write a persistent message to the replication queue on 
each event(send message, ack message and so on), hence the overhead of 
the transactions is not that noticeable in comparison to the extra write 
to the disk.

On 9/15/22 12:48, Matt Pavlovich wrote:
> CAUTION: This email originated from outside of the organization. Do not click links \
> or open attachments unless you can confirm the sender and know the content is safe. \
>  
> 
> Can you share the results of the testing?  I'm having a hard time tracking how \
> adding the transaction does not incur latency. 
> > On Sep 14, 2022, at 2:03 PM, Nikita Shupletsov <nshuplet@amazon.com.INVALID> \
> > wrote: 
> > Yeah, I agree that there is some overhead because of transactions. but our tests \
> > didn't show any significant impact on the performance. 
> > We use JMS transactions, not XA, so there is no transaction prepare operations. \
> > From my understanding nothing gets written to disk for non-persistent messages \
> > regardless the transaction they are in. 
> > And our tests also didn't show any issues with non-persistent messages.
> > 
> > On 9/13/22 18:31, Étienne Hossack wrote:
> > > CAUTION: This email originated from outside of the organization. Do not click \
> > > links or open attachments unless you can confirm the sender and know the \
> > > content is safe. 
> > > 
> > > 
> > > An issue one could possible see with adding transactions is the memory and \
> > > speed overhead. 
> > > Have you tested this with very large messages or high throughput? What's the \
> > > impact there? I would expect that because the transaction "prepare"s MUST be \
> > > written to disk, it would limit the modes in which this could be run (not that \
> > > it really would make sense to enable replication for a broker with in-memory \
> > > only messages), but you couldn't for example have delayed paging right?. 
> > > --
> > > Étienne
> > > 
> > > On Tue, 13 Sep 2022, at 4:55 PM, Nikita Shupletsov wrote:
> > > > Greetings.
> > > > 
> > > > It's a gentle reminder of the change I have been working on - an
> > > > Asynchronous Replication plugin for ActiveMQ "Classic"
> > > > https://issues.apache.org/jira/browse/AMQ-8354.
> > > > The latest changes can be found here:
> > > > https://github.com/apache/activemq/pull/848. I would really appreciate
> > > > comments and feedback.
> > > > 
> > > > We started wrapping the original message and the corresponding
> > > > replication message in a transaction(if the original message already has
> > > > a transaction, we will add the replication message to the same
> > > > transaction. If not, we will create a new one). The idea behind is to
> > > > make both writes atomic so that we can preserve consistency.
> > > > I would be glad to get any opinion on that.
> > > > 
> > > > We are still in the phase of active development and bug fixing, so any
> > > > input is welcome!
> > > > 
> > > > On 6/10/22 17:15, Nikita Shupletsov wrote:
> > > > > Greetings.
> > > > > 
> > > > > My name is Nikita Shupletsov. I am a software development engineer at
> > > > > Amazon MQ.
> > > > > 
> > > > > I have been working on an Asynchronous Replication plugin for ActiveMQ
> > > > > "Classic" that was proposed by Étienne Hossack last year:
> > > > > https://issues.apache.org/jira/browse/AMQ-8354. Étienne published a
> > > > > high level design document
> > > > > (https://github.com/ehossack-aws/activemq-replication-design/tree/initial-design)
> > > > >  though I'm not sure if many of you would have reviewed it or provided
> > > > > feedback.
> > > > > 
> > > > > I have published my latest changes
> > > > > here:https://github.com/amazon-mq/upstreaming-activemq/pull/1 and
> > > > > would appreciate comments and feedback from anyone.
> > > > > 
> > > > > To be perfectly clear, this repository is only going to be used for
> > > > > working on changes that we (Amazon MQ) plan to submit upstream to the
> > > > > Apache ActiveMQ community. We are not planning to maintain a fork of
> > > > > ActiveMQ - the purpose is solely to upstream changes.
> > > > > 
> > > > > Currently the plugin supports:
> > > > > 
> > > > > * create/delete destinations
> > > > > * send message to queues and topics(except temporary destinations
> > > > > and advisory topics)
> > > > > * durable subscribers
> > > > > * message acknowledge for queues and topic durable subscribers
> > > > > * transactions and XA transaction
> > > > > 
> > > > > We are still working on the infrastructure part of the story such as:
> > > > > 
> > > > > * error handling and message retries
> > > > > * JMX metrics to monitor the health of the replication
> > > > > * the plugin structure
> > > > > * testing, looking for missed functionality and bugs
> > > > > 
> > > > > 
> > > > > We are still in the phase of active development, so any early input is
> > > > > welcome!


[prev in list] [next in list] [prev in thread] [next in thread]