[prev in list] [next in list] [prev in thread] [next in thread] 

List:       activemq-dev
Subject:    Re: Interesting/surprising performance wrt. NON_PERSISTENT messaging
From:       Robbie Gemmell <robbie.gemmell () gmail ! com>
Date:       2022-02-03 10:55:39
Message-ID: CAFitrpRzVQBTSRisFj6QDeZCeN_XH_5drmQsuMZ8WEvTmT03bA () mail ! gmail ! com
[Download RAW message or body]

Given it is using JMS 2 it wont yet work with the client being used here.

On Thu, 3 Feb 2022 at 09:56, Francesco Nigro <nigro.fra@gmail.com> wrote:
>
> HI Endre,
>
> you can try using https://github.com/apache/activemq-artemis/pull/3857 th=
e
> perf client tool to quickly check these differences too.
> Giving me some feedback on the tool eheh :P
>
> It's based on JMS 2 asynchronous send, although not so much adopted, has
> the interesting feature to allow many in-flight persistent messages to be
> sent, in a non-blocking fashion, helping to get the best from the
> Artemis journal (and disk) that naturally batches them on the disk.
>
> Il giorno gio 3 feb 2022 alle ore 10:49 Endre St=C3=B8lsvik <Endre@stolsv=
ik.com>
> ha scritto:
>
> > You are totally correct, sorry about that. I actually find the JMS API =
a
> > tad convoluted. Since I do actually change the delivery mode per messag=
e in
> > my library, I knew it was possible, and was like "ah, there it is" when=
 I
> > found that setter on the message.
> >
> > When changing to using 'producer.send(msg, deliveryMode, pri, ttl)', th=
at
> > "further finding" disappears! Great.
> >
> > The original and main point still seems to stand though. If you want me=
 to
> > send that to the user list then I'll do that.
> >
> > I've updated the repo.
> >
> > Each test is time for sending and in case of 'Transactional', committin=
g,
> > [1000] messages
> > no store, non-Transactional, non-Persistent:   [4.36516976] ms
> > no store, non-Transactional, Persistent:   [27.665534] ms
> > no store, Transactional, non-Persistent:   [29.056998] ms
> > no store, Transactional, Persistent:   [30.520276] ms
> > ALWAYS, non-Transactional, non-Persistent:   [4.41806886] ms  <-- (This=
 is
> > great, avoiding the hit.)
> > ALWAYS, non-Transactional, Persistent:   [4107.414315] ms
> > ALWAYS, Transactional, non-Persistent:   [4118.322262] ms     <-- This =
one
> > is unfortunate!
> > ALWAYS, Transactional, Persistent:   [4143.173852] ms
> > PERIODIC, non-Transactional, non-Persistent:   [4.453423] ms
> > PERIODIC, non-Transactional, Persistent:   [58.694872] ms
> > PERIODIC, Transactional, non-Persistent:   [28.286106] ms
> > PERIODIC, Transactional, Persistent:   [64.690608] ms
> >
> >
> >
> > On Tue, Feb 1, 2022 at 7:22 PM Robbie Gemmell <robbie.gemmell@gmail.com=
>
> > wrote:
> >
> > > You should post such things to the users list.
> > >
> > > Your further finding is not as interesting as it may first seem, it
> > > makes an invalid assumption that you get 'semantically exactly the
> > > same messages sent' by using the jmsMessage.setJMSDeliveryMode(..)
> > > method, which JMS defines that you do not. This method (and several
> > > others like it) is specifically not meant for application usage and
> > > actually shouldnt have any influence on what happens during send. It
> > > is only for use by a JMS implementation itself to stamp the actual
> > > details for observation after send returns, and exist to allow for th=
e
> > > case when it is operating on message objects created by another JMS
> > > implementation. Only the producer's own setting or its send method
> > > override argument has influence on the deliveryMode when sending.
> > >
> > >
> > >
> > https://docs.oracle.com/javaee/7/api/javax/jms/Message.html#setJMSDeliv=
eryMode-int-
> > >
> > > setJMSDeliveryMode(int deliveryMode)
> > > Sets the DeliveryMode value for this message.
> > >
> > > This method is for use by JMS providers only to set this field when a
> > > message is sent. This message cannot be used by clients to configure
> > > the delivery mode of the message. This method is public to allow a JM=
S
> > > provider to set this field when sending a message whose implementatio=
n
> > > is not its own.
> > >
> > >
> > > On Sat, 29 Jan 2022 at 14:20, Endre St=C3=B8lsvik <Endre@stolsvik.com=
> wrote:
> > > >
> > > > Further interesting finding:
> > > > Only the jmsProducer.setDeliveryMode(..) have any effect on the tim=
ing:
> > > If
> > > > employing the jmsMessage.setJMSDeliveryMode(..) method to get
> > > semantically
> > > > exactly the same messages sent with same DeliveryMode (that is,
> > > PERSISTENT
> > > > vs. NON_PERSISTENT), you lose out on those blazing fast
> > > "non-Transactional,
> > > > non-Persistent" timings.
> > > >
> > > > Each test is time for sending and in case of 'Transactional',
> > committing,
> > > > [1000] messages
> > > > no store, non-Transactional, non-Persistent:   [32.237903] ms   <- =
This
> > > is
> > > > no longer superfast
> > > > no store, non-Transactional, Persistent:   [27.756596] ms
> > > > no store, Transactional, non-Persistent:   [37.082023] ms
> > > > no store, Transactional, Persistent:   [35.110782] ms
> > > > ALWAYS, non-Transactional, non-Persistent:   [4108.457421] ms   <- =
...
> > > and
> > > > neither this
> > > > ALWAYS, non-Transactional, Persistent:   [4102.393857] ms
> > > > ALWAYS, Transactional, non-Persistent:   [3954.603762] ms
> > > > ALWAYS, Transactional, Persistent:   [3978.435732] ms
> > > > PERIODIC, non-Transactional, non-Persistent:   [58.224914] ms   <- =
..
> > nor
> > > > this
> > > > PERIODIC, non-Transactional, Persistent:   [81.982655] ms
> > > > PERIODIC, Transactional, non-Persistent:   [67.837032] ms
> > > > PERIODIC, Transactional, Persistent:   [58.78441] ms
> > > >
> > > > On Sat, Jan 29, 2022 at 4:26 AM Endre St=C3=B8lsvik <Endre@stolsvik=
.com>
> > > wrote:
> > > >
> > > > > Hi!
> > > > >
> > > > > I observed a performance/timing situation that I found a bit
> > > surprising,
> > > > > and made a little test for it. You may find that here:
> > > > > https://github.com/stolsvik/activemq_perf_issue
> > > > >
> > > > > When running ActiveMQ with default config, it uses KahaDB as back=
end
> > > with
> > > > > JournalDiskSyncStrategy.ALWAYS which flushes to disk after every
> > > > > operation. This is to live up to the JMS promises of persistent
> > > messaging
> > > > > where operations should actually have been stored to disk when e.=
g.
> > > > > producer.send() or session.commit() returns.
> > > > >
> > > > > This constant flushing gives a very heavy performance impact, and=
 one
> > > > > should really consider the pros and cons of using PERIODIC instea=
d.
> > > > >
> > > > > However, I was a bit surprised by an observation: If you run the =
JMS
> > > > > Session in Transactional mode, but also do
> > > > > producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT), one might
> > > believe
> > > > > that there wouldn't be a need to flush the disk, as one has
> > > specifically
> > > > > told the system that "this doesn't really matter". However, the
> > > performance
> > > > > impact from "Transactional, non-Persistent" is just as heavy as w=
ith
> > > > > non-Transactional+Persistent and Transactional+Persistent.
> > > > >
> > > > > With non-Transactional + non-Persistent, even with
> > > > > JournalDiskSyncStrategy.ALWAYS, the speed is blazing.
> > > > >
> > > > > One might argue, well don't use Transactional then. But Transacti=
onal
> > > also
> > > > > brings another feature, whereby you may receive a message, and se=
nd a
> > > > > message, and have these two operations inside a transactional
> > boundary.
> > > > > This aspect of Transactional is valuable even if though the messa=
ges
> > > are
> > > > > non-Persistent. And the broker evidently handles this logic alrea=
dy
> > > even
> > > > > without persistence, since if you say
> > > BrokerService.setPersistent(false)
> > > > > and thus turn off the entire storage of the broker, it still hand=
les
> > > this
> > > > > aspect of Transactional messaging.
> > > > >
> > > > > It would have been valuable to be able to "selectively" turn off =
this
> > > > > pretty massive hit of flushing to disk if the messages inside the
> > > > > Transaction was non-Persistent.
> > > > >
> > > > > Note: The Java class in this repo uses an in-vm broker, with the
> > > "vm://"
> > > > > connection. However, this situation was first observed with a rem=
ote
> > > > > broker, so the effects of interest are the same over TCP or over =
VM.
> > > > >
> > > > >  Each test is time for sending and in case of 'Transactional',
> > > committing, [1000] messages
> > > > >
> > > > >  no store, non-Transactional, non-Persistent:   [4.26321216] ms
> > > > >  no store, non-Transactional, Persistent:   [26.784713] ms
> > > > >  no store, Transactional, non-Persistent:   [32.404009] ms
> > > > >  no store, Transactional, Persistent:   [46.314925] ms
> > > > >  ALWAYS, non-Transactional, non-Persistent:   [3.9833712400000003=
] ms
> > > > >  ALWAYS, non-Transactional, Persistent:   [4235.887625] ms
> > > > >  ALWAYS, Transactional, non-Persistent:   [4040.549101] ms    <--
> > This
> > > one is unfortunate!
> > > > >  ALWAYS, Transactional, Persistent:   [3902.525469] ms
> > > > >  PERIODIC, non-Transactional, non-Persistent:   [3.85038125] ms
> > > > >  PERIODIC, non-Transactional, Persistent:   [78.012904] ms
> > > > >  PERIODIC, Transactional, non-Persistent:   [39.030052] ms
> > > > >  PERIODIC, Transactional, Persistent:   [94.339609] ms
> > > > >
> > > > >
> > > > >
> > >
> >
[prev in list] [next in list] [prev in thread] [next in thread]