[prev in list] [next in list] [prev in thread] [next in thread] 

List:       activemq-dev
Subject:    [jira] [Closed] (AMQ-3593) Messages with empty marshalledProperties
From:       "Timothy Bish (JIRA)" <jira () apache ! org>
Date:       2013-09-30 19:38:25
Message-ID: JIRA.12531359.1321297496800.5744.1380569905984 () arcas
[Download RAW message or body]


     [ https://issues.apache.org/jira/browse/AMQ-3593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel \
]

Timothy Bish closed AMQ-3593.
-----------------------------

    Resolution: Cannot Reproduce

There's no reproducible test case and no other reports of seeing this.  Testing with \
a recent 5.9-SNAPSHOT might show if the fixes for the concurrent store and dispatch \
functionality when using message groups is your issue or not.  

> Messages with empty marshalledProperties
> ----------------------------------------
> 
> Key: AMQ-3593
> URL: https://issues.apache.org/jira/browse/AMQ-3593
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.4.2, 5.5.1
> Environment: * OS: Linux version 2.6.9-67.0.4.ELlargesmp \
> (brewbuilder@hs20-bc1-6.build.redhat.com) (gcc version 3.4.6 20060404 (Red Hat \
>                 3.4.6-9)) #1 SMP Fri Jan 18 05:10:01 EST 2008
> * JVM (-server option in use): Java(TM) SE Runtime Environment (build 1.6.0_24-b07) \
>                 Java HotSpot(TM) Server VM (build 19.1-b02, mixed mode)
> Reporter: Luca Zenti
> Attachments: PublisherAndBroker.java, Subscriber.java
> 
> 
> Sometimes messages arrive on the client with an empty "marshalledProperties" field. \
> This causes an exception trying to unmarshall them. This happens very rarely, but \
> we've got at least 2-3 cases a day per subscriber in our production environment. We \
> use a broker embedded with a server component that publishes messages on topics. \
> The subscribers are remote. I was unable to replicate the problem on my Windows \
> workstation, nor to reproduce it precisely. Attached you can find a simple pair of \
> programs (the publisher and the subscriber) that can reproduce the issue within 5 \
> minutes at most on my environment. This happens only if at least one of the \
> subscribers runs on a remote machine and it seems to happen quickly with more \
> clients connected. The publishing program has a command line arguments that allows \
> to adjust the publishing rate, in my case with 10,000 messages per second and 2 \
> clients the problem happens within a couple of minutes. I also found a workaround \
> for this problem: as far as I can understand, my original message is copied as a \
> Java object to the broker (the broker is embedded, so there is no need to serialize \
> it), then it is copied using its copy method to one or more new ActiveMQMessage \
> instances and these are pushed into the consumer queues and then serialized to be \
> sent over tcp. I think that under some particular conditions the properties are not \
> serialized into the marshalledProperties field or this happens after the message \
> has been actually sent. My workaround is to force the marshalling of properties \
> into the marshalledProperties field when the message is copied using the copy \
> method. This is not very clean, but still acceptable. The problem is that, in order \
> for this to work, I need to create messages of a subclass of ActiveMqBytesMessage \
> using an explicit constructor call rather that session.createBytesMessage() and I \
> need to set the connection into them. Here is the code of my redefined class (this \
> is the BytesMessage version, but the same is applicable to other kinds of \
> messages): {noformat}
> import java.io.DataOutputStream;
> import java.io.IOException;
> import javax.jms.Session;
> import org.apache.activemq.ActiveMQSession;
> import org.apache.activemq.command.ActiveMQBytesMessage;
> import org.apache.activemq.command.Message;
> import org.apache.activemq.util.ByteArrayOutputStream;
> import org.apache.activemq.util.MarshallingSupport;
> public class EarlySerializingBytesMessage extends ActiveMQBytesMessage
> {
> private static final org.apache.log4j.Logger logger = \
> org.apache.log4j.Logger.getLogger(EarlySerializingBytesMessage.class); 
> public EarlySerializingBytesMessage(Session session)
> {
> ActiveMQSession activeMqSession = (ActiveMQSession)session;
> setConnection(activeMqSession.getConnection());
> }
> 
> /*
> * We redefine <code>copy</code> to force the marshalling of properties here and \
>                 avoid a problem we've found with serialized properties 
> * arriving empty on the client side.
> * This is a workaround to make sure the properties are already serialized when \
>                 ActiveMQ actually sends the message to
> * the client.
> * This happens very rarely and it is very difficult to spot this error in a \
>                 "normal" testing session.
> * To have reasonable changes to reproduce it, we use a stress test tool that sends \
>                 5000 messages a second and at that rate
> * the error happens usually within 10 minutes, so we can say that one message every \
>                 million is typically affected.
> * The actual problem is probably due to a bug in the ActiveMQ broker. Basically, it \
>                 serializes every message just before
> * sending it and it happens that the marshalled properties are missing the \
>                 resulting message, so we get an error on the client
> * as soon as we try to read them.
> * Since we use an embedded broker, our original message is never serialized, it is \
>                 passed into the publishing queue as a Java object,
> * then it is copied into a new message of the same base type (ActiveMQBytesMessage, \
>                 in this case) and pushed into the subscribers' 
> * queues (a topic is actually implemented as a series of queues, one for each \
>                 subscriber). In order to do this, the <code>copy</code>
> * method is invoked, we redefine it to early serialize the properties and make sure \
>                 that this step is not skipped.
> */
> @Override
> public Message copy()
> {
> try
> {
> ByteArrayOutputStream baos = new ByteArrayOutputStream();
> DataOutputStream os = new DataOutputStream(baos);
> MarshallingSupport.marshalPrimitiveMap(properties, os);
> os.close();
> marshalledProperties = baos.toByteSequence();
> }
> catch(IOException exc)
> {
> logger.error("Error marhalling message properties, message was " + this, exc);
> }
> 
> return super.copy();
> }
> }
> {noformat}
> The problem happens both on Java and C# clients, with the following stacktraces:
> Java:
> {noformat}
> javax.jms.JMSException: java.io.EOFException
> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:199)
>  at org.apache.activemq.command.ActiveMQMessage.getObjectProperty(ActiveMQMessage.java:509)
>  at org.apache.activemq.command.ActiveMQMessage.getStringProperty(ActiveMQMessage.java:604)
>  at activemq.emptypropertiestest.Subscriber$1.onMessage(Subscriber.java:65)
> at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1230)
>  at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:134)
>  at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:205)
>  at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:127)
> at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:83)
>  at org.apache.activemq.util.MarshallingSupport.unmarshalPrimitiveMap(MarshallingSupport.java:73)
>  at org.apache.activemq.command.Message.unmarsallProperties(Message.java:202)
> at org.apache.activemq.command.Message.getProperty(Message.java:159)
> at org.apache.activemq.filter.PropertyExpression.evaluate(PropertyExpression.java:197)
>                 
> ... 11 more
> {noformat}
> C#:
> {noformat}
> System.IO.EndOfStreamException: Unable to read beyond the end of the stream.
> at System.IO.MemoryStream.InternalReadInt32()
> at System.IO.BinaryReader.ReadInt32()
> at Apache.NMS.Util.EndianBinaryReader.ReadInt32()
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(BinaryReader dataIn)
> at Apache.NMS.Util.PrimitiveMap.UnmarshalPrimitiveMap(Byte[] data)
> at Apache.NMS.Util.PrimitiveMap.Unmarshal(Byte[] data)
> at Apache.NMS.ActiveMQ.Commands.ActiveMQMessage.get_Properties()
> ...
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)


[prev in list] [next in list] [prev in thread] [next in thread]