[prev in list] [next in list] [prev in thread] [next in thread] 

List:       activemq-dev
Subject:    [jira] [Created] (AMQ-4239) Possible Race Condition in FilePendingMessageCursor when sending message
From:       "Joseph Chang (JIRA)" <jira () apache ! org>
Date:       2012-12-28 19:58:12
Message-ID: JIRA.12625440.1356724671829.59673.1356724692701 () arcas
[Download RAW message or body]

Joseph Chang created AMQ-4239:
---------------------------------

             Summary: Possible Race Condition in FilePendingMessageCursor when \
sending messages quickly  Key: AMQ-4239
                 URL: https://issues.apache.org/jira/browse/AMQ-4239
             Project: ActiveMQ
          Issue Type: Bug
          Components: Broker
    Affects Versions: 5.7.0
         Environment: OS           : CentOS release 6.3 (Final)
KERNEL       : 2.6.32-279.el6.x86_64
CPU          : Intel(R) Xeon(R) CPU           X3470  @ 2.93GHz
CPU COUNT    : 8
CPU TYPE     : x86_64
RAM          : 32 GB

Manufacturer : Dell Inc.
Model        : PowerEdge R310
BIOS Version : 1.8.2


            Reporter: Joseph Chang


When sending persistent messages extremely quickly, we often see the following stack \
trace.

Broker Persistence is turned on, but not using KahaDB.  Instead, using JDBC \
Persistence (MySQL).  KahaDB is commented out on the xml configuration file.

Reading through source, seems like the Kaha engine is still used for temporary \
storage of pending messages (/data/**/tmp_storage).  However, in reading through \
code, seems like the "loaded" member variable is not intialized in time, causing this \
stack to be thrown.

Any help appreciated, simply gums up our logging, and the exceptions make it back to \
the client.

[04 Dec 2012 08:02:08] [scheduler_Worker-2] 36391186 ERROR
(com.ms.jms.publisher.BaseMessagePublisher:143) -
java.lang.IllegalStateException: TheListIndex is not loaded
javax.jms.JMSException: java.lang.IllegalStateException: TheListIndex
is not loaded
        at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
                
        at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1377)
                
        at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1305)
                
        at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1795)
        at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:277)
                
        at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:212)
                
        at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:269)
                
        at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:121)
                
        at com.ms.jms.publisher.BaseMessagePublisher.talker(BaseMessagePublisher.java:133)
                
        at com.ms.jms.publisher.BaseMessagePublisher.sendMessage(BaseMessagePublisher.java:63)
                
        at com.ms.mscm.bus.ClientBasedMessagePublisher.sendMessage(ClientBasedMessagePublisher.java:169)
                
        at com.ms.cart.ScheduledCartProcessor.checkout(ScheduledCartProcessor.java:101)
                
        at com.ms.cart.ScheduledCartJob.doExecuteInternal(ScheduledCartJob.java:63)
        at com.ms.cart.ScheduledCartJob.executeInternal(ScheduledCartJob.java:50)
        at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86)
  at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
 Caused by: java.lang.RuntimeException:
java.lang.IllegalStateException: TheListIndex is not loaded
        at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:239)
                
        at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:202)
                
        at org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1669)
        at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:785)
        at org.apache.activemq.broker.region.Queue.send(Queue.java:717)
        at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:407)
                
        at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:503)
        at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:311)
                
        at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
        at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
                
        at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:317)
                
        at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
        at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
                
        at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:450)
                
        at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:680)
                
        at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294)
                
        at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:152)
                
        at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
                
        at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
                
        at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:256)
                
        at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
                
        at org.apache.activemq.transport.nio.NIOTransport.serviceRead(NIOTransport.java:138)
                
        at org.apache.activemq.transport.nio.NIOTransport$1.onSelect(NIOTransport.java:69)
                
        at org.apache.activemq.transport.nio.SelectorSelection.onSelect(SelectorSelection.java:94)
                
        at org.apache.activemq.transport.nio.SelectorWorker$1.run(SelectorWorker.java:119)
                
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
                
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalStateException: TheListIndex is not loaded
        at org.apache.kahadb.index.ListIndex.assertLoaded(ListIndex.java:301)
        at org.apache.kahadb.index.ListIndex.add(ListIndex.java:199)
        at org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:84)
        at org.apache.kahadb.page.Transaction.execute(Transaction.java:769)
        at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:82)
        at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.flushToDisk(FilePendingMessageCursor.java:438)
                
        at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:226)
                
        ... 27 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira


[prev in list] [next in list] [prev in thread] [next in thread] 

Configure | About | News | Add a list | Sponsored by KoreLogic