[prev in list] [next in list] [prev in thread] [next in thread] 

List:       avro-dev
Subject:    [jira] [Updated] (AVRO-1027) NettyTransceiver will deadlock when attempting transceive/disconnect on
From:       "James Baldassari (Updated) (JIRA)" <jira () apache ! org>
Date:       2012-02-25 23:01:49
Message-ID: 1137428417.20181.1330210909282.JavaMail.tomcat () hel ! zones ! apache ! org
[Download RAW message or body]


     [ https://issues.apache.org/jira/browse/AVRO-1027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel \
]

James Baldassari updated AVRO-1027:
-----------------------------------

    Fix Version/s: 1.6.3
           Status: Patch Available  (was: Open)
    
> NettyTransceiver will deadlock when attempting transceive/disconnect on the same \
>                 thread
> ---------------------------------------------------------------------------------------
>  
> Key: AVRO-1027
> URL: https://issues.apache.org/jira/browse/AVRO-1027
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.6.1
> Reporter: Simon Wilkinson
> Assignee: James Baldassari
> Fix For: 1.6.3
> 
> Attachments: AVRO-1027-v2.patch, AVRO-1027.patch
> 
> 
> If an Exception is caught while trying to write to a Channel, Netty can deliver the \
> Exception to a ChannelUpstreamHandler on the same thread that attempted to write to \
> the Channel. If this occurs with the NettyClientAvroHandler implementation of \
> ChannelUpstreamHandler then the thread will deadlock. Specifically, \
> NettyClientAvroHandler overrides the ChannelUpstreamHandler.exceptionCaught() \
> method to perform a disconnect, which requires the NettyTransceiver's write lock. \
> However, in the above situation, the thread will already have locked the \
> NettyTransceiver's read lock to write to the Channel. ReentrantReadWriteLock does \
> not allow upgrading from a read to a write lock, hence the thread deadlocks. \
> Example stack trace (simplified): "SessionManager-TimeoutPoller" prio=10 \
>                 tid=0x7b689c00 nid=0x375d waiting on condition \
>                 [0x7b0ad000..0x7b0ade70]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for  <0xf2a944d8> (a \
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) at \
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) at \
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:747)
>  at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:778)
>  at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1114)
>  at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:807)
> 
> > > > [Acquire write lock] at \
> > > > org.apache.avro.ipc.NettyTransceiver.disconnect(NettyTransceiver.java:285)
> at org.apache.avro.ipc.NettyTransceiver.access$2(NettyTransceiver.java:281)
> at org.apache.avro.ipc.NettyTransceiver$NettyClientAvroHandler.exceptionCaught(NettyTransceiver.java:499)
>  at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122)
>  at org.apache.avro.ipc.NettyTransceiver$NettyClientAvroHandler.handleUpstream(NettyTransceiver.java:473)
>  at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>  at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783)
>  at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:238)
>  at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122)
>  at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>  at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
>  at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:432)
> at org.jboss.netty.channel.socket.nio.NioWorker.cleanUpWriteBuffer(NioWorker.java:661)
>  at org.jboss.netty.channel.socket.nio.NioWorker.writeFromUserCode(NioWorker.java:372)
>  at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:117)
>  at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:771)
>  at org.jboss.netty.channel.Channels.write(Channels.java:632)
> at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:70)
>  at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591)
>  at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582)
>  at org.jboss.netty.channel.Channels.write(Channels.java:611)
> at org.jboss.netty.channel.Channels.write(Channels.java:578)
> at org.jboss.netty.channel.AbstractChannel.write(AbstractChannel.java:251)
> > > > [Acquire read lock] at \
> > > > org.apache.avro.ipc.NettyTransceiver.writeDataPack(NettyTransceiver.java:413) \
> > > > [Acquire read lock] at \
> > > > org.apache.avro.ipc.NettyTransceiver.transceive(NettyTransceiver.java:394)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:147)
> at org.apache.avro.ipc.Requestor.request(Requestor.java:129)
> at org.apache.avro.ipc.specific.SpecificRequestor.invoke(SpecificRequestor.java:68)
> <snip>
> Note, in Avro 1.6.1 the read lock is acquired in both NettyTransceiver.transceive() \
> and NettyTransceiver.writeDataPack(). AVRO-1013 fixes this so that it is acquired \
> only once in NettyTransceiver.transceive(). I've attached a patch that demonstrates \
> a potential fix for the deadlock; the patch assumes that AVRO-1013 has also been \
> applied.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: \
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more \
information on JIRA, see: http://www.atlassian.com/software/jira

        


[prev in list] [next in list] [prev in thread] [next in thread]