[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-dev
Subject:    [jira] [Created] (HADOOP-14262) rpcTimeOut is not set up correctly in Client thus client doesn't tim
From:       "Yongjun Zhang (JIRA)" <jira () apache ! org>
Date:       2017-03-31 8:09:41
Message-ID: JIRA.13060601.1490947772000.173431.1490947781707 () Atlassian ! JIRA
[Download RAW message or body]

Yongjun Zhang created HADOOP-14262:
--------------------------------------

             Summary: rpcTimeOut is not set up correctly in Client thus client \
doesn't time out  Key: HADOOP-14262
                 URL: https://issues.apache.org/jira/browse/HADOOP-14262
             Project: Hadoop Common
          Issue Type: Bug
            Reporter: Yongjun Zhang
            Assignee: Yongjun Zhang


NameNodeProxies.createNNProxyWithClientProtocol  does

{code}
  ClientNamenodeProtocolPB proxy = RPC.getProtocolProxy(
        ClientNamenodeProtocolPB.class, version, address, ugi, conf,
        NetUtils.getDefaultSocketFactory(conf),
        org.apache.hadoop.ipc.Client.getTimeout(conf), defaultPolicy,
        fallbackToSimpleAuth).getProxy();
{code}
which calls Client.getTimeOut(conf) to get timeout value. 

Client.getTimeOut(conf) doesn't consider IPC_CLIENT_RPC_TIMEOUT_KEY right now. Thus \
rpcTimeOut doesn't take effect for relevant RPC calls, and they hang!

For example, receiveRpcResponse blocked forever at:
{code}
Thread 16127: (state = BLOCKED)                                                       \
                
 - sun.nio.ch.SocketChannelImpl.readerCleanup() @bci=6, line=279 (Compiled frame)     \
                
 - sun.nio.ch.SocketChannelImpl.read(java.nio.ByteBuffer) @bci=205, line=390 \
                (Compiled frame)       
 - org.apache.hadoop.net.SocketInputStream$Reader.performIO(java.nio.ByteBuffer) \
                @bci=5, line=57 (Compiled frame)
 - org.apache.hadoop.net.SocketIOWithTimeout.doIO(java.nio.ByteBuffer, int) @bci=35, \
                line=142 (Compiled frame)
 - org.apache.hadoop.net.SocketInputStream.read(java.nio.ByteBuffer) @bci=6, line=161 \
                (Compiled frame)
 - org.apache.hadoop.net.SocketInputStream.read(byte[], int, int) @bci=7, line=131 \
                (Compiled frame) 
 - java.io.FilterInputStream.read(byte[], int, int) @bci=7, line=133 (Compiled frame) \
                
 - java.io.FilterInputStream.read(byte[], int, int) @bci=7, line=133 (Compiled frame) \
                
 - org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(byte[], int, int) \
                @bci=4, line=521 (Compiled frame)
 - java.io.BufferedInputStream.fill() @bci=214, line=246 (Compiled frame)             \
                
 - java.io.BufferedInputStream.read() @bci=12, line=265 (Compiled frame)              \
                
 - java.io.DataInputStream.readInt() @bci=4, line=387 (Compiled frame)                \
                
 - org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse() @bci=19, line=1081 \
                (Compiled frame) 
 - org.apache.hadoop.ipc.Client$Connection.run() @bci=62, line=976 (Compiled frame)   \
 {code}

Filing this jira to fix it.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


[prev in list] [next in list] [prev in thread] [next in thread]