[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    Re: cannot create files in hdfs when -put command issued on a datanode which is in exclude list
From:       Rainer Toebbicke <rtb () pclella ! cern ! ch>
Date:       2015-01-28 10:06:59
Message-ID: 5D41D02C-8EE2-434B-983D-22E858BCDED2 () pclella ! cern ! ch
[Download RAW message or body]


Hello,

adding to this: the hbase regionserver does not survive either when it runs into that \
situation! When putting a node into "decomissioning", if a regionserver has a file \
open on that node, it dies:


2015-01-28 10:11:18,178 FATAL [regionserver60020.logRoller] \
regionserver.HRegionServer: ABORTING region server xxxxx.cern.ch,60020,1422371469606: \
                Failed log close in log roller
org.apache.hadoop.hbase.regionserver.wal.FailedLogCloseException: #1422436277964
        at org.apache.hadoop.hbase.regionserver.wal.FSHLog.cleanupCurrentWriter(FSHLog.java:787)
                
        at org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:575)
                
        at org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:97)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File \
/hbase/WALs/xxxxx.cern.ch,60020,1422371469606/xxxxx.cern.ch%2C60020%2C1422371469606.1422436277964 \
could only be replicated to 0 nodes instead of minRepl ication (=1).  There are 17 \
                datanode(s) running and 17 node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1492)
                
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3027)
                
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:614)
                
        at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.addBlock(AuthorizationProviderProxyClientProtocol.java:188)
                
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:476)
                
....


[prev in list] [next in list] [prev in thread] [next in thread]