[prev in list] [next in list] [prev in thread] [next in thread]
List: hadoop-user
Subject: Query in hive error!
From: Tije Kusnadi <tijekusnadi () adskom ! com>
Date: 2017-11-29 3:38:59
Message-ID: CA+gdq5OwoFW4mCeSLx=TQNsdXadhn=Uh_f5ET73-Xxx-OVLE=w () mail ! gmail ! com
[Download RAW message or body]
*Diagnostic Messages for this Task:
Error: java.io.IOException: java.io.IOException: Cannot obtain block
length for LocatedBlock{BP-1781152160-172.16.0.170-1483968868061:blk_1076822902_3082215;
getBlockSize()=153218; corrupt=false; offset=0;
locs=[DatanodeInfoWithStorage[172.16.0.175:50010
<http://172.16.0.175:50010>,DS-c03666a5-2f54-4085-b635-e36b25daedca,DISK],
DatanodeInfoWithStorage[172.16.0.109:50010
<http://172.16.0.109:50010>,DS-11410139-58aa-4160-9c89-2459884e7f51,DISK]]}
at org.apache.hadoop.hive.io
<http://org.apache.hadoop.hive.io>.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
at org.apache.hadoop.hive.io
<http://org.apache.hadoop.hive.io>.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
at org.apache.hadoop.hive.ql.io
<http://org.apache.hadoop.hive.ql.io>.HiveInputFormat.getRecordReader(HiveInputFormat.java:252)
at org.apache.hadoop.hive.ql.io
<http://org.apache.hadoop.hive.ql.io>.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:703)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.io.IOException: Cannot obtain block length for
LocatedBlock{BP-1781152160-172.16.0.170-1483968868061:blk_1076822902_3082215;
getBlockSize()=153218; corrupt=false; offset=0;
locs=[DatanodeInfoWithStorage[172.16.0.175:50010
<http://172.16.0.175:50010>,DS-c03666a5-2f54-4085-b635-e36b25daedca,DISK],
DatanodeInfoWithStorage[172.16.0.109:50010
<http://172.16.0.109:50010>,DS-11410139-58aa-4160-9c89-2459884e7f51,DISK]]}
at org.apache.hadoop.hdfs.DFSInputStream.readBlockLength(DFSInputStream.java:427)
at org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:335)
at org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:271)
at org.apache.hadoop.hdfs.DFSInputStream.<init>(DFSInputStream.java:263)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1584)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:309)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:305)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:305)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:780)
at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:109)
at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
at org.apache.hadoop.hive.ql.io
<http://org.apache.hadoop.hive.ql.io>.HiveInputFormat.getRecordReader(HiveInputFormat.java:250)
... 9 more
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 8 Reduce: 3 Cumulative CPU: 29.62 sec HDFS
Read: 143256373 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 29 seconds 620 msec*
*i try running the command :*
*hdfs fsck -openforwrite -files -blocks -locations / | grep
blk_1076822902_3082215**
*the result is this:*
*
0. BP-1781152160-172.16.0.170-1483968868061:blk_1076822902_3082215{blockUCState=UNDER_CONSTRUCTION,
primaryNodeIndex=-1,
replicas=[ReplicaUnderConstruction[[DISK]DS-c03666a5-2f54-4085-b635-e36b25daedca:NORMAL:172.16.0.175:50010|RWR],
ReplicaUnderConstruction[[DISK]DS-11410139-58aa-4160-9c89-2459884e7f51:NORMAL:172.16.0.109:50010|RWR]]}
len=153218 MISSING!*
What can i do ? Looking forward for an answer!
Thanks
[Attachment #3 (text/html)]
<div dir="ltr"><pre><font color="#000000" face="courier new, courier, \
monospace"><span style="font-size:16px"><b>Diagnostic Messages for \
this Task:
Error: java.io.IOException: java.io.IOException: Cannot obtain block length for \
LocatedBlock{BP-1781152160-<wbr>172.16.0.170-1483968868061:<wbr>blk_1076822902_3082215; \
getBlockSize()=153218; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[<a \
href="http://172.16.0.175:50010" \
target="_blank"><wbr>172.16.0.175:50010</a>,DS-<wbr>c03666a5-2f54-4085-b635-<wbr>e36b25daedca,DISK], \
DatanodeInfoWithStorage[<a href="http://172.16.0.109:50010" \
target="_blank">172.<wbr>16.0.109:50010</a>,DS-11410139-<wbr>58aa-4160-9c89-2459884e7f51,<wbr>DISK]]}
at <a href="http://org.apache.hadoop.hive.io">org.apache.hadoop.hive.io</a>.<wbr>Hiv \
eIOExceptionHandlerChain.<wbr>handleRecordReaderCreationExce<wbr>ption(<wbr>HiveIOExceptionHandlerChain.<wbr>java:97)
at <a href="http://org.apache.hadoop.hive.io">org.apache.hadoop.hive.io</a>.<wbr>Hiv \
eIOExceptionHandlerUtil.<wbr>handleRecordReaderCreationExce<wbr>ption(<wbr>HiveIOExceptionHandlerUtil.<wbr>java:57)
at <a href="http://org.apache.hadoop.hive.ql.io">org.apache.hadoop.hive.ql.io</a>.<wbr>HiveInputFormat.<wbr>getRecordReader(<wbr>HiveInputFormat.java:252)
at <a href="http://org.apache.hadoop.hive.ql.io">org.apache.hadoop.hive.ql.io</a>.<w \
br>CombineHiveInputFormat.<wbr>getRecordReader(<wbr>CombineHiveInputFormat.java:<wbr>703)
at org.apache.hadoop.mapred.<wbr>MapTask$TrackedRecordReader.<<wbr>init>(MapTask.java:169)
at org.apache.hadoop.mapred.<wbr>MapTask.runOldMapper(MapTask.<wbr>java:432)
at org.apache.hadoop.mapred.<wbr>MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.<wbr>YarnChild$2.run(YarnChild.<wbr>java:164)
at java.security.<wbr>AccessController.doPrivileged(<wbr>Native Method)
at javax.security.auth.Subject.<wbr>doAs(Subject.java:422)
at org.apache.hadoop.security.<wbr>UserGroupInformation.doAs(<wbr>UserGroupInformation.java:<wbr>1698)
at org.apache.hadoop.mapred.<wbr>YarnChild.main(YarnChild.java:<wbr>158)
Caused by: java.io.IOException: Cannot obtain block length for \
LocatedBlock{BP-1781152160-<wbr>172.16.0.170-1483968868061:<wbr>blk_1076822902_3082215; \
getBlockSize()=153218; corrupt=false; offset=0; locs=[DatanodeInfoWithStorage[<a \
href="http://172.16.0.175:50010" \
target="_blank"><wbr>172.16.0.175:50010</a>,DS-<wbr>c03666a5-2f54-4085-b635-<wbr>e36b25daedca,DISK], \
DatanodeInfoWithStorage[<a href="http://172.16.0.109:50010" \
target="_blank">172.<wbr>16.0.109:50010</a>,DS-11410139-<wbr>58aa-4160-9c89-2459884e7f51,<wbr>DISK]]}
at org.apache.hadoop.hdfs.<wbr>DFSInputStream.<wbr>readBlockLength(<wbr>DFSInputStream.java:427)
at org.apache.hadoop.hdfs.<wbr>DFSInputStream.<wbr>fetchLocatedBlocksAndGetLastBl<wbr>ockLength(DFSInputStream.java:<wbr>335)
at org.apache.hadoop.hdfs.<wbr>DFSInputStream.openInfo(<wbr>DFSInputStream.java:271)
at org.apache.hadoop.hdfs.<wbr>DFSInputStream.<init>(<wbr>DFSInputStream.java:263)
at org.apache.hadoop.hdfs.<wbr>DFSClient.open(DFSClient.java:<wbr>1584)
at org.apache.hadoop.hdfs.<wbr>DistributedFileSystem$3.<wbr>doCall(DistributedFileSystem.<wbr>java:309)
at org.apache.hadoop.hdfs.<wbr>DistributedFileSystem$3.<wbr>doCall(DistributedFileSystem.<wbr>java:305)
at org.apache.hadoop.fs.<wbr>FileSystemLinkResolver.<wbr>resolve(<wbr>FileSystemLinkResolver.java:<wbr>81)
at org.apache.hadoop.hdfs.<wbr>DistributedFileSystem.open(<wbr>DistributedFileSystem.java:<wbr>305)
at org.apache.hadoop.fs.<wbr>FileSystem.open(FileSystem.<wbr>java:780)
at org.apache.hadoop.mapred.<wbr>LineRecordReader.<init>(<wbr>LineRecordReader.java:109)
at org.apache.hadoop.mapred.<wbr>TextInputFormat.<wbr>getRecordReader(<wbr>TextInputFormat.java:67)
at <a href="http://org.apache.hadoop.hive.ql.io">org.apache.hadoop.hive.ql.io</a>.<wbr>HiveInputFormat.<wbr>getRecordReader(<wbr>HiveInputFormat.java:250)
... 9 more
FAILED: Execution Error, return code 2 from \
org.apache.hadoop.hive.ql.<wbr>exec.mr.MapRedTask MapReduce Jobs Launched:
Stage-Stage-1: Map: 8 Reduce: 3 Cumulative CPU: 29.62 sec HDFS Read: 143256373 \
HDFS Write: 0 FAIL Total MapReduce CPU Time Spent: 29 seconds 620 \
msec<br></b></span></font></pre><pre><font color="#000000" face="courier new, \
courier, monospace"><span \
style="font-size:16px"><b><br></b></span></font></pre><pre><font color="#000000" \
face="courier new, courier, monospace"><span \
style="font-size:16px"><b><br></b></span></font></pre><pre><font color="#000000" \
face="courier new, courier, monospace"><span style="font-size:16px"><b>i try running \
the command :</b></span></font></pre><pre><font color="#000000" face="courier new, \
courier, monospace"><span \
style="font-size:16px"><b><br></b></span></font></pre><pre><font color="#000000" \
face="courier new, courier, monospace"><span style="font-size:16px"><b>hdfs fsck \
-openforwrite -files -blocks -locations / | grep blk_1076822902_3082215*</b></span><b \
style="font-size:16px"><br></b></font></pre><pre><font color="#000000" face="courier \
new, courier, monospace"><span style="font-size:16px"><b>the result is \
this:</b></span></font></pre><pre><font color="#000000" face="courier new, courier, \
monospace"><span style="font-size:16px"><b> 0. \
BP-1781152160-172.16.0.170-<wbr>1483968868061:blk_1076822902_<wbr>3082215{blockUCState=UNDER_<wbr>CONSTRUCTION, \
primaryNodeIndex=-1, \
replicas=[<wbr>ReplicaUnderConstruction[[<wbr>DISK]DS-c03666a5-2f54-4085-<wbr>b635-e36b25daedca:NORMAL:172.<wbr>16.0.175:50010|RWR], \
ReplicaUnderConstruction[[<wbr>DISK]DS-11410139-58aa-4160-<wbr>9c89-2459884e7f51:NORMAL:172.<wbr>16.0.109:50010|RWR]]} \
len=153218 MISSING!</b></span><b \
style="font-size:16px"><br></b></font></pre><pre>What can i do ? Looking forward for \
an answer!</pre><pre>Thanks</pre></div>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic