[prev in list] [next in list] [prev in thread] [next in thread]
List: hadoop-user
Subject: Re: Error importing hbase table on new system
From: Håvard_Wahl_Kongsgård <haavard.kongsgaard () gmail ! com>
Date: 2015-09-27 16:52:55
Message-ID: CAKH910-EStBn5z+9KGbRpyYfc5WnuM=hyNKjqBjWhwg+xzs_dA () mail ! gmail ! com
[Download RAW message or body]
But is not that tool for reading regions not export (dumps) ??
-Håvard
On Sun, Sep 27, 2015 at 6:23 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> Have you used HFile tool ?
>
> The tool would print out that information as part of metadata.
>
> Cheers
>
> On Sun, Sep 27, 2015 at 9:19 AM, Håvard Wahl Kongsgård
> <haavard.kongsgaard@gmail.com> wrote:
> >
> > Yes, I have tried to read them on another system as well. It worked
> > there. But I don't know if they are HFilev1 or HFilev2 format(any way
> > to check ?? )
> >
> > This is the first lines from one of the files
> >
> > SEQ
> > 1org.apache.hadoop.hbase.io.ImmutableBytesWritable%org.apache.hadoop.hbase.client.Result
> >
> > *org.apache.hadoop.io.compress.DefaultCodec���N��
> > $��a&t ��wb%!100000107712083-10152358443612846x��� P ]�6:� w \
> > |pw ��� �$��K 0� Npw�$x�@� \
> > ���s�;�Ɦ���V�^��ݽW� �� � �қ ��<� \
> > /��0/ � ?'/ � ����� /��� �� 7{kG [(��� \
> > ���w�� OY^I ���}9 � �l��;�TJ�� �� �J� ‹ \
> > ���� pu���V� ӡm�\E @ ��V6�oe45U ���,�3 \
> > ���Ͻ�w��O���zڼ�/��歇�KȦ/ ����?�� \
> > Y;����� / ��� �� �� }� \
> > ��룫-�'_�k����� ��q� $ ��˨� � ���^ \
> > ��� i��� tH$/��e.J��{S �\��S���� >G \
> > d���1~ p#�� o �� ��M �! ��;c��I kQ
> > �A)|d�i�(Z�f��o Pb �j {� �x��� � \
> > `�����b���cbb`�"�}�������� ����� \
> > HCG��&�����JG�%��',*!!�� �� � � ��& \
> > �����_Q��R�2�1��_��~>:� b � �������w \
> > @�B � ~Y�H�(�h/FR ����_+��nX `#�
> > > D��� �j���� f ��ƨT��k/ 颚h ��4` \
> > > +Q#�ⵕ�,Z�80�V:�
> > )Y)4Lq��[� z#���T<b� �-*.�����/��m{? �8|�] \
> > �6� �sk�t�L\UjT=
> > -Håvard
> >
> > On Sun, Sep 27, 2015 at 4:06 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > > Have you verified that the files to be imported are in HFilev2 format ?
> > >
> > > http://hbase.apache.org/book.html#_hfile_tool
> > >
> > > Cheers
> > >
> > > On Sun, Sep 27, 2015 at 4:47 AM, Håvard Wahl Kongsgård
> > > <haavard.kongsgaard@gmail.com> wrote:
> > > >
> > > > > Is the single node system secure ?
> > > >
> > > > No have not activated, just defaults
> > > >
> > > > the mapred conf.
> > > >
> > > > <?xml version="1.0"?>
> > > >
> > > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > > >
> > > >
> > > > <configuration>
> > > >
> > > > <property>
> > > >
> > > > <name>mapred.job.tracker</name>
> > > >
> > > > <value>rack3:8021</value>
> > > >
> > > > </property>
> > > >
> > > >
> > > > <!-- Enable Hue plugins -->
> > > >
> > > > <property>
> > > >
> > > > <name>mapred.jobtracker.plugins</name>
> > > >
> > > > <value>org.apache.hadoop.thriftfs.ThriftJobTrackerPlugin</value>
> > > >
> > > > <description>Comma-separated list of jobtracker plug-ins to be
> > > > activated.
> > > >
> > > > </description>
> > > >
> > > > </property>
> > > >
> > > > <property>
> > > >
> > > > <name>jobtracker.thrift.address</name>
> > > >
> > > > <value>0.0.0.0:9290</value>
> > > >
> > > > </property>
> > > >
> > > > </configuration>
> > > >
> > > >
> > > > > > Have you checked hdfs healthiness ?
> > > >
> > > >
> > > > sudo -u hdfs hdfs dfsadmin -report
> > > >
> > > > Configured Capacity: 2876708585472 (2.62 TB)
> > > >
> > > > Present Capacity: 1991514849280 (1.81 TB)
> > > >
> > > > DFS Remaining: 1648230617088 (1.50 TB)
> > > >
> > > > DFS Used: 343284232192 (319.71 GB)
> > > >
> > > > DFS Used%: 17.24%
> > > >
> > > > Under replicated blocks: 52
> > > >
> > > > Blocks with corrupt replicas: 0
> > > >
> > > > Missing blocks: 0
> > > >
> > > >
> > > > -------------------------------------------------
> > > >
> > > > Datanodes available: 1 (1 total, 0 dead)
> > > >
> > > >
> > > > Live datanodes:
> > > >
> > > > Name: 127.0.0.1:50010 (localhost)
> > > >
> > > > Hostname: rack3
> > > >
> > > > Decommission Status : Normal
> > > >
> > > > Configured Capacity: 2876708585472 (2.62 TB)
> > > >
> > > > DFS Used: 343284232192 (319.71 GB)
> > > >
> > > > Non DFS Used: 885193736192 (824.40 GB)
> > > >
> > > > DFS Remaining: 1648230617088 (1.50 TB)
> > > >
> > > > DFS Used%: 11.93%
> > > >
> > > > DFS Remaining%: 57.30%
> > > >
> > > > Last contact: Sun Sep 27 13:44:45 CEST 2015
> > > >
> > > >
> > > > > > To which release of hbase were you importing ?
> > > >
> > > > Hbase 0.94 (CHD 4)
> > > >
> > > > the new one is CHD 5.4
> > > >
> > > > On Sun, Sep 27, 2015 at 1:32 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > > > > Is the single node system secure ?
> > > > > Have you checked hdfs healthiness ?
> > > > > To which release of hbase were you importing ?
> > > > >
> > > > > Thanks
> > > > >
> > > > > > On Sep 27, 2015, at 3:06 AM, Håvard Wahl Kongsgård
> > > > > > <haavard.kongsgaard@gmail.com> wrote:
> > > > > >
> > > > > > Hi, Iam trying to import a old backup to a new smaller system (just
> > > > > > single node, to get the data out)
> > > > > >
> > > > > > when I use
> > > > > >
> > > > > > sudo -u hbase hbase -Dhbase.import.version=0.94
> > > > > > org.apache.hadoop.hbase.mapreduce.Import crawler
> > > > > > /crawler_hbase/crawler
> > > > > >
> > > > > > I get this error in the tasks . Is this a permission problem?
> > > > > >
> > > > > >
> > > > > > 2015-09-26 23:56:32,995 ERROR
> > > > > > org.apache.hadoop.security.UserGroupInformation:
> > > > > > PriviledgedActionException as:mapred (auth:SIMPLE)
> > > > > > cause:java.io.IOException: keyvalues=NONE read 4096 bytes, should
> > > > > > read
> > > > > > 14279
> > > > > > 2015-09-26 23:56:32,996 WARN org.apache.hadoop.mapred.Child: Error
> > > > > > running child
> > > > > > java.io.IOException: keyvalues=NONE read 4096 bytes, should read
> > > > > > 14279
> > > > > > at
> > > > > >
> > > > > > org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2221)
> > > > > > at
> > > > > >
> > > > > > org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.nextKeyValue(SequenceFileRecordReader.java:74)
> > > > > > at
> > > > > >
> > > > > > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:483)
> > > > > > at
> > > > > >
> > > > > > org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:76)
> > > > > > at
> > > > > >
> > > > > > org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:85)
> > > > > > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:139)
> > > > > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
> > > > > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
> > > > > > at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
> > > > > > at java.security.AccessController.doPrivileged(Native Method)
> > > > > > at javax.security.auth.Subject.doAs(Subject.java:415)
> > > > > > at
> > > > > >
> > > > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
> > > > > > at org.apache.hadoop.mapred.Child.main(Child.java:262)
> > > > > > 2015-09-26 23:56:33,002 INFO org.apache.hadoop.mapred.Task: Runnning
> > > > > > cleanup for the task
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Håvard Wahl Kongsgård
> > > > > > Data Scientist
> > > >
> > > >
> > > >
> > > > --
> > > > Håvard Wahl Kongsgård
> > > > Data Scientist
> > >
> > >
> >
> >
> >
> > --
> > Håvard Wahl Kongsgård
> > Data Scientist
>
>
--
Håvard Wahl Kongsgård
Data Scientist
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic