[prev in list] [next in list] [prev in thread] [next in thread] 

List:       hadoop-user
Subject:    HDFS seek perf question
From:       Adam Silberstein <adam () trifacta ! com>
Date:       2015-01-29 1:33:20
Message-ID: CAKcNZ_Rt0f2sPoc0YOfoh+1=S-=y0aMEOq5xjQfuRyhWD0e4kQ () mail ! gmail ! com
[Download RAW message or body]

Hi,
I have a question about hdfs seek performance.  I see some info on this
periodically, but nothing too recent.

How do these costs compare?
A) seeking to the start of an HDFS block and reading about 10MB of data
B) reading the entire HDFS block

Assuming A is faster, how many random seeks can you do against an HDFS
block before that is slower than reading the whole thing?  On paper that
can be computed using the disk's speed numbers but would like to know how
well in practice HDFS matches that behavior.

Thanks,
Adam

[Attachment #3 (text/html)]

<div dir="ltr"><div>Hi,</div><div>I have a question about hdfs seek performance.   I \
see some info on this periodically, but nothing too \
recent.</div><div><br></div><div>How do these costs compare?  </div><div>A) seeking \
to the start of an HDFS block and reading about 10MB of data</div><div>B) reading the \
entire HDFS block</div><div><br></div><div>Assuming A is faster, how many random \
seeks can you do against an HDFS block before that is slower than reading the whole \
thing?   On paper that can be computed using the disk&#39;s speed numbers but would \
like to know how well in practice HDFS matches that \
behavior.</div><div><br></div><div>Thanks,</div><div>Adam</div></div>



[prev in list] [next in list] [prev in thread] [next in thread]