[prev in list] [next in list] [prev in thread] [next in thread] 

List:       lustre-discuss
Subject:    [lustre-discuss] How Lustre stores hyperslabs and chunks of HDF5?
From:       andreas.dilger () intel ! com (Dilger, Andreas)
Date:       2015-07-21 23:53:09
Message-ID: 042590F4-381E-4118-BC91-61CF884EB83B () intel ! com
[Download RAW message or body]

Currently there is no direct connection between Lustre layout and HDF5 file layout.  \
The only option is RAID-0 striping across OST objects with a fixed stripe size. If \
HDF5 is aware of this stripe size and can take advantage of it, that is great.

There is a project that has started to implement Progressive File Layout (PFL) that \
allows different extents of a file to have different stripe counts and stripe sizes, \
which could be leveraged by libraries like HDF5 in the future.

See http://cdn.opensfs.org/wp-content/uploads/2015/04/Progressive-File-Layouts_Hammond.pdf \
and/or https://www.youtube.com/watch?v=5rm6Nlmqdp0 for more details on PFL prototype \
development.

Cheers, Andreas

On Jul 19, 2015, at 20:37, "Prakrati.Agrawal at shell.com<mailto:Prakrati.Agrawal at \
shell.com>" <Prakrati.Agrawal at shell.com<mailto:Prakrati.Agrawal at shell.com>> \
wrote:

Hi,

I wanted to understand how Lustre stores the chunks and hyperslabs in HDF5 framework \
on the OSTs? If I set the chunk size and each rank is writing a hyperslab, then OST0 \
has chunk0, OST1 has chunk1 and so on or is it that OST0 has hyperslab0, OST1 has \
hyperslab1. Is there any way of finding that out?

Thanks and Regards,
Prakrati
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


[prev in list] [next in list] [prev in thread] [next in thread]