[prev in list] [next in list] [prev in thread] [next in thread]
List: macports-users
Subject: Re: HDF5 Problem
From: "Barry McInnes (NOAA Affiliate)" <barry.j.mcinnes () noaa ! gov>
Date: 2016-10-18 14:25:51
Message-ID: cf91c5be-9c63-d727-d012-cbebe7c5d2ae () noaa ! gov
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
We were almost out of version issues, until the latest selfupdate yesterday
The problem is that the netcdf-C library
(/opt/local/lib/libnetcdf.dylib) is now trying to dynamically link to
the HDF5 1.10 library in /opt/local/lib. It should instead be linking
to the HDF5 1.8.x library in /opt/local/lib/hdf5-18/lib
The port info for netcdf states the dependency is hdf5-18 ?
mac56:~/fortran/simple-examples/nc-ex 14> port info netcdf
netcdf @4.4.1_2 (science)
Variants: clang33, clang34, clang35, clang36, clang37,
clang38, clang39, [+]dap, debug, dragonegg33, dragonegg34, gcc44, gcc45,
gcc46, gcc47, gcc48, gcc49, gcc5, gcc6, gcc7, hdf4, llvm,
mpich, mpich_devel, [+]netcdf4, openmpi,
openmpi_devel, universal
Description: NetCDF (network Common Data Form) is a set of
software libraries and machine-independent data formats that support the
creation, access, and sharing of array-oriented scientific
data.
Homepage: http://www.unidata.ucar.edu/software/netcdf/
Build Dependencies: cmake
Library Dependencies: hdf5-18, curl
Platforms: darwin
License: Permissive
Maintainers: takeshi@macports.org, openmaintainer@macports.org
mac56:
On 10/18/16 07:48, Adam Dershowitz wrote:
>
>
>> On Oct 13, 2016, at 11:32 AM, Ryan Schmidt <ryandesign@macports.org
>> <mailto:ryandesign@macports.org>> wrote:
>>
>>>
>>> On Oct 13, 2016, at 10:14 AM, Adam Dershowitz <dersh@alum.mit.edu
>>> <mailto:dersh@alum.mit.edu>> wrote:
>>>
>>> I just updated hdf5 and hdf5-18:
>>>
>>> hdf5 1.10.0_1 < 1.10.0-patch1_0
>>> hdf5-18 1.8.16_6 < 1.8.17_0
>>>
>>> Now when I try to use pandas to open an HDF5 file I get this error:
>>>
>>> Warning! ***HDF5 library version mismatched error***
>>> The HDF5 header files used to compile this application do not match
>>> the version used by the HDF5 library to which this application is
>>> linked.
>>> Data corruption or segmentation faults may occur if the application
>>> continues.
>>> This can happen when an application was compiled by one version of
>>> HDF5 but
>>> linked with a different version of static or shared HDF5 library.
>>> You should recompile the application or check your shared library
>>> related
>>> settings such as 'LD_LIBRARY_PATH'.
>>> You can, at your own risk, disable this warning by setting the
>>> environment
>>> variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
>>> Setting it to 2 or higher will suppress the warning messages totally.
>>> Headers are 1.8.16, library is 1.8.17
>>
>> Sounds like whatever library generated that error (you mentioned
>> py27-pandas and py27-tables but I don't know) is being possibly
>> overzealous in its hdf5 version checking. Sounds like we should
>> increase the revision of whichever port is triggering this error, and
>> add a comment to the hdf5 port to remind us to do that every time the
>> version of hdf5 is increased.
>>
>> It looks to me like py27-tables is the one that depends on hdf5-18,
>> so that's the one that probably needs to be rebuilt. py27-pandas only
>> depends on py27-tables and other python modules.
>>
>>
>>> I tried uninstalling and reinstalling pandas and py27-tables, but
>>> that didn’t help.
>>
>> If you received a binary from our server, it would be a binary built
>> with the old version of hdf5. We need to increase that port's
>> revision to cause it to rebuild with the new version of hdf5.
>>
>> You can try rebuilding from source on your system to identify the
>> problem.
>>
>> sudo port -ns upgrade --force py27-tables
>>
>> If that works, let us know so we can increase its revision.
>
> Yes, forcing the rebuild of py27-tables seems to fix the problem.
> Although, it was slightly more complicated because that rebuild
> complained that py27-tables can’t be built with hdf5 active, so i had
> to force deactivate that, then it built fine, and reactivated hdf5 after.
>
>
>>
>>
>> Looking a little further, I see that the "HDF5 library version
>> mismatched error" actually comes from the hdf5-18 (and hdf5) source
>> code. This makes me think that *any* port that uses hdf5-18 (or hdf5)
>> will encounter this problem, and they should all be revbumped any
>> time the version of hdf5-18 (or hdf5) is increased. This is a pain
>> and the developers of hdf5 should be contacted to explain this to
>> them, so that they can weigh in on why they are inflicting this pain
>> on us and what they can do to mitigate it in subsequent versions of
>> their software.
>>
>>
>>> So, I’m not sure where the bug actually is, what to do about it, or
>>> what port to file a bug report against.
>>> Any suggestions?
>
>
>
> _______________________________________________
> macports-users mailing list
> macports-users@lists.macosforge.org
> https://lists.macosforge.org/mailman/listinfo/macports-users
--
Barry McInnes
325 Broadway
Boulder CO 80304
(303)4976231
barry.j.mcinnes@noaa.gov
[Attachment #5 (text/html)]
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
We were almost out of version issues, until the latest selfupdate
yesterday<br>
<br>
The problem is that the netcdf-C library
(/opt/local/lib/libnetcdf.dylib) is now trying to dynamically link
to the HDF5 1.10 library in /opt/local/lib. It should instead
be linking to the HDF5 1.8.x library in /opt/local/lib/hdf5-18/lib<br>
The port info for netcdf states the dependency is hdf5-18 ?<br>
mac56:~/fortran/simple-examples/nc-ex 14> port info netcdf<br>
netcdf @4.4.1_2 (science)<br>
Variants: clang33, clang34, clang35, clang36, clang37,
clang38, clang39, [+]dap, debug, dragonegg33, dragonegg34, gcc44,
gcc45, gcc46, gcc47, gcc48, gcc49, gcc5, gcc6, gcc7, hdf4, llvm,<br>
mpich, mpich_devel, [+]netcdf4, openmpi,
openmpi_devel, universal<br>
<br>
Description: NetCDF (network Common Data Form) is a set of
software libraries and machine-independent data formats that support
the creation, access, and sharing of array-oriented scientific<br>
data.<br>
Homepage: <a class="moz-txt-link-freetext" \
href="http://www.unidata.ucar.edu/software/netcdf/">http://www.unidata.ucar.edu/software/netcdf/</a><br>
<br>
Build Dependencies: cmake<br>
Library Dependencies: hdf5-18, curl<br>
Platforms: darwin<br>
License: Permissive<br>
Maintainers: <a class="moz-txt-link-abbreviated" \
href="mailto:takeshi@macports.org">takeshi@macports.org</a>, <a \
class="moz-txt-link-abbreviated" \
href="mailto:openmaintainer@macports.org">openmaintainer@macports.org</a><br> \
mac56:<br> <div class="moz-cite-prefix">On 10/18/16 07:48, Adam Dershowitz
wrote:<br>
</div>
<blockquote
cite="mid:BB162B88-9CA3-426B-8067-AA6C85E293BD@alum.mit.edu"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<br class="">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Oct 13, 2016, at 11:32 AM, Ryan Schmidt <<a
moz-do-not-send="true"
href="mailto:ryandesign@macports.org" \
class="">ryandesign@macports.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<blockquote type="cite" style="font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=""><br class="Apple-interchange-newline">
On Oct 13, 2016, at 10:14 AM, Adam Dershowitz <<a
moz-do-not-send="true" href="mailto:dersh@alum.mit.edu"
class="">dersh@alum.mit.edu</a>> wrote:<br class="">
<br class="">
I just updated hdf5 and hdf5-18:<br class="">
<br class="">
hdf5 1.10.0_1 <
1.10.0-patch1_0 <br class="">
hdf5-18 1.8.16_6 < 1.8.17_0
<br class="">
<br class="">
Now when I try to use pandas to open an HDF5 file I get
this error:<br class="">
<br class="">
Warning! ***HDF5 library version mismatched error***<br
class="">
The HDF5 header files used to compile this application do
not match<br class="">
the version used by the HDF5 library to which this
application is linked.<br class="">
Data corruption or segmentation faults may occur if the
application continues.<br class="">
This can happen when an application was compiled by one
version of HDF5 but<br class="">
linked with a different version of static or shared HDF5
library.<br class="">
You should recompile the application or check your shared
library related<br class="">
settings such as 'LD_LIBRARY_PATH'.<br class="">
You can, at your own risk, disable this warning by setting
the environment<br class="">
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.<br
class="">
Setting it to 2 or higher will suppress the warning
messages totally.<br class="">
Headers are 1.8.16, library is 1.8.17<br class="">
</blockquote>
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px; float: none; display:
inline !important;" class="">Sounds like whatever library
generated that error (you mentioned py27-pandas and
py27-tables but I don't know) is being possibly
overzealous in its hdf5 version checking. Sounds like we
should increase the revision of whichever port is
triggering this error, and add a comment to the hdf5 port
to remind us to do that every time the version of hdf5 is
increased.</span><br style="font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class="">
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px; float: none; display:
inline !important;" class="">It looks to me like
py27-tables is the one that depends on hdf5-18, so that's
the one that probably needs to be rebuilt. py27-pandas
only depends on py27-tables and other python modules.<span
class="Apple-converted-space"> </span></span><br
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<blockquote type="cite" style="font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class="">I tried uninstalling and reinstalling pandas and
py27-tables, but that didn’t help.<br class="">
</blockquote>
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px; float: none; display:
inline !important;" class="">If you received a binary from
our server, it would be a binary built with the old
version of hdf5. We need to increase that port's revision
to cause it to rebuild with the new version of hdf5.</span><br
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px; float: none; display:
inline !important;" class="">You can try rebuilding from
source on your system to identify the problem.<span
class="Apple-converted-space"> </span></span><br
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px; float: none; display:
inline !important;" class="">sudo port -ns upgrade --force
py27-tables</span><br style="font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class="">
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px; float: none; display:
inline !important;" class="">If that works, let us know so
we can increase its revision.</span><br
style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
</div>
</blockquote>
<div><br class="">
</div>
Yes, forcing the rebuild of py27-tables seems to fix the
problem. </div>
<div>Although, it was slightly more complicated because that
rebuild complained that py27-tables can’t be built with hdf5
active, so i had to force deactivate that, then it built fine,
and reactivated hdf5 after. </div>
<div><br class="">
</div>
<div><br class="">
<blockquote type="cite" class="">
<div class=""><br style="font-family: Helvetica; font-size:
12px; font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px; float: none; display:
inline !important;" class="">Looking a little further, I
see that the "HDF5 library version mismatched error"
actually comes from the hdf5-18 (and hdf5) source code.
This makes me think that *any* port that uses hdf5-18 (or
hdf5) will encounter this problem, and they should all be
revbumped any time the version of hdf5-18 (or hdf5) is
increased. This is a pain and the developers of hdf5
should be contacted to explain this to them, so that they
can weigh in on why they are inflicting this pain on us
and what they can do to mitigate it in subsequent versions
of their software.</span><br style="font-family:
Helvetica; font-size: 12px; font-style: normal;
font-variant-caps: normal; font-weight: normal;
letter-spacing: normal; orphans: auto; text-align: start;
text-indent: 0px; text-transform: none; white-space:
normal; widows: auto; word-spacing: 0px;
-webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<br style="font-family: Helvetica; font-size: 12px;
font-style: normal; font-variant-caps: normal;
font-weight: normal; letter-spacing: normal; orphans:
auto; text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; widows: auto; word-spacing:
0px; -webkit-text-stroke-width: 0px;" class="">
<blockquote type="cite" style="font-family: Helvetica;
font-size: 12px; font-style: normal; font-variant-caps:
normal; font-weight: normal; letter-spacing: normal;
orphans: auto; text-align: start; text-indent: 0px;
text-transform: none; white-space: normal; widows: auto;
word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class="">So, I’m not sure where the bug actually is, what
to do about it, or what port to file a bug report against.<br
class="">
Any suggestions? </blockquote>
</div>
</blockquote>
</div>
<br class="">
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
macports-users mailing list
<a class="moz-txt-link-abbreviated" \
href="mailto:macports-users@lists.macosforge.org">macports-users@lists.macosforge.org</a>
<a class="moz-txt-link-freetext" \
href="https://lists.macosforge.org/mailman/listinfo/macports-users">https://lists.macosforge.org/mailman/listinfo/macports-users</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<p>Barry McInnes<br>
325 Broadway<br>
Boulder CO 80304<br>
(303)4976231<br>
<a class="moz-txt-link-abbreviated" \
href="mailto:barry.j.mcinnes@noaa.gov">barry.j.mcinnes@noaa.gov</a></p> </div>
</body>
</html>
_______________________________________________
macports-users mailing list
macports-users@lists.macosforge.org
https://lists.macosforge.org/mailman/listinfo/macports-users
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic