From kde-buildsystem Tue Oct 30 14:17:40 2007 From: Thiago Macieira Date: Tue, 30 Oct 2007 14:17:40 +0000 To: kde-buildsystem Subject: Re: Some libraries are not found without LD_LIBRARY_PATH mangling Message-Id: <200710301517.53727.thiago () kde ! org> X-MARC-Message: https://marc.info/?l=kde-buildsystem&m=119375393925286 MIME-Version: 1 Content-Type: multipart/mixed; boundary="--===============0630236830==" --===============0630236830== Content-Type: multipart/signed; boundary="nextPart1530983.8q1gYqjfpP"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit --nextPart1530983.8q1gYqjfpP Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Em Tuesday 30 October 2007 14:44:51 Alexander Neundorf escreveu: > > We're talking about running uninstalled applications that link to > > uninstalled libraries. Linking to installed libraries has never been a > > problem -- the RPATH to the install tree is present anyways. > > But not necessarily to RPATH to libraries that library links to, see belo= w. It doesn't have to. See below. > > > then A will link just to that and not know about other libraries > > > libsoprano.so links to (and their directories, which are required for > > > the RPATH). If libsoprano.so doesn't find its libraries without RPATH > > > (or RUNPATH), there are two options: > > > > Again, stop. libsoprano.so's dependencies are loaded by libsoprano.so > > itself, not by the application. So it's the dependent library's job to > > find the libraries the right way (environment variable, ld.so.conf > > modification, RPATH, whatever). > > Yes, we agree here. > What I still don't know is for which steps is the RPATH valid ? DT_RPATH is only valid for the libraries loaded by that module's DT_NEEDED= =20 tags. It doesn't work indirectly. > Say APP links to libA, and libA links to libB, and APP has an RPATH, is > this RPATH also valid for finding libB from libA ? No. libA has to know how to find libB on its own. Test case: $ ls -lR =2E: total 12 =2Drw-r--r-- 1 tmacieir tmacieir 45 2007-10-30 14:57 main.c drwxr-xr-x 2 tmacieir tmacieir 100 2007-10-30 14:57 ONE/ drwxr-xr-x 2 tmacieir tmacieir 100 2007-10-30 14:58 TWO/ =2E/ONE: total 12 lrwxrwxrwx 1 tmacieir tmacieir 11 2007-10-30 14:57 libONE.so -> libONE.so= =2E1* =2Drwxr-xr-x 1 tmacieir tmacieir 5236 2007-10-30 14:54 libONE.so.1* =2Drw-r--r-- 1 tmacieir tmacieir 21 2007-10-30 14:54 main.c =2E/TWO: total 12 lrwxrwxrwx 1 tmacieir tmacieir 11 2007-10-30 14:58 libTWO.so -> libTWO.so= =2E1* =2Drwxr-xr-x 1 tmacieir tmacieir 5375 2007-10-30 14:57 libTWO.so.1* =2Drw-r--r-- 1 tmacieir tmacieir 53 2007-10-30 14:55 main.c $ cat ONE/main.c int symbol_ONE() { } $ ldd ONE/libONE.so linux-gate.so.1 =3D> (0xb7fe7000) libc.so.6 =3D> /lib/i686/libc.so.6 (0xb7e85000) /lib/ld-linux.so.2 (0x80000000) $ cat TWO/main.c int symbol_ONE(); int symbol_TWO() { symbol_ONE(); } $ ldd TWO/libTWO.so linux-gate.so.1 =3D> (0xb7fcd000) libONE.so.1 =3D> not found libc.so.6 =3D> /lib/i686/libc.so.6 (0xb7e6b000) /lib/ld-linux.so.2 (0x80000000) So, at this point above, you can see that libTWO was compiled linking to=20 libONE, but *without* any RPATH setting whatsoever. If we try to link our test program: $ cat main.c int symbol_TWO(); int main() { symbol_TWO; } $ gcc -o main -LTWO -lTWO main.c /usr/bin/ld: warning: libONE.so.1, needed by TWO/libTWO.so, not found (try= =20 using -rpath or -rpath-link) TWO/libTWO.so: undefined reference to `symbol_ONE' collect2: ld returned 1 exit status Oh, well, let's add the -rpath: $ gcc -o main -LTWO -lTWO main.c -Wl,-rpath,$PWD/ONE:$PWD/TWO $ ./main =2E/main: error while loading shared libraries: libONE.so.1: cannot open sh= ared=20 object file: No such file or directory So you see that main's RPATH does *not* apply to libTWO's dependencies. Now, if we make it a direct dependency: $ gcc -o main -LTWO -LONE -lTWO -lONE main.c -Wl,-rpath,$PWD/ONE:$PWD/TWO $ ./main Works just fine. Also works if -rpath is set in the dependent library: $ ldd TWO/libTWO.so linux-gate.so.1 =3D> (0xb7f50000) libONE.so.1 =3D> /tmp/testdir/TWO/../ONE/libONE.so.1 (0xb7f4b000) libc.so.6 =3D> /lib/i686/libc.so.6 (0xb7dec000) /lib/ld-linux.so.2 (0x80000000) [rpath was added, so it finds its dependency] $ gcc -o main -LTWO -lTWO main.c -Wl,-rpath,$PWD/TWO $ ./main Conclusions: 1) when linking to uninstalled libraries, we should link to them *all* (so = as=20 to make all uninstalled dependencies direct ones) 2) each one should clean up after itself: each library should be loadable o= n=20 its own after installed. Having a "not found" in ldd's output means you=20 messed up. > Yes, but for anything except kdelibs any library from kdelibs is an > "already installed library". Currently they don't get an RPATH at all > (except with FULL_RPATH option), but when linking to them the full list of > libs is used (which should produce the correct RPATH for the executable). > Once the libraries get an RPATH, they already have it in the build tree or > they will be relinked. I didn't understand. > > > > - on ELF platforms without DT_RUNPATH (IRIX, HP-UX): > > > > RUN_UNINSTALLED executables are relinked at install time > > > > > > This could make some things easier, I think we could even completely > > > get rid of the RUN_UNINSTALLED argument. Everything, apps and libs > > > would get the full RPATH (RUNPATH) for the install tree, the shell > > > scripts which set LD_LIBRARY_PATH are created anyway. > > > > That's what I proposed for the platforms using DT_RUNPATH. But mind you: > > So we agree, right ? Yes. > > the real target (the ELF executable) should have a different name -- the > > shell script should have the target's name inside the build tree. > > I don't feel like changing this now and I don't see a real reason for it. > The shell script is currently .shell, if you use tab completion y= ou > notice that there are two executables. I see a reason for it: if those programs are run uninstalled, it's because= =20 there's a rule in CMake somewhere that is running them. Are we absolutely=20 sure that all places calling them have the ".shell" if it's inside kdelibs= =20 but no ending if it's not? > Yes, FULL_RPATH is ok, my only problem is the time the relinking takes. > It's even better than RUNPATH+LD_LIBRARY_PATH (because you can run it > directly and don't have to use a script). =46ULL_RPATH+RUNPATH+LD_LIBRARY_PATH is the best solution I see. Distributi= ons,=20 however, probably won't use RPATH at all, so LD_LIBRARY_PATH also applies t= o=20 them and works fine. (Then again, distributions don't have the installed=20 libraries in their build environments, so even if RPATH is there, it will=20 find no library) As far as I can see, relinking is necessary only on AIX and on ELF without= =20 RUNPATH. > I'll put chrpath on my TODO. I did take a look at it once and my conclusion was that increasing the RPAT= H=20 entry would require rewriting the entire ELF file. That's not an easy task= =20 and requires the tool to actually understand ELF. Don't forget that there a= re=20 some ELF binaries that are several megabytes in size. However, since we're talking about *removing* path entries from RPATH=20 (removing the build tree path), it should be always possible, since you=20 decrease the length of the tag. =2D-=20 =A0 Thiago Macieira =A0- =A0thiago (AT) macieira.info - thiago (AT) kde.org =A0 =A0 PGP/GPG: 0x6EF45358; fingerprint: =A0 =A0 E067 918B B660 DBD1 105C =A0966C 33F5 F005 6EF4 5358 --nextPart1530983.8q1gYqjfpP Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) iD8DBQBHJz0FM/XwBW70U1gRAvCSAJ9WjUpeSf9rc0+ac99PPYILoNkU8gCgomMt EgjsRkbUVzMVLy7fw5aRUco= =R1jZ -----END PGP SIGNATURE----- --nextPart1530983.8q1gYqjfpP-- --===============0630236830== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Kde-buildsystem mailing list Kde-buildsystem@kde.org https://mail.kde.org/mailman/listinfo/kde-buildsystem --===============0630236830==--