[prev in list] [next in list] [prev in thread] [next in thread] 

List:       python-dev
Subject:    Re: [Python-Dev] versioned .so files for Python 3.2
From:       Barry Warsaw <barry () python ! org>
Date:       2010-06-30 22:10:24
Message-ID: 20100630181024.1e46b667 () heresy
[Download RAW message or body]

[Attachment #2 (multipart/signed)]


On Jun 30, 2010, at 10:35 PM, M.-A. Lemburg wrote:

>The Python default installation dir for
>libs (including site-packages) is $prefix/lib/pythonX.X, so you
>already have separate and properly versioned directory paths.
>
>What difference would the extra version on the .so file make in
>such a setup ?

So on Debian (inherted by Ubuntu), that would be /usr/lib/pythonX.Y and in
fact if you look there, you'll see the normal standard library layout you'd
expect.  There is no site-packages under there though because we install
add-ons to dist-packages[*] but functionally it's what you'd expect.

However, if you look inside dist-packages, you'll see something a little
weird.  All the .py files .egg-info files will actually be symlinks into a
parallel hierarchy under /usr/share/pyshared.  Also, under dist-packages
you'll see a directory layout that looks like it normally would under a
standard site-packages layout, except that again, all the .py files are
symlinked to the pyshared location.  The .so and .pyc files in dist-packages
will be the actual .so and .pyc files.

Why is that?  Well, it's because the .py files can be shared but the .pyc a=
nd
.so files can't.  Getting rid of these crufty symlink farms was the initial
motivation for PEP 3147.

In a PEP 3147 world, pure-Python packages don't need the directory
/usr/lib/pythonX.Y/dist-packages.  We can just install packages to
/usr/share/pyshared and none of the .pyc files will collide between Python
versions.  It makes distro packaging simpler (no symlinks to keep track of =
or
post-install linking and post-removal clean up to do) and smaller (only one
.py file for all supported Python versions).  All we have to do is a
post-install byte-compilation and we're done.  When we remove such a package
we only *have* to remove the .py source file because PEP 3147 will not load=
 a
__pycache__ pyc if the source is missing (though it's easy enough to clean =
up
the pyc files too).

Throw extension modules into the mix and we have the file collision problem
again.  We can't install _foo.so into /usr/share/pyshared because that's go=
ing
to be Python ABI-specific.  If we can ABI-tag the .so and ensure it will get
imported, then we're back to the simpler layout where /usr/share/pyshared is
the destination for all installed Python packages.

-Barry

[*] That's done as a compromise between Debian's interpretation of the FHS =
and
the conflict of that interpretation to from-source installs of Python.
Specifically, Debian puts /usr/local/lib/pythonX.Y/dist-packages on the
*system* Python's path because that's where it expects system administrators
to put their own add-ons to the system Python.  That used to be
/usr/local/lib/pythonX.Y/site-packages but then if you did a from-source
altinstall for example of the same Python version, it would be possible for=
 a
"/usr/local/bin/pythonX.Y setup.py install" of a third-party package to bre=
ak
your system Python.  Not good!  (And yes, it happened to me :).


["signature.asc" (application/pgp-signature)]

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40progressive-comp.com


[prev in list] [next in list] [prev in thread] [next in thread]