[prev in list] [next in list] [prev in thread] [next in thread] 

List:       python-distutils-sig
Subject:    Re: [Distutils] Proper handling of PEP420 namespace packages with setuptools and pip
From:       PJ Eby <pje () telecommunity ! com>
Date:       2015-04-25 19:08:54
Message-ID: CALeMXf7J6Do6xVoo1Kh3qqeiGC93rxpkt38bnapnZR-efMuxUQ () mail ! gmail ! com
[Download RAW message or body]

On Wed, Apr 22, 2015 at 5:20 PM, Robert Collins
<robertc@robertcollins.net> wrote:
> Ah, ok so I think this is the crux - I'm arguing that Python version
> isn't a big enough check. Because anything installed with a current
> version of setuptools, or any wheel built likewise, is going to not
> have that per-Python-version check.
>
> And it seems to me that that implies that bringing in a
> per-Python-version check in a new release of setuptools or pip is
> going to result in mixed mode installs:
>
> install name.A with setuptools-X [legacy]
> upgrade setuptools
> install name.B with setuptools-Y [does a Python version check]
> -> boom
>
> But perhaps sufficient glue can be written to make it all work.

When I wrote PEP 402 (the precursor to 420), the idea was that in a
mixed environment you would need to:

1. Change pkg_resources' namespace system to support non-__init__
directories  (and likewise pkgutil.extend_path)
2. Change easy_install's .pth generation magic to not do any magic or
import a PEP 420 emulation module on old systems
3. Change package building tools to stop injecting __init__.py files

This basically solves the mixed installation problem because if you
have an __init__.py that uses existing magic, the empty dirs get
folded in.  You basically have a transitional state with mixed
__init__ and non-__init__ stuff.  If there happens to be an
__init__.py, then as long as it declares the namespace then the local
*runtime* takes care of making the runtime environment work.  The
*installation* tools don't have to manage mixed modes, they should
just blindly install whatever package they have, and over the long
term the packages all end up shipped without __init__.py's, but the
__init__.py approach will continue to work basically forever.

> My personal preferred migration strategy is:
>  - have a flag day amongst the cooperating packages that make up the namespace
>  - release versions that are all in the new layout in a batch to PyPI.
>
> It would be nice if PEP-426 had a conflicts stanza, so you could say
> conflicts: [name.A < version_with_new_X] without that implying that
> name.A *should* be installed.

This is all *really* unnecessary.  Setuptools has essentially *always*
built non-egg, non-exe binary distributions in a PEP 420-compatible
way (i.e., without __init__.py).  And pkg_resources already builds a
namespace path by asking importers if they can import a package at
that location.  So if PEP 420 importers say "yes" when asked to
find_module('somepkg') in the case of an __init__-less subdirectory
named 'somepkg', then pkg_resources *already* supports mixed-mode
installations under PEP 420, and doesn't even need to be updated!

I haven't checked whether that's the case, but if it is, then the only
thing that setuptools neds to change is its .pth generation magic, to
not do the magic if it's on a PEP 420 platform at runtime, and to stop
including __init__.py's for namespace packages.
_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
[prev in list] [next in list] [prev in thread] [next in thread]