'Re: [Distutils] more thoughts on python package management'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       python-distutils-sig
Subject:    Re: [Distutils] more thoughts on python package management
From:       kiorky <kiorky () cryptelium ! net>
Date:       2008-09-30 20:11:34
Message-ID: 48E287F6.9010605 () cryptelium ! net
[Download RAW message or body]

[Attachment #2 (multipart/signed)]

Chris Withers a =E9crit :
> > Hi All,
> >
> > I've been trying to catch up on all the packaging discussions but
> > couldn't find the right place to reply so thought I'd just do so
> > seperately...
Maybe, we have all our own definition ...
> >
> > Probably the biggest thing that strikes me now is that
> > distutils/setuptools/distribute/pacman/whatever should aim to do much=

> > less...
I don't totally agree, we must provide a tool to build from start to end =
our
package even if we let means (eg via complete metadatas) to do the job in=

another way, by other tools.
> >
> > In fact, I get the feeling what we really need is a way for package
> > maintainers to provide the following metadata:
> >
> > - where the docs are
> >
> > - where the tests are and how they're run
> >
> > - how anything not-python should be built
> >
> > - what the dependencies are
> >   (maybe even what the non-python dependencies are!)
> >
> > - what version of the package this is
> >
> > This should be in a build-tool independent fashion such that any buil=
d
> > tools, but especially those of operating system maintainers, can run
> > over the same metadata and build their packages.
+1 for more explicit metadata on python packages,
But i don't think our target is only the target "OS" and its relative pac=
kages
manager. I really can't see how to get running a lot of projects with
inter-related incompatibilities on the same system without some sort of i=
solation
So we cannot say we will be able to install whatever we want with everyth=
ing
else at the same time, system wide. And selecting versions by hand, in ea=
ch
project, can be painful. I'd prefer to have a "project" or "environment
specific" approach like buildout, or virtualenv, or the combination of th=
ese
tools, even maybe a wrap-up tool like minitage [1].

> >
> > The only other critical thing for me is that *all* of the above metad=
ata
> >  should be available post-install.
> >
I 'd prefer to know what i am installing before installing it. For me, th=
e
metadata are predictable things, which must be there before installing an=
y part
of a project.

> > With the above in place, we free up the evolution of build tools and =
let
> > the OS-specific packaging tools play nicely.
> >
> > I think a good aim would also be to have some "one-way-to-do-it" pyth=
on
> > tools too for:
> >
> > - installing a package
> >
> > - uploading a package to PyPI
> >
> > - getting a package from PyPI
> >
> >
+1, but see under
> > ...without any silly big plugin system in the way distutils currently=

> > works.
> >
> > What do other people feel?
> >
> > cheers,
> >
> > Chris
> >

IMHO, a good management suite for python packages is a collection of tool=
s,
plugins and API bindings which:
 - Can do all its job in total isolation (installation in /prefix even wh=
ere our
only privilege is a filesystem write access)
   There two levels i can see there:
     - Isolation from the OS
     - Isolation in the "target environment" (/package1/the-stuff and
/package2/the-stuff)
 - Is OS independant (maybe the most hard thing to do)
 - Is failure tolerant:
    - Handle offline mode
    - Know how to go backward
    - Use a sandbox before (un)installing
 - Let us means to add, remove and override the packages 's repositories =
and
even the packages themselves.
 - Can use a download cache
 - Deal with dependencies
 - Can distribute the package
 - Can fetch the package
 - Can build the package
 - Can Test the package
 - Can install the package from source or binary
 - Trace and store what is installed
 - Can uninstall the package
 - Know how to deal with reverse dependencies problems, even if this is a=
 tool
with just rebuild them like the gentoo's ones [2].
 - Provide goods metadata
 - Have all the classical features cli tools like apt, emerge ... have:
 - Search on installed and available softwares
 - List installed files
 - Knows from which package a file is belonging to
 - Have an API to let us build, query, distribute, get, ... our packages.=

 - Have some hook/plugin registration system like the egg's entry points.=

And the according packages's metadata must contain:
 - package name
 - package version
 - Detailed dependencies requirements which include also incompatibilitie=
s (like
OS ones)
 - Maybe some knobs system like the USES in gentoo.
 - where we get the package
 - The way we build/install/uninstall it
 - Author/website/license and so on
 - Where are the tests
 - How the tests are run
 - hooks registration information

There is nothing new there, Those are classical package manager needs. Ma=
ybe
more "source packages manager" needs because we are dealing with somethin=
g which
can be built on the target. What is different there is that i try  introd=
uce an
"environment" approach more than a "system centric" one.

[1] http://www.minitage.org/doc/rst
[2]
 - emerge @preserved-rebuild:
http://r0bertz.blogspot.com/2008/06/portage-22-preserve-libs-features.htm=
l
 - revdep-rebuild : http://gentoo-wiki.com/TIP_Control_revdep-rebuild

--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF

["signature.asc" (application/pgp-signature)]

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig

[prev in list] [next in list] [prev in thread] [next in thread] 