[prev in list] [next in list] [prev in thread] [next in thread] 

List:       python-distutils-sig
Subject:    [Distutils] (no subject)
From:       "Stanley A. Klein" <sklein () cpcug ! org>
Date:       2007-07-30 23:37:06
Message-ID: 3600.207.188.248.157.1185838626.squirrel () www ! cpcug ! org
[Download RAW message or body]

Phillip J. Eby wrote:
> At 03:15 PM 7/30/2007 -0400, Stanley A. Klein wrote:
>>> I don't need to build the .so files.  They are already built.  That
had to
>>> be done using the build-in-place and the numpy distutils for reasons I
>>> don't fully understand but are related to the use of numpy.
>
>> Have you tried building them with setuptools, using the numpy
>> distutils 'build_ext' command, using:
>>
>>     setup(
>>        cmdclass = dict(build_ext = numpy_build_ext_class_here),
>>        ext_modules = list_of_Extension_objects,
>>        ...
>>     )
>>
>> Unless there is a radical difference between numpy distutils and the
>> regular distutils, you should be able to do this.  Just find numpy's
>> "build_ext" class, and define the appropriate Extension() objects
>> (for the things to build) in your setup script.  Setuptools will then
>> delegate the building to numpy, but handle the installing itself.
>
>> Again, this is assuming that numpy's distutils extensions don't do
>> anything unfriendly like completely redefine how extension objects
>> work or assume that their commands will be only mixed with other
>> numpy commands.  (Setuptools doesn't make such assumptions, and tries
>> to leave the normal distutils stuff alone as much as possible.)

>I think we're getting into confusing territory by trying to get
workarounds >for workarounds. Let me try to take us a step back and focus
on the initial >problem which is that bdist_rpm is not working with
enthought.kiva. The >existing setup script already does build extensions
just fine; they're just >not being picked up by bdist_rpm. A suggestion
from a coworker of mine >prompted Stanley to look at using a script that
we have for building >enthought.kiva inplace (there are a few more
options that are needed beyond >"python setup.py develop"); however, it
wasn't really a suggestion to use >that as basis for building an RPM.
>
>numpy.distutils extends distutils in three ways which are important for
>enthought.kiva:
>
>  * automatically adds the location of the numpy headers to the
>include_dirs of Extensions. (easily replaced)
>
>  * adds a build_src command that allows users to give Python functions
in >the sources list of an Extension. These functions will be called to
>actually
>generate the real source files. (hard to replace)
>
>  * allows subpackages to have their own build information which is
>assembled by the top-level setup.py script. This is mostly legacy from
when >the enthought package was monolithic and doesn't strictly need to
continue. >I won't go into details since I don't think it's part of the
problem. >(straightforward, but time-consuming to replace)

>numpy.distutils tries hard to not step on setuptools' toes. We actually
>check if setuptools is in sys.modules and use its command classes
instead >of distutils' as the base classes for our commands. However,
it's possible >that neglect of our bdist_rpm implementation has caused
the implementations >to diverge and some toe-stepping has taking place.

>The main problem is that bdist_rpm is not working on enthought.kiva. Most
>likely, this is the fault of numpy.distutils. However, this is a bug that
>needs to be caught and fixed. Working around it by doing an --inplace
build >and then trying to include the extension modules as package_data
is not >likely to work and is not a solution.
>
>I'm not usually a Redhat guy, so I don't have much experience with
>bdist_rpm; however, numpy.distutils has had problems with bdist_rpm in
the >past. I'm trying to get an environment working on a Redhat machine,
and >will try to build an RPM for enthought.kiva and try to see the
problem >first-hand. I've looked over Stanley's emails on the subject,
and don't see >enough information for me to really pin down the problem.

Robert -

Thanks for illuminating the issue.

The problem I had was as follows.  Fedora (also RedHat) uses SE-Linux,
which needs to know all the files expected to be in sensitive directories
such as the Python site-packages.  This includes the pyc and pyo files
ordinarily generated by Python as the .py files are executed.

It turns out that to do a bdist_rpm for Fedora, it is necessary to create
a setup.cfg file containing the lines:

[install]
optimize = 1

or to add these lines to the existing setup.cfg file.

If that is not done, the result in Fedora is an unpackaged files error. 
This is due to the fact that if distutils/setuptools doesn't cause the pyc
and pyo files to be created, Fedora will create them but they won't be
properly handled in the spec file created by distutils/setuptools.

In trying to do bdist_rpm with kiva, I got the unpackaged files error. 
This implies that numpy distutils did not properly handle the optimize=1
in the setup.cfg (when I did "python setup.py bdist_rpm").  That's when I
went to the workaround that resulted in this thread.

I hope this clarifies the problem.

Thanks.


Stan Klein

_______________________________________________
Distutils-SIG maillist  -  Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
[prev in list] [next in list] [prev in thread] [next in thread]