[prev in list] [next in list] [prev in thread] [next in thread]
List: python-distutils-sig
Subject: Re: [Distutils] Installing a file into sitepackages
From: Ionel_Cristian_Mărieș <contact () ionelmc ! ro>
Date: 2015-03-27 9:56:38
Message-ID: CANkHFr8-8kNjDj1CdezUGUVSbg22jDgtFe8v8wqEKtjqDgQ_Tw () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Also, a similar command subclass can be written for `develop`. So far i got
3 subclasses, for: build, easy_install and develop. Did I miss something
important?
Thanks,
-- Ionel Cristian Mărieș, http://blog.ionelmc.ro
On Wed, Mar 25, 2015 at 2:51 PM, Stuart Axon <stuaxo2@yahoo.com> wrote:
> That looks much cleaner than my one, I'll give it a try.. does it work on
> python3, just found out my one does not.
>
> S++
>
>
>
> On Wednesday, March 25, 2015 9:03 AM, Ionel Cristian Mărieș <
> contact@ionelmc.ro> wrote:
>
>
>
> This seems to do the trick:
>
> class EasyInstallWithPTH(easy_install):
> def run(self):
> easy_install.run(self)
> for path in glob(join(dirname(__file__), 'src', '*.pth')):
> dest = join(self.install_dir, basename(path))
> self.copy_file(path, dest)
>
>
> Thanks,
> -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
>
> On Tue, Mar 24, 2015 at 11:36 AM, Stuart Axon <stuaxo2@yahoo.com> wrote:
>
> Hi,
> This works from pypi - but not when installing from source with python
> setup.py install which stops this nifty thing from working:
>
> PYTHON_HUNTER="module='os.path'" python yourapp.py
>
>
> Sandbox monkeypatches os.file, so I think it catches you using copy.
> Maybe we need a common API for code that runs at startup?
>
> S++
>
>
>
> On Tuesday, March 24, 2015 3:56 PM, Ionel Cristian Mărieș <
> contact@ionelmc.ro> wrote:
>
>
>
> Hey,
>
> If you just want to copy a out-of-package file into site-package you could
> just override the build command and copy it there (in the build dir).
> Here's an example:
> https://github.com/ionelmc/python-hunter/blob/master/setup.py#L27-L31 -
> it seems to work fine with wheels.
>
>
>
> Thanks,
> -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
>
> On Mon, Mar 16, 2015 at 11:02 AM, Stuart Axon <stuaxo2@yahoo.com> wrote:
>
> Hi All
> This, and another memory-leak bug were triggered by the sandbox.
> Would it be possible to either add an API to exempt files, or just allow
> writing within site packages, even if just for .pth files ?
>
> I'm monkey patching around these for now
> https://github.com/stuaxo/vext/blob/master/setup.py#L16
>
> S++
>
>
>
> On Thursday, March 12, 2015 2:54 PM, Stuart Axon <stuaxo2@yahoo.com>
> wrote:
>
>
>
> For closure: The solution was to make a Command class + implement
> finalize_options to fixup the paths in distribution.data_files.
>
>
> Source:
>
> # https://gist.github.com/stuaxo/c76a042cb7aa6e77285b
> """
> Install a file into the root of sitepackages on windows as well as linux.
>
> Under normal operation on win32 path_to_site_packages
> gets changed to '' which installs inside the .egg instead.
> """
>
> import os
>
> from distutils import sysconfig
> from distutils.command.install_data import install_data
> from setuptools import setup
>
> here = os.path.normpath(os.path.abspath(os.path.dirname(__file__)))
>
> site_packages_path = sysconfig.get_python_lib()
> site_packages_files = ['TEST_FILE.TXT']
>
> class _install_data(install_data):
> def finalize_options(self):
> """
> On win32 the files here are changed to '' which
> ends up inside the .egg, change this back to the
> absolute path.
> """
> install_data.finalize_options(self)
> global site_packages_files
> for i, f in enumerate(list(self.distribution.data_files)):
> if not isinstance(f, basestring):
> folder, files = f
> if files == site_packages_files:
> # Replace with absolute path version
> self.distribution.data_files[i] = (site_packages_path,
> files)
>
> setup(
> cmdclass={'install_data': _install_data},
> name='test_install',
> version='0.0.1',
>
> description='',
> long_description='',
> url='https://example.com',
> author='Stuart Axon',
> author_email='stuaxo2@yahoo.com',
> license='PD',
> classifiers=[],
> keywords='',
> packages=[],
>
> install_requires=[],
>
> data_files=[
> (site_packages_path, site_packages_files),
> ],
>
> )
>
>
>
> On Tue, 10 Mar, 2015 at 11:29 PM, Stuart Axon <stuaxo2@yahoo.com> wrote:
>
> I had more of a dig into this, with a minimal setup.py:
> https://gist.github.com/stuaxo/c76a042cb7aa6e77285b setup calls
> install_data On win32 setup.py calls install_data which copies the file
> into the egg - even though I have given the absolute path to sitepackages
> C:\> python setup.py install .... running install_data creating
> build\bdist.win32\egg copying TEST_FILE.TXT -> build\bdist.win32\egg\ ....
> On Linux the file is copied to the right path: $ python setup.py install
> ..... installing package data to build/bdist.linux-x86_64/egg running
> install_data copying TEST_FILE.TXT ->
> /mnt/data/home/stu/.virtualenvs/tmpv/lib/python2.7/site-packages ....
> *something* is normalising my absolute path to site packages into just '' -
> it's possible to see by looking at self.data_files in the 'run' function
> in: distutils/command/install_data.py - on windows it the first part has
> been changed to '' unlike on linux where it's the absolute path I set...
> still not sure where it's happening though. *This all took a while, as
> rebuilt VM and verified on 2.7.8 and 2.7.9.. S++
>
> On Monday, March 9, 2015 12:17 AM, Stuart Axon <stuaxo2@yahoo.com> wrote:
> > I had a further look - and on windows the file ends up inside the .egg
> file, on linux it ends up inside the site packages as intended. At a guess
> it seems like there might be a bug in the path handling on windows. .. I
> wonder if it's something like this
> http://stackoverflow.com/questions/4579908/cross-platform-splitting-of-path-in-python
> which seems an easy way to get an off-by-one error in a path ?
>
>
>
>
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
>
>
>
>
>
>
[Attachment #5 (text/html)]
<div dir="ltr"><div class="gmail_default" style="font-family:trebuchet \
ms,sans-serif">Also, a similar command subclass can be written for `develop`. So far \
i got 3 subclasses, for: build, easy_install and develop. Did I miss something \
important?<br></div></div><div class="gmail_extra"><br clear="all"><div><div \
class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div><span \
style="font-family:trebuchet ms,sans-serif"><span \
style="color:rgb(51,51,51)"><br><font><span \
style="color:rgb(51,51,51)">Thanks,</span><br><span style="color:rgb(153,153,153)">-- \
Ionel</span></font></span><font><font style="color:rgb(153,153,153)"> Cristian \
Mărieș, <a href="http://blog.ionelmc.ro" \
target="_blank">http://blog.ionelmc.ro</a><br></font></font></span></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Wed, Mar 25, 2015 at 2:51 PM, Stuart Axon <span \
dir="ltr"><<a href="mailto:stuaxo2@yahoo.com" \
target="_blank">stuaxo2@yahoo.com</a>></span> wrote:<br><blockquote \
class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc \
solid;padding-left:1ex"><div><div \
style="color:#000;background-color:#fff;font-family:HelveticaNeue,Helvetica \
Neue,Helvetica,Arial,Lucida Grande,sans-serif;font-size:16px">That looks much cleaner \
than my one, I'll give it a try.. does it work on python3, just found out my \
one does not.<span class="HOEnZb"><font \
color="#888888"><br><div><span></span></div><div> \
</div><div><div>S++</div></div></font></span><div><div class="h5"> \
<br><div><br><br></div><div style="display:block"> <div \
style="font-family:HelveticaNeue,Helvetica Neue,Helvetica,Arial,Lucida \
Grande,sans-serif;font-size:16px"> <div style="font-family:HelveticaNeue,Helvetica \
Neue,Helvetica,Arial,Lucida Grande,sans-serif;font-size:16px"> <div dir="ltr"> <font \
face="Arial"> On Wednesday, March 25, 2015 9:03 AM, Ionel Cristian Mărieș <<a \
href="mailto:contact@ionelmc.ro" target="_blank">contact@ionelmc.ro</a>> \
wrote:<br> </font> </div> <blockquote style="border-left:2px solid \
rgb(16,16,255);margin-left:5px;margin-top:5px;padding-left:5px"> <br><br> \
<div><div><div><div dir="ltr"><div style="font-family:trebuchet ms,sans-serif">This \
seems to do the trick:<br clear="none"><br clear="none"><span \
style="font-family:monospace,monospace">class EasyInstallWithPTH(easy_install):<br \
clear="none"> def run(self):<br clear="none"> \
easy_install.run(self)<br clear="none"> for path in \
glob(join(dirname(__file__), 'src', '*.pth')):<br clear="none"> \
dest = join(self.install_dir, basename(path))<br clear="none"> \
self.copy_file(path, dest)</span><br clear="none"></div></div><div><br \
clear="all"><div><div><div dir="ltr"><div><div dir="ltr"><div><div><span \
style="font-family:trebuchet ms,sans-serif"><span style="color:rgb(51,51,51)"><br \
clear="none"><font><span style="color:rgb(51,51,51)">Thanks,</span><br \
clear="none"><span style="color:rgb(153,153,153)">-- \
Ionel</span></font></span><font><font style="color:rgb(153,153,153)"> Cristian \
Mărieș, <a rel="nofollow" shape="rect" href="http://blog.ionelmc.ro/" \
target="_blank">http://blog.ionelmc.ro</a><br \
clear="none"></font></font></span></div></div></div></div></div></div></div> <br \
clear="none"><div><div>On Tue, Mar 24, 2015 at 11:36 AM, Stuart Axon <span \
dir="ltr"><<a rel="nofollow" shape="rect" href="mailto:stuaxo2@yahoo.com" \
target="_blank">stuaxo2@yahoo.com</a>></span> wrote:<br clear="none"><blockquote \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div \
style="color:#000;background-color:#fff;font-family:HelveticaNeue,Helvetica \
Neue,Helvetica,Arial,Lucida Grande,sans-serif;font-size:16px"><div><div><div \
style="color:#000;background-color:#fff;font-family:HelveticaNeue,Helvetica \
Neue,Helvetica,Arial,Lucida Grande,sans-serif;font-size:16px"><div>Hi,</div><div \
dir="ltr"> This works from pypi - but not when installing from source with python \
setup.py install which stops this nifty thing from working:<br \
clear="none"></div><div dir="ltr"><br \
clear="none"></div><pre>PYTHON_HUNTER="module='os.path'" python \
yourapp.py </pre><div> </div><div><div><div dir="ltr">Sandbox monkeypatches os.file, \
so I think it catches you using copy. Maybe we need a common API for code that \
runs at startup?<span><font color="#888888"><br \
clear="none"></font></span></div><span><font color="#888888"><br \
clear="none"></font></span><div>S++</div></div></div> <br clear="none"><div><br \
clear="none"><br clear="none"></div><div></div></div></div></div><div><div><div> <div \
style="font-family:HelveticaNeue,Helvetica Neue,Helvetica,Arial,Lucida \
Grande,sans-serif;font-size:16px"> <div style="font-family:HelveticaNeue,Helvetica \
Neue,Helvetica,Arial,Lucida Grande,sans-serif;font-size:16px"> <div dir="ltr"> <font \
face="Arial"> On Tuesday, March 24, 2015 3:56 PM, Ionel Cristian Mărieș <<a \
rel="nofollow" shape="rect" href="mailto:contact@ionelmc.ro" \
target="_blank">contact@ionelmc.ro</a>> wrote:<br clear="none"> </font> </div> \
<blockquote style="border-left:2px solid \
rgb(16,16,255);margin-left:5px;margin-top:5px;padding-left:5px"> <br \
clear="none"><br clear="none"> <div><div><div><div dir="ltr"><div \
style="font-family:trebuchet ms,sans-serif">Hey,<br clear="none"><br \
clear="none"></div><div style="font-family:trebuchet ms,sans-serif">If you just want \
to copy a out-of-package file into site-package you could just override the build \
command and copy it there (in the build dir). Here's an example: <a \
rel="nofollow" shape="rect" \
href="https://github.com/ionelmc/python-hunter/blob/master/setup.py#L27-L31" \
target="_blank">https://github.com/ionelmc/python-hunter/blob/master/setup.py#L27-L31</a> \
- it seems to work fine with wheels.<br clear="none"><br \
clear="none"></div></div><div><br clear="all"><div><div><div dir="ltr"><div><div \
dir="ltr"><div><div><span style="font-family:trebuchet ms,sans-serif"><span \
style="color:rgb(51,51,51)"><br clear="none"><font><span \
style="color:rgb(51,51,51)">Thanks,</span><br clear="none"><span \
style="color:rgb(153,153,153)">-- Ionel</span></font></span><font><font \
style="color:rgb(153,153,153)"> Cristian Mărieș, <a rel="nofollow" shape="rect" \
href="http://blog.ionelmc.ro/" target="_blank">http://blog.ionelmc.ro</a><br \
clear="none"></font></font></span></div></div></div></div></div></div></div> <br \
clear="none"><div>On Mon, Mar 16, 2015 at 11:02 AM, Stuart Axon <span \
dir="ltr"><<a rel="nofollow" shape="rect" href="mailto:stuaxo2@yahoo.com" \
target="_blank">stuaxo2@yahoo.com</a>></span> wrote:<br clear="none"><blockquote \
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div \
style="color:#000;background-color:#fff;font-family:HelveticaNeue,Helvetica \
Neue,Helvetica,Arial,Lucida Grande,sans-serif;font-size:16px"><div>Hi All</div><div \
dir="ltr"> This, and another memory-leak bug were triggered by the sandbox. \
Would it be possible to either add an API to exempt files, or just allow writing \
within site packages, even if just for .pth files ?<br clear="none"></div><div \
dir="ltr"><br clear="none"></div><div dir="ltr">I'm monkey patching around these \
for now<br clear="none"></div><div dir="ltr"><a rel="nofollow" shape="rect" \
href="https://github.com/stuaxo/vext/blob/master/setup.py#L16" \
target="_blank">https://github.com/stuaxo/vext/blob/master/setup.py#L16</a><span><font \
color="#888888"><br clear="none"></font></span></div><span><font \
color="#888888"></font></span><div><span></span></div><div><br \
clear="none"><span></span></div><div><div>S++</div></div><div><div> <br \
clear="none"><div><br clear="none"><br clear="none"></div><div style="display:block"> \
<div style="font-family:HelveticaNeue,Helvetica Neue,Helvetica,Arial,Lucida \
Grande,sans-serif;font-size:16px"> <div style="font-family:HelveticaNeue,Helvetica \
Neue,Helvetica,Arial,Lucida Grande,sans-serif;font-size:16px"> <div dir="ltr"> <font \
face="Arial"> On Thursday, March 12, 2015 2:54 PM, Stuart Axon <<a rel="nofollow" \
shape="rect" href="mailto:stuaxo2@yahoo.com" \
target="_blank">stuaxo2@yahoo.com</a>> wrote:<br clear="none"> </font> </div> \
<blockquote style="border-left:2px solid \
rgb(16,16,255);margin-left:5px;margin-top:5px;padding-left:5px"> <br \
clear="none"><br clear="none"> <div><div><div>For closure: The solution was to make \
a Command class + implement finalize_options to fixup the paths in \
distribution.data_files.<div><br clear="none"></div><div><br \
clear="none"></div><div>Source:</div><div><br clear="none"></div><div># <a \
rel="nofollow" shape="rect" \
href="https://gist.github.com/stuaxo/c76a042cb7aa6e77285b" \
target="_blank">https://gist.github.com/stuaxo/c76a042cb7aa6e77285b</a></div><div>"""</div><div>Install \
a file into the root of sitepackages on windows as well as linux.</div><div><br \
clear="none"></div><div>Under normal operation on win32 \
path_to_site_packages</div><div>gets changed to '' which installs inside the \
.egg instead.</div><div>"""</div><div><br \
clear="none"></div><div>import os</div><div><br clear="none"></div><div>from \
distutils import sysconfig</div><div>from distutils.command.install_data import \
install_data</div><div>from setuptools import setup</div><div><br \
clear="none"></div><div>here = \
os.path.normpath(os.path.abspath(os.path.dirname(__file__)))</div><div><br \
clear="none"></div><div>site_packages_path = \
sysconfig.get_python_lib()</div><div>site_packages_files = \
['TEST_FILE.TXT']</div><div><br clear="none"></div><div>class \
_install_data(install_data):</div><div> def finalize_options(self):</div><div> \
"""</div><div> On win32 the files here are changed to \
'' which</div><div> ends up inside the .egg, change this back to \
the</div><div> absolute path.</div><div> \
"""</div><div> \
install_data.finalize_options(self)</div><div> global \
site_packages_files</div><div> for i, f in \
enumerate(list(self.distribution.data_files)):</div><div> if not \
isinstance(f, basestring):</div><div> folder, files = \
f</div><div> if files == site_packages_files:</div><div> \
# Replace with absolute path version</div><div> \
self.distribution.data_files[i] = (site_packages_path, files)</div><div><br \
clear="none"></div><div>setup(</div><div> cmdclass={'install_data': \
_install_data},</div><div> name='test_install',</div><div> \
version='0.0.1',</div><div><br clear="none"></div><div> \
description='',</div><div> long_description='',</div><div> \
url='<a rel="nofollow" shape="rect" href="https://example.com/" \
target="_blank">https://example.com</a>',</div><div> author='Stuart \
Axon',</div><div> author_email='<a rel="nofollow" shape="rect" \
href="mailto:stuaxo2@yahoo.com" \
target="_blank">stuaxo2@yahoo.com</a>',</div><div> \
license='PD',</div><div> classifiers=[],</div><div> \
keywords='',</div><div> packages=[],</div><div><br \
clear="none"></div><div> install_requires=[],</div><div><br \
clear="none"></div><div> data_files=[</div><div> (site_packages_path, \
site_packages_files),</div><div> ],</div><div><br \
clear="none"></div><div>)</div><div><br clear="none"></div><div><br \
clear="none"><div><br clear="none">On Tue, 10 Mar, 2015 at 11:29 PM, Stuart Axon \
<<a rel="nofollow" shape="rect" href="mailto:stuaxo2@yahoo.com" \
target="_blank">stuaxo2@yahoo.com</a>> wrote:<br clear="none"> <blockquote \
type="cite"><div style="white-space:pre-wrap">I had more of a dig into this, with a \
minimal setup.py:
<a rel="nofollow" shape="rect" \
href="https://gist.github.com/stuaxo/c76a042cb7aa6e77285b" \
target="_blank">https://gist.github.com/stuaxo/c76a042cb7aa6e77285b</a>
setup calls install_data
On win32 setup.py calls install_data which copies the file into the egg - even though \
I have given the absolute path to sitepackages
C:\> python setup.py install
....
running install_data
creating build\bdist.win32\egg
copying TEST_FILE.TXT -> build\bdist.win32\egg\
....
On Linux the file is copied to the right path:
$ python setup.py install
.....
installing package data to build/bdist.linux-x86_64/egg
running install_data
copying TEST_FILE.TXT -> \
/mnt/data/home/stu/.virtualenvs/tmpv/lib/python2.7/site-packages
....
*something* is normalising my absolute path to site packages into just '' - \
it's possible to see by looking at self.data_files in the 'run' function \
in:
distutils/command/install_data.py
- on windows it the first part has been changed to '' unlike on linux where \
it's the absolute path I set... still not sure where it's happening though.
*This all took a while, as rebuilt VM and verified on 2.7.8 and 2.7.9..
S++
<blockquote> On Monday, March 9, 2015 12:17 AM, Stuart Axon <<a rel="nofollow" \
shape="rect" href="mailto:stuaxo2@yahoo.com" \
target="_blank">stuaxo2@yahoo.com</a>> wrote: > I had a further look - and on \
windows the file ends up inside the .egg file, on linux it ends up inside the site \
packages as intended.
At a guess it seems like there might be a bug in the path handling on windows.
.. I wonder if it's something like this
<a rel="nofollow" shape="rect" \
href="http://stackoverflow.com/questions/4579908/cross-platform-splitting-of-path-in-python" \
target="_blank">http://stackoverflow.com/questions/4579908/cross-platform-splitting-of-path-in-python</a>
which seems an easy way to get an off-by-one error in a path ?
</blockquote></div></blockquote></div></div></div></div><br clear="none"><br \
clear="none"></div> </blockquote> </div> </div> \
</div></div></div></div></div></div><br \
clear="none">_______________________________________________<br clear="none"> \
Distutils-SIG maillist - <a rel="nofollow" shape="rect" \
href="mailto:Distutils-SIG@python.org" \
target="_blank">Distutils-SIG@python.org</a><br clear="none"> <a rel="nofollow" \
shape="rect" href="https://mail.python.org/mailman/listinfo/distutils-sig" \
target="_blank">https://mail.python.org/mailman/listinfo/distutils-sig</a><br \
clear="none"> <br clear="none"></blockquote></div><br \
clear="none"></div></div></div><br clear="none"><br clear="none"></div> </blockquote> \
</div> </div> </div></div></div></div></div></blockquote></div></div><br \
clear="none"></div></div></div><br><br></div> </blockquote> </div> </div> \
</div></div></div></div></div></blockquote></div><br></div>
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic