[prev in list] [next in list] [prev in thread] [next in thread]
List: kde-core-devel
Subject: Re: Review Request 126345: PythonMacros: specify destination directory in byte-compiled files
From: "Raphael Kubo da Costa" <rakuco () FreeBSD ! org>
Date: 2015-12-16 17:30:44
Message-ID: 20151216173044.7434.3706 () mimi ! kde ! org
[Download RAW message or body]
--===============7747995590863427699==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://git.reviewboard.kde.org/r/126345/
-----------------------------------------------------------
(Updated Dec. 16, 2015, 5:30 p.m.)
Status
------
This change has been marked as submitted.
Review request for Build System, kdelibs and Alex Merry.
Changes
-------
Submitted with commit 94f1d2fa9582a2942d5154b85c849cc3c6140e31 by Raphael Kubo da \
Costa to branch KDE/4.14.
Repository: kdelibs
Description
-------
The `PYTHON_INSTALL()` macro is a wrapper around the `py_compile` Python module that \
also installs the byte-code (.pyc) file it generates.
However, when a .py file is passed to `py_compile` without any additional arguments, \
its full path is recorded in the .pyc file. This is problematic, as most \
distributions install all files into a build root instead of simply copying files to \
`/` as part of the packaging process. In this case, the generated .pyc file will have \
something like /wrkdir/buildroot/usr/lib/python2.7/site-packages/Foo/my_module.py
in it. Not only does this show up in exception tracebacks, but if the user later \
invokes `my_module.py` and has write access to `my_module`'s directory, \
`my_module.pyc` will be rewritten with the right path to `my_module.py` (without the \
build root). This can lead to uninstallation errors if the package management system \
checks each file before removal, for example.
Fix it by rewritting the `PythonCompile.py` script so that it takes a \
`--destination-dir` argument that we use to pass the full path to `my_module.py` \
instead of letting it be (wrongly) deduced.
Diffs
-----
cmake/modules/PythonCompile.py 156fea2
cmake/modules/PythonMacros.cmake 6a82d88
Diff: https://git.reviewboard.kde.org/r/126345/diff/
Testing
-------
Fedora has packaging macros that will regenerate .pyc and .pyo files with the right \
paths as part of the build, so it is not affected. Debian disables this macro in \
pykde4, FreeBSD and openSUSE contain wrong paths in its .pyc files. With this patch, \
I was able to verify with `hexdump` that the right path is present in the .pyc files \
installed by pykde4.
Thanks,
Raphael Kubo da Costa
--===============7747995590863427699==
MIME-Version: 1.0
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 7bit
<html>
<body>
<div style="font-family: Verdana, Arial, Helvetica, Sans-Serif;">
<table bgcolor="#f9f3c9" width="100%" cellpadding="12" style="border: 1px #c9c399 \
solid; border-radius: 6px; -moz-border-radius: 6px; -webkit-border-radius: 6px;"> \
<tr> <td>
This is an automatically generated e-mail. To reply, visit:
<a href="https://git.reviewboard.kde.org/r/126345/">https://git.reviewboard.kde.org/r/126345/</a>
</td>
</tr>
</table>
<br />
<table bgcolor="#e0e0e0" width="100%" cellpadding="12" style="border: 1px gray solid; \
border-radius: 6px; -moz-border-radius: 6px; -webkit-border-radius: 6px;"> <tr>
<td>
<h1 style="margin: 0; padding: 0; font-size: 10pt;">This change has been marked as \
submitted.</h1> </td>
</tr>
</table>
<br />
<table bgcolor="#fefadf" width="100%" cellspacing="0" cellpadding="12" style="border: \
1px #888a85 solid; border-radius: 6px; -moz-border-radius: 6px; \
-webkit-border-radius: 6px;"> <tr>
<td>
<div>Review request for Build System, kdelibs and Alex Merry.</div>
<div>By Raphael Kubo da Costa.</div>
<p style="color: grey;"><i>Updated Dec. 16, 2015, 5:30 p.m.</i></p>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Changes</h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: \
1px solid #b8b5a0"> <tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;">Submitted with commit 94f1d2fa9582a2942d5154b85c849cc3c6140e31 by \
Raphael Kubo da Costa to branch KDE/4.14.</pre> </td>
</tr>
</table>
<div style="margin-top: 1.5em;">
<b style="color: #575012; font-size: 10pt;">Repository: </b>
kdelibs
</div>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Description </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" \
style="border: 1px solid #b8b5a0"> <tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;"><p style="padding: 0;text-rendering: inherit;margin: 0;line-height: \
inherit;white-space: inherit;">The <code style="text-rendering: inherit;color: \
#4444cc;padding: 0;white-space: normal;margin: 0;line-height: \
inherit;">PYTHON_INSTALL()</code> macro is a wrapper around the <code \
style="text-rendering: inherit;color: #4444cc;padding: 0;white-space: normal;margin: \
0;line-height: inherit;">py_compile</code> Python module that also installs the \
byte-code (.pyc) file it generates.</p> <p style="padding: 0;text-rendering: \
inherit;margin: 0;line-height: inherit;white-space: inherit;">However, when a .py \
file is passed to <code style="text-rendering: inherit;color: #4444cc;padding: \
0;white-space: normal;margin: 0;line-height: inherit;">py_compile</code> without any \
additional arguments, its full path is recorded in the .pyc file. This is \
problematic, as most distributions install all files into a build root instead of \
simply copying files to <code style="text-rendering: inherit;color: #4444cc;padding: \
0;white-space: normal;margin: 0;line-height: inherit;">/</code> as part of the \
packaging process. In this case, the generated .pyc file will have something like \
/wrkdir/buildroot/usr/lib/python2.7/site-packages/Foo/my_module.py in it. Not only \
does this show up in exception tracebacks, but if the user later invokes <code \
style="text-rendering: inherit;color: #4444cc;padding: 0;white-space: normal;margin: \
0;line-height: inherit;">my_module.py</code> and has write access to <code \
style="text-rendering: inherit;color: #4444cc;padding: 0;white-space: normal;margin: \
0;line-height: inherit;">my_module</code>'s directory, <code style="text-rendering: \
inherit;color: #4444cc;padding: 0;white-space: normal;margin: 0;line-height: \
inherit;">my_module.pyc</code> will be rewritten with the right path to <code \
style="text-rendering: inherit;color: #4444cc;padding: 0;white-space: normal;margin: \
0;line-height: inherit;">my_module.py</code> (without the build root). This can lead \
to uninstallation errors if the package management system checks each file before \
removal, for example.</p> <p style="padding: 0;text-rendering: inherit;margin: \
0;line-height: inherit;white-space: inherit;">Fix it by rewritting the <code \
style="text-rendering: inherit;color: #4444cc;padding: 0;white-space: normal;margin: \
0;line-height: inherit;">PythonCompile.py</code> script so that it takes a <code \
style="text-rendering: inherit;color: #4444cc;padding: 0;white-space: normal;margin: \
0;line-height: inherit;">--destination-dir</code> argument that we use to pass the \
full path to <code style="text-rendering: inherit;color: #4444cc;padding: \
0;white-space: normal;margin: 0;line-height: inherit;">my_module.py</code> instead of \
letting it be (wrongly) deduced.</p></pre> </td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Testing </h1>
<table width="100%" bgcolor="#ffffff" cellspacing="0" cellpadding="10" style="border: \
1px solid #b8b5a0"> <tr>
<td>
<pre style="margin: 0; padding: 0; white-space: pre-wrap; white-space: \
-moz-pre-wrap; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: \
break-word;"><p style="padding: 0;text-rendering: inherit;margin: 0;line-height: \
inherit;white-space: inherit;">Fedora has packaging macros that will regenerate .pyc \
and .pyo files with the right paths as part of the build, so it is not affected. \
Debian disables this macro in pykde4, FreeBSD and openSUSE contain wrong paths in its \
.pyc files. With this patch, I was able to verify with <code style="text-rendering: \
inherit;color: #4444cc;padding: 0;white-space: normal;margin: 0;line-height: \
inherit;">hexdump</code> that the right path is present in the .pyc files installed \
by pykde4.</p></pre> </td>
</tr>
</table>
<h1 style="color: #575012; font-size: 10pt; margin-top: 1.5em;">Diffs</b> </h1>
<ul style="margin-left: 3em; padding-left: 0;">
<li>cmake/modules/PythonCompile.py <span style="color: grey">(156fea2)</span></li>
<li>cmake/modules/PythonMacros.cmake <span style="color: grey">(6a82d88)</span></li>
</ul>
<p><a href="https://git.reviewboard.kde.org/r/126345/diff/" style="margin-left: \
3em;">View Diff</a></p>
</td>
</tr>
</table>
</div>
</body>
</html>
--===============7747995590863427699==--
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic