[prev in list] [next in list] [prev in thread] [next in thread]
List: kde-bindings
Subject: Re: [Kde-bindings] Progress report on Py*5 binding generation
From: Shaheed Haque <srhaque () theiet ! org>
Date: 2016-04-21 16:54:50
Message-ID: CAHAc2jeC3WmA850Auh1enYn6XJo6prJDipTN34OtJaC8BxRKAA () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi Steve,
On 20 April 2016 at 22:27, Stephen Kelly <steveire@gmail.com> wrote:
> Shaheed Haque wrote:
>
>> Specifically, the core logic in sip_generator.py has almost no
>> references to anything Qt or KDE specific (there are a few, though I
>> expect them to be generally non-intrusive, and they could be factored
>> out if anybody cared). The obvious exception is that the CLI defaults
>> are Qt and KDE centric, but they are just defaults. For normal sized
>> projects with a sane number of files to process, this may be all you
>> need, along with your custom rules_XXX.py.
>
> Yes, the defaults got in my way already :).
>
>>> I tried running it on just one header so I can learn some fundamentals
of
>>> what is happening here. I got an error when I tried to generate:
>>>
>>> $ ./sip_generator.py /usr/include/KF5/KItemModels/kselectionproxymodel.h
>>> > sip/kselectionproxymodel.sip
>>> ERROR: Parse error /usr/include/x86_64-linux-
>>> gnu/qt5/QtCore/qglobal.h:38[10] 'stddef.h' file not found
>>
>> Yes; the Clang LLVM compiler front end works like a real compiler :-).
>
> I could get past the above error by specifying on the command line include
> directories which are usually built-in when using the clang driver.
>
> However, after that I get more errors:
>
> /usr/include/x86_64-linux-gnu/c++/4.9.3/bits/os_defines.h:44:5: error:
> function-like macro '__GLIBC_PREREQ' is not defined
>
> Is there no way to tell libclang to use the built-in includes and defines?
You are treading a well-work path :-) in that I did exactly the same thing
before getting to the point where I gave up on the basis that the failure
to find stddef.h *appears* not to cause any other issues. I tried several
things, including giving it a local dummy stddef.h (which later on grew to
include a few macros and suchlike), along with a few other sibling empty
system header files before I decided I was on a hiding to nothing. So far,
apart from that scary looking error, it seems to all work.
> Anyway, we should be able to pass the correct includes to the tool with
some
> cmake macro.
In due course, avoiding the error message would be nice indeed.
>> However, the specific reason for the failure in your case is that the
>> individual SIP files generated from .h files are not intended to be
>> compiled standalone (note the bit at the end where it thinks you are
>> compiling a C module?). Instead, in each directory, there is a
>> <whatever>mod.sip file which collects together all the other files
>> into a single compilable unit. Note how the file <whatever>mod.sip
>> file is the one that is compiled by default:
>
> Note that I am trying to avoid the built-in KF5 stuff in the tools, so
that
>
> 1) I can understand the tool and help with the cmake stuff
> 2) I can know how to use this with non-KF5 libraries
Makes sense.
> To that end, in addition to sip/kselectionproxymodel.sip I now have:
>
> $ cat sip/kitemmodelsmod.sip
>
> %Module PyKDE5.kitemmodels
>
> %ModuleHeaderCode
> #pragma GCC visibility push(default)
> %End
>
> %Import QtCore/QtCoremod.sip
>
> %Include kselectionproxymodel.sip
>
>
> Is this what I am supposed to do? Here is the output I get when attempting
> to use it:
Yeeessss...it is a bit tricky...
First, if you use the "-v" flag to the sip_compiler.py, it will show you
the exact command like for SIP and later the C++ compiler/linker. That
might help you see what is going on.
Next, if you look at the XXXmod.sip (and the XXXmod.sip.tmp) that I
generate, the latter includes two things ine "individual" .sip files do not:
- A set of %Feature declarations.
- The funny "includes" extract thing you get an error on below.
The %Feature thing you may be able to ignore. It is used along with the SIP
compiler's -x (lower case X) option to try to work around the problem where
in KF5, you often get AAAmod.sip and BBBmod.sip which need to import each
other: SIP does not like this, and I use the features (and a
"modules.features" output file which you might have spotted) to deal with
issue.
For your problem, the important thing to get is that my code auto-generates
the %Imports based on analysing the first-level #includes of the original
header files (as reported to me by Clang) and crucially this analysis is
also used to track the #includes which the C++ compile steps needs for
those same dependencies. Now, in order to pass this information from the
sip_bulk_generator.py stage to the sip_compiler stage via processing
through the SIP compiler, I include the needed information in the
XXXmod.sip file in an %Extract section named "includes". Then, in order to
unpack this information for use by the C++ compiler...
> $ ./sip_compiler.py --select @kitemmodelsmod.sip sip cxx
> INFO: Creating cxx/
> ERROR: sip: There is no extract defined with the identifier "includes"
while
> processing sip/kitemmodelsmod.sip
> Traceback (most recent call last):
> File "./sip_compiler.py", line 211, in main
> d.process_one_module(args.select)
> File "./sip_compiler.py", line 145, in process_one_module
> self._run_command(cmd)
> File "./sip_compiler.py", line 171, in _run_command
> raise RuntimeError(stdout)
> RuntimeError: sip: There is no extract defined with the identifier
> "includes"
>
> The error seems to come from sip. Do you know what it means?
...I used the SIP compiler's -X (upper case X) option to extract the
%Extract data, and generate the "-I foo -I bar" paths for the C++ compiler.
So, you can just put a %Extract section like this into your file:
%Extract(id=includes)
/usr/include/KF5/KDBusAddons
/usr/include/x86_64-linux-gnu/qt5/QtCore
/usr/include/x86_64-linux-gnu/qt5/QtDBus
%End
Possibly easier would be to just let sip_bulk_generator.py do its thing. If
you want to only process part of KF5, you could just use the --selector
option to say something like:
./sip_bulk_generator --select KItemModels <sip-output-dir>
./sip_compiler --select KItemModels <sip-input-dir-from-above>
<cxx-output-dir>
Equally, you are welcome to just run stuff by hand (as noted, use the -v
option to see what the compiler stage is doing).
>> I believe this is related to needing to link against the underlying
>> KF5 library, but I have not automated the hookup needed to test this
>> theory:
>
> I can help with the cmake/linking side, but I'm still struggling to get
the
> tool to work for a trivial case above. Can you help with that?
Hope the above helps, if needed we can work together interactively
timezones etc. permitting. Send me a PM if you think that is needed (I'm in
the UK).
Thanks, Shaheed
> Thanks,
>
> Steve.
>
> _______________________________________________
> Kde-bindings mailing list
> Kde-bindings@kde.org
> https://mail.kde.org/mailman/listinfo/kde-bindings
[Attachment #5 (text/html)]
<div dir="ltr"><div><div>Hi Steve,<br><br>On 20 April 2016 at 22:27, Stephen Kelly \
<<a href="mailto:steveire@gmail.com" target="_blank">steveire@gmail.com</a>> \
wrote:<br>> Shaheed Haque wrote:<br>><br>>> Specifically, the core logic \
in sip_generator.py has almost no<br>>> references to anything Qt or KDE \
specific (there are a few, though I<br>>> expect them to be generally \
non-intrusive, and they could be factored<br>>> out if anybody cared). The \
obvious exception is that the CLI defaults<br>>> are Qt and KDE centric, but \
they are just defaults. For normal sized<br>>> projects with a sane number of \
files to process, this may be all you<br>>> need, along with your custom \
rules_XXX.py.<br>><br>> Yes, the defaults got in my way already \
:).<br>><br>>>> I tried running it on just one header so I can learn some \
fundamentals of<br>>>> what is happening here. I got an error when I tried \
to generate:<br>>>><br>>>> $ ./sip_generator.py \
/usr/include/KF5/KItemModels/kselectionproxymodel.h<br>>>> > \
sip/kselectionproxymodel.sip<br>>>> ERROR: Parse error \
/usr/include/x86_64-linux-<br>>>> gnu/qt5/QtCore/qglobal.h:38[10] \
'stddef.h' file not found<br>>><br>>> Yes; the Clang LLVM \
compiler front end works like a real compiler :-).<br>><br>> I could get past \
the above error by specifying on the command line include<br>> directories which \
are usually built-in when using the clang driver.<br>><br>> However, after that \
I get more errors:<br>><br>> \
/usr/include/x86_64-linux-gnu/c++/4.9.3/bits/os_defines.h:44:5: error:<br>> \
function-like macro '__GLIBC_PREREQ' is not defined<br>><br>> Is there \
no way to tell libclang to use the built-in includes and defines?<br><br>You are \
treading a well-work path :-) in that I did exactly the same thing before getting to \
the point where I gave up on the basis that the failure to find stddef.h *appears* \
not to cause any other issues. I tried several things, including giving it a local \
dummy stddef.h (which later on grew to include a few macros and suchlike), along with \
a few other sibling empty system header files before I decided I was on a hiding to \
nothing. So far, apart from that scary looking error, it seems to all \
work.<br><br>> Anyway, we should be able to pass the correct includes to the tool \
with some<br>> cmake macro.<br><br>In due course, avoiding the error message would \
be nice indeed.<br><br>>> However, the specific reason for the failure in your \
case is that the<br>>> individual SIP files generated from .h files are not \
intended to be<br>>> compiled standalone (note the bit at the end where it \
thinks you are<br>>> compiling a C module?). Instead, in each directory, there \
is a<br>>> <whatever>mod.sip file which collects together all the other \
files<br>>> into a single compilable unit. Note how the file \
<whatever>mod.sip<br>>> file is the one that is compiled by \
default:<br>><br>> Note that I am trying to avoid the built-in KF5 stuff in the \
tools, so that<br>><br>> 1) I can understand the tool and help with the cmake \
stuff<br>> 2) I can know how to use this with non-KF5 libraries<br><br>Makes \
sense.<br><br>> To that end, in addition to sip/kselectionproxymodel.sip I now \
have:<br>><br>> $ cat sip/kitemmodelsmod.sip<br>><br>> %Module \
PyKDE5.kitemmodels<br>><br>> %ModuleHeaderCode<br>> #pragma GCC \
visibility push(default)<br>> %End<br>><br>> %Import \
QtCore/QtCoremod.sip<br>><br>> %Include \
kselectionproxymodel.sip<br>><br>><br>> Is this what I am supposed to do? \
Here is the output I get when attempting<br>> to use it:<br><br>Yeeessss...it is a \
bit tricky...<br><br>First, if you use the "-v" flag to the \
sip_compiler.py, it will show you the exact command like for SIP and later the C++ \
compiler/linker. That might help you see what is going on.<br><br>Next, if you look \
at the XXXmod.sip (and the XXXmod.sip.tmp) that I generate, the latter includes two \
things ine "individual" .sip files do not:<br><ul><li>A set of %Feature \
declarations.</li><li>The funny "includes" extract thing you get an error \
on below.<br></li></ul>The %Feature thing you may be able to ignore. It is used along \
with the SIP compiler's -x (lower case X) option to try to work around the \
problem where in KF5, you often get AAAmod.sip and BBBmod.sip which need to import \
each other: SIP does not like this, and I use the features (and a \
"modules.features" output file which you might have spotted) to deal with \
issue.<br><br>For your problem, the important thing to get is that my code \
auto-generates the %Imports based on analysing the first-level #includes of the \
original header files (as reported to me by Clang) and crucially this analysis is \
also used to track the #includes which the C++ compile steps needs for those same \
dependencies. Now, in order to pass this information from the sip_bulk_generator.py \
stage to the sip_compiler stage via processing through the SIP compiler, I include \
the needed information in the XXXmod.sip file in an %Extract section named \
"includes". Then, in order to unpack this information for use by the C++ \
compiler...<br><br>> $ ./sip_compiler.py --select @kitemmodelsmod.sip sip \
cxx<br>> INFO: Creating cxx/<br>> ERROR: sip: There is no extract defined with \
the identifier "includes" while<br>> processing \
sip/kitemmodelsmod.sip<br>> Traceback (most recent call last):<br>> File \
"./sip_compiler.py", line 211, in main<br>> \
d.process_one_module(args.select)<br>> File "./sip_compiler.py", line \
145, in process_one_module<br>> self._run_command(cmd)<br>> File \
"./sip_compiler.py", line 171, in _run_command<br>> raise \
RuntimeError(stdout)<br>> RuntimeError: sip: There is no extract defined with the \
identifier<br>> "includes"<br>><br>> The error seems to come from \
sip. Do you know what it means?<br><br></div>...I used the SIP compiler's -X \
(upper case X) option to extract the %Extract data, and generate the "-I foo -I \
bar" paths for the C++ compiler. So, you can just put a %Extract section like \
this into your file:<br><br>%Extract(id=includes)<br>/usr/include/KF5/KDBusAddons<br>/ \
usr/include/x86_64-linux-gnu/qt5/QtCore<br>/usr/include/x86_64-linux-gnu/qt5/QtDBus<br>%End<br><br></div><div>Possibly \
easier would be to just let sip_bulk_generator.py do its thing. If you want to only \
process part of KF5, you could just use the --selector option to say something \
like:<br><br></div><div>./sip_bulk_generator --select KItemModels \
<sip-output-dir><br></div><div>./sip_compiler --select KItemModels \
<sip-input-dir-from-above> \
<cxx-output-dir><br></div><div><br></div><div>Equally, you are welcome to just \
run stuff by hand (as noted, use the -v option to see what the compiler stage is \
doing).<br></div><div><div><br>>> I believe this is related to needing to link \
against the underlying<br>>> KF5 library, but I have not automated the hookup \
needed to test this<br>>> theory:<br>><br>> I can help with the \
cmake/linking side, but I'm still struggling to get the<br>> tool to work for \
a trivial case above. Can you help with that?<br><br></div><div>Hope the above helps, \
if needed we can work together interactively timezones etc. permitting. Send me a PM \
if you think that is needed (I'm in the UK).<br><br></div><div>Thanks, \
Shaheed<br></div><div><br>> Thanks,<br>><br>> Steve.<br>><br>> \
_______________________________________________<br>> Kde-bindings mailing \
list<br>> <a href="mailto:Kde-bindings@kde.org" \
target="_blank">Kde-bindings@kde.org</a><br>> <a \
href="https://mail.kde.org/mailman/listinfo/kde-bindings" \
target="_blank">https://mail.kde.org/mailman/listinfo/kde-bindings</a><br><br></div></div></div>
[Attachment #6 (text/plain)]
_______________________________________________
Kde-bindings mailing list
Kde-bindings@kde.org
https://mail.kde.org/mailman/listinfo/kde-bindings
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic