[prev in list] [next in list] [prev in thread] [next in thread]
List: gnuradio-discuss
Subject: Re: Regression in GR3.10
From: ikjtel <ikj1234i () yahoo ! com>
Date: 2022-06-23 18:01:42
Message-ID: 1525757522.6343575.1656007302913 () mail ! yahoo ! com
[Download RAW message or body]
> I think the way to properly handle GIL from long running calls
> is to add py::call_guard<py::gil_scoped_release>()
Josh
Full success! I snarfed the gr code for msg_queue, msg_handler, and message,
and added this call guard. There is no more GIL deadlock in the python OP25
apps.
Max
==============================================================================
tl;dr : following geek material may be safely skipped
Initial attempts to compile msg_handler OOT resulted in a compile error:
/usr/include/pybind11/detail/init.h:63:64: error: invalid new-expression of abstract \
class type ‘gr::op25::msg_handler'
63 | inline Class *construct_or_initialize(Args &&...args) { return new \
Class{std::forward<Args>(args)...}; } | \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In file included from \
/home/op25/repo/op25/src4/op25/gr-op25/python/op25/bindings/msg_handler_python.cc:26:
/home/op25/repo/op25/src4/op25/gr-op25/python/op25/bindings/../../../include/gnuradio/op25/msg_handler.h:27:16: \
note: because the following virtual functions are pure within \
‘gr::op25::msg_handler':
==============================================================================
In gr3.8 (swig) you get this error:
from gnuradio import gr
> > > h=gr.msg_handler()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.6/dist-packages/gnuradio/gr/runtime_swig.py", line \
4629, in __init__ raise AttributeError("No constructor defined - class is abstract")
AttributeError: No constructor defined - class is abstract
==============================================================================
In gr3.10 (pybind11) with the in tree version of msg_handler you get
$ python3
Python 3.10.4 (main, Apr 2 2022, 09:04:19) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
> > > from gnuradio import gr
> > > h=gr.msg_handler()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: gnuradio.gr.gr_python.msg_handler: No constructor defined!
> > >
==============================================================================
Eventually the OOT tree version was "fixed" after noticing that some of
the lines in the python bindings file in the in-tree version were commented:
// .def(py::init<>(),D(msg_handler,msg_handler,0))
// .def(py::init<gr::msg_handler const &>(), \
py::arg("arg0"), // D(msg_handler,msg_handler,1)
// )
==============================================================================
py::call_guard<py::gil_scoped_release>()
Me of little faith, I wanted to ensure this would re-acquire the lock on
the way back from C++ to python. Yes, RAII style, its destructor makes sure
of that
if (active)
PyEval_RestoreThread(tstate);
==============================================================================
Finally, the following "gdb" command will identify the thread that holds(held)
the Python GIL:
p *(PyThreadState*)_PyRuntime.ceval.gil.last_holder._value
The "python3-dbg" and "libpython3-dbg" packages must be installed in order to
enable the gdb python extensions.
==============================================================================
Thanks Josh
[Attachment #3 (text/html)]
<html><head></head><body><div class="yahoo-style-wrap" style="font-family:Helvetica \
Neue, Helvetica, Arial, sans-serif;font-size:13px;"><div dir="ltr" \
data-setdir="false"><div>> I think the way to properly handle GIL from long \
running calls<br>> is to add \
py::call_guard<py::gil_scoped_release>()<br><br>Josh<br><br>Full success! \
I snarfed the gr code for msg_queue, msg_handler, and message,<br>and added this call \
guard. There is no more GIL deadlock in the python OP25 \
<br>apps.<br><br>Max<br>==============================================================================<br>tl;dr \
: following geek material may be safely skipped <br><br>Initial attempts to compile \
msg_handler OOT resulted in a compile \
error:<br>/usr/include/pybind11/detail/init.h:63:64: error: invalid new-expression of \
abstract class type ‘gr::op25::msg_handler'<br> 63 | inline Class \
*construct_or_initialize(Args &&...args) { return new \
Class{std::forward<Args>(args)...}; }<br> \
| & \
nbsp; &nb \
sp;   \
; \
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>In file included from \
/home/op25/repo/op25/src4/op25/gr-op25/python/op25/bindings/msg_handler_python.cc:26:< \
br>/home/op25/repo/op25/src4/op25/gr-op25/python/op25/bindings/../../../include/gnuradio/op25/msg_handler.h:27:16: \
note: because the following virtual functions are pure within \
‘gr::op25::msg_handler':<br><br>==============================================================================<br>In \
gr3.8 (swig) you get this error:<br><br>from gnuradio import gr<br>>>> \
h=gr.msg_handler()<br>Traceback (most recent call last):<br> File \
"<stdin>", line 1, in <module><br> File \
"/usr/local/lib/python3.6/dist-packages/gnuradio/gr/runtime_swig.py", line 4629, in \
__init__<br> raise AttributeError("No constructor defined - class \
is abstract")<br>AttributeError: No constructor defined - class is \
abstract<br><br>==============================================================================<br>In \
gr3.10 (pybind11) with the in tree version of msg_handler you get<br>$ \
python3<br>Python 3.10.4 (main, Apr 2 2022, 09:04:19) [GCC 11.2.0] on \
linux<br>Type "help", "copyright", "credits" or "license" for more \
information.<br>>>> from gnuradio import gr<br>>>> \
h=gr.msg_handler()<br>Traceback (most recent call last):<br> File \
"<stdin>", line 1, in <module><br>TypeError: \
gnuradio.gr.gr_python.msg_handler: No constructor \
defined!<br>>>><br><br>==============================================================================<br>Eventually \
the OOT tree version was "fixed" after noticing that some of<br>the lines in the \
python bindings file in the in-tree version were \
commented:<br> // \
.def(py::init<>(),D(msg_handler,msg_handler,0))<br> \
// .def(py::init<gr::msg_handler const \
&>(), \
py::arg("arg0"),<br> // \
D(msg_handler,msg_handler,1)<br> // \
)<br><br><br>==============================================================================<br> \
py::call_guard<py::gil_scoped_release>()<br><br>Me of little faith, I wanted to \
ensure this would re-acquire the lock on <br>the way back from C++ to python. \
Yes, RAII style, its destructor makes sure<br>of \
that<br> if \
(active)<br> \
PyEval_RestoreThread(tstate);<br><br>==============================================================================<br>Finally, \
the following "gdb" command will identify the thread that holds(held)<br>the Python \
GIL:<br> p \
*(PyThreadState*)_PyRuntime.ceval.gil.last_holder._value<br>The "python3-dbg" and \
"libpython3-dbg" packages must be installed in order to<br>enable the gdb python \
extensions.<br><br><br>==============================================================================<br>Thanks \
Josh</div><div><br></div></div></div></body></html>
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic