[prev in list] [next in list] [prev in thread] [next in thread]
List: rpm-devel
Subject: Re: [CVS] RPM: rpm-5_3: rpm/rpmio/ rpmruby.c rpmruby.h
From: Jeff Johnson <n3npq () mac ! com>
Date: 2010-09-20 18:15:35
Message-ID: 80C6703F-A07C-4764-A6E6-499B94C1F432 () mac ! com
[Download RAW message or body]
OK, I mostly have a co-routine exchange running
a rubyi-1.9.2p0 (+ruby_bind_stack patch) interpreter
using yarnLock/yarnThread.
See rpmio/xruby.c on the -r rpm-5_3 branch.
The basic technique seems to be to get a thread
running with its own stack.
The entire ruby interpreter execution needs to be in
one routine with one constant stack: i.e. one cannot split
initialize <-> run <-> terminate
across separate routines. Feeding a single ruby
interpreter on its own thread SHOULD work.
The major flaw is that one cannot restart a new interpreter
after the first exits. I tried and failed (but I could
likely succeed if I studied _EXACTLY_ what was needed:
certainly a dlopen(3) to re-initialize all of -lruby's
address space SHOULD re-initialize everything.
I've also wired up rpmio/rpmzlog.h logging, which is
needed for reasonable thread development so that multiple
threads don't clobber stdout/stderr. An rpmio/xruby run looks
like this (from a gdb session):
...
#<Thread:0x833b5b4> recursing at 499...
[Thread 0xb71c5b70 (LWP 9155) exited]
GOOD JOB! :-)
[Thread 0xb7b5cb70 (LWP 9145) exited]
Main: Goodbye!
trace 1285004868.906429 -- rpmrubyThread: running
trace 1285004868.910619 -- rpmrubyThread: interpreter starting
trace 1285004868.950908 -- rpmrubyThread: interpreter started
trace 1285004868.950933 -- rpmrubyThread: require '../ruby/hello.rb' begin
trace 1285004874.132776 -- rpmrubyThread: require '../ruby/hello.rb' end
trace 1285004874.132818 -- rpmrubyThread: interpreter terminating
trace 1285004874.143521 -- rpmrubyThread: interpreter terminated
trace 1285004874.143626 -- rpmrubyThread: ended
[Thread 0xb7fddb70 (LWP 9144) exited]
Program exited normally.
Feel free to dink a bit, loading stringio and fishing the result from ruby
is gonna be needed for the RPM embedded macro syntax. But for debugging, you can just
see whatever is displayed on stdout.
The other hard part (with co-routine interlocks) is that there needs
to be some way to kill the ruby interpreter thread. With interlocks,
one is very likely to hang on odd corner cases.
There's also going to be a need to feed strings (not just file paths) to a
persistently running interpreter instance on a thread. That will likely lead
to dropping a resumable co-routine model and just queuing input/output to a ruby
interpreter on a thread imho.
One approach is through socketpair(2) (but ensuring that every input leads to
SOME output is tricky). The other way is through input/output queues.
If interested in queues, go read through the Marc Adler's PIGZ code to see how yarnLocks/yarnThreads
(and spaces and pools and jobs) are used to achieve multi-threaded zlib compression.
All of the PIGZ routines are present in -lrpmio, just unused so far directly by RPM
(and I've not looked at in ~1.5 years, expect some minor bit-rot).
I gotta get back to rpm-5.3.4 release engineering, will revisit embedded
ruby next month.
hth
73 de Jeff
On Sep 20, 2010, at 1:50 PM, Jeff Johnson wrote:
> RPM Package Manager, CVS Repository
> http://rpm5.org/cvs/
> ____________________________________________________________________________
>
> Server: rpm5.org Name: Jeff Johnson
> Root: /v/rpm/cvs Email: jbj@rpm5.org
> Module: rpm Date: 20-Sep-2010 19:50:50
> Branch: rpm-5_3 Handle: 2010092017505000
>
> Modified files: (Branch: rpm-5_3)
> rpm/rpmio rpmruby.c rpmruby.h
>
> Log:
> - ruby: sanity.
>
> Summary:
> Revision Changes Path
> 2.12.2.6 +33 -1 rpm/rpmio/rpmruby.c
> 2.7.2.3 +18 -4 rpm/rpmio/rpmruby.h
> ____________________________________________________________________________
>
> patch -p0 <<'@@ .'
> Index: rpm/rpmio/rpmruby.c
> ============================================================================
> $ cvs diff -u -r2.12.2.5 -r2.12.2.6 rpmruby.c
> --- rpm/rpmio/rpmruby.c 19 Sep 2010 21:30:37 -0000 2.12.2.5
> +++ rpm/rpmio/rpmruby.c 20 Sep 2010 17:50:50 -0000 2.12.2.6
> @@ -34,11 +34,43 @@
>
> /*@unchecked@*/
> int _rpmruby_debug = 0;
> -#define RUBYDBG(_l) if (_rpmruby_debug) fprintf _l
>
> /*@unchecked@*/ /*@relnull@*/
> rpmruby _rpmrubyI = NULL;
>
> +/*==============================================================*/
> +/* puts the Ruby coroutine in control */
> +void _rpmruby_main_to_ruby(rpmruby ruby)
> +{
> + rpmzLog zlog = ruby->zlog;
> +
> + yarnRelease(ruby->ruby_coroutine_lock);
> + yarnPossess(ruby->main_coroutine_lock);
> +if (_rpmruby_debug < 0) Trace((zlog, "-> %s", __FUNCTION__));
> +}
> +
> +/* puts the main C program in control */
> +unsigned long _rpmruby_ruby_to_main(rpmruby ruby, unsigned long self)
> +{
> + rpmzLog zlog = ruby->zlog;
> +
> + yarnRelease(ruby->main_coroutine_lock);
> + yarnPossess(ruby->ruby_coroutine_lock);
> +if (_rpmruby_debug < 0) Trace((zlog, "<- %s", __FUNCTION__));
> + return Qnil;
> +}
> +
> +#ifdef NOTYET
> +/* Using _rpmrubyI, puts the main C program in control */
> +static VALUE relay_from_ruby_to_main(VALUE self)
> +{
> + /* XXX FIXME: _rpmrubyI is global */
> + return _rpmruby_ruby_to_main(_rpmrubyI, self);
> +}
> +#endif
> +
> +/*==============================================================*/
> +
> static void rpmrubyFini(void * _ruby)
> /*@globals fileSystem @*/
> /*@modifies *_ruby, fileSystem @*/
> @@ .
> patch -p0 <<'@@ .'
> Index: rpm/rpmio/rpmruby.h
> ============================================================================
> $ cvs diff -u -r2.7.2.2 -r2.7.2.3 rpmruby.h
> --- rpm/rpmio/rpmruby.h 19 Sep 2010 21:30:39 -0000 2.7.2.2
> +++ rpm/rpmio/rpmruby.h 20 Sep 2010 17:50:50 -0000 2.7.2.3
> @@ -7,6 +7,8 @@
>
> #include <rpmiotypes.h>
> #include <rpmio.h>
> +#include <argv.h>
> +#include <rpmzlog.h>
> #include <yarn.h>
>
> typedef /*@abstract@*/ /*@refcounted@*/ struct rpmruby_s * rpmruby;
> @@ -18,19 +20,26 @@
> extern rpmruby _rpmrubyI;
>
> #if defined(_RPMRUBY_INTERNAL)
> +#define RUBYDBG(_l) if (_rpmruby_debug) fprintf _l
> +#define Trace(_x) do { rpmzLogAdd _x; } while (0)
> struct rpmruby_s {
> struct rpmioItem_s _item; /*!< usage mutex and pool identifier. */
> void * I;
> size_t nstack;
> void * stack;
>
> - const char * fn;
> - yarnThread ruby_coroutine;
> + ARGV_t av;
> + int ac;
> +
> + struct timeval start; /*!< starting time of day for tracing */
> +/*@refcounted@*/ /*@null@*/
> + rpmzLog zlog; /*!< trace logging */
> +
> + unsigned more;
> + yarnThread thread;
> yarnLock main_coroutine_lock;
> yarnLock ruby_coroutine_lock;
>
> - unsigned ruby_coroutine_finished;
> -
> unsigned long state;
> #if defined(__LCLINT__)
> /*@refs@*/
> @@ -112,6 +121,11 @@
> /*@globals fileSystem, internalState @*/
> /*@modifies ruby, *resultp, fileSystem, internalState @*/;
>
> +void _rpmruby_main_to_ruby(rpmruby ruby)
> + /*@*/;
> +unsigned long _rpmruby_ruby_to_main(rpmruby ruby, unsigned long _self)
> + /*@*/;
> +
> #ifdef __cplusplus
> }
> #endif
> @@ .
> ______________________________________________________________________
> RPM Package Manager http://rpm5.org
> CVS Sources Repository rpm-cvs@rpm5.org
______________________________________________________________________
RPM Package Manager http://rpm5.org
Developer Communication List rpm-devel@rpm5.org
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic