[prev in list] [next in list] [prev in thread] [next in thread] 

List:       arts
Subject:    Re: Multithreaded audioiooss for arts
From:       Stefan Westerfeld <stefan () space ! twc ! de>
Date:       2001-06-10 4:58:02
[Download RAW message or body]

   Hi!

On Wed, May 30, 2001 at 11:40:24PM +0200, Matthias Welwarsky wrote:
> > So the only way I see (which is not trivial, I agree, but probably the only
> > way to go), is putting the production of new data (= flow system) and the
> > output of the data (= audio thread) into a seperate thread. Inside this
> > thread, allocations would not be allowed, nor anything else that could block
> > for a long time (open, read, write, hostname lookup, ...).
> 
> Well, not necessarily. A certain amount of buffering is allowed, if you
> do
> it in the right place. We won't be able to maintain realtime over the 
> complete flow from the producer to the consumer, because often we will
> have no control about the producer's realtime behaviour. We need to 
> identify the places where realtime is needed. Feeding the soundcard is 
> clearly critical, an mp3-decoder is clearly not critical.

Well, for mp3-decoders reaction is not critical. But there are other
producers where it is. For instance, consider the case where you are singing
into a microphone and applying an effect. There, you want the minimum latency
between the producer and consumer. So simply creating an audio output thread
and buffering a lot between the producers (microphone, mp3 player) and
consumers (soundcard) will not help you there.

Instead, what the mpeglib MP3PlayObject currently does is that it creates
an own thread for decoding.

mpeglib thread:  [ mp3 decoder ]
                     |
                     |   (ringbuffer)
					 |
main thread:  [ mp3 player ] ------------------->  |  ----> [ soundcard ]
              [ microphone ] ---> [ effect ] --->  |

That way, if the decoder blocks (for instance because it reads from nfs), then
the main thread is not blocked. Now there is still the problem what happens
if an incoming MCOP request is dispatched and takes a long time, but that is
what I think can only be solved by putting the whole filter graph into
another thread. Even so, long running MCOP requests will have an impact on
how long it takes to dispatch another MCOP request (as the MCOP stuff can't
deal with multithreaded servers), and since this could be a realtime event,
such as a midi event, long running MCOP requests are generally harmful.

> > - linux has difficulties measuring multithreaded programs as opposed to
> >   single threaded programs
> 
> Huh? I don't understand this.

Well, I saw code in some mp3 player once:

  /* fool users into thinking that our decoder only takes 0.1% cpu */
  usleep(10000);

or something like this. And it worked. Removing the usleep increased the CPU
usage to 5%. So I don't really trust measurements of threaded programs made
with top. ;)

> > About the IOManager:
> > 
> > You might want to have a look at defining the latency debugging in the
> > IOManager code for getting figures how long it takes between two select()
> > cycles. Priorities are an interesting concept, although I can not say how
> > much the gain would be if you priorize every action of the IOManager, and
> > rewrite the main cycle like:
> > 
> >         1. determine parameters for select (prepare)
> >         2. select
> >         3. determine which activities need to be done (check)
> >         4. pick the first (and only the first) activity and do it (dispatch)
> 
> Given a duplex soundcard, which of the events would you serve first?
> Capture
> or playback? Or a timer notification event? You must prioritize, and you
> must preempt, too.

Well, if you say so ;). It's still the question how often we really have ten
events to serve at the same time, so that the priorization would do something
useful.

Capture and playback are no problem, btw.

 * if playback is served first, the graph backpropagates until it hits the
   module where it reads from the soundcard - if there is something in the
   input queue, it uses this, if not, it reads from the soundcard directly

 * if capture is served first, the data is read from the soundcard and put
   into the input queue

So, given the current implementation, no matter which event is served first,
the result will be the same.

But of course, you currently can bomb down artsd with MCOP requests, since
they have the same "priority" as playing audio data. So you are right, you
could improve some of the behaviour with priorities.

> > The important thing here is that to be really fine grained, we should rather
> > only do one activity, as in the time that passes for doing this, another,
> > higher priority event might have arrived, which we should do first. There
> > is also a considerable overhead through additional syscalls that we might
> > have through this. Adding a gettimeofday after doing each activity (to allow
> > clustering of some activities) could help making a priorized IOManager
> > efficient.
> 
> OK, but I must be able to interrupt a low level activity in favour of a
> high
> priority activity. That is, I must be able to stop decoding an mp3 in
> order
> to feed the soundcard. That cannot simply be done with select().
> However, 
> doing this with SCHED_FIFO threads is rather simple, because you just
> give
> the thread that feeds the soundcard a higher priority, and it will
> interrupt 
> the mp3 decoder once it needs more data. No additional syscalls needed.
> The
> feeder thread simply returns from the blocking write(audio_fd) call and
> gets served instantly. Plus, you don't need extensive buffering for
> that.
> You just need to have enough spare cycles so that the producer can
> generate
> data while you serve the soundcard from your buffer. You don't even need
> a "short" producer loop because you can interrupt it.

Well, you can do that right now. Thread your producer, make it buffer
internally and let it work in big chunks. I know that this is not too
convenient for the programmer, but I see no really easy way out of that.

The MCOP protocol stack is not thread-safe, nor is the rest of aRts. So
basically, you can't simply move some producer generating an asynchronous
byte stream into another thread through the framework. One could look into
providing means to do so.

As I have said, it will not work with all producers anyway, because

 * you might have too many producers to make this feasible (a typical piece
   of midi synthesis easily can use 300 aRts modules)
 * producers may use some aRts APIs which are not threadsafe

and there are some producers (i.e. an effect which just delays the signal a
few samples) where the cost of doing such a simple task in a seperate thread
would be too high, and no benefit would occur anyway. So it's a question of
how fine-grained your modules are whether threading them is useful or not.

   Cu... Stefan
-- 
  -* Stefan Westerfeld, stefan@space.twc.de (PGP!), Hamburg/Germany
     KDE Developer, project infos at http://space.twc.de/~stefan/kde *-         

[prev in list] [next in list] [prev in thread] [next in thread]