[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-multimedia
Subject:    KDE3.0 Multimedia Meeting Summary
From:       Stefan Westerfeld <stefan () space ! twc ! de>
Date:       2001-10-22 21:19:27
[Download RAW message or body]

        The KDE3.0 Multimedia Meeting, IRC discussion summary
====================================================================

This summary provides an overview of the things that we talked about
during the KDE3.0 multimedia meeting which took place on IRC (2001-09-04).

I will summarize this document to a list list of keywords for the KDE3
planned features document (which is relevant for the freeze), but this is
the more verbose source of what will be done.

The original IRC log can be found at:

  http://space.twc.de/~stefan/kde/download/kde30-mm-log.txt

Participants (in no particular order):

  Carsten Niehaus
  Stefan Schimanski
  Andreas Burman 
  Robert Gogolok
  Achim Bohnet
  Michael Seiwert
  Stefan Westerfeld
  Matthias Kretz
  Nikolas Zimmermann
  Neil Stevens
  Tim Janik
  Jan Wuerthner
  Daniel Haas
  Malte Starostik
  Martin Vogt
  (... there was a lot of joining and leaving on the channel, so maybe
       I forgot somebody ...)

Overview:
=========

We talked our way to more or less what was posted before on the list, the
list of topics we talked about here was:

1. Making the MCOP component model more open to the world
2. Compatibility with legacy/broken OSS drivers
3. Enabling i18n
4. Recording
5. Video embedding / streaming
6. PlayObjects (special point ;)
7. MIDI
8. GSL Engine
9. GSL other stuff
10. Samples
11. Environments
12. Decisions & Open Discussion
13. Conclusion

1. Making the MCOP component model more open to the world
=========================================================

 * MCOP<->DCOP bridging: the plan is to provide access to MCOP based services
   from DCOP - thats useful for scripting, as well as development, and desktop
   integration

 ==> responsible: Nikolas Zimmermann

 * JMCOP: There is a working but unreleased partial Java MCOP implementation,
   so if you want to work on something like this, don't start from scratch -
   however, there is currently no fixed completion date for this, nor will
   it probably be distributed bundled with KDE3.0

 * UCOM: There was a bit discussion about this one, but nobody seemed to
   to have looked too much into UCOM yet

2. Compatibility with legacy/broken OSS drivers
===============================================

  * Threaded OSS support: some applications acquire better results with
    respect to compatibility with various possible kinds of hardware than
	aRts, because of their threaded implementation of accessing the sound
	card. The threaded oss patch is a patch that opens the driver in blocking
	mode and spawns a seperate thread.
	
	While there was discussion whether the observed increase in quality (less
	dropouts, better support on more hardware) was caused
	  - additional buffering gained through this (especially for cards with
	    small hardware buffer)
      - brokenness of various OSS drivers for completely non-blocking I/O
	  - the usage of select(...) which might be badly implemented for some
	    OSS drivers
    or a combination of those, it generally was considered a good idea to
	do it.

  ==> responsible: Matthias Welwarsky

      [ I wrote him here because he wrote the patch, but since he wasn't
	    at the meeting, we might need to change this ]

3. Enabling i18n
================

  * All strings emitted by aRts should be internationalizable in KDE3.0 - they
    are currently not, because aRts does neither use libkdecore (so it can't
	use the standard i18n), nor has own means to deal with that.

    While workarounds might be possible (such as artsd only emitting error
	codes, and seperate in-the-gui-layer-translation), the right solution,
	which was also somewhat discussed on the lists will be adding own i18n
	capabilities to libmcop. Whether or not libc i18n can be used, or whether
	the kdecore code could be copied, is still to be evaluated. As there are
	no unicode strings in aRts (no QString), utf8 is probably the way to go
	for transport to the GUI.

  ==> responsible: Jeff Tranter

      [ Volunteered on the lists recently ]

4. Recording
============

 * The aRts C API didn't implement clientside recording until now, but it
   should. Likewise should the Arts::SoundServer interface. It might be good
   finally following the KDE2 plan to provide a KAudioStream class in
   libartskde, to provide recording/playing streams in a nice, canned,
   easy-to-use way. Including an end user sound recorder into kdemultimedia
   might be useful, too.

  ==> responsible: Matthias Kretz

5. Video embedding / streaming
==============================

 * There is an VideoPlayObject interface in KDE2.2 already, which supports
   playing videos via X11 embedding. This is a quite simple solution, but
   using it more in KDE applications will make viewing videos much more
   user friendly than it is now.

 * There was some discussion about videos and noatun. Noatun currently doesn't
   use the embedding interface, and this might be hard to do, because every
   skin needs to be changed for that. On the other hand, noatun is the
   official KDE media player, so it should probably be able to embed videos.

 * There was some discussion about eventually implementing video editing, but
   we currently lack mainly the codecs to do it. The codecs don't support
   precise frame based seeking and delivery of the raw image as data. Besides
   this, video support would greatly benefit from introducing more structured
   asynchronous data delivery (with timestamps and frames) into aRts.

 * Extending Brahms towards embedding on the other hand seems less hard, so
   that it might be reasonable simple to compose film sound tracks with Brahms
   eventually, or - if we have better codecs - even edit films the same way
   Brahms can edit music.

6. PlayObjects
==============

 * Malte does have a RealMedia PlayObject ready (based on the Real library, so
   the implementation is not entierly open source). Currently, there are
   unsolved licensing issues.

 ==> responsible: Malte Starostik

 * There was some discussions on other ways to get codecs. Basically, writing
   a PlayObject for one of
    - ffmpeg
	- libmpeg3
	- xine
	- gstreamer
   could extend the fileformats aRts will be able to play. Martin Vogt is
   working on xine, combined with code to standarize rendering and output
   more, to make integration of new video codecs, and later, video editing,
   simpler. There is experimental libmpeg3 code in the CVS.

7. MIDI
=======

 * Midi sequencers (Brahms/Rosegarden/Anthem) have started to use aRts
   as way of outputting midi, whereas other applications (kmid) still use
   libkmid, and some things (for instance ALSA) are not supported by aRts
   at this time. For KDE3.0, aRts should provide a complete solution.

 * The midi interfaces should finally support accessing sound card hardware of
   all fashions (ALSA, OSS-hw-synth) - that will mean a rewrite of some stuff,
   but is necessary for making the midi interfaces reasonable complete.

 * Precise synchronization between midi/audio should be adressed.

 * Libkmid could finally be moved upon the new midi interfaces.

 * The midi libs should be mainstream (i.e. kdelibs) and stable after KDE3.

 * A midi PlayObject would be helpful, to enable media players like noatun 
   to deal with midis.

 ==> responsible (all midi issues): Antonio Larrosa, Stefan Westerfeld

8. GSL Engine
=============

 * GSL will be the new engine for scheduling synthesis modules. This will
   not cause a glib dependancy.

 ==> responsible: Tim Janik, Stefan Westerfeld

-- more lengthy explanation about what/why/how --

   aRts currently has an engine that decides which modules are to be
   calculated when, given a certain flow graph - this is called scheduling
   (and implemented in kdelibs/arts/flow/synthschedule.cc)

   The GSL engine is a complete replacement of the algorithm with the
   following characteristics:

    - it should be quite a bit more efficient in the common case

    - it supports dispatching calculations in multiple threads, which for
      instance allows to fully exploit multi processor systems

    - it allows putting module calculations in a seperate thread from the
      aRts main thread, thereby avoiding dropouts from lengthy aRts operations,
      such as for instance calling open() on an NFS drive

    - it is designed in a way to support advanced scheduling features that
      aRts currently does not support, that is
        * feedback loops (circles)
        * sample precise timing

    - it should allow achieving very very low latencies reliable (where very
      very low is something like 3ms) - aRts currently can't do this
      
    - it is implemented in plain C, using some glibisms, however it can be
      integrated into aRts with a glib-compat source in a way that it does
      not bring additional dependancies (no glib required)

   The GSL engine should first act as a drop-in replacement - however to take
   advantage of its capabilities, it will be necessary to make sure that the
   modules are implemented in a way that they can run in a seperate thread
   without interfering with anything else.

   That means, that there will two types of modules: the old modules (which
   will still be running in the aRts main thread), and the hard-rt-modules
   (which will be able to run in the engine threaded).

   You will get the very very low latency benefit only if you use only hard-
   rt-modules, but everything should work with any mix of modules.

   Where is GSL from?

   GSL is designed by Tim Janik and me, and contains a lot of know how about
   music apps (Tim Janik has been working for years on Beast - best.gtk.org,
   and I have been working for years on aRts).

   GSL code is in the CVS and working (although yet somewhat slow) already.

9. GSL other stuff
==================

 * GSL provides a sample cache which supports partial caching and preloading,
   the integration of that will get rid of the long outstanding issue that
   aRts always needs to load whole WAV files (or other formats), and support
   GigaSampler like "read-from-HD-and-play" style sample sets of huge size.

 * GSL provides filter design code for butterworth, tschebyscheff-1/2
   and soon elliptic low/high and soon bandpass/stop filters, which can be
   wrapped in new aRts modules (aka Arts::Synth_HIGHPASS).

10. Samples
===========

 * While aRts is good at creating synthetic sounds and using these as
   instruments in a sequencer (i.e. Brahms), playing samples will greatly
   enhance the possibilities.

 * I have a prototype of a .PAT loader - you can import a whole timidity
   style patchset with that, which means that we have a useful GM-sample-set
   for aRts. There is also AKAI loading code in kmusic/ksynth. GSL provides
   another native format. All of these probably should use the GSL sample
   cache for efficient, preloaded, replaying.

 * Combined with an aRts PlayObject, media players (noatun, kaboodle) gain
   not only the capability to do similar replay than kmid (output midi via
   card synth or external midi), but also kmidi (output midi as calculated
   audio data from sample sets).

 ==> responsible (all points): Antonio Larrosa, Stefan Westerfeld

11. Environments
================

 * The work started with mixers/environments should be completed - for instance
   it would be very useful if the mixer could save/restore itself properly, and
   if it could also contain insert-stereoeffects and a master volume. Also,
   the mixer look more like a real-mixer(tm), for instance groups or panning
   is not there right now.

 ==> responsible: Matthias Kretz

 * Visualizations (led-bars) would be nice, but currently have the technical
   problem that they overload the MCOP layer easily, this can probably only
   be fixed by changing MCOP to deal more intelligently with change
   notifications.

 ==> responsible: Matthias Kretz

12. Decisions, Open Discussion
==============================

 * aRts will break binary compatibility (KDE does so, so we should, too). 

 * the general consensus seems to be that moving aRts in an extra CVS module
   (so that it is easier to obtain, compile, develop, use, package without
   KDE) seems to be a good thing do to

 * whether to merge the SoundServerV2 interface into SoundServer (and so on),
   is still an open issue
    - advantages: simpler code, cleaner interfaces
	- disadvantages: wire compatibility (running KDE2 and KDE3 apps on artsd)
	  will break - or is at least very hard to keep
              
13. Conclusion
==============

There are about three major areas which the work focuses on. First of all,
it's important to continue to improve everyday usage for casual users. Mostly,
recording, i18n and the threaded OSS support fall in that category.

On the other hand, a lot of things should be simply unified, made more
consistent, and merged. A media player which can play everything might sound
a lot better to the user than noatun, kaboodle, kmid, kmidi and aktion.

Finally, there is a lot of work to be done especially for music applications,
which includes the gsl engine, more support for samples, midi improvements,
and environments/mixers.

I hope that with the way we split up the work/responsibility on several
developers, we'll be able to achieve the goals listed here in time for
KDE3.0.

    Cu... Stefan
-- 
  -* Stefan Westerfeld, stefan@space.twc.de (PGP!), Hamburg/Germany
     KDE Developer, project infos at http://space.twc.de/~stefan/kde *-         
_______________________________________________
kde-multimedia mailing list
kde-multimedia@mail.kde.org
http://mail.kde.org/mailman/listinfo/kde-multimedia

[prev in list] [next in list] [prev in thread] [next in thread]