[prev in list] [next in list] [prev in thread] [next in thread] 

List:       openjdk-2d-dev
Subject:    [OpenJDK 2D-Dev] DRAFT PROPOSAL: Implement XRender pipeline for Java2D
From:       linuxhippy () gmail ! com (Clemens Eisserer)
Date:       2008-02-29 17:47:44
Message-ID: 194f62550802290947o600adab5x66e2fd0c1026eac () mail ! gmail ! com
[Download RAW message or body]

Hi,

Also sorry for cross-posting, I thought it wouldn't be a bad idea to
let you guys on the 2d-dev list know about my proposal. Sorry for the
traffic.

I would be very happy about suggestions and ideas howto enhance the
proposal as well as maybe some patronage at
challenge-discuss at openjdk.java.net.
I have to admit the best that I could imagine is something like a
supporter directly from Sun, like  what "Charles Oliver Nutter" did
for the Better-native-Interface-Proposal ;)

Thanks for reading, lg Clemens

---------------------------------------------------------------------------------------
1.) Introduction
The more advanced features of Java2D are used in a lot new key
technologies like JavaFX, Nimbus, SVG, Arith and even native Swing
themes like the GtkLookAndFeel and without any doubt their use will
grow further.
Since its introduction, Java2D supports very advanced rendering
operations, although only very basic operations were accelerated by
hardware, because the underlying APIs were very poor.
However software-rendering leads to low performance and bad user
experience in this case.

Modern 3D graphic APIs, like OpenGL or Direct3D, allow to map most of
Java2D's functionality without the need for expensive read-backs
and/or software rendering.
To make that functionality available to the Java platform two new
?pipelines" where created:
The OpenGL pipeline for non Windows platforms, and the Direct3D
pipeline for the Microsoft Windows operating systems only.
However OpenGL on Unix in general has some disadvantages compared to
X11 for typical desktop applications, because of poor drivers its
often not possible to use that new functionality and Java falls back
to the default X11 rendering pipeline.

An Xrender backend for Java2D would provide almost the same features
as the existing OpenGL pipeline, however would run on a much larger
set of hardware and would integrate better into existing X11 based
desktops.
With the Xrender backend developers and users could rely on almost the
same set of functionality accelerated by default on all platforms
supported currently by OpenJDK, e.g. JavaFX animations could run at
almost the same speed everywhere.


2.) Status Quo
The existing X11 pipeline used on Unix systems is only capable of
accelerating basic operations and falls back to software rendering
even for often used operations like antialiasing (even for text!) or
translucent images often causing expensive VRAM readbacks.
Because of X11's very limited rendering capabilities Xrender was
introduced back in 2001, and after a long delay first drivers are
available which accelerate large areas of Xrender.

XRender allows to accelerate most of Java2D's functionality, and it is
considered to be the de-facto standard for advanced 2D graphics,
already used by a large set of libraries and toolkits like Cairo,
GTK+, QT4 and KDE4.
Because of the broad and extensive use of Xrender there is currently
major work ongoing to modify drivers for better Xrender performance.

There is already an OpenGL backend for Java2D which is about as
powerful as its Direct3D counterpart on Windows.
Unlike on Windows where Microsoft starting with Windows Vista
more or less deprecated all interfaces except Direct3D, and therefor
forces GPU/driver manufacturers to create drivers well suited also for
2D rendering, on Unix good
OpenGL drivers are still rare and only proprietary drivers are till
now able to run the OpenGL pipeline.
However those proprietary drivers are often optimized for heavy-weight 3D
applications like games and are not really suited for rendering
light-weight simple 2D applications.

A small overview about the advantages/disadvantages Xrender has
compared to OpenGL:

+ Lower per-primitive overhead:
Because Xrender is compared to OpenGL a lot simpler, there is less
work for the accelerating driver to do for each primitive rendering
operation. This is especially important for complex shapes which are
transformed to many rectangles which then are passed to the underlying
graphic API.
To make it short: Because Xrender focuses on only a small set of
?simple" operations, driver implementers can focus on making those few
operations as fast as possible.

+ Support for higher-level primitives:
Xrender has support for client-side font rendering, which means glyphs
which are cached in vram can be shared across different applications.
For OpenGL each application has to upload its glyphs into a private
texture wasting large amounts of Vram.
Because it was designed with 2D graphics in mind many features can be
directly mapped to Xrender without the need of complex shaders.

+ Reliable AntiAliasing:
Because Xrender specifies exactly how antialiasing has to be done,
Java2D can rely on Xrender's antialiasing capabilities. The Direct3D
and OpenGL pipelines render a alpha-mask in software, upload it to
vram and only let the GPU only do the final composition step.
Because of this, the amount of data transferred between the Xserver
and the Java application would be reduced a lot, making antialising
even usable over slow remote connections.

+ Better driver support:
On Linux only proprietary OpenGL implementations provide enough
functionality to let the OpenGL pipeline run. However even those
proprietary drivers were/are buggy in the code-paths exercised by the
OpenGL pipeline - sometimes new drivers even broke the OpenGL
pipeline.
For Xrender there are already many optimized drivers, both open-source
(Intel, ATI, ...) and proprietary (Nvidia). Because of the high
adaption rate of Xrender lately (KDE4, Cairo, GTK+, ...) it can be
expected that soon more and more drivers will support Xrender
acceleration.

+ Better desktop integration:
Xrender applications are ?native" X11 applications, and therefor work
well with composition managers. They also get visuals optimized for
2D, and do not waste Vram with additional buffers they get, although
they are not used.

+ Better network transparency:
Although OpenGL can also be made network-transparent via GLX, the
OpenGL API was not designed to perform especially well in a network
environment.
X11/XRender relies on the Client-Server approach and therefor is
optimized for exactly this case.

- Not programmable:
A disadvantage of Xrender is that its not as flexible as OpenGL.
OpenGL allows to use shaders to implement features not provided by
fixed functionality, the only solution for a Xrender based backend is
to fall back to software rendering.


3.) Goals
The goal of this proposal is to make use of Xrender for accelerating
even advanced Java2D functionality.
This would allow multimedia applications/content like JavaFX to take
advantage of modern hardware, and would bring JDK6Update10 graphic
features also to X11/Unix based platforms.

Some of the most important enhancements compared to the X11 pipeline include:
* Accelerated text rendering for normal, antialiased and subpixel
antialiased text.
* Support for translucent images
* Composition (colors with alpha values, ...)
* Antialiasing

A long-term goal would be to make the existing X11 pipeline obsolete
once the XRender backend has prooven to be stable, removing the burden
of maintaining two, in their dedication similar, backends.


4.) Approach
XRender has a lot in common with OpenGL/Direct3D and is a very
low-level API which supports only a limited number of drawing
primitives but it is very flexible in the way of operations it
operates with those operations. Therefor the approach would be to
create a brand new backend based on the shared code developed for the
OpenGL/Direct3D pipelines in JDK6-u10.
The advantage of this approach would be a ?clean start", as well as a
lot shared code between OpenGL/Direct3D/XRender backends.
The downside is that some highly optimized primitives which are
supported by X11's core drawing (Arcs, Elipses, ..) maybe perform a
little bit less well. Furthermore new code needs a lot of testing.


5.) Milestones
5.1) Create a skeleton which is able to use the shared OpenGL/Direct3D code
5.2) Implement only enough to functionality to make
software-rendering-loops work.
5.3) Extend the created skeleton, accelerate basic operations -> Make
typical Swing/Ocean applications work well.
5.4) Accelerate more advanced functionality like gradients,
composition and antialising -> Accalerate most of Java2Demo, make the
Nimbus LnF work well.
5.5) Extensive profiling and compatibility testing, using self
written, Sun's jtreg and real- world tests, reporting performance and
implementation problems back to driver programmers. Make sure large
graphical applications work well.


6.) Developer
Clemens Eisserer is Student at the Technical University of Vienna
(Information Technology) and:
* Has submitted patches and regression tests to jdk-collaboration
(plugin, webstart) and OpenJDK (Java2D, Corelibs)
* Implemented JLayeredPane for SwingWT: http://swingwt.sourceforge.net/
* Ported Kaffe's XAWT to GCJ: http://sf.net/projects/xawt
* Is working on an open-source remote-swing project:
http://juibrowser.sourceforge.net/
* Compiled and packaged a free JVM for the Nokia770 internet tablet.
* Is working on enhancing GTK's theme API and later on adapting
GtkLookAndFeel to take advantage of those enhancements:
http://bugzilla.gnome.org/show_bug.cgi?id=515600
* Finished some commercial projects written in Java with focus on
client-server architectures


7.) Links, References, ...
http://keithp.com/~keithp/render/
http://www.nabble.com/Best-way-organizing-XRender-additions-for-the-X11-pipeline--td14152122.html
http://www.phoronix.com/scan.php?page=article&item=934&num=1
http://bugs.sun.com/view_bug.do?bug_id=5086814 (RFE: Java 2D: consider
Xrender for anti-aliased text on remote X11)
http://bugs.sun.com/view_bug.do?bug_id=6307603 ([X11] Use RENDER
extension for complex operations done in software)

[prev in list] [next in list] [prev in thread] [next in thread]