[prev in list] [next in list] [prev in thread] [next in thread]
List: cairo
Subject: Re: [cairo] Time for a release ?
From: Heiko Lewin <hlewin () gmx ! de>
Date: 2020-11-20 2:17:53
Message-ID: 0aead640-1a9a-ad37-9b32-8e5f959d787e () gmx ! de
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Am 20.11.2020 um 00:14 schrieb Bryce Harrington:
> On Thu, Nov 19, 2020 at 10:23:33PM +0000, Stuart Axon wrote:
>> [1] Locking issues
>>
>> I ran the tests in a Windows VM with Meson and quite a few timed out, i=
t
>> seems like quite a few will be the ones with locking issues.
>>
>> https://gist.github.com/stuaxo/1d125dc83888a28f60c684718619a112
>>
>> There were a couple of tests that popped up dialog boxes under Windows =
when
>> they failed, which seems not idea for CI.
>>
>> I'll see if I can do the same under Linux and possibly MacOS to see if =
the
>> same ones time out.
>>
>> [2]=C2=A0 Rendering issues:
>>
>> It sounds like there are different sources for these, so maybe it's wor=
th
>> having tickets for the different kinds of failure, once a baseline is
>> established.
>>
>> Maybe we could make sanity checks for the different failure types (or
>> designate existing tests for these) ?
>>
>> Pixman:=C2=A0 As mentioned elsewhere, changes that affect rendering her=
e will
>> mean ref images need updating.
> For rendering issues in the image backend, I tended to find that a
> common root cause would often be a changed assumption about how
> drawing operations should behave. For example, algorithm changes in
> pixman can result in a pixel to flip on or off, or to be set to a
> slightly different shade of grey. Legitimate performance or quality
> improvements to blending or dithering algorithms can cause these kinds
> of changes too. Typically for those cases the rendering behavior is
> kind of a judgment call, so not much to do except redo the reference
> images to match the new behavior.
Latest pixman version also has a regression in downscaling
https://gitlab.freedesktop.org/pixman/pixman/-/issues/43
>> Fonts and operating systems:
>>
>> Do we expect output on differing operating systems to be the same?=C2=
=A0 If not,
>> maybe per-OS reference images is could work (e.g. Linux/MacOS/Windows w=
ould
>> be expected to be similar, but whichever OS you are running on should b=
e an
>> exact match).
> A foundational principle for Cairo is that rendering output should be
> reliably identical not only cross-platform but also cross-device
> (i.e. what's on your monitor matches what's printed on paper, and
> rendering to PNG, PDF, or SVG should all be identical.)
>
> In practice, no of course actual renderings will differ from one
> platform to the next.
>
> Honestly, though, it's even more complicated than that due to dependence
> on fonts, dependency versions, etc. So I wouldn't expect images on
> Ubuntu to match ones from Fedora, nor even ones rendered on Ubuntu 16.04
> to match those on Ubuntu 20.04. Achieving decent apples-to-apples
> comparison gets to be a bit of an combinatorial explosion...
A bit of the variance problem is rooted in cairo, though. For example
CAIRO_HINT_STYLE_DEFAULT
Use the default hint style for
font backend and target device, since 1.0
means that cairo will not set up anything, but use whatever
freetype/fontconfig, CoreText or
GDI will come up with. The tests do not do that.
To get halfway stable results you'd have to manually configure all
options to
a non-default value I guess.
So, as I understand now, the procedure for testing can only be:
- Generate reference images
- Mess with code
- Run tests again
This even is documented. Now, that is enlightening.
[Attachment #5 (text/html)]
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">Am 20.11.2020 um 00:14 schrieb Bryce
Harrington:<br>
</div>
<blockquote type="cite"
cite="mid:20201119231442.GK2602109@bryceharrington.org">
<pre class="moz-quote-pre" wrap="">On Thu, Nov 19, 2020 at 10:23:33PM +0000, \
Stuart Axon wrote: </pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">[1] Locking issues
I ran the tests in a Windows VM with Meson and quite a few timed out, it
seems like quite a few will be the ones with locking issues.
<a class="moz-txt-link-freetext" \
href="https://gist.github.com/stuaxo/1d125dc83888a28f60c684718619a112">https://gist.github.com/stuaxo/1d125dc83888a28f60c684718619a112</a>
There were a couple of tests that popped up dialog boxes under Windows when
they failed, which seems not idea for CI.
I'll see if I can do the same under Linux and possibly MacOS to see if the
same ones time out.
[2] Rendering issues:
It sounds like there are different sources for these, so maybe it's worth
having tickets for the different kinds of failure, once a baseline is
established.
Maybe we could make sanity checks for the different failure types (or
designate existing tests for these) ?
Pixman: As mentioned elsewhere, changes that affect rendering here will
mean ref images need updating.
</pre>
</blockquote>
For rendering issues in the image backend, I tended to find that a
common root cause would often be a changed assumption about how
drawing operations should behave. For example, algorithm changes
in pixman can result in a pixel to flip on or off, or to be set to
a slightly different shade of grey. Legitimate performance or
quality improvements to blending or dithering algorithms can cause
these kinds of changes too. Typically for those cases the
rendering behavior is kind of a judgment call, so not much to do
except redo the reference images to match the new behavior. </blockquote>
Latest pixman version also has a regression in downscaling<br>
<a class="moz-txt-link-freetext"
href="https://gitlab.freedesktop.org/pixman/pixman/-/issues/43">https://gitlab.freedesktop.org/pixman/pixman/-/issues/43</a>
<blockquote type="cite"
cite="mid:20201119231442.GK2602109@bryceharrington.org">
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">Fonts and operating systems:
Do we expect output on differing operating systems to be the same? If not,
maybe per-OS reference images is could work (e.g. Linux/MacOS/Windows would
be expected to be similar, but whichever OS you are running on should be an
exact match).
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">A foundational principle for Cairo is that \
rendering output should be reliably identical not only cross-platform but also \
cross-device (i.e. what's on your monitor matches what's printed on paper, and
rendering to PNG, PDF, or SVG should all be identical.)
In practice, no of course actual renderings will differ from one
platform to the next.
Honestly, though, it's even more complicated than that due to dependence
on fonts, dependency versions, etc. So I wouldn't expect images on
Ubuntu to match ones from Fedora, nor even ones rendered on Ubuntu 16.04
to match those on Ubuntu 20.04. Achieving decent apples-to-apples
comparison gets to be a bit of an combinatorial explosion...</pre>
</blockquote>
<br>
<p>A bit of the variance problem is rooted in cairo, though. For
example</p>
<pre>CAIRO_HINT_STYLE_DEFAULT
Use the default hint style for
font backend and target device, since 1.0</pre>
means that cairo will not set up anything, but use whatever
freetype/fontconfig, CoreText or<br>
GDI will come up with. The tests do not do that.<br>
To get halfway stable results you'd have to manually configure all
options to<br>
a non-default value I guess.
<p>So, as I understand now, the procedure for testing can only be:</p>
- Generate reference images<br>
- Mess with code<br>
- Run tests again<br>
<p>This even is documented. Now, that is enlightening. </p>
</body>
</html>
--
cairo mailing list
cairo@cairographics.org
https://lists.cairographics.org/mailman/listinfo/cairo
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic