'Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)'

[prev in list] [next in list] [prev in thread] [next in thread] 

List:       webkit-dev
Subject:    Re: [webkit-dev] WebKit position on Paint Timing / (first paint, first contentful paint)
From:       Maciej Stachowiak <mjs () apple ! com>
Date:       2020-03-02 0:37:54
Message-ID: 1C8CD54C-991B-4948-9E6E-CA6261A41202 () apple ! com
[Download RAW message or body]

[Attachment #2 (multipart/alternative)]

> On Mar 1, 2020, at 4:19 PM, Noam Rosenthal <noam@webkit.org> wrote:
> 
> 
> 
> On Mon, Mar 2, 2020 at 1:18 AM Maciej Stachowiak <mjs@apple.com \
> <mailto:mjs@apple.com>> wrote: 
> 
> > On Mar 1, 2020, at 2:57 PM, Noam Rosenthal <noam@webkit.org \
> > <mailto:noam@webkit.org>> wrote: 
> > 
> > 
> > On Mon, Mar 2, 2020 at 12:21 AM Maciej Stachowiak <mjs@apple.com \
> > <mailto:mjs@apple.com>> wrote: 
> > 
> > > On Mar 1, 2020, at 2:07 PM, Noam Rosenthal <noam@webkit.org \
> > > <mailto:noam@webkit.org>> wrote: 
> > > 
> > > The first visually non-empty milestone almost always happens way before this \
> > > point. The above is just a fallback to make sure we eventually hit this \
> > > milestone. For example, if a document is totally blank even after loading the \
> > > document and all subresources, we want to paint it instead of waiting forever. 
> > > The visually non-empty heuristic is elsewhere.
> > > 
> > > Note that WebKit would consider the paint triggered by the above fallback code \
> > > to be both a first paint and "first visually non-empty paint" or "first \
> > > meaningful paint", which somewhat corresponds to Chrome's notion of "first \
> > > contentful paint". 
> > > First paint can only happen earlier under even more unusual circumstances, I \
> > > believe there is a timeout after which we will paint even if all we can paint \
> > > is blank white or background color. 
> > > > 
> > > > But let's take the specifics to Slack/bugzilla?
> > > 
> > > I think it would be good for you to talk to people who understand WebKit's \
> > > layout/paint milestones in detail before taking things to bugzilla. Ask on \
> > > Slack, and I'll point you to the right people. Oops, just saw this, an initial \
> > > (not for review) patch is already in bugzilla :) But I'll continue the \
> > > discussion - I have better idea of what to ask now. Who would be the right \
> > > people to ask?
> > 
> > Alan, Simon, and Ryosuke should all know about this.
> > Awesomeness.
> > 
> > 
> > A few of us sat down and reviewed this spec. We think that, as written, the \
> > definition of "first contentful paint" is underspecified and in some cases buggy. \
> > For example, it says a "non-white canvas" should count as contentful. But \
> > canvases don't start as white, they start as fully transparent black (all zeros). \
> > And checking the color of every pixel in a canvas is expensive. The intent here \
> > is probably that a canvas that has ever been painted into is contentful (or one \
> > that has been painted into more recently than the last time it was cleared). Yes, \
> > I also thought that part of the spec was strange. I'll help revise it. btw the \
> > FirstMeaningfulPaint in webkit doesn't look at canvas content at all - rather on \
> > the creation of a RenderCanvasElement... maybe being closer to the spec here \
> > would be better rendering-wise? Also, the current layout milestones doen't \
> > consider background images as "rendered pixels". Is that on purpose?
> 
> We think background images (and SVG, if not included yet) should count as \
> meaningful content. And canvases should probably be limited to non-empty ones. OK - \
> that's more like the spec. 
> 
> > 
> > 
> > In any case, it definitely does not match the WebKit notion of first visually \
> > non-empty layout / first meaningful paint, in some cases in ways that we regret. \
> > Regret, as in - it was better if the spec was closer to what webkit was doing?
> 
> No, I mean in some cases we think we should do what the spec says.
> 
> > I think it's OK if the spec's FCP and webkit's FMP are not the same, and if FCP \
> > is fired according to spec, and after the actual painting in webkit was done. 
> 
> I think we should try to align with the spec. Otherwise, because WebKit usually \
> won't paint at all until webkit-FMP time, FCP won't fire until that time (since it \
> is only fired when an actual paint happens). 
> 
> > 
> > Spec issues to follow.
> > 
> > We also think exposing "first paint" is harmful and we'd rather not implement it. \
> > We don't agree that painting non-contentful content quickly is a good idea, \
> > either for browsers or for websites. And this API will inevitably be used to \
> > compare browsers, regardless of disclaimers that it shouldn't be. Harmful in the \
> > sense of comparing browsers? I don't think it can, regardless of disclaimers - \
> > since the hardware used for the browsers (at least on mobile) is vastly \
> > different, and also the networks. How about exposing a prefixed entry? e.g. \
> > "-webkit-first-paint" - something that doesn't try to seem comparable? The goal \
> > here is to help people optimize their site and make sure it doesn't regress on \
> > webkit, rather than create meaningless cross-browser comparisons... 
> 
> We will probably take up our complaint in the form of a spec issue. In the \
> meantime, it would be nice if first paint was controlled by a separate flag. 
> > 
> > In any case even having first-contentful-paint would be useful - with or without \
> > first-paint. 
> > 
> > It's probably unwise to implement this until the spec is fixed. And once it is, \
> > we should change our "first visually non-empty layout" notion to match it before \
> > exposing the corresponding paint timing milestone. Not sure that's necessary. We \
> > can match the spec without changing the way we render. For example, "first \
> > visually non-empty layout" needs a minimum amount of pixels/characters - do we \
> > want to change the spec to have this heuristic, or change that heuristic in \
> > webkit, or neither?
> 
> We want to change one or both of them to match, if at all possible.
> OK, to summarize what I got from this
> - we want the spec and webkit painting to be as close as possible
> - The spec needs to be clearer/less buggy about a few things, such as "White" \
>                 canvas, spec issues to be files
> - WebKit should be closer to the spec wrt canvas, backgrounds and potentially \
>                 pixel/character threshold (TBD)
> - FP is more sensitive than FCP, because it exposes browser differences and may \
> lead to unwanted comparisons and misunderstanding. Thus, it should be exposed as a \
> different runtime feature flag. 
> One thing I'm wondering about - would it be better to change the rendering \
> heuristics together with implementing the paint API reporting? Or would it be \
> better to separate those concerns a bit in terms of implementation? I mean, having \
> the performance APIs in the code behind 2 flags with failed tests conforming to the \
> spec might help iterate on the actual rendering. What would you consider a better \
> approach here?

If I were doing this myself, I'd first change rendering heuristics (probably without \
a flag) and then add the web-facing API.

What you describe might be reasonable if there are WPT conformance tests that can \
distinguish cases where the FCP timing is too late or too soon. It's probably \
possible to make such tests, but I am not sure if they already exist.

- Maciej

[Attachment #5 (unknown)]

<html><head><meta http-equiv="Content-Type" content="text/html; \
charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; \
line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote \
type="cite" class=""><div class="">On Mar 1, 2020, at 4:19 PM, Noam Rosenthal &lt;<a \
href="mailto:noam@webkit.org" class="">noam@webkit.org</a>&gt; wrote:</div><br \
class="Apple-interchange-newline"><div class=""><br \
class="Apple-interchange-newline"><br style="caret-color: rgb(0, 0, 0); font-family: \
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; \
font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; \
text-transform: none; white-space: normal; word-spacing: 0px; \
-webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div \
class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; \
font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none;"><div dir="ltr" class="gmail_attr">On Mon, Mar 2, 2020 at 1:18 \
AM Maciej Stachowiak &lt;<a href="mailto:mjs@apple.com" target="_blank" \
class="">mjs@apple.com</a>&gt; wrote:<br class=""></div><blockquote \
class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; \
border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: \
1ex;"><div class=""><br class=""><div class=""><br class=""><blockquote type="cite" \
class=""><div class="">On Mar 1, 2020, at 2:57 PM, Noam Rosenthal &lt;<a \
href="mailto:noam@webkit.org" target="_blank" class="">noam@webkit.org</a>&gt; \
wrote:</div><br class=""><div class=""><br class=""><br style="font-family: \
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; \
font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; \
text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" \
class=""><div class="gmail_quote" style="font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; text-decoration: none;"><div dir="ltr" \
class="gmail_attr">On Mon, Mar 2, 2020 at 12:21 AM Maciej Stachowiak &lt;<a \
href="mailto:mjs@apple.com" target="_blank" class="">mjs@apple.com</a>&gt; wrote:<br \
class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; \
border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, \
204); padding-left: 1ex;"><div class=""><br class=""><div class=""><br \
class=""><blockquote type="cite" class=""><div class="">On Mar 1, 2020, at 2:07 PM, \
Noam Rosenthal &lt;<a href="mailto:noam@webkit.org" target="_blank" \
class="">noam@webkit.org</a>&gt; wrote:</div><br class=""><div class=""><div \
dir="ltr" class=""><div class="gmail_quote"><blockquote class="gmail_quote" \
style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; \
border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div \
class=""><br class="">The first visually non-empty milestone almost always happens \
way before this point. The above is just a fallback to make sure we eventually hit \
this milestone. For example, if a document is totally blank even after loading the \
document and all subresources, we want to paint it instead of waiting \
forever.</div><div class=""><br class=""></div><div class="">The visually non-empty \
heuristic is elsewhere.</div><div class=""><br class=""></div><div class="">Note that \
WebKit would consider the paint triggered by the above fallback code to be both a \
first paint and "first visually non-empty paint" or "first meaningful paint", which \
somewhat corresponds to Chrome's notion of "first contentful paint".</div><div \
class=""><br class=""></div><div class="">First paint can only happen earlier under \
even more unusual circumstances, I believe there is a timeout after which we will \
paint even if all we can paint is blank white or background color.</div><br \
class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" \
style="font-family: Helvetica; font-size: 12px; font-style: normal; \
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: \
start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: \
0px; text-decoration: none;" class=""><div class="gmail_quote"><div class=""><div \
style="margin: 0px; font-variant-numeric: normal; font-variant-east-asian: normal; \
font-stretch: normal; font-size: 12px; line-height: normal; font-family: Menlo; \
color: rgb(155, 35, 147);" class=""><span class=""><br class="">But let's take the \
specifics to Slack/bugzilla?</span></div></div></div></div></div></blockquote><div \
class=""><br class=""></div><div class="">I think it would be good for you to talk to \
people who understand WebKit's layout/paint milestones in detail before taking things \
to bugzilla. Ask on Slack, and I'll point you to the right \
people.</div></div></blockquote><div class="">Oops, just saw this, an initial (not \
for review) patch is already in bugzilla :)<br class="">But I'll continue the \
discussion - I have better idea of what to ask now.<br class="">Who would be the \
right people to ask?</div></div></div></div></blockquote></div><br class=""><div \
class="">Alan, Simon, and Ryosuke should all know about \
this.</div></div></blockquote><div class="">Awesomeness.<br \
class="">&nbsp;</div><blockquote class="gmail_quote" style="margin: 0px 0px 0px \
0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, \
204, 204); padding-left: 1ex;"><div class=""><div class=""><br class=""></div><div \
class="">A few of us sat down and reviewed this spec. We think that, as written, the \
definition of "first contentful paint" is underspecified and in some cases buggy. For \
example, it says a "non-white canvas" should count as contentful. But canvases don't \
start as white, they start as fully transparent black (all zeros). And checking the \
color of every pixel in a canvas is expensive. The intent here is probably that a \
canvas that has ever been painted into is contentful (or one that has been painted \
into more recently than the last time it was cleared).</div></div></blockquote><div \
class="">Yes, I also thought that part of the spec was strange. I'll help revise \
it.<br class="">btw the FirstMeaningfulPaint in webkit doesn't look at canvas content \
at all - rather on the creation of a RenderCanvasElement... maybe being closer to the \
spec here would be better rendering-wise? Also, the current layout milestones doen't \
consider background images as "rendered pixels". Is that on purpose?<br \
class=""></div></div></div></blockquote><div class=""><br class=""></div><div \
class="">We think background images (and SVG, if not included yet) should count as \
meaningful content. And canvases should probably be limited to non-empty \
ones.</div></div></div></blockquote><div class="">OK - that's more like the spec.<br \
class="">&nbsp;</div><blockquote class="gmail_quote" style="margin: 0px 0px 0px \
0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, \
204, 204); padding-left: 1ex;"><div class=""><div class=""><br class=""><blockquote \
type="cite" class=""><div class=""><div class="gmail_quote" style="font-family: \
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; \
font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; \
text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: \
none;"><div class="">&nbsp;</div><blockquote class="gmail_quote" style="margin: 0px \
0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: \
rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div class=""><br \
class=""></div><div class="">In any case, it definitely does not match the WebKit \
notion of first visually non-empty layout / first meaningful paint, in some cases in \
ways that we regret.</div></div></blockquote><div class="">Regret, as in - it was \
better if the spec was closer to what webkit was doing?<br \
class=""></div></div></div></blockquote><div class=""><br class=""></div><div \
class="">No, I mean in some cases we think we should do what the spec says.</div><br \
class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" \
style="font-family: Helvetica; font-size: 12px; font-style: normal; \
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: \
start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: \
0px; text-decoration: none;"><div class="">I think it's OK if the spec's FCP and \
webkit's FMP are not the same, and if FCP is fired according to spec, and after the \
actual painting in webkit was done.&nbsp;</div></div></div></blockquote><div \
class=""><br class=""></div><div class="">I think we should try to align with the \
spec. Otherwise, because WebKit usually won't paint at all until webkit-FMP time, FCP \
won't fire until that time (since it is only fired when an actual paint \
happens).</div></div></div></blockquote><div class="">&nbsp;</div><blockquote \
class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; \
border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: \
1ex;"><div class=""><div class=""><br class=""><blockquote type="cite" class=""><div \
class=""><div class="gmail_quote" style="font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: \
normal; word-spacing: 0px; text-decoration: none;"><blockquote class="gmail_quote" \
style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; \
border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div \
class=""><br class=""></div><div class="">Spec issues to follow.</div><div \
class=""><br class=""></div><div class="">We also think exposing "first paint" is \
harmful and we'd rather not implement it. We don't agree that painting non-contentful \
content quickly is a good idea, either for browsers or for websites. And this API \
will inevitably be used to compare browsers, regardless of disclaimers that it \
shouldn't be.</div></div></blockquote><div class="">Harmful in the sense of comparing \
browsers? I don't think it can, regardless of disclaimers - since the hardware used \
for the browsers (at least on mobile) is vastly different, and also the networks. How \
about exposing a prefixed entry? e.g. "-webkit-first-paint" - something that doesn't \
try to seem comparable? The goal here is to help people optimize their site and make \
sure it doesn't regress on webkit, rather than create meaningless cross-browser \
comparisons...&nbsp;<br class=""></div></div></div></blockquote><div class=""><br \
class=""></div><div class="">We will probably take up our complaint in the form of a \
spec issue. In the meantime, it would be nice if first paint was controlled by a \
separate flag.</div></div></div></blockquote><blockquote class="gmail_quote" \
style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; \
border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div \
class=""><br class=""><blockquote type="cite" class=""><div class=""><div \
class="gmail_quote" style="font-family: Helvetica; font-size: 12px; font-style: \
normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; \
text-align: start; text-indent: 0px; text-transform: none; white-space: normal; \
word-spacing: 0px; text-decoration: none;"><div class=""><br class="">In any case \
even having first-contentful-paint would be useful - with or without first-paint.<br \
class=""><br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px \
0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: \
rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div class=""><br \
class=""></div><div class="">It's probably unwise to implement this until the spec is \
fixed. And once it is, we should change our "first visually non-empty layout" notion \
to match it before exposing the corresponding paint timing \
milestone.</div></div></blockquote><div class="">Not sure that's necessary. We can \
match the spec without changing the way we render. For example,&nbsp;"first visually \
non-empty layout" needs a minimum amount of pixels/characters - do we want to change \
the spec to have this heuristic, or change that heuristic in webkit, or \
neither?</div></div></div></blockquote><div class=""><br class=""></div><div \
class="">We want to change one or both of them to match, if at all \
possible.</div></div></div></blockquote><div class="">OK, to summarize what I got \
from this<br class="">- we want the spec and webkit painting to be as close as \
possible<br class="">- The spec needs to be clearer/less buggy about a few things, \
such as "White" canvas, spec issues to be files<br class="">- WebKit should be closer \
to the spec wrt canvas, backgrounds and potentially pixel/character threshold \
(TBD)<br class="">- FP is more sensitive than FCP, because it exposes browser \
differences and may lead to unwanted comparisons and misunderstanding. Thus, it \
should be exposed as a different runtime feature flag.<br class=""><br class="">One \
thing I'm wondering about - would it be better to change the rendering heuristics \
together with implementing the paint API reporting? Or would it be better to separate \
those concerns a bit in terms of implementation? I mean, having the performance APIs \
in the code behind 2 flags with failed tests conforming to the spec might help \
iterate on the actual rendering. What would you consider a better approach \
here?</div></div></div></blockquote></div><br class=""><div class="">If I were doing \
this myself, I'd first change rendering heuristics (probably without a flag) and then \
add the web-facing API.</div><div class=""><br class=""></div><div class="">What you \
describe might be reasonable if there are WPT conformance tests that can distinguish \
cases where the FCP timing is too late or too soon. It's probably possible to make \
such tests, but I am not sure if they already exist.</div><div class=""><br \
class=""></div><div class="">- Maciej</div><div class=""><br class=""></div><div \
class=""><br class=""></div></body></html>

_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[prev in list] [next in list] [prev in thread] [next in thread] 