[prev in list] [next in list] [prev in thread] [next in thread]
List: webkit-dev
Subject: Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer
From: Noam Rosenthal <noam () webkit ! org>
Date: 2019-11-12 13:03:28
Message-ID: CAGttnEV3H4h0E_3RLHqW3jCQdHSGfdMCSxEnDCB_6YpPE7Mq=g () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak <mjs@apple.com> wrote:
>
>
> On Nov 11, 2019, at 4:37 AM, Noam Rosenthal <noam@webkit.org> wrote:
>
>
>
> On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak <mjs@apple.com> wrote:
>
>> - Is there a specification for Content-DPR? All I could find in bugzila
>> and elsewhere is links to <
>> https://httpwg.org/http-extensions/client-hints.html>, which does not
>> mention Content-DPR at all).
>>
> In a nutshell, the spec is in transition from IETF to HTML/fetch, and the
> client-hints aspects of it are still in progress (unlike conent-dpr, which
> is much simpler hasn't changed since introduced). It's documented here:
> https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md
> .
> I will answer more about this separately if required, waiting for some
> info from the people working on the spec originally.
>
>
> It looks like these are the relevant Pull Requests:
>
>
> HTML: https://github.com/whatwg/html/pull/3774 |
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density
> Fetch: https://github.com/whatwg/fetch/pull/773 |
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density
>
> It looks like these are in a somewhat messy state. For example, Fetch
> places the Content-DPR value in an "image density" variable, but it doesn't
> look like the Pull Request to HTML doesn't use it anywhere. As another
> example, HTML directly references Content-DPR as setting the "current pixel
> density", but does not specify that it would take priority over a density
> derived from srcset. There are also no diffs to CSS Images or to SVG, which
> provide distinct ways to reference images and which presumably should
> respect Content-DPR.
>
>
>
>> - Can you give us some examples of how the CDN would make the decision of
>> what density of image to serve without knowing the client's DPR via
>> client-hints?
>>
> Some examples from https://github.com/eeeps/content-dpr-explainer:
> - making decisions by user agent, like choosing to cap certain images for
> user-agents known to have smaller displays
> - making decisions by traffic/geo, like serving lower-resolution images
> when traffic is bogged down
> - A client may ask for "placeholder image" (e.g. ?placeholder=true in the
> URL), and the CDN would decide whether to serve a lower-quality JPEG or to
> scale it down, or do both.
>
>
> These seem like reasonable use cases.
>
>
> - Is this presuming situations where the CDN serves the images but not the
>> markup or CSS (which presumably could be rewritten to account for DPR)?
>>
> Correct.
>
>
>> - What happens if Content-DPR is provided, but the markup or CSS has
>> conflicting explicit info? For example, <img srcset="image-2x.jpg 2x,
>> image.jpg 1x">, and image-2x.jpg is served with a Content-DPR header of 3x.
>> Or the analogous thing with CSS.
>>
> When image size is decided, css/markup takes precedence, then content DPR,
> and finally srcset-selection DPR. This is actually a bonus value of this
> feature - if the server happens to serve an image that differs from the
> ratio requested in the srcset tag, the intrinsic size match the served
> content and not the request (which is more of a recommendation).
>
> - Does Content-DPR have any effect on images where an explicit size is
>> provided, either in markup or in CSS? (I'm assuming no.)
>>
> No, it only has effect on intrinsic size.
>
>
> Overall, this seems like a feature with valid use cases. But
> unfortunately, it's a (currently) Blink-only feature with no clear
> specification. To the extent there is a spec, it's mixed in with multiple
> Pull Requests. These pull requests mix in Client Hints, a feature unlikely
> to gain multi implementor support, so they probably won't land any time
> soon. And the language about Content-DPR is broken and does not not seem to
> match Chrome's actual implementation.
>
> So in summary, there is no proper spec, and Chrome has shown willingness
> to have their implementation change faster than even draft spec language in
> PRs can keep up with.
>
> Although the use case seems valid, I don't think it's a good idea for
> WebKit to implement the feature in this state. It seems likely to cause
> interop headaches and a need to reverse engineer Blink rather than
> developing against specs and test cases.
>
> I would like to see a clean spec, either separating Content-DPR PRs from
> client-hints or making a standalone spec. And either way it should be fixed
> to match what Chrome has implemented, and have WPT tests that reflect the
> spec and intended behavior.
>
I hear you. Let me get back to this thread once we're cleaner on the spec
front.
Thank you for the detailed response, it gives a clear path forward.
~Noam
>
> Regards,
> Maciej
>
>
>
>
>
[Attachment #5 (text/html)]
<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak <<a \
href="mailto:mjs@apple.com">mjs@apple.com</a>> wrote:<br></div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: \
break-word;"><br><div><br><blockquote type="cite"><div>On Nov 11, 2019, at 4:37 AM, \
Noam Rosenthal <<a href="mailto:noam@webkit.org" \
target="_blank">noam@webkit.org</a>> wrote:</div><br><div><div dir="ltr"><div \
dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On \
Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak <<a href="mailto:mjs@apple.com" \
target="_blank">mjs@apple.com</a>> wrote:</div><blockquote class="gmail_quote" \
style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div><div><div>- Is there a specification for \
Content-DPR? All I could find in bugzila and elsewhere is links to <<a \
href="https://httpwg.org/http-extensions/client-hints.html" \
target="_blank">https://httpwg.org/http-extensions/client-hints.html</a>>, which \
does not mention Content-DPR at all).</div></div></div></blockquote><div>In a \
nutshell, the spec is in transition from IETF to HTML/fetch, and the client-hints \
aspects of it are still in progress (unlike conent-dpr, which is much simpler \
hasn't changed since introduced). It's documented here: <a \
href="https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md" \
rel="noopener noreferrer" \
style="box-sizing:inherit;font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)" \
target="_blank">https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md</a>.</div><div><div>I \
will answer more about this separately if required, waiting for some info from the \
people working on the spec \
originally.</div></div></div></div></div></blockquote><div><br></div>It looks like \
these are the relevant Pull Requests:</div><div><br></div><div><br></div><div>HTML: \
<a href="https://github.com/whatwg/html/pull/3774" \
target="_blank">https://github.com/whatwg/html/pull/3774</a> | <a \
href="https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density" \
target="_blank">https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density</a></div><div>Fetch: \
<a href="https://github.com/whatwg/fetch/pull/773" \
target="_blank">https://github.com/whatwg/fetch/pull/773</a> | <a \
href="https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density" \
target="_blank">https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density</a></div><div><br></div><div>It \
looks like these are in a somewhat messy state. For example, Fetch places the \
Content-DPR value in an "image density" variable, but it doesn't look like the Pull \
Request to HTML doesn't use it anywhere. As another example, HTML directly references \
Content-DPR as setting the "current pixel density", but does not specify that it \
would take priority over a density derived from srcset. There are also no diffs to \
CSS Images or to SVG, which provide distinct ways to reference images and which \
presumably should respect Content-DPR.</div><div><br><blockquote \
type="cite"><div><div dir="ltr"><div class="gmail_quote"><div><div></div></div><div> \
</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px \
solid rgb(204,204,204);padding-left:1ex"><div><div><div>- Can you give us some \
examples of how the CDN would make the decision of what density of image to serve \
without knowing the client's DPR via client-hints? \
</div></div></div></blockquote><div> Some examples from <a \
href="https://github.com/eeeps/content-dpr-explainer" \
target="_blank">https://github.com/eeeps/content-dpr-explainer</a>:</div><div>- \
making decisions by user agent, like choosing to cap certain images for user-agents \
known to have smaller displays</div><div>- making decisions by traffic/geo, like \
serving lower-resolution images when traffic is bogged down</div><div>- A client may \
ask for "placeholder image" (e.g. ?placeholder=true in the URL), and the \
CDN would decide whether to serve a lower-quality JPEG or to scale it down, or do \
both.</div></div></div></div></blockquote><div><br></div>These seem like reasonable \
use cases.</div><div><br><blockquote type="cite"><div><div dir="ltr"><div \
class="gmail_quote"><div><br></div><blockquote class="gmail_quote" style="margin:0px \
0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div><div><div>- Is this presuming situations \
where the CDN serves the images but not the markup or CSS (which presumably could be \
rewritten to account for DPR)?</div></div></div></blockquote><div>Correct.</div><div> \
</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px \
solid rgb(204,204,204);padding-left:1ex"><div><div><div>- What happens if Content-DPR \
is provided, but the markup or CSS has conflicting explicit info? For example, \
<img srcset="image-2x.jpg 2x, image.jpg 1x">, and image-2x.jpg is served with a \
Content-DPR header of 3x. Or the analogous thing with \
CSS.</div></div></div></blockquote><div>When image size is decided, css/markup takes \
precedence, then content DPR, and finally srcset-selection DPR. This is actually a \
bonus value of this feature - if the server happens to serve an image that differs \
from the ratio requested in the srcset tag, the intrinsic size match the served \
content and not the request (which is more of a \
recommendation).</div><div><br></div><blockquote class="gmail_quote" \
style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div><div><div>- Does Content-DPR have any effect \
on images where an explicit size is provided, either in markup or in CSS? (I'm \
assuming no.)</div></div></div></blockquote><div>No, it only has effect on intrinsic \
size. </div></div></div></div></blockquote><br></div><div>Overall, this seems like a \
feature with valid use cases. But unfortunately, it's a (currently) Blink-only \
feature with no clear specification. To the extent there is a spec, it's mixed in \
with multiple Pull Requests. These pull requests mix in Client Hints, a feature \
unlikely to gain multi implementor support, so they probably won't land any time \
soon. And the language about Content-DPR is broken and does not not seem to match \
Chrome's actual implementation.</div><div><br></div><div>So in summary, there is no \
proper spec, and Chrome has shown willingness to have their implementation change \
faster than even draft spec language in PRs can keep up \
with.</div><div><br></div><div>Although the use case seems valid, I don't think it's \
a good idea for WebKit to implement the feature in this state. It seems likely to \
cause interop headaches and a need to reverse engineer Blink rather than developing \
against specs and test cases.</div><div><br></div><div>I would like to see a clean \
spec, either separating Content-DPR PRs from client-hints or making a standalone \
spec. And either way it should be fixed to match what Chrome has implemented, and \
have WPT tests that reflect the spec and intended \
behavior.</div></div></blockquote><div><br></div><div>I hear you. Let me get back to \
this thread once we're cleaner on the spec front.</div><div>Thank you for the \
detailed response, it gives a clear path forward.</div><div>~Noam </div><blockquote \
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid \
rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: \
break-word;"><div><br></div><div>Regards,</div><div>Maciej</div><div><br></div><div><br></div><div><br></div><br></div></blockquote></div></div>
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic