[prev in list] [next in list] [prev in thread] [next in thread]
List: webkit-dev
Subject: Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer
From: Maciej Stachowiak <mjs () apple ! com>
Date: 2020-03-05 2:24:26
Message-ID: 352F4099-2129-491F-8542-F3C3153B34A6 () apple ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
> On Mar 2, 2020, at 11:22 PM, Noam Rosenthal <noam@webkit.org> wrote:
>
>
>
> On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak <mjs@apple.com <mailto:mjs@apple.com>> wrote:
>
>
> > On Nov 11, 2019, at 4:37 AM, Noam Rosenthal <noam@webkit.org <mailto:noam@webkit.org>> wrote:
> >
> >
> >
> > On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak <mjs@apple.com <mailto:mjs@apple.com>> wrote:
> > - Is there a specification for Content-DPR? All I could find in bugzila and elsewhere is links to \
> > <https://httpwg.org/http-extensions/client-hints.html \
> > <https://httpwg.org/http-extensions/client-hints.html>>, which does not mention Content-DPR at all). \
> > In a nutshell, the spec is in transition from IETF to HTML/fetch, and the client-hints aspects of it \
> > are still in progress (unlike conent-dpr, which is much simpler hasn't changed since introduced). \
> > It's documented here: \
> > https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md \
> > <https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md>. I \
> > will answer more about this separately if required, waiting for some info from the people working on \
> > the spec originally.
>
> It looks like these are the relevant Pull Requests:
>
>
> HTML: https://github.com/whatwg/html/pull/3774 <https://github.com/whatwg/html/pull/3774> | \
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density \
> <https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density>
> Fetch: https://github.com/whatwg/fetch/pull/773 <https://github.com/whatwg/fetch/pull/773> | \
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density \
> <https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density>
> It looks like these are in a somewhat messy state. For example, Fetch places the Content-DPR value in \
> an "image density" variable, but it doesn't look like the Pull Request to HTML doesn't use it anywhere. \
> As another example, HTML directly references Content-DPR as setting the "current pixel density", but \
> does not specify that it would take priority over a density derived from srcset. There are also no \
> diffs to CSS Images or to SVG, which provide distinct ways to reference images and which presumably \
> should respect Content-DPR.
>
>
> >
> > - Can you give us some examples of how the CDN would make the decision of what density of image to \
> > serve without knowing the client's DPR via client-hints? Some examples from \
> > https://github.com/eeeps/content-dpr-explainer \
> > <https://github.com/eeeps/content-dpr-explainer>:
> > - making decisions by user agent, like choosing to cap certain images for user-agents known to have \
> > smaller displays
> > - making decisions by traffic/geo, like serving lower-resolution images when traffic is bogged down
> > - A client may ask for "placeholder image" (e.g. ?placeholder=true in the URL), and the CDN would \
> > decide whether to serve a lower-quality JPEG or to scale it down, or do both.
>
> These seem like reasonable use cases.
>
> >
> > - Is this presuming situations where the CDN serves the images but not the markup or CSS (which \
> > presumably could be rewritten to account for DPR)? Correct.
> >
> > - What happens if Content-DPR is provided, but the markup or CSS has conflicting explicit info? For \
> > example, <img srcset="image-2x.jpg 2x, image.jpg 1x">, and image-2x.jpg is served with a Content-DPR \
> > header of 3x. Or the analogous thing with CSS. When image size is decided, css/markup takes \
> > precedence, then content DPR, and finally srcset-selection DPR. This is actually a bonus value of \
> > this feature - if the server happens to serve an image that differs from the ratio requested in the \
> > srcset tag, the intrinsic size match the served content and not the request (which is more of a \
> > recommendation).
> > - Does Content-DPR have any effect on images where an explicit size is provided, either in markup or \
> > in CSS? (I'm assuming no.) No, it only has effect on intrinsic size.
>
> Overall, this seems like a feature with valid use cases. But unfortunately, it's a (currently) \
> Blink-only feature with no clear specification. To the extent there is a spec, it's mixed in with \
> multiple Pull Requests. These pull requests mix in Client Hints, a feature unlikely to gain multi \
> implementor support, so they probably won't land any time soon. And the language about Content-DPR is \
> broken and does not not seem to match Chrome's actual implementation.
> So in summary, there is no proper spec, and Chrome has shown willingness to have their implementation \
> change faster than even draft spec language in PRs can keep up with.
> Although the use case seems valid, I don't think it's a good idea for WebKit to implement the feature \
> in this state. It seems likely to cause interop headaches and a need to reverse engineer Blink rather \
> than developing against specs and test cases.
> There has been a lot of work done on the spec front since the comments above, and I think we're ready \
> for a re-review the state.
> * The next pull request: https://github.com/whatwg/html/pull/5112 \
> <https://github.com/whatwg/html/pull/5112> has been thumbed up to make it into the HTML spec, and is \
> separate from client-hints, but requires second implementor interest, which is what this email thread \
> is for :)
I don't think it has? I'm seeing:
"[ ] At least two implementers are interested (and none opposed):"
And satisfying this is a requirement for adding features to WHATWG Living Standards.
And the review block at the bottom says "Review required" and "Merging is Blocked with big red X's.
> * Web platform tests are comprehensive, separated from client-hints, and match both the spec and the \
> Chrome implementation: https://github.com/web-platform-tests/wpt/tree/master/content-dpr \
> <https://github.com/web-platform-tests/wpt/tree/master/content-dpr>. We can always add more if we find \
> more use-cases.
> Would love to hear more thoughts!
Sounds like a good time to re-review.
Regards,
Maciej
[Attachment #5 (unknown)]
<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body \
style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br \
class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Mar 2, 2020, at 11:22 PM, \
Noam Rosenthal <<a href="mailto:noam@webkit.org" class="">noam@webkit.org</a>> wrote:</div><br \
class="Apple-interchange-newline"><div class=""><br class="Apple-interchange-newline"><br \
style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; \
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: \
0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none;" class=""><div class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: \
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; \
word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div dir="ltr" \
class="gmail_attr">On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak <<a href="mailto:mjs@apple.com" \
class="">mjs@apple.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: \
0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, \
204); padding-left: 1ex;"><div style="overflow-wrap: break-word;" class=""><br class=""><div class=""><br \
class=""><blockquote type="cite" class=""><div class="">On Nov 11, 2019, at 4:37 AM, Noam Rosenthal \
<<a href="mailto:noam@webkit.org" target="_blank" class="">noam@webkit.org</a>> wrote:</div><br \
class=""><div class=""><div dir="ltr" class=""><div dir="ltr" class=""><br class=""></div><br \
class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 11, 2019 at 12:13 AM \
Maciej Stachowiak <<a href="mailto:mjs@apple.com" target="_blank" class="">mjs@apple.com</a>> \
wrote:</div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; \
border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div \
class=""><div class="">- Is there a specification for Content-DPR? All I could find in bugzila and \
elsewhere is links to <<a href="https://httpwg.org/http-extensions/client-hints.html" target="_blank" \
class="">https://httpwg.org/http-extensions/client-hints.html</a>>, which does not mention Content-DPR \
at all).</div></div></div></blockquote><div class="">In a nutshell, the spec is in transition from IETF \
to HTML/fetch, and the client-hints aspects of it are still in progress (unlike conent-dpr, which is much \
simpler hasn't changed since introduced). It's documented here: <a \
href="https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md" \
rel="noopener noreferrer" target="_blank" style="box-sizing: inherit; font-family: Slack-Lato, appleLogo, \
sans-serif; font-size: 15px; font-variant-ligatures: common-ligatures; background-color: rgb(248, 248, \
248);" class="">https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md</a>.</div><div \
class=""><div class="">I will answer more about this separately if required, waiting for some info from \
the people working on the spec originally.</div></div></div></div></div></blockquote><div class=""><br \
class=""></div>It looks like these are the relevant Pull Requests:</div><div class=""><br \
class=""></div><div class=""><br class=""></div><div class="">HTML:<span \
class="Apple-converted-space"> </span><a href="https://github.com/whatwg/html/pull/3774" \
target="_blank" class="">https://github.com/whatwg/html/pull/3774</a><span \
class="Apple-converted-space"> </span>|<span class="Apple-converted-space"> </span><a \
href="https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density" target="_blank" \
class="">https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density</a></div><div \
class="">Fetch:<span class="Apple-converted-space"> </span><a \
href="https://github.com/whatwg/fetch/pull/773" target="_blank" \
class="">https://github.com/whatwg/fetch/pull/773</a><span \
class="Apple-converted-space"> </span>|<span class="Apple-converted-space"> </span><a \
href="https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density" target="_blank" \
class="">https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density</a></div><div \
class=""><br class=""></div><div class="">It looks like these are in a somewhat messy state. For example, \
Fetch places the Content-DPR value in an "image density" variable, but it doesn't look like the Pull \
Request to HTML doesn't use it anywhere. As another example, HTML directly references Content-DPR as \
setting the "current pixel density", but does not specify that it would take priority over a density \
derived from srcset. There are also no diffs to CSS Images or to SVG, which provide distinct ways to \
reference images and which presumably should respect Content-DPR.</div></div></blockquote><div \
class=""><br class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; \
border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: \
1ex;"><div style="overflow-wrap: break-word;" class=""><div class=""><br class=""><blockquote type="cite" \
class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div class=""><div \
class=""></div></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px \
0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); \
padding-left: 1ex;"><div class=""><div class=""><div class="">- Can you give us some examples of how the \
CDN would make the decision of what density of image to serve without knowing the client's DPR via \
client-hints? </div></div></div></blockquote><div class=""> Some examples from <span \
class="Apple-converted-space"> </span><a href="https://github.com/eeeps/content-dpr-explainer" \
target="_blank" class="">https://github.com/eeeps/content-dpr-explainer</a>:</div><div class="">- making \
decisions by user agent, like choosing to cap certain images for user-agents known to have smaller \
displays</div><div class="">- making decisions by traffic/geo, like serving lower-resolution images when \
traffic is bogged down</div><div class="">- A client may ask for "placeholder image" (e.g. \
?placeholder=true in the URL), and the CDN would decide whether to serve a lower-quality JPEG or to scale \
it down, or do both.</div></div></div></div></blockquote><div class=""><br class=""></div>These seem like \
reasonable use cases.</div><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div \
dir="ltr" class=""><div class="gmail_quote"><div class=""><br class=""></div><blockquote \
class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; \
border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div class=""><div class="">- Is \
this presuming situations where the CDN serves the images but not the markup or CSS (which presumably \
could be rewritten to account for DPR)?</div></div></div></blockquote><div class="">Correct.</div><div \
class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: \
1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div \
class=""><div class=""><div class="">- What happens if Content-DPR is provided, but the markup or CSS has \
conflicting explicit info? For example, <img srcset="image-2x.jpg 2x, image.jpg 1x">, and \
image-2x.jpg is served with a Content-DPR header of 3x. Or the analogous thing with \
CSS.</div></div></div></blockquote><div class="">When image size is decided, css/markup takes precedence, \
then content DPR, and finally srcset-selection DPR. This is actually a bonus value of this feature - if \
the server happens to serve an image that differs from the ratio requested in the srcset tag, the \
intrinsic size match the served content and not the request (which is more of a \
recommendation).</div><div class=""><br class=""></div><blockquote class="gmail_quote" style="margin: 0px \
0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); \
padding-left: 1ex;"><div class=""><div class=""><div class="">- Does Content-DPR have any effect on \
images where an explicit size is provided, either in markup or in CSS? (I'm assuming \
no.)</div></div></div></blockquote><div class="">No, it only has effect on intrinsic \
size. </div></div></div></div></blockquote><br class=""></div><div class="">Overall, this seems like \
a feature with valid use cases. But unfortunately, it's a (currently) Blink-only feature with no clear \
specification. To the extent there is a spec, it's mixed in with multiple Pull Requests. These pull \
requests mix in Client Hints, a feature unlikely to gain multi implementor support, so they probably \
won't land any time soon. And the language about Content-DPR is broken and does not not seem to match \
Chrome's actual implementation.</div><div class=""><br class=""></div><div class="">So in summary, there \
is no proper spec, and Chrome has shown willingness to have their implementation change faster than even \
draft spec language in PRs can keep up with. </div></div></blockquote><blockquote \
class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; \
border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="overflow-wrap: break-word;" \
class=""><div class=""><br class=""></div><div class="">Although the use case seems valid, I don't think \
it's a good idea for WebKit to implement the feature in this state. It seems likely to cause interop \
headaches and a need to reverse engineer Blink rather than developing against specs and test \
cases.</div><div class=""><br class=""></div></div></blockquote>There has been a lot of work done on the \
spec front since the comments above, and I think we're ready for a re-review the state.<br class="">* The \
next pull request: <a href="https://github.com/whatwg/html/pull/5112" \
class="">https://github.com/whatwg/html/pull/5112</a> has been thumbed up to make it into the HTML \
spec, and is separate from client-hints, but requires second implementor interest, which is what this \
email thread is for :)<br class=""></div></div></blockquote><div><br class=""></div><div>I don't think it \
has? I'm seeing:</div><div>"[ ] At least two implementers are interested (and none \
opposed):"</div><div><br class=""></div><div>And satisfying this is a requirement for adding features to \
WHATWG Living Standards.</div><div><br class=""></div><div>And the review block at the bottom says \
"Review required" and "Merging is Blocked with big red X's.</div><br class=""><blockquote type="cite" \
class=""><div class=""><div class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; \
font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: \
normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: \
0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class="">* Web platform tests are \
comprehensive, separated from client-hints, and match both the spec and the Chrome \
implementation: <a href="https://github.com/web-platform-tests/wpt/tree/master/content-dpr" \
class="">https://github.com/web-platform-tests/wpt/tree/master/content-dpr</a>. We can always add more if \
we find more use-cases.<br class=""><br class="">Would love to hear more thoughts!<br \
class=""></div></div></div></blockquote></div><br class=""><div class="">Sounds like a good time to \
re-review.</div><div class=""><br class=""></div><div class="">Regards,</div><div \
class="">Maciej</div><div class=""><br class=""></div></body></html>
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic