[prev in list] [next in list] [prev in thread] [next in thread]
List: webkit-dev
Subject: Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer
From: Maciej Stachowiak <mjs () apple ! com>
Date: 2020-03-05 2:24:26
Message-ID: 352F4099-2129-491F-8542-F3C3153B34A6 () apple ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
> On Mar 2, 2020, at 11:22 PM, Noam Rosenthal <noam@webkit.org> wrote:
>
>
>
> On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak <mjs@apple.com <mailto:mjs@apple.com>> \
> wrote:
>
> > On Nov 11, 2019, at 4:37 AM, Noam Rosenthal <noam@webkit.org <mailto:noam@webkit.org>> \
> > wrote:
> >
> >
> > On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak <mjs@apple.com <mailto:mjs@apple.com>> \
> > wrote:
> > - Is there a specification for Content-DPR? All I could find in bugzila and elsewhere is \
> > links to <https://httpwg.org/http-extensions/client-hints.html \
> > <https://httpwg.org/http-extensions/client-hints.html>>, which does not mention Content-DPR \
> > at all). In a nutshell, the spec is in transition from IETF to HTML/fetch, and the \
> > client-hints aspects of it are still in progress (unlike conent-dpr, which is much simpler \
> > hasn't changed since introduced). It's documented here: \
> > https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md \
> > <https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md>.
> > I will answer more about this separately if required, waiting for some info from the \
> > people working on the spec originally.
>
> It looks like these are the relevant Pull Requests:
>
>
> HTML: https://github.com/whatwg/html/pull/3774 <https://github.com/whatwg/html/pull/3774> | \
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density \
> <https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density>
>
> Fetch: https://github.com/whatwg/fetch/pull/773 <https://github.com/whatwg/fetch/pull/773> | \
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density \
> <https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density>
> It looks like these are in a somewhat messy state. For example, Fetch places the Content-DPR \
> value in an "image density" variable, but it doesn't look like the Pull Request to HTML \
> doesn't use it anywhere. As another example, HTML directly references Content-DPR as setting \
> the "current pixel density", but does not specify that it would take priority over a density \
> derived from srcset. There are also no diffs to CSS Images or to SVG, which provide distinct \
> ways to reference images and which presumably should respect Content-DPR.
>
>
> >
> > - Can you give us some examples of how the CDN would make the decision of what density of \
> > image to serve without knowing the client's DPR via client-hints? Some examples from \
> > https://github.com/eeeps/content-dpr-explainer \
> > <https://github.com/eeeps/content-dpr-explainer>:
> > - making decisions by user agent, like choosing to cap certain images for user-agents known \
> > to have smaller displays
> > - making decisions by traffic/geo, like serving lower-resolution images when traffic is \
> > bogged down
> > - A client may ask for "placeholder image" (e.g. ?placeholder=true in the URL), and the CDN \
> > would decide whether to serve a lower-quality JPEG or to scale it down, or do both.
>
> These seem like reasonable use cases.
>
> >
> > - Is this presuming situations where the CDN serves the images but not the markup or CSS \
> > (which presumably could be rewritten to account for DPR)? Correct.
> >
> > - What happens if Content-DPR is provided, but the markup or CSS has conflicting explicit \
> > info? For example, <img srcset="image-2x.jpg 2x, image.jpg 1x">, and image-2x.jpg is served \
> > with a Content-DPR header of 3x. Or the analogous thing with CSS. When image size is \
> > decided, css/markup takes precedence, then content DPR, and finally srcset-selection DPR. \
> > This is actually a bonus value of this feature - if the server happens to serve an image \
> > that differs from the ratio requested in the srcset tag, the intrinsic size match the \
> > served content and not the request (which is more of a recommendation).
> > - Does Content-DPR have any effect on images where an explicit size is provided, either in \
> > markup or in CSS? (I'm assuming no.) No, it only has effect on intrinsic size.
>
> Overall, this seems like a feature with valid use cases. But unfortunately, it's a \
> (currently) Blink-only feature with no clear specification. To the extent there is a spec, \
> it's mixed in with multiple Pull Requests. These pull requests mix in Client Hints, a feature \
> unlikely to gain multi implementor support, so they probably won't land any time soon. And \
> the language about Content-DPR is broken and does not not seem to match Chrome's actual \
> implementation.
> So in summary, there is no proper spec, and Chrome has shown willingness to have their \
> implementation change faster than even draft spec language in PRs can keep up with.
> Although the use case seems valid, I don't think it's a good idea for WebKit to implement the \
> feature in this state. It seems likely to cause interop headaches and a need to reverse \
> engineer Blink rather than developing against specs and test cases.
> There has been a lot of work done on the spec front since the comments above, and I think \
> we're ready for a re-review the state.
> * The next pull request: https://github.com/whatwg/html/pull/5112 \
> <https://github.com/whatwg/html/pull/5112> has been thumbed up to make it into the HTML spec, \
> and is separate from client-hints, but requires second implementor interest, which is what \
> this email thread is for :)
I don't think it has? I'm seeing:
"[ ] At least two implementers are interested (and none opposed):"
And satisfying this is a requirement for adding features to WHATWG Living Standards.
And the review block at the bottom says "Review required" and "Merging is Blocked with big red \
X's.
> * Web platform tests are comprehensive, separated from client-hints, and match both the spec \
> and the Chrome implementation: \
> https://github.com/web-platform-tests/wpt/tree/master/content-dpr \
> <https://github.com/web-platform-tests/wpt/tree/master/content-dpr>. We can always add more \
> if we find more use-cases.
> Would love to hear more thoughts!
Sounds like a good time to re-review.
Regards,
Maciej
[Attachment #5 (unknown)]
<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body \
style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" \
class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Mar \
2, 2020, at 11:22 PM, Noam Rosenthal <<a href="mailto:noam@webkit.org" \
class="">noam@webkit.org</a>> wrote:</div><br class="Apple-interchange-newline"><div \
class=""><br class="Apple-interchange-newline"><br style="caret-color: rgb(0, 0, 0); \
font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; \
font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; \
text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none;" class=""><div class="gmail_quote" style="caret-color: rgb(0, 0, 0); \
font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; \
font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; \
text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none;"><div dir="ltr" class="gmail_attr">On Mon, Nov 11, 2019 at 10:13 PM \
Maciej Stachowiak <<a href="mailto:mjs@apple.com" class="">mjs@apple.com</a>> wrote:<br \
class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; \
border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); \
padding-left: 1ex;"><div style="overflow-wrap: break-word;" class=""><br class=""><div \
class=""><br class=""><blockquote type="cite" class=""><div class="">On Nov 11, 2019, at 4:37 \
AM, Noam Rosenthal <<a href="mailto:noam@webkit.org" target="_blank" \
class="">noam@webkit.org</a>> wrote:</div><br class=""><div class=""><div dir="ltr" \
class=""><div dir="ltr" class=""><br class=""></div><br class=""><div class="gmail_quote"><div \
dir="ltr" class="gmail_attr">On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak <<a \
href="mailto:mjs@apple.com" target="_blank" class="">mjs@apple.com</a>> \
wrote:</div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; \
border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); \
padding-left: 1ex;"><div class=""><div class=""><div class="">- Is there a specification for \
Content-DPR? All I could find in bugzila and elsewhere is links to <<a \
href="https://httpwg.org/http-extensions/client-hints.html" target="_blank" \
class="">https://httpwg.org/http-extensions/client-hints.html</a>>, which does not mention \
Content-DPR at all).</div></div></div></blockquote><div class="">In a nutshell, the spec is in \
transition from IETF to HTML/fetch, and the client-hints aspects of it are still in progress \
(unlike conent-dpr, which is much simpler hasn't changed since introduced). It's documented \
here: <a href="https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md" \
rel="noopener noreferrer" target="_blank" style="box-sizing: inherit; font-family: Slack-Lato, \
appleLogo, sans-serif; font-size: 15px; font-variant-ligatures: common-ligatures; \
background-color: rgb(248, 248, 248);" \
class="">https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md</a>.</div><div \
class=""><div class="">I will answer more about this separately if required, waiting for some \
info from the people working on the spec \
originally.</div></div></div></div></div></blockquote><div class=""><br class=""></div>It looks \
like these are the relevant Pull Requests:</div><div class=""><br class=""></div><div \
class=""><br class=""></div><div class="">HTML:<span \
class="Apple-converted-space"> </span><a href="https://github.com/whatwg/html/pull/3774" \
target="_blank" class="">https://github.com/whatwg/html/pull/3774</a><span \
class="Apple-converted-space"> </span>|<span class="Apple-converted-space"> </span><a \
href="https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density" \
target="_blank" class="">https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density</a></div><div \
class="">Fetch:<span class="Apple-converted-space"> </span><a \
href="https://github.com/whatwg/fetch/pull/773" target="_blank" \
class="">https://github.com/whatwg/fetch/pull/773</a><span \
class="Apple-converted-space"> </span>|<span class="Apple-converted-space"> </span><a \
href="https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density" \
target="_blank" class="">https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density</a></div><div \
class=""><br class=""></div><div class="">It looks like these are in a somewhat messy state. \
For example, Fetch places the Content-DPR value in an "image density" variable, but it doesn't \
look like the Pull Request to HTML doesn't use it anywhere. As another example, HTML directly \
references Content-DPR as setting the "current pixel density", but does not specify that it \
would take priority over a density derived from srcset. There are also no diffs to CSS Images \
or to SVG, which provide distinct ways to reference images and which presumably should respect \
Content-DPR.</div></div></blockquote><div class=""><br class=""> </div><blockquote \
class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; \
border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div \
style="overflow-wrap: break-word;" class=""><div class=""><br class=""><blockquote type="cite" \
class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div class=""><div \
class=""></div></div><div class=""> </div><blockquote class="gmail_quote" style="margin: \
0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: \
rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div class=""><div class="">- Can you \
give us some examples of how the CDN would make the decision of what density of image to serve \
without knowing the client's DPR via \
client-hints? </div></div></div></blockquote><div class=""> Some examples \
from <span class="Apple-converted-space"> </span><a \
href="https://github.com/eeeps/content-dpr-explainer" target="_blank" \
class="">https://github.com/eeeps/content-dpr-explainer</a>:</div><div class="">- making \
decisions by user agent, like choosing to cap certain images for user-agents known to have \
smaller displays</div><div class="">- making decisions by traffic/geo, like serving \
lower-resolution images when traffic is bogged down</div><div class="">- A client may ask for \
"placeholder image" (e.g. ?placeholder=true in the URL), and the CDN would decide whether to \
serve a lower-quality JPEG or to scale it down, or do \
both.</div></div></div></div></blockquote><div class=""><br class=""></div>These seem like \
reasonable use cases.</div><div class=""><br class=""><blockquote type="cite" class=""><div \
class=""><div dir="ltr" class=""><div class="gmail_quote"><div class=""><br \
class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; \
border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); \
padding-left: 1ex;"><div class=""><div class=""><div class="">- Is this presuming situations \
where the CDN serves the images but not the markup or CSS (which presumably could be rewritten \
to account for DPR)?</div></div></div></blockquote><div class="">Correct.</div><div \
class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; \
border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); \
padding-left: 1ex;"><div class=""><div class=""><div class="">- What happens if Content-DPR is \
provided, but the markup or CSS has conflicting explicit info? For example, <img \
srcset="image-2x.jpg 2x, image.jpg 1x">, and image-2x.jpg is served with a Content-DPR \
header of 3x. Or the analogous thing with CSS.</div></div></div></blockquote><div class="">When \
image size is decided, css/markup takes precedence, then content DPR, and finally \
srcset-selection DPR. This is actually a bonus value of this feature - if the server happens to \
serve an image that differs from the ratio requested in the srcset tag, the intrinsic size \
match the served content and not the request (which is more of a recommendation).</div><div \
class=""><br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; \
border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); \
padding-left: 1ex;"><div class=""><div class=""><div class="">- Does Content-DPR have any \
effect on images where an explicit size is provided, either in markup or in CSS? (I'm assuming \
no.)</div></div></div></blockquote><div class="">No, it only has effect on intrinsic \
size. </div></div></div></div></blockquote><br class=""></div><div class="">Overall, this \
seems like a feature with valid use cases. But unfortunately, it's a (currently) Blink-only \
feature with no clear specification. To the extent there is a spec, it's mixed in with multiple \
Pull Requests. These pull requests mix in Client Hints, a feature unlikely to gain multi \
implementor support, so they probably won't land any time soon. And the language about \
Content-DPR is broken and does not not seem to match Chrome's actual implementation.</div><div \
class=""><br class=""></div><div class="">So in summary, there is no proper spec, and Chrome \
has shown willingness to have their implementation change faster than even draft spec language \
in PRs can keep up with. </div></div></blockquote><blockquote class="gmail_quote" \
style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; \
border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div style="overflow-wrap: \
break-word;" class=""><div class=""><br class=""></div><div class="">Although the use case \
seems valid, I don't think it's a good idea for WebKit to implement the feature in this state. \
It seems likely to cause interop headaches and a need to reverse engineer Blink rather than \
developing against specs and test cases.</div><div class=""><br \
class=""></div></div></blockquote>There has been a lot of work done on the spec front since the \
comments above, and I think we're ready for a re-review the state.<br class="">* The next pull \
request: <a href="https://github.com/whatwg/html/pull/5112" \
class="">https://github.com/whatwg/html/pull/5112</a> has been thumbed up to make it into \
the HTML spec, and is separate from client-hints, but requires second implementor interest, \
which is what this email thread is for :)<br class=""></div></div></blockquote><div><br \
class=""></div><div>I don't think it has? I'm seeing:</div><div>"[ ] At least two \
implementers are interested (and none opposed):"</div><div><br class=""></div><div>And \
satisfying this is a requirement for adding features to WHATWG Living Standards.</div><div><br \
class=""></div><div>And the review block at the bottom says "Review required" and "Merging is \
Blocked with big red X's.</div><br class=""><blockquote type="cite" class=""><div class=""><div \
class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; \
font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; \
text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: \
0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div class="">* Web platform tests \
are comprehensive, separated from client-hints, and match both the spec and the Chrome \
implementation: <a \
href="https://github.com/web-platform-tests/wpt/tree/master/content-dpr" \
class="">https://github.com/web-platform-tests/wpt/tree/master/content-dpr</a>. We can always \
add more if we find more use-cases.<br class=""><br class="">Would love to hear more \
thoughts!<br class=""></div></div></div></blockquote></div><br class=""><div class="">Sounds \
like a good time to re-review.</div><div class=""><br class=""></div><div \
class="">Regards,</div><div class="">Maciej</div><div class=""><br \
class=""></div></body></html>
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic