[prev in list] [next in list] [prev in thread] [next in thread]
List: webkit-dev
Subject: Re: [webkit-dev] Content-DPR: Image resolution optimization at CDN/transport layer
From: Maciej Stachowiak <mjs () apple ! com>
Date: 2020-03-05 2:24:26
Message-ID: 352F4099-2129-491F-8542-F3C3153B34A6 () apple ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
> On Mar 2, 2020, at 11:22 PM, Noam Rosenthal <noam@webkit.org> wrote:
>
>
>
> On Mon, Nov 11, 2019 at 10:13 PM Maciej Stachowiak <mjs@apple.com \
> <mailto:mjs@apple.com>> wrote:
>
> > On Nov 11, 2019, at 4:37 AM, Noam Rosenthal <noam@webkit.org \
> > <mailto:noam@webkit.org>> wrote:
> >
> >
> > On Mon, Nov 11, 2019 at 12:13 AM Maciej Stachowiak <mjs@apple.com \
> > <mailto:mjs@apple.com>> wrote:
> > - Is there a specification for Content-DPR? All I could find in bugzila and \
> > elsewhere is links to <https://httpwg.org/http-extensions/client-hints.html \
> > <https://httpwg.org/http-extensions/client-hints.html>>, which does not mention \
> > Content-DPR at all). In a nutshell, the spec is in transition from IETF to \
> > HTML/fetch, and the client-hints aspects of it are still in progress (unlike \
> > conent-dpr, which is much simpler hasn't changed since introduced). It's \
> > documented here: \
> > https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md \
> > <https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md>.
> > I will answer more about this separately if required, waiting for some info from \
> > the people working on the spec originally.
>
> It looks like these are the relevant Pull Requests:
>
>
> HTML: https://github.com/whatwg/html/pull/3774 \
> <https://github.com/whatwg/html/pull/3774> | \
> https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density \
> <https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density>
>
> Fetch: https://github.com/whatwg/fetch/pull/773 \
> <https://github.com/whatwg/fetch/pull/773> | \
> https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density \
> <https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density>
> It looks like these are in a somewhat messy state. For example, Fetch places the \
> Content-DPR value in an "image density" variable, but it doesn't look like the Pull \
> Request to HTML doesn't use it anywhere. As another example, HTML directly \
> references Content-DPR as setting the "current pixel density", but does not specify \
> that it would take priority over a density derived from srcset. There are also no \
> diffs to CSS Images or to SVG, which provide distinct ways to reference images and \
> which presumably should respect Content-DPR.
>
>
> >
> > - Can you give us some examples of how the CDN would make the decision of what \
> > density of image to serve without knowing the client's DPR via client-hints? \
> > Some examples from https://github.com/eeeps/content-dpr-explainer \
> > <https://github.com/eeeps/content-dpr-explainer>:
> > - making decisions by user agent, like choosing to cap certain images for \
> > user-agents known to have smaller displays
> > - making decisions by traffic/geo, like serving lower-resolution images when \
> > traffic is bogged down
> > - A client may ask for "placeholder image" (e.g. ?placeholder=true in the URL), \
> > and the CDN would decide whether to serve a lower-quality JPEG or to scale it \
> > down, or do both.
>
> These seem like reasonable use cases.
>
> >
> > - Is this presuming situations where the CDN serves the images but not the markup \
> > or CSS (which presumably could be rewritten to account for DPR)? Correct.
> >
> > - What happens if Content-DPR is provided, but the markup or CSS has conflicting \
> > explicit info? For example, <img srcset="image-2x.jpg 2x, image.jpg 1x">, and \
> > image-2x.jpg is served with a Content-DPR header of 3x. Or the analogous thing \
> > with CSS. When image size is decided, css/markup takes precedence, then content \
> > DPR, and finally srcset-selection DPR. This is actually a bonus value of this \
> > feature - if the server happens to serve an image that differs from the ratio \
> > requested in the srcset tag, the intrinsic size match the served content and not \
> > the request (which is more of a recommendation).
> > - Does Content-DPR have any effect on images where an explicit size is provided, \
> > either in markup or in CSS? (I'm assuming no.) No, it only has effect on \
> > intrinsic size.
>
> Overall, this seems like a feature with valid use cases. But unfortunately, it's a \
> (currently) Blink-only feature with no clear specification. To the extent there is \
> a spec, it's mixed in with multiple Pull Requests. These pull requests mix in \
> Client Hints, a feature unlikely to gain multi implementor support, so they \
> probably won't land any time soon. And the language about Content-DPR is broken and \
> does not not seem to match Chrome's actual implementation.
> So in summary, there is no proper spec, and Chrome has shown willingness to have \
> their implementation change faster than even draft spec language in PRs can keep up \
> with.
> Although the use case seems valid, I don't think it's a good idea for WebKit to \
> implement the feature in this state. It seems likely to cause interop headaches and \
> a need to reverse engineer Blink rather than developing against specs and test \
> cases.
> There has been a lot of work done on the spec front since the comments above, and I \
> think we're ready for a re-review the state.
> * The next pull request: https://github.com/whatwg/html/pull/5112 \
> <https://github.com/whatwg/html/pull/5112> has been thumbed up to make it into the \
> HTML spec, and is separate from client-hints, but requires second implementor \
> interest, which is what this email thread is for :)
I don't think it has? I'm seeing:
"[ ] At least two implementers are interested (and none opposed):"
And satisfying this is a requirement for adding features to WHATWG Living Standards.
And the review block at the bottom says "Review required" and "Merging is Blocked \
with big red X's.
> * Web platform tests are comprehensive, separated from client-hints, and match both \
> the spec and the Chrome implementation: \
> https://github.com/web-platform-tests/wpt/tree/master/content-dpr \
> <https://github.com/web-platform-tests/wpt/tree/master/content-dpr>. We can always \
> add more if we find more use-cases.
> Would love to hear more thoughts!
Sounds like a good time to re-review.
Regards,
Maciej
[Attachment #5 (unknown)]
<html><head><meta http-equiv="Content-Type" content="text/html; \
charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; \
line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote \
type="cite" class=""><div class="">On Mar 2, 2020, at 11:22 PM, Noam Rosenthal <<a \
href="mailto:noam@webkit.org" class="">noam@webkit.org</a>> wrote:</div><br \
class="Apple-interchange-newline"><div class=""><br \
class="Apple-interchange-newline"><br style="caret-color: rgb(0, 0, 0); font-family: \
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; \
font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; \
text-transform: none; white-space: normal; word-spacing: 0px; \
-webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div \
class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; \
font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none;"><div dir="ltr" class="gmail_attr">On Mon, Nov 11, 2019 at \
10:13 PM Maciej Stachowiak <<a href="mailto:mjs@apple.com" \
class="">mjs@apple.com</a>> wrote:<br class=""></div><blockquote \
class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; \
border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: \
1ex;"><div style="overflow-wrap: break-word;" class=""><br class=""><div class=""><br \
class=""><blockquote type="cite" class=""><div class="">On Nov 11, 2019, at 4:37 AM, \
Noam Rosenthal <<a href="mailto:noam@webkit.org" target="_blank" \
class="">noam@webkit.org</a>> wrote:</div><br class=""><div class=""><div \
dir="ltr" class=""><div dir="ltr" class=""><br class=""></div><br class=""><div \
class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 11, 2019 at 12:13 \
AM Maciej Stachowiak <<a href="mailto:mjs@apple.com" target="_blank" \
class="">mjs@apple.com</a>> wrote:</div><blockquote class="gmail_quote" \
style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; \
border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div \
class=""><div class="">- Is there a specification for Content-DPR? All I could find \
in bugzila and elsewhere is links to <<a \
href="https://httpwg.org/http-extensions/client-hints.html" target="_blank" \
class="">https://httpwg.org/http-extensions/client-hints.html</a>>, which does not \
mention Content-DPR at all).</div></div></div></blockquote><div class="">In a \
nutshell, the spec is in transition from IETF to HTML/fetch, and the client-hints \
aspects of it are still in progress (unlike conent-dpr, which is much simpler hasn't \
changed since introduced). It's documented here: <a \
href="https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md" \
rel="noopener noreferrer" target="_blank" style="box-sizing: inherit; font-family: \
Slack-Lato, appleLogo, sans-serif; font-size: 15px; font-variant-ligatures: \
common-ligatures; background-color: rgb(248, 248, 248);" \
class="">https://github.com/yoavweiss/client-hints-infrastructure/blob/master/specification_situation.md</a>.</div><div \
class=""><div class="">I will answer more about this separately if required, waiting \
for some info from the people working on the spec \
originally.</div></div></div></div></div></blockquote><div class=""><br \
class=""></div>It looks like these are the relevant Pull Requests:</div><div \
class=""><br class=""></div><div class=""><br class=""></div><div class="">HTML:<span \
class="Apple-converted-space"> </span><a \
href="https://github.com/whatwg/html/pull/3774" target="_blank" \
class="">https://github.com/whatwg/html/pull/3774</a><span \
class="Apple-converted-space"> </span>|<span \
class="Apple-converted-space"> </span><a \
href="https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density" \
target="_blank" class="">https://whatpr.org/fetch/773/6a644c6...5067615.html#concept-response-image-density</a></div><div \
class="">Fetch:<span class="Apple-converted-space"> </span><a \
href="https://github.com/whatwg/fetch/pull/773" target="_blank" \
class="">https://github.com/whatwg/fetch/pull/773</a><span \
class="Apple-converted-space"> </span>|<span \
class="Apple-converted-space"> </span><a \
href="https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density" \
target="_blank" class="">https://whatpr.org/html/3774/e32a6f8...ddb0544/images.html#current-pixel-density</a></div><div \
class=""><br class=""></div><div class="">It looks like these are in a somewhat messy \
state. For example, Fetch places the Content-DPR value in an "image density" \
variable, but it doesn't look like the Pull Request to HTML doesn't use it anywhere. \
As another example, HTML directly references Content-DPR as setting the "current \
pixel density", but does not specify that it would take priority over a density \
derived from srcset. There are also no diffs to CSS Images or to SVG, which provide \
distinct ways to reference images and which presumably should respect \
Content-DPR.</div></div></blockquote><div class=""><br \
class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px \
0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, \
204, 204); padding-left: 1ex;"><div style="overflow-wrap: break-word;" class=""><div \
class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" \
class=""><div class="gmail_quote"><div class=""><div class=""></div></div><div \
class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px \
0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, \
204, 204); padding-left: 1ex;"><div class=""><div class=""><div class="">- Can you \
give us some examples of how the CDN would make the decision of what density of image \
to serve without knowing the client's DPR via \
client-hints? </div></div></div></blockquote><div class=""> Some \
examples from <span class="Apple-converted-space"> </span><a \
href="https://github.com/eeeps/content-dpr-explainer" target="_blank" \
class="">https://github.com/eeeps/content-dpr-explainer</a>:</div><div class="">- \
making decisions by user agent, like choosing to cap certain images for user-agents \
known to have smaller displays</div><div class="">- making decisions by traffic/geo, \
like serving lower-resolution images when traffic is bogged down</div><div class="">- \
A client may ask for "placeholder image" (e.g. ?placeholder=true in the URL), and the \
CDN would decide whether to serve a lower-quality JPEG or to scale it down, or do \
both.</div></div></div></div></blockquote><div class=""><br class=""></div>These seem \
like reasonable use cases.</div><div class=""><br class=""><blockquote type="cite" \
class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div \
class=""><br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px \
0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: \
rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div class=""><div class="">- \
Is this presuming situations where the CDN serves the images but not the markup or \
CSS (which presumably could be rewritten to account for \
DPR)?</div></div></div></blockquote><div class="">Correct.</div><div \
class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px \
0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, \
204, 204); padding-left: 1ex;"><div class=""><div class=""><div class="">- What \
happens if Content-DPR is provided, but the markup or CSS has conflicting explicit \
info? For example, <img srcset="image-2x.jpg 2x, image.jpg 1x">, and \
image-2x.jpg is served with a Content-DPR header of 3x. Or the analogous thing with \
CSS.</div></div></div></blockquote><div class="">When image size is decided, \
css/markup takes precedence, then content DPR, and finally srcset-selection DPR. This \
is actually a bonus value of this feature - if the server happens to serve an image \
that differs from the ratio requested in the srcset tag, the intrinsic size match the \
served content and not the request (which is more of a recommendation).</div><div \
class=""><br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px \
0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: \
rgb(204, 204, 204); padding-left: 1ex;"><div class=""><div class=""><div class="">- \
Does Content-DPR have any effect on images where an explicit size is provided, either \
in markup or in CSS? (I'm assuming no.)</div></div></div></blockquote><div \
class="">No, it only has effect on intrinsic \
size. </div></div></div></div></blockquote><br class=""></div><div \
class="">Overall, this seems like a feature with valid use cases. But unfortunately, \
it's a (currently) Blink-only feature with no clear specification. To the extent \
there is a spec, it's mixed in with multiple Pull Requests. These pull requests mix \
in Client Hints, a feature unlikely to gain multi implementor support, so they \
probably won't land any time soon. And the language about Content-DPR is broken and \
does not not seem to match Chrome's actual implementation.</div><div class=""><br \
class=""></div><div class="">So in summary, there is no proper spec, and Chrome has \
shown willingness to have their implementation change faster than even draft spec \
language in PRs can keep up with. </div></div></blockquote><blockquote \
class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; \
border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: \
1ex;"><div style="overflow-wrap: break-word;" class=""><div class=""><br \
class=""></div><div class="">Although the use case seems valid, I don't think it's a \
good idea for WebKit to implement the feature in this state. It seems likely to cause \
interop headaches and a need to reverse engineer Blink rather than developing against \
specs and test cases.</div><div class=""><br class=""></div></div></blockquote>There \
has been a lot of work done on the spec front since the comments above, and I think \
we're ready for a re-review the state.<br class="">* The next pull request: <a \
href="https://github.com/whatwg/html/pull/5112" \
class="">https://github.com/whatwg/html/pull/5112</a> has been thumbed up to \
make it into the HTML spec, and is separate from client-hints, but requires second \
implementor interest, which is what this email thread is for :)<br \
class=""></div></div></blockquote><div><br class=""></div><div>I don't think it has? \
I'm seeing:</div><div>"[ ] At least two implementers are interested (and none \
opposed):"</div><div><br class=""></div><div>And satisfying this is a requirement for \
adding features to WHATWG Living Standards.</div><div><br class=""></div><div>And the \
review block at the bottom says "Review required" and "Merging is Blocked with big \
red X's.</div><br class=""><blockquote type="cite" class=""><div class=""><div \
class="gmail_quote" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; \
font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; \
letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; \
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; \
text-decoration: none;"><div class="">* Web platform tests are comprehensive, \
separated from client-hints, and match both the spec and the Chrome \
implementation: <a \
href="https://github.com/web-platform-tests/wpt/tree/master/content-dpr" \
class="">https://github.com/web-platform-tests/wpt/tree/master/content-dpr</a>. We \
can always add more if we find more use-cases.<br class=""><br class="">Would love to \
hear more thoughts!<br class=""></div></div></div></blockquote></div><br \
class=""><div class="">Sounds like a good time to re-review.</div><div class=""><br \
class=""></div><div class="">Regards,</div><div class="">Maciej</div><div \
class=""><br class=""></div></body></html>
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic