[prev in list] [next in list] [prev in thread] [next in thread]
List: webkit-dev
Subject: Re: [webkit-dev] Supporting <link rel=> for finding ref tests
From: Robert Ma <robertma () chromium ! org>
Date: 2019-11-12 19:21:53
Message-ID: CAOPAaNKYJHJSPzvCN+-4Hc4666bq0v2vWmR-j4OwEhN5YdTb6w () mail ! gmail ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
MANIFEST.json mostly stays out of the way / happens under the hood. It is
cached as GitHub releases, and ignored by .gitignore. One of its purposes
is a cache of the reftest graph.
Once you run any test or explicitly run `./wpt manifest`, the manifest will
be generated/updated in the root of your WPT checkout.
On Tue, Nov 12, 2019 at 2:06 PM Simon Fraser <simon.fraser@apple.com> wrote=
:
> > On Nov 12, 2019, at 4:52 AM, Fr=C3=A9d=C3=A9ric Wang <fwang@igalia.com>=
wrote:
> >
> > On 09/11/2019 04:02, Ryosuke Niwa wrote:
> >>
> >>>>> - Requires us modifying each port's DRT to support this format
> >>>>>
> >>>>> No, it just requires webkitpy hacking which I've done in the patch.
> >>>> I'm not certain writing a bunch of regular expressions in webkitpy i=
s
> >>>> a reliable mechanism to find expected results. Another issue I found
> >>>> back then was that it significantly slowed run-webkit-tests' startup
> >>>> time because WPT has a workflow to find all tests & their expected
> >>>> results upfront before any tests could run.
> >>> The patch uses html5lib (via BeautifulSoup), which is exactly what
> WPT, and our importer use to find the ref tests.
> >>>
> >>> We don't find references up-front; only when running each test. This
> patch does add some overhead for parsing each test file,
> >>> which I measured to be about 1-2 sec on a directory which took 30s to
> run. I think this slight slowdown is worthwhile (we could
> >>> probably eliminate it with some webkitpy optimizations).
> >> Hm... that's ~3% overhead.
> >
> > @Simon: I agree with Ryosuke that 3% sounds big. IIUC you are parsing
> > the HTML file when running each test? I thought that there is a
> > MANIFEST.json file which is supposed to cache that information, why
> > can't we use it?
>
> I don't see any files call MANIFEST.json in the wpt repo.
>
> There are reftest.list files but these are obsolete.
>
> I hope to get that 3% back by other webkitpy perf optimizations (python
> optimization hints welcome).
>
> Simon
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
>
[Attachment #5 (text/html)]
<div dir="ltr">MANIFEST.json mostly stays out of the way / happens under the hood. \
It is cached as GitHub releases, and ignored by .gitignore. One of its purposes is a \
cache of the reftest graph.<div><br></div><div>Once you run any test or explicitly \
run `./wpt manifest`, the manifest will be generated/updated in the root of your WPT \
checkout.</div><div></div></div><br><div class="gmail_quote"><div dir="ltr" \
class="gmail_attr">On Tue, Nov 12, 2019 at 2:06 PM Simon Fraser <<a \
href="mailto:simon.fraser@apple.com">simon.fraser@apple.com</a>> \
wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px \
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">> On Nov 12, 2019, \
at 4:52 AM, Frédéric Wang <<a href="mailto:fwang@igalia.com" \
target="_blank">fwang@igalia.com</a>> wrote:<br> > <br>
> On 09/11/2019 04:02, Ryosuke Niwa wrote:<br>
>> <br>
>>>>> - Requires us modifying each port's DRT to support this \
format<br> >>>>> <br>
>>>>> No, it just requires webkitpy hacking which I've done in the \
patch.<br> >>>> I'm not certain writing a bunch of regular \
expressions in webkitpy is<br> >>>> a reliable mechanism to find expected \
results. Another issue I found<br> >>>> back then was that it \
significantly slowed run-webkit-tests' startup<br> >>>> time because \
WPT has a workflow to find all tests & their expected<br> >>>> \
results upfront before any tests could run.<br> >>> The patch uses html5lib \
(via BeautifulSoup), which is exactly what WPT, and our importer use to find the ref \
tests.<br> >>> <br>
>>> We don't find references up-front; only when running each test. This \
patch does add some overhead for parsing each test file,<br> >>> which I \
measured to be about 1-2 sec on a directory which took 30s to run. I think this \
slight slowdown is worthwhile (we could<br> >>> probably eliminate it with \
some webkitpy optimizations).<br> >> Hm... that's ~3% overhead.<br>
> <br>
> @Simon: I agree with Ryosuke that 3% sounds big. IIUC you are parsing<br>
> the HTML file when running each test? I thought that there is a<br>
> MANIFEST.json file which is supposed to cache that information, why<br>
> can't we use it?<br>
<br>
I don't see any files call MANIFEST.json in the wpt repo.<br>
<br>
There are reftest.list files but these are obsolete.<br>
<br>
I hope to get that 3% back by other webkitpy perf optimizations (python optimization \
hints welcome).<br> <br>
Simon<br>
<br>
_______________________________________________<br>
webkit-dev mailing list<br>
<a href="mailto:webkit-dev@lists.webkit.org" \
target="_blank">webkit-dev@lists.webkit.org</a><br> <a \
href="https://lists.webkit.org/mailman/listinfo/webkit-dev" rel="noreferrer" \
target="_blank">https://lists.webkit.org/mailman/listinfo/webkit-dev</a><br> \
</blockquote></div>
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic