[prev in list] [next in list] [prev in thread] [next in thread]
List: webkit-dev
Subject: Re: [webkit-dev] Buildbot upgrade on build.webkit.org
From: Maciej Stachowiak <mjs () apple ! com>
Date: 2018-01-08 23:02:26
Message-ID: 47ADA6A1-0BA7-4B08-AE2A-DAFB46CC4F0C () apple ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
> On Jan 8, 2018, at 1:33 PM, Aakash Jain <aakash_jain@apple.com> wrote:
>
> Hi Everyone,
>
> I filed the Buildbot bug regarding Waterfall view not displaying steps information \
> by default: https://github.com/buildbot/buildbot/issues/3884 \
> <https://github.com/buildbot/buildbot/issues/3884> "Waterfall view should display \
> steps information without hovering over".
> I got the response that the issue with displaying all the steps information by \
> default in waterfall view is that it would require a lot of API calls, and so it \
> wouldn't be scalable.
> Can you guys look at the new console and grid view to see if they can serve as an \
> alternative? e.g.:
> https://nine.buildbot.net/#/console <https://nine.buildbot.net/#/console>
> https://nine.buildbot.net/#/grid <https://nine.buildbot.net/#/grid>
I wish the bubbles in those showed something more useful than the build ID, like \
revision number and/or number of failures. The build number is not really useful for \
seeing whether a given patch has passed tests, since the person who committed won't \
know the build number to look for and it won't even be the same for all bots. Also it \
doesn't show anything at all useful on hover, you have to click through.
It seems like this style of grid and console view would make it very hard to answer \
questions like "has my patch built and passed tests successfully on all bots yet", \
which is my main interest at least out of a build not view.
Console view is close to being good for that, but:
- Doesn't include the revision number on the summary line, you have to click through.
- Gives PR number instead a one-liner summary of the patch without expanding.
- Has the mostly-useless build numbers in bubbles.
- Can't tell easily when a build or test step is still pending (some boxes have more \
than one bubble in them, but I don't seee
I'm not totally sure of the use cases for waterfall so I don't know if it would be \
good for that.
>
>
> Here is the complete response:
>
> "I agree the new waterfall is not very useful.
> It is not really a problem of UI change, it is rather a problem of the number of \
> rest API needed to get the information needed. In order to display the old \
> waterfall, you need to get the whole details for every builds (steps, logs), that \
> would be dozens of REST api calls, which would take a lot of time.
> The old waterfall was not very scalable too, on my old buildbot eight at work, we \
> disabled it, because it was stalling the the master for 10s every time one user \
> would look at it.
> The question you have to ask your users is what is the information that is really \
> needed. Maybe we can load the steps only for the failed builds?
>
> We find the console_view and grid_view much more useful? They don't have details of \
> which step failed, but the details of which builder is failing sorted by change, \
> and usually people have been doing one builder per "kind of test"
> You can see the example here, and the categorisation of builders, which I think \
> could be quite useful as a replacement for https://build.webkit.org/waterfall \
> <https://build.webkit.org/waterfall>
> https://nine.buildbot.net/#/console <https://nine.buildbot.net/#/console>
>
> What user might request is probably the reason for failure, which can be done \
> without having to fetch more information (by using the build results_string) In the \
> current version you can still click on a build, and have all the information for \
> that build, including tail log of the failing steps (without changing web page)."
>
> -Aakash
>
>
> > On Dec 13, 2017, at 6:51 PM, Aakash Jain <aakash_jain@apple.com \
> > <mailto:aakash_jain@apple.com>> wrote:
> >
> >
> > > On Dec 11, 2017, at 1:10 AM, Konstantin Tokarev <annulen@yandex.ru \
> > > <mailto:annulen@yandex.ru>> wrote:
> > >
> > >
> > > 11.12.2017, 02:49, "Carlos Alberto Lopez Perez" <clopez@igalia.com \
> > > <mailto:clopez@igalia.com>>:
> > > > On 07/12/17 21:47, Aakash Jain wrote:
> > > > > For people using build.webkit.org <http://build.webkit.org/> \
> > > > > <http://build.webkit.org/ <http://build.webkit.org/>>, I would like to know \
> > > > > what pages you use most of the time (e.g.: builder page, console view etc.) \
> > > > > and what are your primary use-cases (purpose to visit build.webkit.org \
> > > > > <http://build.webkit.org/> <http://build.webkit.org/ \
> > > > > <http://build.webkit.org/>>). Also if you have any feedback/concern about \
> > > > > new Buildbot please let me know.
> > > >
> > > > I use 99% of the time the waterfall view, and I think the new waterfall
> > > > view of buildbot 9 is a step back in terms of usefulness.
> > > >
> > > > More details on about why I think this here:
> > > > https://bugs.webkit.org/show_bug.cgi?id=175056#c1 \
> > > > <https://bugs.webkit.org/show_bug.cgi?id=175056#c1>
> > >
> > > I also use waterfall 99% of time, and new waterfall totally sucks.
> > >
> > > I wonder if it's possible to get rid of stupid sidebar, if not it's a complete
> > > disaster.
> >
> > The sidebar can be collapsed by clicking on the pin button on top (<pin.png>).
> >
> > Are there any other suggestions for changes we could try in new buildbot \
> > waterfall view to make it suitable?
> > Since we are on a very old version of buildbot, at some point we would have to \
> > upgrade. It would be nice if we could identify changes which can be made either \
> > to buildbot or to our workflow to better suit our requirements. We can also \
> > consider passing on the feedback directly to Buildbot by filing bugs at \
> > https://github.com/buildbot/buildbot/issues \
> > <https://github.com/buildbot/buildbot/issues>, so that Buildbot can fix them.
> > Thanks
> > Aakash
> >
> > >
> > > >
> > > > ,
> > > >
> > > > _______________________________________________
> > > > webkit-dev mailing list
> > > > webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
> > > > https://lists.webkit.org/mailman/listinfo/webkit-dev \
> > > > <https://lists.webkit.org/mailman/listinfo/webkit-dev>
> > >
> > > --
> > > Regards,
> > > Konstantin
> > > _______________________________________________
> > > webkit-dev mailing list
> > > webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
> > > https://lists.webkit.org/mailman/listinfo/webkit-dev \
> > > <https://lists.webkit.org/mailman/listinfo/webkit-dev>
> >
> > _______________________________________________
> > webkit-dev mailing list
> > webkit-dev@lists.webkit.org <mailto:webkit-dev@lists.webkit.org>
> > https://lists.webkit.org/mailman/listinfo/webkit-dev
>
> _______________________________________________
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
[Attachment #5 (text/html)]
<html><head><meta http-equiv="Content-Type" content="text/html; \
charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: \
space; line-break: after-white-space;" class=""><br class=""><div><br \
class=""><blockquote type="cite" class=""><div class="">On Jan 8, 2018, at 1:33 PM, \
Aakash Jain <<a href="mailto:aakash_jain@apple.com" \
class="">aakash_jain@apple.com</a>> wrote:</div><br \
class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" \
content="text/html; charset=us-ascii" class=""><div style="word-wrap: break-word; \
-webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Everyone,<div \
class=""><br class=""></div><div class="">I filed the Buildbot bug regarding \
Waterfall view not displaying steps information by default: <a \
href="https://github.com/buildbot/buildbot/issues/3884" \
class="">https://github.com/buildbot/buildbot/issues/3884</a> "Waterfall view should \
display steps information without hovering over".<div class=""><br \
class=""></div><div class="">I got the response that the issue with displaying all \
the steps information by default in waterfall view is that it would require a lot of \
API calls, and so it wouldn't be scalable.</div><div class=""><br class=""></div><div \
class="">Can you guys look at the new console and grid view to see if they can serve \
as an alternative?</div><div class="">e.g.:</div><div class=""><a \
href="https://nine.buildbot.net/#/console" \
class="">https://nine.buildbot.net/#/console</a></div><div class=""><a \
href="https://nine.buildbot.net/#/grid" \
class="">https://nine.buildbot.net/#/grid</a></div></div></div></div></blockquote><div><br \
class=""></div><div>I wish the bubbles in those showed something more useful than the \
build ID, like revision number and/or number of failures. The build number is not \
really useful for seeing whether a given patch has passed tests, since the person who \
committed won't know the build number to look for and it won't even be the same for \
all bots. Also it doesn't show anything at all useful on hover, you have to click \
through.</div><div><br class=""></div><div>It seems like this style of grid and \
console view would make it very hard to answer questions like "has my patch built and \
passed tests successfully on all bots yet", which is my main interest at least out of \
a build not view.</div><div><br class=""></div><div>Console view is close to being \
good for that, but:</div><div>- Doesn't include the revision number on the summary \
line, you have to click through.</div><div>- Gives PR number instead a one-liner \
summary of the patch without expanding.</div><div>- Has the mostly-useless build \
numbers in bubbles.</div><div>- Can't tell easily when a build or test step is still \
pending (some boxes have more than one bubble in them, but I don't seee</div><div><br \
class=""></div><div>I'm not totally sure of the use cases for waterfall so I don't \
know if it would be good for that.</div><div><br class=""></div><blockquote \
type="cite" class=""><div class=""><div style="word-wrap: break-word; \
-webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><div \
class=""><br class=""></div><div class=""><br class=""></div><div class="">Here is \
the complete response:</div><div class=""><br class=""></div><div class="">"I agree \
the new waterfall is not very useful.</div><div class="">It is not really a problem \
of UI change, it is rather a problem of the number of rest API needed to get the \
information needed. In order to display the old waterfall, you need to get the whole \
details for every builds (steps, logs), that would be dozens of REST api calls, which \
would take a lot of time.</div><div class=""><br class=""></div><div class="">The old \
waterfall was not very scalable too, on my old buildbot eight at work, we disabled \
it, because it was stalling the the master for 10s every time one user would look at \
it.</div><div class=""><br class=""></div><div class="">The question you have to ask \
your users is what is the information that is really needed.</div><div class="">Maybe \
we can load the steps only for the failed builds?</div><div class=""><br \
class=""></div><div class="">We find the console_view and grid_view much more useful? \
They don't have details of which step failed, but the details of which builder is \
failing sorted by change, and usually people have been doing one builder per "kind of \
test"</div><div class=""><br class=""></div><div class="">You can see the example \
here, and the categorisation of builders, which I think could be quite useful as a \
replacement for <a href="https://build.webkit.org/waterfall" \
class="">https://build.webkit.org/waterfall</a></div><div class=""><br \
class=""></div><div class=""><a href="https://nine.buildbot.net/#/console" \
class="">https://nine.buildbot.net/#/console</a></div><div class=""><br \
class=""></div><div class="">What user might request is probably the reason for \
failure, which can be done without having to fetch more information (by using the \
build results_string)</div><div class="">In the current version you can still click \
on a build, and have all the information for that build, including tail log of the \
failing steps (without changing web page).<span style="color: rgb(36, 41, 46); \
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, \
Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", \
"Segoe UI Symbol"; font-size: 14px;" class="">"</span></div><div \
class=""><br class=""></div><div class=""><br class=""><div \
class="">-Aakash</div><div class=""><br class=""></div><div class=""><br \
class=""><blockquote type="cite" class=""><div class="">On Dec 13, 2017, at 6:51 PM, \
Aakash Jain <<a href="mailto:aakash_jain@apple.com" \
class="">aakash_jain@apple.com</a>> wrote:</div><br \
class="Apple-interchange-newline"><div class=""><div style="word-wrap: break-word; \
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br \
class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On \
Dec 11, 2017, at 1:10 AM, Konstantin Tokarev <<a href="mailto:annulen@yandex.ru" \
class="">annulen@yandex.ru</a>> wrote:</div><br \
class="Apple-interchange-newline"><div class=""><div class=""><br class=""><br \
class="">11.12.2017, 02:49, "Carlos Alberto Lopez Perez" <<a \
href="mailto:clopez@igalia.com" class="">clopez@igalia.com</a>>:<br \
class=""><blockquote type="cite" class="">On 07/12/17 21:47, Aakash Jain wrote:<br \
class=""><blockquote type="cite" class=""> For people using <a \
href="http://build.webkit.org/" class="">build.webkit.org</a> <<a \
href="http://build.webkit.org/" class="">http://build.webkit.org/</a>>, I would<br \
class=""> like to know what pages you use most of the time (e.g.: builder \
page,<br class=""> console view etc.) and what are your primary use-cases \
(purpose to<br class=""> visit <a href="http://build.webkit.org/" \
class="">build.webkit.org</a> <<a href="http://build.webkit.org/" \
class="">http://build.webkit.org/</a>>). Also if you have<br class=""> any \
feedback/concern about new Buildbot please let me know.<br class=""></blockquote><br \
class="">I use 99% of the time the waterfall view, and I think the new waterfall<br \
class="">view of buildbot 9 is a step back in terms of usefulness.<br class=""><br \
class="">More details on about why I think this here:<br class=""><a \
href="https://bugs.webkit.org/show_bug.cgi?id=175056#c1" \
class="">https://bugs.webkit.org/show_bug.cgi?id=175056#c1</a><br \
class=""></blockquote><br class="">I also use waterfall 99% of time, and new \
waterfall totally sucks.<br class=""><br class="">I wonder if it's possible to get \
rid of stupid sidebar, if not it's a complete<br class="">disaster.<br \
class=""></div></div></blockquote><div class=""><br class=""></div><div class="">The \
sidebar can be collapsed by clicking on the pin button on top (<span \
id="cid:D3015B70-FF4F-454A-8A36-4741A681A623@apple.com" \
class=""><pin.png></span>).</div><div class=""><br class=""></div><div \
class="">Are there any other suggestions for changes we could try in new buildbot \
waterfall view to make it suitable?</div><div class=""><br class=""></div><div \
class="">Since we are on a very old version of buildbot, at some point we would have \
to upgrade. It would be nice if we could identify changes which can be made either to \
buildbot or to our workflow to better suit our requirements. We can also consider \
passing on the feedback directly to Buildbot by filing bugs at <a \
href="https://github.com/buildbot/buildbot/issues" \
class="">https://github.com/buildbot/buildbot/issues</a>, so that Buildbot can fix \
them.</div><div class=""><br class=""></div><div class="">Thanks</div><div \
class="">Aakash</div><br class=""><blockquote type="cite" class=""><div class=""><div \
class=""><br class=""><blockquote type="cite" class=""><br class="">,<br class=""><br \
class="">_______________________________________________<br class="">webkit-dev \
mailing list<br class=""><a href="mailto:webkit-dev@lists.webkit.org" \
class="">webkit-dev@lists.webkit.org</a><br class=""><a \
href="https://lists.webkit.org/mailman/listinfo/webkit-dev" \
class="">https://lists.webkit.org/mailman/listinfo/webkit-dev</a><br \
class=""></blockquote><br class="">-- <br class="">Regards,<br class="">Konstantin<br \
class="">_______________________________________________<br class="">webkit-dev \
mailing list<br class=""><a href="mailto:webkit-dev@lists.webkit.org" \
class="">webkit-dev@lists.webkit.org</a><br class=""><a \
href="https://lists.webkit.org/mailman/listinfo/webkit-dev" \
class="">https://lists.webkit.org/mailman/listinfo/webkit-dev</a><br \
class=""></div></div></blockquote></div><br \
class=""></div>_______________________________________________<br class="">webkit-dev \
mailing list<br class=""><a href="mailto:webkit-dev@lists.webkit.org" \
class="">webkit-dev@lists.webkit.org</a><br class=""><a \
href="https://lists.webkit.org/mailman/listinfo/webkit-dev" \
class="">https://lists.webkit.org/mailman/listinfo/webkit-dev</a><br \
class=""></div></blockquote></div><br \
class=""></div></div></div>_______________________________________________<br \
class="">webkit-dev mailing list<br class=""><a \
href="mailto:webkit-dev@lists.webkit.org" class="">webkit-dev@lists.webkit.org</a><br \
class="">https://lists.webkit.org/mailman/listinfo/webkit-dev<br \
class=""></div></blockquote></div><br class=""></body></html>
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic