[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-panel-devel
Subject:    Re: Thoughts about a better Quality Management process for Plasma
From:       David Edmundson <david () davidedmundson ! co ! uk>
Date:       2013-01-17 11:22:04
Message-ID: CAGeFrHDinaEDu0eACPw41VH13XFi4TZ3QbBZSnhV1Ur7_+Mt-w () mail ! gmail ! com
[Download RAW message or body]

On Sun, Jan 13, 2013 at 2:50 PM, Martin Gr=E4=DFlin <mgraesslin@kde.org> wr=
ote:
> Hi all,
>
> *warning* long mail! Please take time reading it. Given how long I am now
> writing on it: schedule half an hour or so ;-)
>
> I will try to formulate some thoughts on what I think might help to impro=
ve
> the Quality Management in Plasma. Some ideas are based on what works well=
 in
> KWin.
>
> First of all I want to say that I find it awesome how many mentioned Qual=
ity
> in Aaron's 4.10 reflection thread. It means that we identified an area for
> improvement. I think that's the most important part to actually get somet=
hing
> changed. And considering where we have been just a few years ago, I find =
it
> even more awesome that we consider Quality as the issue of the release cy=
cle.
> We are complaining here on a very high scale. Yes we had regressions, but=
 we
> also fixed them in time for the release.
>
> I think the problem with our QM process is, that we don't have a tool to
> support it. Our bugtracker is (in it's current state) totally useless. Le=
t me
> just show a few stats for Plasma:
> * 858 bugs (including wishlist) created since 4.9.0 tagging
> ** 343 are marked as resolved
> *** 43 (!) are fixed - that are 5 % of all reported bugs
> *** 29 are invalid
> *** 222 are duplicates - a quote of 26 %
> ** 384 are still unconfirmed (45 %)
> *** 96 of those are crash reports
> ** 130 of the not resolved bugs are still in component general (25 % of t=
he
> not resolved bugs)
>
> now for comparison I show the stats of KWin:
> * 432 bugs (including wishlist) created since 4.9.0 tagging
> ** 329  are marked as resolved
> *** 87 are fixed - that are 20 % of all reported bugs
> *** 28 are invalid
> *** 149 are duplicates - a quote of 34 %
> ** 42 are still unconfirmed (9.7 %)
> *** 4 of those are crash reports
> ** 17 of the not resolved (or needsinfo) bugs are still in component gene=
ral
>
> Two people work on the KWin bugs. Plasma gets twice the amount of bug rep=
orts
> than KWin, so it would need 4 people to keep the bug tracker in a managab=
le
> state. Looking at the stats it would need 4 people triaging one bug per d=
ay to
> keep it clean. That is doable!
>
> I have two requirements for bug reports:
> 1. There should not be any unconfirmed bugs
> 2. There should not be any bugs in component general
>
> What is an unconfirmed bug? It's a bug which has not been reproduced/veri=
fied
> that it is valid! A bug report should not stay in this status for a long =
time.
> Our users spend time to report it, so we should process them. If the user
> provided the correct input, it should be possible to get the bug to status
> "CONFIRMED". If the user cannot provide the necessary data to get the bug=
 into
> status CONFIRMED, it has to be set to RESOLVED WAITINGFORINFO. A bug shou=
ld
> never be longer than one week in status UNCONFIRMED without any action. I=
f the
> user provided input which is not sufficient: ask for more. If he cannot
> provide it -> WAITINGFORINFO. Yes it sounds harsh, but you want to keep t=
he
> tracker clean.
>
> What is a bug in component general? A bug that applies to everything of
> Plasma? Impossible! There is no general. A bug in general means that it is
> unknown where it belongs to. It should be moved into the best fitting
> component. If it doesn't exist create a new component! But don't keep it =
in
> general - nobody is responsible for those bugs.
>
This morning I spent an hour going through the list of "general", I
found a few problems.

There were several bugs I looked at where the relevant component
didn't exist (webslice plasmoid, timer plasmoid).
All plasmoids should have a component, it's much easier to manage lots
of small lists than one big one, especially for delegating. I created
components for the items I found (hope that's OK), and will continue
to do so. If anyone else finds one of these, ping someone with
bugzilla superpowers (like me)

There are a lot of crashes filed against general, looking at the
backtrace it's impossible to see who's at fault. The backtrace only
shows some timer firing, or a python binding call or something generic
- it's impossible to tell what's at fault. I have no idea how we can
fix this.

Some bugs weren't even against plasma (as devs know it). As we use the
phrase "plasma workspaces" I guess it's natural for bugs to end up
here which are totally generic. I found bugs on kwin, dolphin, and
other KDE things. I don't think there's really a fix for this.

There's a lot of wishlists.
I found these very hard to triage because I can't say on someone
else's project saying "we're never doing this, wontfix", which I would
do on my projects (maybe slightly more politely). I imagine many
others are in the same position, how can we manage this?


> Now the action items I would suggest to do:
>
> *Re-evaluate all components for Plasma*
> Let's check whether the components we have are still valid and fitting. L=
et's
> rename what can be improved and create what makes sense to create. Maybe =
even
> rethink whether it makes sense to split the product "plasma" into multiple
> products.
>
> *Find a maintainer for each component*
> The current maintainer model of Aaron and Marco for everything doesn't sc=
ale.
> We have so many people who could take over maintainership for a small
> component. Whoever is maintainer of a component, should become the default
> assignee for bugs in that component. No longer a one address for all assi=
gnee.
> Let's make the people responsible for their components.
>
> *Assign some "super" maintainers*
> Having a maintainer for each component carries the risk of people droppin=
g out
> without being noticed and that would render the component unmaintained
> (example: all the kde-workspace components currently maintained by Lubos)=
. We
> need some people who regularly check whether there is activity and poke t=
he
> right people when lots of bugs get unmanaged (hey Jimmy are you still wor=
king
> on those? No? Should we get a new maintainer?). In Bugzilla talk this is
> called a QA contact IIRC. Random thought: give those super maintainers the
> power to pull the plug on a component if the quality decreases.
>
> *Restrict the available version numbers*
> At KWin we only allow bugs to be reported for:
> * the latest beta/RC
> * the last two minor versions of the current stable
> * the last two minor version of the last stable
> That is currently: 4.8.4, 4.8.5, 4.9.4, 4.9.5, 4.10 RC2
>
> We won't do a release for 4.8 or 4.9 any more, so we hardly care about th=
ese
> versions. There are just too many changes to properly support those versi=
ons.
> I would restrict even further, but there are users out there and it's not=
 nice
> telling them that the distro they installed two months ago is already so
> outdated that we don't support it. The restriction on the latest two minor
> versions is to catch all the .1 bug reports we get through distro install
> media, but are fixed in .2 or .3. It helps the user to tell them: look th=
ere
> are bug fix releases, you might want to update.
>
> *Close all existing bug reports*
> We are currently in the process of developing Plasma 2 and porting everyt=
hing
> to QML. Do we really care about a bug reported for the C++ version? Do we
> really care about a crash report which is for an old version and the back=
trace
> doesn't match the code any more? Do we really want to spend person years =
of
> going through the bugs to confirm them? Now with the step towards Qt 5/KF
> 5/libplasma2/QML we have the chance to do the clear cut. We did it before
> (KDesktop/Kicker). Let's do it again. I'm pretty sure our users prefer the
> honesty that we won't work on it then having the bug open for so long.
> I just checked and there are 811 reports open which haven't changed over =
the
> last 365 days (and all of them are wishlist). And there are an additional=
 561
> bugs which have not changed over the last half year (no comment, nothing).
>
> That's all for the bugzilla part, now let's do some more general ideas I =
have
> :-)
>
> *Every commit should be referenced to a bug*
> What is the motivation for a commit? It's either a bug fix or it is a new
> feature/improvement. If it's a bug it's clear that there has to be a bug
> report for it (out of experience: there is always a bug, if not: create i=
t).
> If it's a feature, it should also have a feature request in the tracker.
> Create it yourself if you need to. Sounds beaurocratic, but it comes quite
> handy as it allows to generate changelogs from it, allows people to easily
> test new features.
>
> *Bug fixes should come with a unit test*
> Plasma so far does not have any unit tests. Why? Let's not fool ourself w=
ith
> "it's UI, that's difficult to test". No it's not, because Plasma has a gr=
eat
> separation between UI and logic. It would not be a problem to create a un=
it
> test for the data engines. If there is a bug do yourself the favor of cre=
ating
> the test. It helps you understanding the problem and fixing it. Also reme=
mber:
> we have now a Jenkins installation running the tests after each commit, s=
o we
> see when they break (currently 3 tests in kde-workspace are failing).
>
> *New features should come with unit tests*
> If you work on something new it's easy to restructure the code in a way t=
hat
> it gets testable. New logic should be covered by tests. GUI is difficult,
> though we could spend a GSoC on getting a test framework.
>
> So that's it. Looking forwards to your comments on it :-)
>
> Cheers
> Martin
>
> _______________________________________________
> Plasma-devel mailing list
> Plasma-devel@kde.org
> https://mail.kde.org/mailman/listinfo/plasma-devel
>
_______________________________________________
Plasma-devel mailing list
Plasma-devel@kde.org
https://mail.kde.org/mailman/listinfo/plasma-devel
[prev in list] [next in list] [prev in thread] [next in thread]