[prev in list] [next in list] [prev in thread] [next in thread] 

List:       meego-community
Subject:    Re: [MeeGo-community] [MeeGo-SDK] Proposed QA rules for
From:       Attila Csipa <meego () csipa ! in ! rs>
Date:       2011-05-29 22:31:55
Message-ID: 201105300131.56492.meego () csipa ! in ! rs
[Download RAW message or body]

On Sunday 29 May 2011 22:40:28 you wrote:
> Too many prescriptive rules leads to an over selection of testers and
> a backlog of ever increasing apps. I might not be able to test (for
> example) an app for participating with a particular piece of hardware,
> but if I see a report from someone I trust saying it works, I might
> vote "it should be in Extras".

Some potential problems with this - one, the issue all-too-present on major 
appstores, being that most of the subjective feedback is simply rubbish.

The other problem is that we (still) retain the same single scale approach 
which mixes popularity with quality, say, comparing an app that has 10 OKs and 
another that has (for the same testing period) 100 OKs and 100 not-OKs 
(remember, if we're in subjective land, the not-OKs might be due to anything). 

Three is the trust used in the way you mention. If you vote on it simply 
because someone else voted on it, your vote is redundant and, worse, puts 
popularity on the same scale as technical prowess. Say, you vote OK, and then 
100 folks, seeing 'hey, if Jaffa says it's ok it must be ok' go the same way. 
But then a random dude discovers an actual showstopper bug and suddenly it 
becomes the question of is that issue more recognized by others than you are 
popular (wtf?). 

Finally, device-specific behavior can ruin your subjective score - I had that 
issue with Mirror on the N900 - I had to note, comment and alert everywhere 
because I systematically got thumbs downs and one stars due to image quality 
(which was the result of physical hardware and firmware on a particular 
device).

> However, I would say that any such descriptions should be shorter than:
> 
>   http://wiki.maemo.org/Extras-testing/QA_checklist
>   http://wiki.maemo.org/Extras-testing#Quality_Assurance_criteria

To underline - I do want apps.meego.com to have somewhat more liberal criteria 
than maemo.org had, and with the extra manpower and tools already has a good 
headstart on it, but let's have a clear perspective (and learn not just from 
Extras, but also other stores and efforts) what certain changes can bring, as 
tweaking is always more difficult than starting with a good set of rules, which 
will ultimately stem from what the target goal of the QA process is - just 
filter out glaring problems and then hope community feedback takes care of the 
rest, or protect users who will have that repository enabled by default and 
prefer tried and tested status to extra features.

Best regards,
Attila Csipa
_______________________________________________
MeeGo-community mailing list
MeeGo-community@meego.com
http://lists.meego.com/listinfo/meego-community
http://wiki.meego.com/Mailing_list_guidelines
[prev in list] [next in list] [prev in thread] [next in thread]