[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-look
Subject:    Re: Icon Usability
From:       "Steven D'Aprano" <dippy () mikka ! net ! au>
Date:       2001-03-24 3:52:19
[Download RAW message or body]

"Aaron J. Seigo" wrote:
> 
> Hi...
> 
> > Timing the task has a technical solution, but not a practical one, since
> > you can't control for Joe User getting up to make a cup of coffee or
> > watch a Babylon Five marathon in the middle of the task.
> 
> This is assuming:
> 
>   1) you accept all data as correct (ignoring that 30 minute lull in activity
> in relation to the 5-30 seconds all the other tasks took) and don't filter
> accordingly

OK, perhaps watching B5 in the middle of the task is an obvious outlier,
but more to the point: how do you allow for people who multitask? You
won't see one or two obvious outliers, but you'll have figures that vary
from (off the top of my head) 5-180 seconds. How do you decide whether a
30 second icon is one that took the user 30 seconds to recognise, or one
where they were distracted by a juicy little email they had to read? Or
whatever.

>   2) you don't tell the tester that time is critical, please focus on the
> task while doing it, otherwise the data is useless (even going so far as to
> be there while they do it)

How are you going to be there when they do it? Isn't the whole point
that people all over the world can go to kde.org and do the test? You
going to fly all over the world and break into their homes? :-)

And as for telling people time is critical, that might help the problem
but it won't solve it completely, since many people (especially the most
and least intelligent ones) don't read instructions in full.

>   3) you don't give the tester a "Pause" button.

Which complicates the coding immensely, and leaves open the opportunity
to cheat. "Heh heh, lets see if I can recognise every icon in under one
second!"

> the idea isn't to trick people into doing usability studies, but to allow
> volunteers to spend time providing data because they want to add to the pool
> of data and then to manipulate the data inteligently.

I never suggested anything about tricking users. But as a wanna-be
statistician, you have to be aware of the certainty of users wanting to
trick YOU.

If you were only asking people to vote on half a dozen icons (just
enough to fit on a single screen with no scrolling) then maybe any of
these solutions would work. But if the original plan is followed and you
ask people to vote on ALL kde icons, dozens or hundreds or them, you
need a better solution:

Firstly, since many people won't sit all the way to vote on every single
icon, you need to allow for icons that don't get a vote.

Secondly, you need to present them in a random order to people. This
means that the data will have no systematic bias as to which icons get
voted on the most and which icons get missed. This also has the
advantage that it will seperate icons from their families, and show them
in isolation (ie instead of showing all aKation file icons one after the
other, they will show up at random intervals).

Thirdly, perhaps you should give the user an *explicit* way of rating
how recognisable an icon is: a second combo box or something that rates
the recognisability as "Obvious/Easy/Moderate/Hard/Impossible".

Finally, you need to recognise that your sample space is composed of an
unrepresentative number of the most intelligent and experienced users,
and badly unrepresented by Joe Hacker's grandmother.

Steps 1 2 and 3 are easy to fix. Step 4 is the doozy.


-- 
Steven D'Aprano

[prev in list] [next in list] [prev in thread] [next in thread]