[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-usability
Subject:    KDE usability study
From:       Robert Watkins <robert_maria () yahoo ! com>
Date:       2001-08-30 20:54:23
[Download RAW message or body]

See my comments below....

> A report maintainer should have the following
>duties:
>
>- read submitted usability problems from users
>and select pertinant issues
>- add these issues to a report
>- liase with the developers to get as many of
the >issues fixed as possible
>
>And optionally...
>
>- conduct tests on the applications usability

I think that the optional part should be our next
step and not an option. The objective part comes
from having the statistics on user performance
during usability tests. Of course, the subjective
part is the choice of test tasks. 

One of the issues I see so far with our approach
is that the usability reports are nothing more
than bug-tracking at this point. The reports
should really be veiwed as a means to create a
usability test as described in Jakob Neilsen's
book "Usability Engineering".

>The report maintainers are the core of the
study, >so we need to develop some guidelines for
what >makes an objective report. Any volunteers
to >write these guidelines?

I think a better goal would be to describe a way
to select items to test and report on based on
user comments (e.g. from the Usability Report as
it exists today)

> Data Gathering <
>
>Another important aspect of the study is to
>gather research from users on what problems and
>issues they have with usability. Also to gather
>research on the type of users that we have and
>their needs. With this data we can make KDE more
>suitable to our audience.
Hopefully, the developers have an audience in
mind when they created the application. If our
idea of the end user is different than theirs,
our results will  be suspect, at least from the
developer's perspective. 

>Data gathering can be done via online
>questionnaires, tests on real world users and
>other methods.

Sorta... I'd suggest using the 'discount
usability' testing methodology as described in
"Usability Engineering" p.175. 
-Start with average users
-test with only a few
-keep good notes
-track some basic metrics (time to complete, #
errors, success ratio, etc.) 
-separate design comments and user experience
comments. "I wish the button was here" is not
helpful. "I wish I could find the button" is a
more useful one.
- etc.

> Usability Research <

>We could do with setting up a KDE Usability
>Research Labproject. In this project we would
>develop test software to develop new and
>innovative usability designs. An example would
be >developing a new method of opening files. We
>would code a prototype, release it, gather
>research on what people think and then publish
>the research. We ideally need usability
>researchers and developers to get involved in
>this. We could then develop new and possibly
>unseen user interface concepts and metaphors.

While R&D is all well and good, we don't even
have common paradigms and metaphors well
implemented. Creating new interfaces is fun, but
can detract from the usability.


> Test Suites <
>
>KDE as it stands forms a lot of content. We need
>to test this content to see where it has flaws.
>Examples of tests could include:

I think we may better be served by putting lots
of efforts into a few applications than putting a
small amount of efforts on a wide array of
applications. Having the reports is only really
helpful if a study is performed to confirm the
problem. A couple of vocal people that don't like
some aspect of a perfectly adequate application
shouldn't be in a position to have their
suggestions implemented without independent
verification that their issue is truly a
usability problem.

>WHAT WE HAVE NOW
>
>I have been developing recently a usability
>tracking database. This would enable the
>following:
>
This should be a means to an end and not an end
in  and of itself.

>
>Another project is the Question API. This
>comprises of a number of typical questionnaire
>questions. These questions can then be simply
>plugged into a questionnaire to form a
>questionnaire. This would save us having to
write >questions every time we need a
questionnaire to >be written.

Don't be fooled into thinking that you can get
accurate, reliable, and complete information from
a quesitonaire. Users may hate those annoying
javascript popup windows when they fill out a
form wrong, but it may lead to a reduction in
their error rate. Taking a questionaire approach,
you may be tempted to remove the popups, but then
your error rate would go up. Where's the benefit.


>
>SO WHERE DO WE GO?
>
>Where to now then? Well first of all, we need
>volunteers for the ideas I mentioned above. We
>also need the report maintainers to keep writing
>reports and sending them to the mailing list and
>relevent developers. When the new database
system >is ready we can then migrate to it.

I'd like to write some guidelines for:
1)Designing a usability test
2)Getting subjects for a usability test
3)Running a usability test
4)Interpreting the results
5)Relaying them to the developers

Jono, Your work does not go unnoticed. I
appreciate the work you have done on keeping this
project going.

Robert Watkins

BTW, getting subjects for tests is not always
easy. It's made simpler by having a token of
appreciation given to the participants. I know
this project has no money set aside for this, but
I'd like to know if anyone has ideas on what we
could do to entice people to do the tests?

My ideas are:
* Having bumper stickers we can pass out (print
them to bumper-sticker paper individually)
* Left-over vendor give-a-ways from conferences
(pens, pencils, keychains, etc.)
* home made cds of Linux w/KDE...
* refer them to paypal for $5 referral bonus.



=====


__________________________________________________
Do You Yahoo!?
Get email alerts & NEW webcam video instant messaging with Yahoo! Messenger
http://im.yahoo.com
_______________________________________________
kde-usability mailing list
kde-usability@mail.kde.org
http://mail.kde.org/mailman/listinfo/kde-usability

[prev in list] [next in list] [prev in thread] [next in thread]