[prev in list] [next in list] [prev in thread] [next in thread] 

List:       kde-usability
Subject:    User Testing Package
From:       Irwin K <emerald-arcana () rogers ! com>
Date:       2002-07-10 2:43:50
[Download RAW message or body]

Here's a sort of "package" that can be used for user-testing.  A lot of credit 
goes to David High-Jones for writing up pretty much tne task list and the 
questionnaire in his template.  This is meant to be a pretty generic package 
that you can hand out to anyone who says, "I have a user I want to test." 

These are all in separate files because they're meant to be handed out 
separately.  Ideally they would be distributed as PDF or another 
easily-printable format for ease of distribution.  I'll work on that.

Finally, the informed consent sheet is NOT included in this package because I 
haven't typed it out yet, but it goes along the lines of:

"I am consenting to KDE Usability testing.  The KDE Usability Project will not 
use this information for malicious reasons.... only for research purposes.... 
they are not liable for any damages.... I understand that this is a 
completely voluntary exercise and that I may back out at any time.... that I 
have no obligation to complete the test or answer any of the questions.... 
please contact <name> if I have any concerns." 

Signed: ___ Date: ___

Please provide comments, feedback, and constructive criticism.  Thanks,

-- 
-- Arcana  (Irwin)
["questionnaire.txt" (text/plain)]

KDE USABILITY TEST

Questionnaire
=============

Please fill in the questionnaire after you have finished executing the tasks.  Return \
the questionnaire to the test administrator.

You do not need to answer all of the questions.


1. How long have you used computers for? Choose the closest answer.

a. Never used a computer before
b. Less than 6 months
c. 6 to 12 months 
d. 1 to 2 years
e. 2 to 5 years
f. Over 5 years


2. How often do you normally use a computer? Choose the closest answer.

a. Never used a computer before
b. Less than once a month
c. Approximately once a month
d. Approximately once a week
e. Every day


3. What sort of computers do you have previous experience with?  Circle all that \
apply.

a. Windows
b. Macintosh
c. Linux/Unix 
d. Other (Specify: _______________________________)
e. I'm not sure


4. How easy was it to log in to the computer?

a. Very easy
b. Somewhat easy
c. About average
d. Somewhat difficult
e. Very difficult

Please provide some comments on why you came to your conclusion.



5. Think back to finding the README.html file on the system. How easy was it to find \
the file?

a. Very easy
b. Somewhat easy
c. About average
d. Somewhat difficult
e. Very difficult

Please provide some comments on why you came to your conclusion.



6. When you had found the "README.html" file, how easy was it to open the?

a. Very easy
b. Somewhat easy
c. About average
d. Somewhat difficult
e. Very difficult

Please provide some comments on why you came to your conclusion.




7. How easy was it to visit the web site, "http://www.kde.org"?

a. Very easy
b. Somewhat easy
c. About average
d. Somewhat difficult
e. Very difficult

Please provide some comments on why you came to your conclusion.




8. How easy was it to save the on-line KDE announcement to your home directory?

a. Very easy
b. Somewhat easy
c. About average
d. Somewhat difficult
e. Very difficult

Please provide some comments on why you came to your conclusion.



9. How easy was it to log out of the computer?

a. Very easy
b. Somewhat easy
c. About average
d. Somewhat difficult
e. Very difficult

Please provide some comments on why you came to your conclusion.



10. Overall, how easy was it to use the computer system?


a. Very easy
b. Somewhat easy
c. About average
d. Somewhat difficult
e. Very difficult


11. Did you enjoy using the new computer system?

a. Enjoyed it very much
b. Enjoyed it somewhat
c. No strong opinion
d. Did not enjoy it very much
e. Did not enjoy it at all


12. Can you think of anything that could be done to make the computer system easier \
to use?






13. Can you think of anything that could be done to make the computer system more \
enjoyable to use?






14. Please use the space below to make any other comments about this computer system.


["setup.txt" (text/plain)]

KDE Usability Test

==========
Hardware and Software Setup
==========

Install KDE on the target machine.  The packages that should be installed are:
	-qt
	-arts
	-kdelibs
	-kdebase
	-kdenetwork
	-kdegraphics

Create a user named "test" with a password of "kdetest" on your machine.

Ensure that the user is visible from the KDM login manager, and that the user can \
connect to the internet as normal.

Create the following directory structure in the user directory.  The script \
"create_environment.sh" will copy the necessary files for you.

~test
	instructions/README.html
	kde.txt
	user_test/questionnaire.txt
	images/konqi.jpg
	images/kdelogo.jpg
	.bash_logout

Install the included logout script, ".bash_logout", in the "test" user's directory to \
ensure that the environment is clean when the user exits the KDE login environment.

==========
Test Administrator Guidelines
==========

Ensure that the user receives a copy of the test subject guidelines, the task \
instructions, the questionnaire, and the consent form.  Ensure that the consent form \
is signed before the user begins the test.

Have a watch ready to monitor how long the test subject takes to perform each of the \
tasks detailed in the user instructions. Take notes on each of the tasks - did \
anything cause the user difficulty? Did the subject do things in a different way from \
what you expected?  Were there system behaviours that frustrated the subject? Did the \
subject get "stuck" and have to ask for help?

If the test administrator cannot personally witness each test, then ask that the \
users read over the documentation carefully before they start.  Ensure that they do \
not feel confused or frustrated as to the purpose of the test.  Remember, they are \
not obligated to perform all of the objectives, and remind them that the test is \
totally voluntary.  It is not a test of their abilities, but instead a test of the \
system.


["tasklist.txt" (text/plain)]

KDE USABILITY TEST

==============
User Task List
==============

1. An account has been created for you on the computer system. The user's login is
"guest" and the password is "abc123". Please login to the new system.

2. Your administrator has given you a README file to tell you about the system. The
file is called README.html and is in a folder called "instructions" in your home directory.
Please find the file and open it.

3. The file refers to a website, "http://www.kde.org".  Use the
computer's web browser to access this web site.

4. When the web site finishes loading, click on one of the
"Announcements" to learn more about the KDE Project.

5. The announcement seems to be useful!  Please save the web page to
your home directory for future reference. 

6. That's all you're going to do for today. Please log out of the computer.

==========


["user-guidelines.txt" (text/plain)]

KDE USABILITY TEST

===============
User Guidelines
===============


Overview of the Test
-------------------

A usability evaluation actively studies specific activities performed by users on a \
computer.  The study gathers experimental evidence by witnessing users and gathering \
user feedback to evaluate the ease of use, the intuitiveness, the functionality, and \
the speed at which users can execute common actions.


Before We Begin
---------------

The test administrator should have briefly explained to you the purpose of the test \
and what the result are used for.  They will also provide you a copy of the following \
documents:

1. A User Guidelines sheet (this document), which summarizes the purpose of the test \
and gives you some background information. 2. A Consent form, which states your \
agreement to participate in the usability study. 3. A list of tasks to perform in \
front of the computer. 4. A questionnaire consisting of questions to answer for \
evaluation purposes.

Please read the provided consent form carefully.  Sign both copies if you agree to \
the terms presented.  Keep a copy for yourself and give the other copy to the test \
administrator.

A list of "User Tasks" is included with the Usability Test Package.  These tasks will \
be evaluated for their usability.  Please review them briefly.  If there are any \
terms or definitions that you are not familiar with, please ask the test \
administrator for clarification before you begin. 


Executing the Test
------------------

As you execute the test, speak aloud as you execute actions.  For example, if you are \
trying to open a menu, say aloud, "I'm going to the menu.  I see 'File'.  I'm going \
to click here."  This helps the evaluator guage your reactions to the tasks and \
elements on the screen.

Do not feel compelled to fulfill all of the actions asked of you by the evaluator.  \
If you cannot perform an action, then announce that you do not know what to do, and \
the test can proceed to the next step.  Remember, the test is for the software, not \
for you.

The following are questions that you may want to ask yourself as you perform tasks in \
front of the computer:

- How does the user interface feel?  Does it look warm, familiar and friendly?  Do \
                you like the colours and the layout?  Is it something that you would \
                like to look at?
- What is the first reaction you have when asked to perform a task?  Does the \
computer conform to your first reaction, or do you need to look farther to perform a \
                task?
- Is the user interface easy to scan?  When you are looking at it, can you easily \
decipher the different parts of the interface?  Are there too many items on the \
                screen?  Too few?
- Are the elements laid out in an order that makes sense?  Does it seem like items \
                have a logical grouping?
- How long does it take you to perform a task?
- How long does it take for the task to finish executing?  Is this time acceptable to \
you as a user?


Finishing the Test
------------------

When the test is finished, please fill out the questionnaire that is included with \
the Usability Test package.  When you are finished, please return the test to the \
test administrator.


_______________________________________________
kde-usability mailing list
kde-usability@mail.kde.org
http://mail.kde.org/mailman/listinfo/kde-usability

[prev in list] [next in list] [prev in thread] [next in thread]