User testing...from class slides Fall 2015 CSE 556A
Tips on Recruiting Users
Find people with the same experience level as the typical user.
Dont get people who are familiar with the product or your views on the product.
Be careful about friendsand family testing.
Public places like libraries, dining halls, coffee shops can be good places to find people who wouldn’t mind helping for a few minutes.
Some companies have user testing labs that they set up and they handle recruiting users.In academia, we often post fliers or set up agreements with local organizationsA small budget to give out gift certificates or something can help. People like reimbursement for their time
Find a quiet distraction free room for usertesting.
Optionally audio or video record the user tests.
Where it will be used -- need similar environment to where it will be used?
Tips on User Instructions:
You are testing a piece of software ... not them
Its ok for them to stop at anytime. How to handle scenario where people leave?
Demonstrate equipment that users will need to use -- unless equipment is what you are testing. Example an occulus or three-D headset.
Tips on Tasks:
The tasks are the ones developed in the previous milestone
For user testing rephrase as "you"
Think Aloud Protocol:
Asking the users to think aloud as they are working.
Explain why -- rich info source for you
You may need to model it once for them
May also want to get them to practice once with an unrelated task.
Observer --> what to do?
Capture the users behavior
Thoughts behind it <-- especially
Lack of intuitiveness for user. Listen for ____???
I am looking for...
I am guessing that...
I am confused...
Oh so this is the ...
When test subject clams up...
Promptthem...want to get them talking:
Can you tell me what you're thinking
Can you tell me why you clicked on x
You cannot provide help
Tell users you will not help them when doing the initial explanation of the test.
User questions...they should ask them anyway. Observers should note these and answer them after test is over.
Some cases you can intercede...know when ahead of time. example user not making progress for 3 minutes.
Evaluation of Results...What to go after:
You should find lots of problems --> what to go after
Difficulty -- is this an easy fix or a major rewrite... major rewrite comes in to play in hi-fi prototypes...lo-fi you should be able to throw it away relatively easily.
Stages of prototyping
Paper prototyping -- can people complete task? cheap at this point to fix.
Digital prototyping --which tasks are the most problematic per cost to change? --timing errors etc.
Deployed Beta -- bug reports, complaints...real world scenarios...more variable data.
Involve the user in brainstorming.
Ask them questions, open ended, to give improvements for the product or application after test is completed.
"How would you make the application better?"."Can you think of anything that needs to be added?"