User Testing 101: CSE 556A HCI WUSTL

• Find people with the same experience level as the typical user.
• Dont get people who are familiar with the product or your views on the product.
• Be careful about friends and family testing.
• Public places like libraries, dining halls, coffee shops can be good places to find people who wouldn’t mind helping for a few minutes.
• Some companies have user testing labs that they set up and they handle recruiting users.
• In academia, we often post fliers or set up agreements with local organizations
• A small budget to give out gift certificates or something can help. People like reimbursement for their time

• Find a quiet distraction free room for user testing.
• Optionally audio or video record the user tests.
• Where it will be used -- need similar environment to where it will be used?

• Tell Users:
• You are testing a piece of software ... not them
• Its ok for them to stop at anytime. How to handle scenario where people leave?
• Demonstrate equipment that users will need to use -- unless equipment is what you are testing. Example an occulus or three-D headset.

• The tasks are the ones developed in the previous milestone
• For user testing rephrase as "you"

• Asking the users to think aloud as they are working.
• Explain why -- rich info source for you
• You may need to model it once for them
• May also want to get them to practice once with an unrelated task.

Capture the users behavior

• What they do
• Thoughts behind it <-- especially

• I am looking for...
• I am guessing that...
• I am confused...
• Oh so this is the ...

• Prompt them...want to get them talking:
• Can you tell me what you're thinking
• Can you tell me why you clicked on x

You cannot provide help

• Tell users you will not help them when doing the initial explanation of the test.
• User questions...they should ask them anyway. Observers should note these and answer them after test is over.
• Some cases you can intercede...know when ahead of time. example user not making progress for 3 minutes.

You should find lots of problems --> what to go after

• Importance -- is this a nit, a minor hurdle or a complete showstopper, in terms of users completing tasks.
• Difficulty -- is this an easy fix or a major rewrite... major rewrite comes in to play in hi-fi prototypes...lo-fi you should be able to throw it away relatively easily.

• Paper prototyping -- can people complete task? cheap at this point to fix.
• Digital prototyping --which tasks are the most problematic per cost to change? --timing errors etc.
• Deployed Beta -- bug reports, complaints...real world scenarios...more variable data.

Ask them questions, open ended, to give improvements for the product or application after test is completed.

• "How would you make the application better?".
• "Can you think of anything that needs to be added?"