Innovative Learning

Usability Testing

The following are examples of methods for testing the usability of computer interfaces (adapted from "Usability Assessment Methods beyond Testing")

Method Stage in Process Users Needed Advantages Disadvantages
Traditional Testing Iterative design, formative evlauation, final testing 3 +   Time consuming and expensive
Heuristic evaluation Competitive analysis, final testing None Finds individual usability problems. Can address expert user issues. Does not involve real users, so does not find "suprises" relating to their needs.
Performance measures   10 + Hard numbers. Results are easy to compare Does not find individual usability problems.
Thinking aloud Iterative design, formative evaluation 3-5 Pinpoints user misconseptions. Inexpensive Unnatural for users. Hard for expert users to verbalize.
Observation Task analysis, follow up studies 3 + Ecological validity; reveals users' real tasks. Suggests functions and features. Appointments hard to set up, No experimenter control
Surveys Task analysis, follow-up studies 30 + Finds subjective user preferences. Easy to repeat Pilot work needed (to prevent misunderstandings).
Interviews Task analysis 5 Flexible, in-depth attitude and experience probing. Time consuming. Hard to analyze and compare
Focus groups Task analysis, user involvement 6-9 per group Spontaneous reactions and group dynamics. Hard to analyze. Low validity
Logging actual product use Final testing, follow-up studies 20 + Finds highly used (or unused) features. Can run continuously. Analysis programs needed for huge mass of data. Violation of users' privacy
User feedback Follow-up studies unlimited Tracks changes in user requirements and views Organization needed to handle responses