Your firm of interaction design consultants is trying to build up a portfolio of impressive work, to enable it to pitch for business convincingly in the future.
Your task is to produce a usability evaluation of an interactive system by carrying out user testing, plus a presentation of your results. You have a completely free choice of what interactive system you evaluate.
Choice of Interactive System
Possibilities include software applications such as CASE tools or games or e-commerce websites or photo editing systems; electronic devices such as remote controls for televisions or DVD players, or digital cameras, or car radios; or control panels for appliances such as microwave ovens or home heating systems; or a self-service system such as an automatic train ticket vending machine. You may, if you wish, choose to evaluate two very similar and directly competing products, and assess ways in which one is superior to the other.
It’s perfectly okay to decide to evaluate a part of a big or complicated system, or consider a limited set of use cases. A reasonable amount of functionality to consider is what you can get one user to work through in one user trial.
The one piece of advice we can give is to choose something that is complicated or difficult to use, or is used to carry out complicated tasks, and preferably has obvious usability problems. Studying more complicated and less frequently used features of a system is likely to be more fruitful than focusing on the standard functions people use all the time. You may choose to interpret ‘interactive system’ very broadly and present a usability evaluation of a static information display, but this would require a sophisticated and detailed analysis of how people use it for practical tasks, and these tasks would need to be complicated enough to give you something to analyse. Ask advice if you consider this.
The Usability Evaluation
Producing the usability evaluation will involve
1.Identifying the use cases or aspects of the functioning of the system to be considered, and briefly describing them in your report. (These don’t need to be a complete set of use cases; for very complicated systems focusing on one part of what they do is just fine. However you should give a clear indication of what subset of the functionality of the system you are considering, and what you are not considering. If in doubt, cover less functionality in more detail.)
2.Defining an evaluation procedure. This will include stating one or several user tasks to be tested with exact descriptions of the scenario and the goal the user is trying to achieve, as well as what the evaluator will do to collect results and produce an evaluation. It requires a description of the procedure to be followed with each test subject from start to finish, including the exact wording of the instructions given to the subjects. The evaluation procedure needs to be described separately from the description of the results. Ideally your procedure should include filming the subjects.
3.Carrying out the evaluation. This will involve following the procedure and documenting what the users do and what problems they have. This should include brief descriptions of your test subjects including what relevant experience they have. Your raw observation notes should be scanned and presented as an appendix. You should test the system with several subjects. You may want to treat your first test or two as a pilot and revise your procedure; if you do this, comment on it.
4.Deriving findings about the usability of the interactive system from the results of the usability evaluation. This should include consideration of how strong and how general the conclusions are. Don’t be afraid of lists.