Testing, usability and evaluation

Testing tips

Test often

Simply put, the more you test, the more you learn. This enables you to make more-informed decisions about code refactoring and design or product iteration. Testing doesn’t have to be expensive or involve users and test houses at every stage. Even informal in-house testing with team members can provide you with valuable insights.

Context can also provide valuable insights. Try to simulate real contexts of mobile use, such as noise, limited lighting, incoming calls, or loss of signal, to ensure that the application will deal gracefully with these types of interruptions.

When possible, test on a real device

Get the design onto a device as early as possible, and test it there throughout the design process. Emulators and remote testing services such as Nokia’s Remote Device Access (RDA) can be extremely useful, but testing on a real device is always best. On-device testing compels you to manipulate the app as your users will — with real fingers. This often uncovers design and usability flaws that aren’t immediately apparent while you’re using a mouse and a large desktop monitor

When possible, test with users

User testing is at the heart of usability studies. Testing a product with users and analysing their actions provides direct information on how real users deal with the product and often uncovers behaviours that even experienced designers cannot foresee.

There are many ways to evaluate a product during the design process. Each method is suited to yield certain types of insights.

Method	Description	Advantages	Disadvantages
Expert evaluation	A usability specialist evaluates the product based on commonly accepted heuristics and guidelines. The result is a list of potential and existing problems, along with recommendations for how to resolve them.	Is quick and low-cost compared to tests with users. Can reveal key design problems and help ensure that your application follows best practices.	Cannot replace insights from your product’s target audience.
Single-user usability testing	Individual users are asked to conduct specific tasks and are observed to determine where and why they encounter problems. It can be helpful for the users to think aloud during this process and explain problems as they occur.	Provides valuable data about natural usage and often reveals common problems that your users may experience.	Requires a skilled facilitator to assist in an impartial way and to prompt users who may be shy or uneasy with the process.
Group usability testing and focus groups	Users from the product’s target group gather to review the product and provide opinions on features or competing products.	Is a fast, cost-effective way to test an application. Can generate helpful dialogue when users are asked to engage in discussion and debate.	Lacks the personal aspect of single-user testing, and participants may  be reluctant to reveal what they really think in front of a group.
Observation	Representative users are observed and interviewed in their own environments.	Is ideal for studying users’ motivations, goals, workflows, task flows, and social and physical environments.	Can be intrusive and may not always represent true behaviour.
Surveys	Prospective users complete questionnaires designed to provide data for statistical analysis.	Is useful for obtaining general information about who your users are and exploring their opinions about a product. Can be completed in private, so can be useful for discussing sensitive topics.	Rarely reveal behavioural patterns or deeper reasons behind opinions and behaviour.

Have you noticed similarities between the methods above and those listed in the Design Research section? When testing occurs throughout product development, each test is likely to have different goals. Some tests are performed to inform design, and others to evaluate, iterate, or adjust a close-to-finished product. Don’t test merely for the sake of testing. Be clear about why you are performing a test, what you hope to measure, how you will measure it — and then choose the best testing method for that task.

Links and resources

Testing

Nokia's User Experience Services offer both expert evaluation and usability testing. Read more about the services.

Nokia’s Remote Device Access (RDA) is a free service that enables developers to test products remotely on real devices. RDA includes access to a collection of new and prototype devices, and its online booking system enables you to reserve devices several days in advance.

DeviceAnywhere is a commercial service that provides remote access to hundreds of devices, including the devices in the extensive Virtual Developer Lab (VDL).

Usability guidelines and practice

The Developer Library’s Design and User Experience Library area provides an extensive collection of usability guidelines.

Developing good games isn’t an easy task. Good playability and usability result from hard work in all design phases. The Mobile game design section of the Design and User Experience Library contain extensive examples.

MeeGo 1.2 Harmattan

The Nokia N9 UX Guidelines section includes an extensive collection of layout, interaction-design, and visual-design guidelines for the Harmattan release of the MeeGo platform, and the Nokia N9 device.

Symbian

Symbian design guidelines contain interaction patterns as well as common UI component descriptions to guide design and development.

Series 40 Touch and Type

Nokia's Series 40 Touch and Type UI Style Guide provides a general overview of UI design principles, along with guidelines for the design of well-integrated, consistent, and usable Series 40 services and applications. For Web apps, see the Series 40 Web App UX Guidelines and the Series 40 Web App UI Graphics Toolkit.

Themes

Theming enables designers to change the visual appearance of S60 and Series 40 devices while leaving their functionality intact. The Design and User Experience Library includes a section on theme design.