Introduction
What characterizes a good product? Before my time in software engineering, I always thought that the requirements for a system were primarily focused on its functionality. In fact, however, it is the combination of functionality and usability that determines the quality of a product from a user's perspective. This article will therefore focus on the far too rarely considered component of a system's usability and how it can be ensured in the world of medical devices.
The concept of usability
The standard DIN EN ISO 9241 Part 11 defines the term usability as follows:
Usability is the extent to which a system, product or service can be used by specific users to achieve specific goals effectively, efficiently and satisfactorily in a specific usage context.
The following components can be assigned to the usability concept:
- User: The user of our product
- Tool: Our product to be developed
- Context: The environment in which the product is used
- Task: The goal of using our product
Good usability is ensured by evaluating our product according to the following criteria:
- Effectiveness: Do users achieve their goal by using our product?
- Efficiency: Do users achieve their goal by using our product without wasting unnecessary resources (time, energy, motivation)?
- Satisfaction: Does our product meet user expectations?
Usability should therefore always play a role wherever users interact with a product. This means that usability refers to user interfaces. These can be digital in nature, such as GUIs and NUIs (Natural User Interfaces, typically touchscreens). But even simpler user interfaces, such as the arrangement of buttons and LEDs on a device, should be developed with good usability in mind.
Getting to know the usage context and task: The Contextual Inquiry
As an external development engineer, it is not always easy to immediately understand the environment in which a product is being worked on. A solution to this is a method called "Contextual Inquiry," which aims to observe and question users while they complete a task in their usual environment. This method follows a dynamic approach and is intended to produce a number of results, which are shown in the table below. For better understanding, I have chosen the example of a newly developed ultrasound device, for whose contextual analysis an orthopedic surgeon is to be observed at work. This example is, of course, greatly reduced in scope. A true Contextual Inquiry would likely produce many more results in this case.
| Result | Analysis object | Core attributes | Example |
|---|---|---|---|
| Get to know the tasks and goals | Approach and action strategies | Activities | Diagnose ligament ruptures, detect fluid retention |
| Frequencies | Average 7 uses per day | ||
| Length of time | 3-7 minutes | ||
| intensity | Thorough investigation requires intensive use | ||
| Special cases | None detected | ||
| Communication and role sharing | Responsibilities | Determination of a patient's findings, instruction of the assistant, procedure of the examination | |
| Communication content | Instructions to assistant, communication of findings | ||
| means of communication | Linguistically, written | ||
| Communication purpose | Practice organization, determination of further steps | ||
| Background information | Domain knowledge | Terminology: Creation of a glossary | |
| Getting to know the context | Physically | premises | Examination room |
| lighting | Normal to bright room lighting | ||
| temperature | Room temperature (approx. 21°C) | ||
| Influences from dirt or other substances | Ultrasound gel, blood if necessary | ||
| Social, cultural | Rules of conduct | Professional appearance, use of understandable language | |
| Other people | Office assistant, patient | ||
| Values | Patient well-being | ||
| organization | Division of roles | Orthopaedist: senior person, primary user
Consultation assistant is: secondary user |
|
| interruptions | By emergency |
This table is intended as a guide for the contextual inquiry and therefore offers no guarantee of completeness. Contextual inquiry merely provides guidelines, from which deviations may or must be made depending on the situation.
Developing Essential Use Cases
Now that we have become familiar with our task and its context in the course of contextual inquiry, we want to derive use cases for the tool to be developed. Before doing so, however, it should be mentioned that for the sake of clarity, I will skip an important step in this article: the identification of user groups and their storage in so-called personas, which represent fictitious people with real characteristics from their respective user group. The Article “Personas” on the usability guidelines page of the US General Services Admission for Technology Transformation Services.
But now back to the use cases: An effective method for writing down use cases for the development of innovative products is the formulation of so-called “essential use cases.” Compared to the conventional use cases, these are more compact and describe the system responsibilities from the User's viewIt becomes clear that the essential use cases originate from the field of human-centered software development.
The following comparison should clarify the difference between essential use cases and conventional use cases. The example chosen is the process "view patient data" in a patient management system.
| Use Case ID: UC101 | |||
|---|---|---|---|
| Use Case Name: “View patient data” | |||
| Personas involved: Kathrin Musterfrau | |||
| Conventional Use Case | Essential Use Case | ||
| User Action | System Response | User Intention | System Responsibility |
| Open application | Request login credentials | To have secure access | Verify identity |
| Enter Username and Password | To view patient data | Offer search possibilities, Show search results |
|
| Press Enter Key | Verify credentials, Show menu |
To logout | Offer logout possibilities |
| Select “Search Patient Database” | Request patient name and DOB | ||
| Enter patient surname and DOB | |||
| Press Enter Key | Parse database, Show results |
||
| Press Logout Button | Logout, Show logout message |
||
In summary, one can say that essential use cases are, on the one hand, more compact and, on the other hand, increase the degree of freedom in deriving requirements, which stimulates innovation in the development process. Essential use cases can be captured, for example, in templates and recorded in a so-called use case diagram. What use case diagrams are and how to model them can be found in our Article about use case diagrams can be read.
Usability requirements
In the next step of the process, we will focus on the requirements for our user interface, which form the basis for a prototype. The user requirements are derived from the essential use cases that we previously captured and documented, e.g., using templates. When creating requirements, the following applies: Usability Guidelines of the German Accreditation Service (DAkks) to ensure compliance with the three quality criteria:
- Objectivity: A user requirement must be formulated in such a way that it is free from stakeholder influences and corresponds to an actual need that has emerged from the analysis of the usage context.
- Consistency: A usage requirement should not contradict other usage requirements. Should this nevertheless occur, this is due to the existence of another user group or another usage context, for which separate current analyses (context and user analyses) must be conducted.
- Validity: A request for use must be based on data that has been confirmed by persons in representative positions and corrected if necessary
The following syntax is suggested for formulating user requirements: “A user mustTask> can.”
For a predicate can be used together with an object.
Some examples, which refer to the patient management use case, should illustrate the formulation:
- “A user must be able to confirm his identity on the system”
- “A user must be able to view patient data on the system”
- “A user must be able to log out of the system”
Prototyping
The prototyping process step involves implementing user requirements through a visual design process, culminating in a testable prototype. The design can be tool-based, with various tools available depending on the interface type. Depending on the required maturity of the prototype, a simple sketch, brought to life using the Wizard of Oz technique, or a wireframe can provide a testable prototype. The advantages of these types of prototypes are obvious: On the one hand, they are very inexpensive, and on the other hand, they are usually available within a very short time and can therefore be used as a basis for discussion in acute situations. Unfortunately, more effort is usually required for user interfaces that are not on screens but, for example, on control panels on a device: Testable prototypes are often just hardware mockups that must be developed using manufacturing processes (e.g., using 3D printing). Nevertheless, I've also heard of companies that have used Lego bricks or cardboard prototypes to create hardware mockups. As always in development work, creativity is required here, too.
Usability Evaluation
The goal of a usability evaluation is to test a prototype for the usability metrics of effectiveness, efficiency, and satisfaction. A variety of methods are available for this purpose. Here, I would like to specifically focus on a particular user-oriented observation method, which, as the name suggests, is based on observing user interactions with the tool prototype. This method is called a usability walkthrough.
The observation takes place in a real usage context, for example in a practice or clinic where the participant, usually a doctor, works. A participant, who should belong to one of our identified user groups, is given tasks during the observation which they must complete without additional assistance from the evaluator. The tasks should be as realistic as possible. An appropriate question in the case of the ultrasound device would be: "Please use the ultrasound device to create an image of the patient's right lateral ligament." It is particularly important that the participant thinks aloud while solving the task so that the evaluator can understand their thought processes and note them down if necessary. In some cases, it is useful to record the test in some form, e.g., with a video. However, data protection aspects should always be clarified in advance. In order to obtain a representative impression of a product's usability, as many of these tests as possible should be conducted with different participants. Any anomalies, so-called usability issues, should be reported back to development so that requirements can be adjusted or new ones can be written if necessary.
Other user-oriented methods for evaluating usability that may be relevant for medical devices:
- Hallway Test
- Pluralistic Walkthrough
- Formal usability test
- questionnaire
Conclusion
Usability is a very important topic for product development in medical technology, as poor usability is reflected in the user's perceived quality of the product. Usability engineering should be viewed as a cyclical process that is repeated several times during the development or redesign of a new product. Usability engineering should begin, if possible, with the project kickoff and should not be postponed. The aspect of prototyping is crucial for this, as it allows for the usability evaluation of a not yet fully developed product.