References:
When IEC 62304 or ISO 14971 is mentioned below, the following editions are meant: IEC 62304:2006 + A1:2015 and EN ISO 14971:2012
Problem:
We are developing software/firmware that is part of a medical device. The software (or software system) is assigned to software safety class B or C. This assignment entails, among other things, the requirement of a software risk management process (Chapter 7) according to IEC 62304.
Claim of this article:
The following article aims to present a practical approach to documenting a software risk management process and to provide suitable methods for identifying potential software errors.
Methods for identification
As is (unfortunately) often the case, IEC 62304 does not provide any specific guidelines regarding the methodology to be used to identify potential software defects. Some companies use methods such as FMEA (Failure Mode and Effect Analysis), as these are mentioned in ISO 14971 along with risk management. But to what extent are these methods suitable for software risk management?
Excursus: FMEA and FMECA
FMEA is described as an analysis technique for the functionality of systems in IEC 60812. The technique is used across industries and is not specific to the medical device industry. More specifically, FMEA originated in the US military and is now widely used, especially in the automotive industry.
If the FMEA is supplemented by the failure significance analysis, then it is called an FMECA (failure modes, effects and criticality analysis)
There probably isn't one true FMEA. Every industry or company uses FMEA in its own way. However, many have in common the use of a risk priority number (RPN).
In German-language literature, the risk priority number is composed of the following three components:
- B: Severity/Significance
- E: Probability of detection
- A: Probability of occurrence
IEC 60812 also refers to a risk acceptance assessment. This sounds familiar to us, and we'll discuss it in more detail later.
The appendix to IEC 60812 provides examples for creating an FMEA or FMECA. This scheme is also familiar to us from ISO 14971.
Requirements of IEC 62304 regarding software risk management
Let's now return to medical devices. IEC 62304 begins by stating:
"Software is often an integral part of medical device technology. Ensuring the safety and effectiveness of a medical device containing software requires knowledge of what the software is intended to do and proof that the software achieves this effect without causing unacceptable risks."
IEC 62304 continues to follow the risk-based approach, which is also known in other areas (quality management systems, classification of medical devices). The methodology behind it is the software safety classification, which divides products into three classes: A, B, and C.
The depth of documentation and verification activities is determined based on the software security classification. Note: This is intended to provide an overview. This article will not delve deeper into the topic at this point; that would require a separate discussion.
For security classes B and C, a software risk management plan is required. Both classes also require planning for the identification and prevention of common software defects.
Differentiation between IEC 62304 and ISO 14971
Another important aspect: dealing with software changes. This is intentionally not discussed here.
It can be seen that the aspects of risk management required in ISO 14971evaluation are not included/required in IEC 62304.
To get straight to the point: This is also the key reason why we consider the use of an FMEA or FMECA to be inappropriate. The use of an RPN in an FMEA is difficult to justify. If the RPN is omitted, in our opinion, there is little left to justify the use of an FMEA. Therefore, we consider it sensible to integrate the software risk analysis into the system risk analysis according to 14971. We will demonstrate how we implement this in practice shortly.
The difficulty with the RPZ:
As mentioned above, the RPN is determined from the severity and P1 and P2 (in some cases only P if the probability is considered combined).
In some FMEAs, RPN values in the range of RPN = 10 * 10 * 10 = 1000 are obtained. A risk acceptance assessment of < 100 and >= 100 is often used. But how can this be justified? In most cases, evidence is lacking, and the rating appears arbitrary, or the rating is used to determine the maximum number of actions to be implemented, which does not fit the "As Low As Possible" principle.
ISO 14971, on the other hand, provides a very good method for creating a risk matrix, as usual. The risk assessment broken down into categories of severity and probability of occurrence is based on the data available in risk management and clinical evaluation.
This data can be:
- Standards
- scientific and technical data;
- Market data of similar medical devices already in use, including published incident reports
- Usability studies with typical users
- clinical data
- Results of appropriate investigations
- Appraise
- external quality assessment programs
We also see our assessment reinforced by the explanation in the appendix of IEC 62304:
“The risk management of software is part of the risk management of the entire medical device and cannot reasonably be considered in isolation.”
From the previous points, we conclude that all entries in an FMEA—that is, potential failures and their associated causes—are considered valuable, regardless of the RPN. Therefore, the FMEA serves only to identify such elements and treats them equally. No quantitative assessment is performed.
Note: This is a subjective assessment from development projects and does not claim to question the use of FMEA in general. In some other areas, such as production, a completely different perspective is applied. Those who have had positive experiences with it in their projects and audits should stick to their strategy. However, we hope this article provides food for thought and sparks some discussion.
Example of a possible documentation of a software risk management process
The way in which documentation is carried out depends essentially on the underlying quality management system, development and software development process as well as the risk management process.
1. Overview of the necessary documentation in connection with SW Risk Management
2. Risk Management File / Risk Analysis Table
The question remains how often we apply the above-mentioned scheme. Should we consider the threats for each software unit or only once for the entire software system? The answer to this question depends on the software architecture. If my system runs on a single piece of hardware and the architecture is accordingly straightforward, we would not recommend splitting software risk management across multiple components. After all, the measures are generally identical, provided the software security classes are the same.
However, if the software system spans multiple, more complex software systems on different hardware, it is recommended to consider these components separately. After all, in such an architecture, the same software errors can result in different effects.