SW metrics from the QMS perspective
Why should software quality metrics be used in the product life cycle?
A software metric maps a software unit into a numerical value. The numerical value represents a measure of the internal quality of software. Software code metrics are used to quantify software properties as quickly as possible. This is done to track a quality measure throughout the product lifecycle and to compare programs using the same or similar programming language.
ISO 13485 and IEC 62304 form the cornerstone of the normative requirements for the development and maintenance of medical software. Depending on the software safety class, the requirements for the development and documentation of safety-critical software become more stringent. ISO 26262-6 applies to the automotive industry and sets out specific requirements for software verification. Tables 14 and 17, among others, list metrics at the software unit and architecture levels. IEC 62304 is less specific in this regard. Nevertheless, metrics can be used to fulfill additional acceptance criteria for software units (Chapter 5.5.4 IEC 62304: 2006) or as a baseline for regression testing and software release readiness. Furthermore, ISO 13485 and 21 CFR 820.250 are very specific regarding the use of statistical methods for assessing product characteristics. Here, software metrics offer a great opportunity to provide quantitative statements and also to quickly and effectively counteract negative trends in software quality.
Weaknesses of software quality metrics
Software metrics can only be meaningfully determined using appropriate tools. The operating effort, as well as any licensing costs, must be factored into the project plan. All tools used must be validated for their intended purpose. This effort must also be sensibly planned.
A single metric can never provide a complete statement about software quality. Metrics should always be evaluated in combination. The quality and maturity of software can only be definitively assessed in combination with static code analysis, code reviews, and functional tests. Software metrics should not be created at the end of development to satisfy the auditor, but rather should be used continuously throughout the product lifecycle to sustainably improve the readability and maintainability of source code.
To use
If the tools are validated and operational, software metrics can be easily integrated into the development and maintenance process. This helps, among other things, to detect potential bugs early on, rather than having to wait much later through laborious system-level debugging sessions. The investment in licensing costs can therefore pay for itself very quickly.
Key performance indicators are useful where a quantitative analysis of software properties is appropriate. This is the case for software properties according to ISO 25010 in the areas of maintainability, efficiency, security, and reliability. Other properties such as functionality, usability, compatibility, and portability can only be verified using appropriate testing methods.
Software metrics can be used as a tool in the field of data analysis (Chapter 8.4 ISO 13485:2016) to determine product characteristics and their trends. This provides the opportunity to counteract negative trends and, at the same time, to implement a general mechanism for improvement. Regulatory-required key performance indicators can be easily, objectively, and efficiently integrated into the process using software metrics.
The use and tracking of software metrics even after the product release can help make the code more robust, increase maintainability and overall contribute to customer satisfaction and secure software.
Literature, links and training
- Basics of Software Testing – Andreas Spillner and Tilo Linz
- Software testing for embedded systems – Stephan Grünfelder
- Basic knowledge of medical software – Christian Johner, Matthias Hölzer-Klüpfel and Sven Wittorf
- Certification: ISTQB Certified Tester
- Certification: Certified Professional for medical software
- Embarc, Architecture Spicker – quantitative analysis: (https://www.embarc.de/architektur-spicker/)