When software meets quality (s-metrics) – Part 1

An overview of common software verification metrics from a developer's perspective

Can software quality be measured?

Software quality is crucial, especially in safety-critical areas such as medical technology or automotive. Especially when software errors can harm people, development processes are needed to prevent errors when writing software and to uncover errors that have already been made. Therefore, the desire to be able to measure software quality is a given. For this reason, software metrics are used. Not only to identify low-quality software components, but also to document the quality of the software. But can the quality of complex software systems really be expressed with just a few numbers? And what value does each metric need to reach to be truly "good"? In this article, we will examine some of the most common metrics and discuss not only how they work, but also their significance and usefulness in a quality management system.

Lines of Code (LOC)

LOC is certainly one of the most trivial metrics. As the name suggests, it represents the number of lines of code. This value can be related to a file or a function, for example. Depending on the tool used to calculate this value, different results can be obtained. It is important that the LOC calculation rule is consistent when comparing different software modules. For example, you can specify whether and to what extent comments are included.

Despite the simple calculation, it is worth keeping an eye on LOC for functions and files. Even if code with fewer lines is not necessarily better than code with a few more lines. If a developer likes to cram complex calculations into one line, it can be considerably more confusing than if they perform the corresponding calculation on several lines. However, it should be clear that code reviews become significantly more difficult for software modules above a certain length. It can therefore make sense to set LOC guidelines in the coding guidelines. How high these values should be set depends on many factors, such as the programming language used. Furthermore, this value can only fulfill its purpose if it is actually checked. For example, as part of a code review or by choosing suitable tools for recording software metrics.

Examples of a limit value are:

LOC per method: 100

LOC per file: 1000

The only question is, what should you do if this value is exceeded? It depends on the specific cause:

• Software architecture: If source files become too large, this can be due, among other things, to the software architecture. If too many functions are assigned to certain modules, they can quickly become bloated. For this reason, it's worthwhile to continually review the architecture, even during development, and adapt it if necessary.
• Coding guidelines: Another reason for high LOC can be the coding guidelines. Either they don't clearly specify how the programmer should write their code, or the guidelines aren't applied. It's also a good idea to regularly review and adapt these guidelines. Furthermore, code reviews should also ensure compliance. The development process should check the status and quality of the design outputs at appropriate points. Phase or milestone reviews should ensure that the software metrics have been recorded and verified against their acceptance criteria.
• Personal programming style: Another reason is simply the programmer's style. Everyone knows those brilliant programmers who are capable of handling extremely large source files, at least if they wrote the code themselves. The problem often only becomes apparent when source files are revisited years later, after the author either no longer works for the company or has completely forgotten what they wrote. Therefore, it is important that even the geniuses among programmers adhere to code size guidelines so as not to make things unnecessarily difficult for their colleagues in the future. Therefore, it is important to train employees accordingly and monitor compliance through regular reviews.

Coverage / Test coverage

Software test coverage is certainly one of the most frequently used metrics for software verification. However, its significance is repeatedly questioned. For this reason, I already discussed test coverage in a previous blog post, which is why I don't want to go into too much detail here. In summary, the coverage of source code by unit tests does not allow any conclusions to be drawn about the quality of the tests or the software itself. Therefore, it only makes limited sense to set a threshold here. However, a very low value can be an indication that the code should be examined more critically. It is possible that something important has actually been overlooked during testing. However, if the coverage is really so low because the untested part is simply not critical, you should learn to live with low numbers.

However, if you really want to set a limit here, I would set it at no more than 60 % establish.

Click here for the article about test coverage

Cyclomatic complexity according to McCabe

In general, highly complex code is harder to master than low-complexity code. Firstly, highly complex code requires more testing. Secondly, complex code is generally harder to read, which reduces the likelihood of finding errors during reviews. The McCabe metric is one way to calculate the complexity of functions. This sounds similar to the LOC metric, so you might think of setting a guideline here as well. The problem with McCabe, however, is that high complexity doesn't necessarily have to be a bad thing. There are often good reasons why code is complex. Also, code with a high McCabe score isn't necessarily less readable than code with a low score.

However, the metric can provide an indication of which code sections require special attention during reviews. However, if you set a maximum value here, you shouldn't be surprised if this value is regularly exceeded without any action being taken.

Example of a common limit is a complexity of 15

Nesting depth of branches

By nesting branches like if and switch Many functions can be implemented with just a few lines of code. Nesting is generally not a bad thing. Alternatives to avoid nesting include:

• Many functions that differ only in a few lines
• Branching out into functions

Either way, with lower nesting depths the number of functions and thus the size of the code increases. Without increasing its complexity. Since even software with many very simple functions can be difficult to test, it can make sense to nest some branches. However, it can be assumed that beyond a certain depth the number of possible function iterations will be almost impossible for a reviewer to understand. The trick here is to find a trade-off between clarity and code size. Measuring an average nesting depth probably makes no sense. It makes more sense to specify a maximum value in the coding guidelines. If this maximum is exceeded, you should check whether it makes sense to rewrite the code.

A common guideline for the maximum nesting depth is 4 proven.

Conclusion

From a programmer's perspective, I find it difficult to rely solely on metrics when evaluating software quality. Very few software systems are similar enough to one another for a comparison based on software metrics to be truly meaningful. Furthermore, adhering to software metrics limits doesn't say much about how well a software is implemented. Nevertheless, I don't want to speak negatively about metrics in general. Incorporating software metrics into the development process can certainly have advantages. "Measuring" your own software can motivate programmers to improve their code. It can also help identify more complex software modules and determine which parts of the software receive special attention during implementation and design reviews. Software metrics do, therefore, offer a certain benefit, but one that shouldn't be overstated.