Code reviews in medical technology

To review code or not to review code? That is the question. There are various approaches to ensuring software quality: static code analysis, automated unit tests, integration tests, smoke tests, and many more. These points are state of the art in today's software development and are indispensable in daily work. However, the medical technology community is divided on code review. Hardly any other testing method is as controversial. Yet, the code review method is often undervalued.

“Talk is cheap, show me the code.”

Linus Torvalds

What does IEC62304 say about this?

IEC 62304 (Medical Device Software – Software Lifecycle Processes) does not require code review. Instead, the software must be assigned a safety classification. Based on the safety classification, you can then determine which testing methods should be used. It is permissible to segregate software components through architectural decisions, thus separating safety-critical from non-safety-critical parts. This reduces effort and is entirely justified. It often leads to code review being the absolute "top method" limited to software of safety class C, while other modules of classes A and B are ignored. After all, less needs to be done for A and B than for C. The question here is whether this decision makes sense. Even if software is incapable of killing or injuring someone, its misbehavior can still have unpleasant consequences.

What are the benefits of code reviews?

• Errors are found early in development (in the programming phase).
• Misunderstood product requirements are identified.
• Developers clean up code that is due for review.
• External experts see things that the developer no longer sees.
• Errors in system testing result in a very high level of effort: finding errors, locating errors, implementing changes, and retesting by making changes (e.g., system tests must be repeated).
• System tests are not exhaustive. It's not possible to test all cases. Therefore, it's better to find error cases through code review.
• A documented code review gives the developer a good feeling and ensures that progress is visible (specification, implementation, review, test, finished, next module)
• Knowledge within the company is growing. Developers are getting better and learning from each other.
• Code maintainability is better when more than one person understands the code. It's also easier for others to learn the code.
• This creates a company-wide standard at a high level.

What speaks against a review?

• There's no more time in the project. Reviews are scheduled too late. The software is supposed to be finished, and now there are reviews that require 100-200 hours of effort. Nobody wants to spend that much time.
• Some modules or components (e.g., Linux) are far too large and it would be uneconomical to review them. This is indeed a good argument against review. On the other hand, it must be clear that this will result in additional effort elsewhere. Therefore, testing must be conducted. With SOUP/OTS (source of unknown provenance/off-the-shelf software), risk management is also required.
• It's just Class A or B software. I strongly advise against this argument! It's very difficult to rule out the possibility that Class A software can also lead to misbehavior. And it's not just about not hurting anyone. The goal is to efficiently develop a good product.

What are my experiences?

• More than 30% of code reviews detect critical errors that can lead to misbehavior in the field.
• If planned and implemented correctly, a complete code review of a class C software requires approximately 5-8% of software development time (based on experience from my projects).
• Young developers in particular benefit enormously from reviews.
• There is a great deal of dynamism and therefore fun in working with colleagues, discussing work results and showing what you can do.
• Many very good and experienced developers advocate for reviews.
• Omitting the review doesn't save any effort; it simply pays for it elsewhere. Potentially, at a higher price.

What else is important?

Developers who haven't yet caught the review bug tend to reject it. They see it as an exam situation, which creates stress. "No one should look at that, I'll take care of it..." This attitude doesn't make sense. The point isn't to put someone down or deny their competence. It's about developing a good product and doing excellent work technologically. Once this is clear, a flow emerges that leads to the ambition to develop software at a high level.

There must be freedom to adapt the review process. Some people need a printed copy of the source code before the review to take notes. Others absolutely need their editor because they can highlight terms throughout the document. Some need peace and quiet to think. Others want to talk about it. Therefore, it doesn't make sense to prescribe a procedure for all cases.

What is the absolute minimum goal?

Code reviews should definitely be performed in the following cases:

• Class C modules
• Complex modules
• Code from inexperienced developers (<3 years of professional experience or participated in less than 20 code reviews) as a kind of mentoring
• Code from external employees
• Code that makes developers feel bad

Consider which modules will be tested and how. Modules may be better tested through unit testing than through review. The test strategy and verification planning must be established early on. The methods must be effective!

Planning code reviews

I plan the code reviews based on the software architecture and the list of software modules. The software modules are listed in a table with their risk classification, and the testing method is defined. This definition is discussed with the software developer. The table below shows the basic structure.

Two approaches are used to estimate the effort required for code reviews. First, the developer estimates the effort for each module in the list. If a colleague is young or inexperienced, experienced colleagues should support the effort. This results in a bottom-up estimation. To check plausibility and accuracy, the effort required for a similar project is compared. This is the top-down estimation. These two values provide a good basis for a sound effort estimate.

Procedure for code reviews

The source code must be cleaned up beforehand to remove any obvious errors. The goal is not to focus on comments (e.g., Doxygen) or spelling mistakes, but rather on the code itself. It is also recommended to perform static code analysis beforehand and check the code to ensure it complies with your organization's coding guidelines.

The flowchart shows a possible approach.

Conclusion

Code reviews are a well-planned and early-stage method for avoiding errors and the associated effort in late project phases. It is recommended to use both a top-down and bottom-up approach when estimating. It is therefore useful to use effort from past projects for estimation. It is also useful to determine the appropriate testing method for qualification. Sometimes a code review is more cost-effective than another method because you simply have to read it.

This approach works and results in better code quality. Not only for the modules being reviewed, but also for the others, because a learning effect occurs and the errors are not repeated in other modules. Quality even improves beyond the project. The code reviews ensure consistent code.

I welcome feedback and would love for you to contact me. Feel free to leave a comment on the article. If you know someone who might also be interested in the blog, I'd be very happy if you would recommend it.

Best regards

Goran Madzar