Automated Tests in Software Development

Software testing is the process of executing a program with the intent of finding errors and defects and verifying that the software meets the expected quality standards. Tests can be automated using specific software developed for the purpose of controlling the execution of an application, comparing actual outcomes with the outcomes expected by the developer.

Unit tests and integration tests are different kinds of automated tests and a good automated test suite should cover them both, together with functional tests ensuring that the software “works” from the user’s perspective. In this blog, we will see more in detail how unit tests ensure that each individual software component works as expected while integration tests ensure that the different components interact as expected.

From waterfall to continuous delivery

In the past, software development followed the waterfall model[1]. The name “waterfall” depicts very well the idea behind this methodology in which the different steps (requirements gathering, design, implementation, verification, maintenance) are executed one at a time so, in a time chart, they appear like a waterfall cascading from one task to the following one.

In software development, this “clean” workflow is hardly applicable: change requests and better understanding of the project scope usually emerge after the delivery and this leads to quick-and-dirt fixes applied without going through the entire process.

            Image  1. The Waterfall model[2]

Conversely, continuous delivery[3] is a methodology that encourages incremental improvements to the software in short cycles, allowing to release changes to production in days rather than months. To keep the delivery process smooth while releasing in short cycles, automated tests are a fundamental tool for the software developers: they catch errors before the software is released, help developers to make changes with no fear of breaking something in the meanwhile and, since the cost of a bug discovered in production is greatly higher than the cost of a bug discovered during development, they have a great impact on the project as a whole.

As a research & innovation project, SOFIE is a perfect example of a context where the cascade model is not applicable: the scope of a research project evolves with findings from the project itself, and the usage of new technologies (like blockchain) requires an additional level of flexibility. For this reason, it’s more important to respond to changes over following a plan or, as software developers like to say, be more “agile”[4].

To respond quickly to the evolving requirements while keeping the software quality high, in our project’s CI/CD environment[5] unit tests are typically executed during the development phase (continuous integration) while integration tests are executed before the continuous delivery phase in the staging environment, and both act as a defence line against software bugs in production.

Unit tests

In unit tests, “units” are single modules or even single functions inside a module to be tested to ensure that individual components work. A good suite of unit tests is, for a developer, a jack of all trades providing out-of-the-box help in API design, feature documentation, and automated quality assurance. They are executed frequently during development, often on every file change, and so provide real-time feedback to developers. To do so, unit tests must run extremely quickly and provide a clear bug report indicating, in case of failure, the component being tested, the expected behaviour, and the actual result.

            Image  2. The test driven development[6]

Integration tests

Integration tests ensure that the various “units” work together and that the interaction with 3^rd party services and APIs is correct. Some examples could be a logger component logging the inbound connections or an API that writes to a database.To do so, integration tests usually require asynchronous operations relying on network connections, file I/O, or database access. This can lead to long running times when there are lots of features being tested and, in such cases, a complete run of a test suite can take several minutes. For this reason, it is important to keep integration tests separate from unit tests in order to keep the execution of unit tests as quick as possible.

Conclusion

In this post, we saw how each type of automated test has its own role in modern software development and how this contributes to the overall software quality. Unit tests provide quick developer feedback, while integration tests cover the interaction among components. A good test suite should include both, together with automated functional tests ensuring that everything works as expected for the end users.

[1] https://www.oxagile.com/article/the-waterfall-model/

[2] https://commons.wikimedia.org/wiki/File:Waterfall_model.svg

[3] https://devops.com/continuous-delivery-pipeline/

[4] https://agilemanifesto.org/

[6] https://upload.wikimedia.org/wikipedia/commons/0/0b/TDD_Global_Lifecycle.png