Enhancing Software Change Impact Analysis
A comprehensive test suite is part of modern software development best practices. Unit tests, integration tests, end-to-end tests, and many others make sure your system keeps working when you need to change the implementation.
But over the lifetime of a software project, you can end up with hundreds of such tests, and every test you add can gradually slow down your CI/CD pipeline. You have high standards for software quality, but with all these tests, your development velocity goes down the drain. How can you get both: quick changes and high quality?
How Does Change Impact Analysis Help?
Change Impact Analysis is one solution to this problem. The goal of it is to minimize the number of tests required to run for a specific change. When you have a massive suite of tests, change impact analysis helps you organize and track which test impacts which source files. Later, when you change one or more of these source files, you can take the change impact analysis list and check which parts of your test suite are impacted by the changes you made.
With change impact analysis, you only run the tests that matter, save time in your CI/CD pipeline, and get results quickly.
Implementing Change Impact Analysis for Node.js
Let’s find out how we can set up change impact analysis for Node.js. For this project, we’ve created an example repository on GitHub that you can clone locally and use as a reference.
This example project is an API that uses the Express framework. It consists of three routes, each having its file and four test files—one for every route and one with tests for two routes.
The unit tests are implemented with the Jest testing framework, which supports code coverage output. This output allows us to check which test file affects which source file.
Running all tests
This is the default behavior, and in our example, this is no problem. But, over time, it could grow slower. That’s why we have to use change impact analysis to find out which test file is concerned with which source file.
Analyzing Change Impact
To analyze change impact, we have to complete the following steps:
- Check which test files exist
- Run each test file on its own
- Check the resulting code coverage of each test run
- Write down which source files the test touched
This is the code for the Jest configuration:
The script gets all of the test files and loops through them. Each loop iteration executes Jest with just one test file and reads the disk’s coverage report. Next, the script adds an entry to the change impact analysis list that contains the test file name and the corresponding source file names. Before writing the source file
names, it transforms the file’s absolute path into a relative one.
Finally, the script writes the change impact analysis list as JSON to disk, so a future execution of the test suite can use it as a filter. The list looks like this:
Running Only Impacted Tests
The idea here is that the script will get a list of changed files from the CI/CD pipeline as an argument. It will then check which tests cover the files and pass the tests to the Jest library. This way, only tests that are related to the changes will be executed.
If we look at the GitHub Action definition, we can see where the changed files come from.
To run the tests, make a change inside a file in the src directory, commit it, and push it to GitHub. GitHub Actions will start automatically right after the push.
Getting More Insight with Foresight
First, create a Foresight account and connect your pipeline by simply installing the GitHub app. After installing the app successfully, you will see the project creation and repositories screen. You should name your first project and select the repositories you want to monitor.
In order to gain insights about our tests such as grouping tests, test suites along with their logs, screenshots, and more to understand why even the most complex integration test failed; we need to update our YAML file with Foresight's report uploader step and troubleshoot our test failures easily.
We can change the GitHub action for our tests to use the Foresight integration.
That’s it. Now we can change a file in the src directory again, commit it, and push it to GitHub to see if everything is working.
It will take a few minutes to get the results in Foresight, but after that push, you can go to the Foresight web console, open your project, and wait for the results.
Foresight provides full visibility and deep insights into the health and performance of your tests and CI/CD pipelines. You can assess the risk of changes, resolve bottlenecks, reduce build times, and deliver high-quality software at speed.
Big test suites give us peace of mind when modifying our code, but they also slow down the delivery of new releases—the bigger the test suite, the slower the pipeline. With change impact analysis, we have a solution to that problem. If we only run tests related to our changes, we can save time while still checking the crucial parts of our code. Sign up for Foresight to improve your test runtimes and gain insight into your tests.