CI/CD Insights and Analytics

Hands on troubleshooting CI workflows and tests

With Foresight's metrics, traces, and logs; we can quickly identify the problematic workflow, discover why it has failed, and get to work fixing the problem.
Serkan Ozal
6 mins read

The quality of your software product makes users choose it over the others. Among all the important components for gaining customers, when you need to retain them, it comes down to product quality. Product quality correlates directly to the tests you run. To maintain quality, we often implement a CI/CD pipeline and run automated tests after every build of the production code. In order to prove that the product meets user expectations, we also add end-to-end testing.

We’ll create GitHub workflows that we’ll then use to build the application i​n this article. Then we’ll run end-to-end tests for the application. Finally, we will apply Foresight, which offers capabilities for monitoring and troubleshooting workflow, ensuring that we can figure out problems quickly and save time during the debugging process.

Building the App

Let’s build an application that enables users to manage their favorite movies. Our service will be able to create new users and authenticate them, as well as add new movies, update movies, get movie content, and delete movies.

We’ll use mongoDB as the database for storage data. To make sure the service can scale, we’ll use the cloud MongoDB service in MongoDB Atlas, and we’ll create a free-tier cluster for MongoDB. Note that to be able to access the database from anywhere, we need to set the inbound rule for the MongoDB cluster.

Figure 1: Network access configuration for MongoDB Atlas

Atlas already generated the three types of connections needed for their users to connect the MongoDB cluster from the application.

Figure 2: Connection to MongoDB cluster options

Since we’ll need to connect to the cluster from our application, we’ll click on “Connect to your application,” then copy the connection string into our application code.

The config file in our application for the database will look like this:


Db.js
module.exports = {
    MONGO_CONNECT_URL:"mongodb+srv://{account}:{password}@{host-name}/{dbname}?retryWrites=true&w=majority"
};

The full details of how this application works are saved in a GitHub repository.

Now we’re ready to bring up our service with the following command:


node index.js

By default, the service will be brought up using port 3000. Let’s test the application to see whether our service will be able to create new users and add a new movie into the system with Postman.

Figure 3: Example Postman request for creating a new user

Note that in order to create a new user (Figure 3), we must set the Authorization header for the API.

Figure 4: Setting the Authorization header for the API request

We saw from Postman that we’re successfully able to create a new user. Now, let’s move on to create a new movie.

Figure 5: Adding a new movie using Postman

We were able to create that new movie, so it seems like our application works. Now we can move to the frontend.

Frontend of the Application

In order for regular users to be able to use and interact with our application, we need to create a frontend service. We’ll implement the user interface with the help of React in JavaScript. Details of the application’s frontend are available in this GitHub repository.

After implementing the UI, we can trigger the frontend by using the following command:


npm run start

The service will be able to access the graphical user interface via port 3001. For better management of the whole application, and to enable users to access the application and its API, we need a few more features.

Setting Up NGINX to Forward Requests

We will define the nginx.conf as shown below, so that  when users go to our web application, the requests will be forwarded into the right place for both frontend rendering and backend requests. The port for the NGINX server will be 7777.


http {

        ##
        # Basic Settings
        ##
        server {
        listen       7777;
        server_name  localhost;

        location / {
            proxy_pass http://127.0.0.1:3001;
        }

        location /posts {
            proxy_pass http://127.0.0.1:3000/v1/posts;
        }
    }
}

Navigate to localhost:7000, and you should be able to see the movies list in the homepage.

Figure 6: Homepage of our sample application

Setting Up Ngrok for Public Address of the Application

Now we’ll set up Ngrok to create a public address for our local IP address with port 7777.

First, we need to create the ngrok.yml file for defining the port to expose, as well as our Ngrok token.


authtoken: {ngrok-token}
tunnels:
  first:
    addr: 7777 // port number
    proto: http

Now we’re ready to expose our port for local addresses to the world.


./ngrok start all
Figure 7: Expose the web application to internet using ngrok
Figure 8: Navigating to the web application from the internet

E2E Test for the Application

Now that the application is up and running, we can write our end-to-end test using Jest (Node.js) with Selenium in the BrowserStack platform.

BrowserStack is a cloud solution that supports automated testing for multiple environments. For convenience, we’ll use their service for our end-to-end test, and we’ll use Selenium because it’s popular and mature. We also need a test runner. We’ll go with Jest because of their helpful features for structuring tests, and because it’s a popular choice for a Node.js testing solution.

Our sample end-to-end test is outlined in this Gist, and the full code for the test is available on GitHub.

Let’s run the test and see if it works.


npm run test

From the BrowserStack dashboard, we can see the test result along with a screenshot and video recording for our test.

Figure 8: BrowserStack video for the end-to-end test

In the video BrowserStack provided, we can see the details of what has happened in our test. If we scroll down, we’ll see that BrowserStack took a screenshot as we instrumented in our Selenium test.

Figure 9: BrowserStack screenshot showing the Selenium test

We’ve successfully executed the end-to-end test for our application. Let’s move on to create GitHub Action workflows for running our application’s continuous integration pipeline.

Creating a Workflow using GitHub Actions

In order to build the application and run the test for every new change automatically, we need to create GitHub Action workflows for the application. The workflows are also integrated with Foresight to make sure we can monitor our application efficiently in the future. We need three workflows for our application: One for the backend service, one for the frontend service, and one for executing the end-to-end testing. Let’s dive in and create just the backend service for today. (Details of workflows for frontend service and end-to-end testing can be found in their GitHub repositories.)

Note: To keep the demonstration simple, let’s assume that we already created the permanent web domain, instead of using a randomly generated domain by ngrok.

The workflow files for the GitHub Action are in YAML format and located in the .GitHub directory in our application. Below are the details of the GitHub Action workflow configuration file for the backend service.


name: Movie Management Workflow

on: [push]

jobs:
  build:

	runs-on: ubuntu-latest

	steps:
  	- name: Collect Workflow Telemetry
    	uses: runforesight/foresight-workflow-kit-action@v1
    	if: success() || failure()
    	with:
      	api_key: b983b8fb-e108-483a-bcd7-fdabdb16944a
  	- uses: actions/checkout@v3
  	- name: Use Node.js
    	uses: actions/setup-node@v3
    	with:
      	node-version: '16.x'
  	- run: npm ci
  	- run: node index.js &
  	- run: npm run test &
  	- name: Foresight test kit
    	if: success() || failure()
    	uses: runforesight/foresight-test-kit-action@v1
    	with:
      	api_key: b983b8fb-e108-483a-bcd7-fdabdb16944a
      	test_format: JUNIT
      	test_framework: JEST
      	test_path: junit.xml
      	coverage_format: COBERTURA/XML
      	coverage_path: ./coverage/cobertura-coverage.xml

Pushing this updated workflow to the GitHub repository will automatically trigger a new GitHub Action for our backend workflow. The workflow is now running successfully.

Figure 10: Our workflow is running automatically

When we look at the Foresight dashboard, we can see a new updated dashboard for our application workflow.

Figure 11: Overview of latest workflows have been executed

When we click on “LATEST WORKFLOW RUN”, we can see more details about recent workflow runs, such as the “Steps” have been executed in the workflow.

Figure 12: Steps that have been executed in the workflow

Click on the “Tests” tab. Here, we can see the change impact analysis of the recently updated code in the workflow.

Figure 13: Change impact analysis captured by Foresight

Clicking on “See Changes” shows us the details of codes that have been updated.

Figure 14: Details of the codes that have been changed

Navigate to the “Processes” tab. Here, we can see detailed information about processes, such as which process arguments have been executed.

Figure 15: Information about processes, such as arguments and their duration

In the “Metrics” tab, we can check how much memory or network has been used in the workflow.

Figure 16: Useful metrics for system resources have been used in the workflow

With all this information from Foresight, we can easily discover why the workflow has failed or debug performance issues in the workflow.

Monitoring & Debugging the Failed Test

In real-world projects, we frequently need to update frontend code in order to implement new features for our application. Let’s look at what happens if, after updating the frontend code for the new feature, one of our end-to-end tests fails. 

Figure 17: Failed workflow in GitHub Action

After checking the information in GitHub Action, we still do not know why the test failed. Let’s go to the Foresight landing page to find out what happened.

Figure 18: Finding out why the test failed, using Foresight

From Foresight’s logs, we can see that the test failed because of a mismatched title: “Mystic River” has been incorrectly rendered as “Mystic Rivera” somewhere in our application. Let’s check the GUI of the application to see how the movie titles are displayed.

Figure 19: Incorrect titles in home page of the movie application

We notice that all the titles have the letter “a” added to the end, which is weird. Let’s check the source code of the frontend repository.

Figure 20: Source code shows the root cause of the issue

Here, we notice there’s an extraneous “a” character in the title. Remove this letter, and our test is successful again.

Figure 21: The test runs successfully

With Foresight’s capabilities, we easily found and fixed the broken workflow.

Foresight for Proactive Workflow Monitoring

Monitoring and fixing the delivery pipeline is not a trivial task, since there are a number of steps in the deployment process and the complexity of the application tends to grow as time goes by. With Foresight’s capabilities, we are presented with carefully chosen metrics and logs so that we can quickly identify the problematic workflow, discover why it has failed, and get to work fixing the problem.

Sign up with Foresight and see just how easy monitoring workflows can be.