Grafana for GitHub Actions Workflows

The observability dashboard for GitHub Actions Workflows

Intro

Responding to the consequences of any changes in DevOps culture is momentous. When you take a panoramic sight of the process of shipping software starting from development until production, metrics such as DORA metrics and the like aim to lead you to a defect-free and fast software delivery.

The foundational components of software development in DevOps and agile are Continuous Integration (CI) and Continuous Delivery (CD). In practice, organizations mostly focus on the CD part of the software development since it is directly related to production and has direct impact on the end users.

Unfortunately, defining and measuring success criteria for CI pipelines often take a backseat. The substantial importance of measuring the health and performance of CI pipelines in software development projects are usually put on the back burner.

Observability tools like Foresight help you keep close tabs on the health and performance of your CI pipelines and tests. If you build your CI pipelines on GitHub Actions, you might need an additional application to keep track of the success status, duration, and cost of your workflows. Cı observability might be critical in many cases that workflows fail.

It's impossible to improve what you can't measure!

Firstly, we started off with the need of monitoring the necessary metrics to keep track of the health and performance of our CI workflows. Management wanted to take healthy decisions, engineering wanted to know what goes right and what goes wrong immediately, Devops wanted to improve the performance.

We needed to ensure that all stakeholders  such as developers, DevOps engineers, SREs, and management have across-the-board visibility into the software delivery process. We understood that this is a common challenge for software organizations. The reason why we needed to have workflow analytics to identify issues and enhance organization-wide visibilty on our workflows is a common requirement.

We had to answer some of the questions below while trying to be well informed about the performance of our CI pipelines.

• How many workflows run in a day/week/months and which ones fail?
• What are the top10 erroneous workflows?
• Which workflows are the most active ones and how successful they are?
• What is the total monthly cost of our workflows?
• What are the most costly workflows?
• What is the average workflow run duration?
• What are the slowest workflows?

By seeing the answers to these questions at a glance, we could be much more eligible to answer the below questions;

• Which workflows fail frequently and need to be optimized?
• What is the duration/success/cost trend in my workflows?
• Do I need to scale my CI up or down?
• Where do we need to start taking actions about healt and optimization?

After working on some use cases, we decided to group our metrics under status, cost, and duration. Because these were the most important questions to answer.

Grafana is a great tool for building dashboards for almost anything. The variety of its use case is huge; you can build both a server monitoring and weather forecasting dashboard. This variety comes with its own challenges. Building such kinds of dashboards requires a substantial development power. Even though you can use Grafana Cloud to remove the burden of setting up the environment, you need to set up APIs and make meaningful reports and charts out of it.

Stay on top of your metrics

Since developing a dashboard with Grafana for our GitHub Actions CI workflows is pretty grueling to create and manage, we built Foresight Highlights and made it free for open source projects to help the community.

As mentioned at the beginning of the article, responding to the consequences of any changes is momentous. Therefore we built Foresight Highlights in an optimized way to allow you to identify the health and performance of your CI workflows in the quickest way possible. Please see the below introduction video for Foresight Highlights.

With Foresight Highlights you don’t need to take the hassle to create and manage Grafana dashboards. Just use it off the shelf to keep track of your CI metrics organization wide.

Quick actions

Quick actions is one of the fastest way to pinpoint your failed and problematic workflow runs. It helps you be notified about your latest runs’ status and more. With quick actions, you can have an overview of the CI performance of your whole organization.

Status

Status section helps you to keep an eye on success and failures, and track how your workflows are performing. Foresight gathers your workflow runs an groups them according to the success, failure and other status information by their dates.

Foresight uses heatmap to visualize your failure trend. Foresight groups failed runs by their execution dates and shows workflow behavior.

Cost

In Foresight Highlights, you see an estimated cost of your workflows. You can detect the most costly workflows. After gathering your workflow run metadata, Foresight combines those information with GitHub's pricing plan.

Duration

Reveal how long it usually takes for your workflow to complete successfully, gather insights to optimize the workflow's duration, and assess the duration trend. 

Foresight calculates the average duration among the successful workflow runs. Statuses other than successful runs may take less or longer than the expected. Therefore they are not inclued in the average duration calculation.

Outro

When we shifted our CI environment to GitHub Actions, we noticed the observability gap there. After speaking to hundreds of GitHub Actions users and understanding their needs and requirements, we decided to introduce a dashboard which solves the common visibility problems.

Foresight “Highlights“ is just like an off-the-shelf Grafana dashboard that helps you identify the issues and enhance organization-wide visibility on your GitHub Actions CI workflows. Foresight users embraced Highlights as a single source of truth when there is an issue with their CI pipelines.

For more information about configuring Foresight, please refer to our docs. You can submit your requests to support@runforesight.com or come to our Discord community for instant chat. You can sign up for Foresight or see the live demo to see Foresight in action.