Learn how to set up code coverage reporting for your own project. In this example we'll report to a channel within a team’s Slack.
February 9, 2024
Here at Headway, we’ve been aiming to better surface metrics from individual projects in a common place for awareness across the team. On the development side, one of these metrics is the amount of code coverage each project’s test suite currently has.
Code coverage gives us project health signals
While code coverage isn’t always a clear indicator of software quality, it can be a helpful pointer to understand a project’s health. High code coverage isn’t a guarantee that the right pieces are tested well, but low code coverage can be a sign that we lack a plan for reliability and maintainability.
Encourage testing practices
Also, we monitor our test coverage to help encourage good testing practices. Not every project needs to have the same percentage of coverage, but watching code coverage metrics can help us to make sure we have a usable and effective test setup for each project. If the configuration and libraries are properly in place, it can be easier to continue writing tests in the future. Without that, it can be a challenge to get started with other project demands at play!
Example of reporting in Slack
For visibility, we report our code coverage to a channel within our team’s Slack. Each update looks something like this:
Let’s walk through how to set this up for a project!
Creating a Slack workflow
First, you’ll need a Slack workflow that can receive events and post messages. I’ll show an overview, but you can follow Slack’s guide for webhooks for more detail.
To start, you’ll want to create a workflow that responds to a webhook. One way to open the workflow builder is to open the settings for a Slack channel, click over to the Integrations tab, and click “Add automation”.
Add two variables to this initial web request step: one for the name of the application (appName) and one for the percentage of code coverage (percent):
Next, add a step to your workflow to post a message.
Select the channel you’d like your messages to appear in and then customize the message text using the variables from the previous step:
When you’re all done, it should look something like this:
Once your workflow is all set, you will need to “Publish” it from Slack’s menu to make it available and ready to use.
Reporting code coverage from GitHub actions
We often use GitHub actions for running our test suite. Other CI tools have their own configuration, but the examples below may still be a helpful starting point.
Depending on your language’s coverage reporting tool, it’s generally straightforward to gather the percentage of code coverage and send that over to Slack.
Here are a few examples steps for adding coverage reporting to your GitHub actions workflow YAML file on merges into main:
Find the final coverage percentage
First, find where your coverage reporter stores the final coverage percentage after test runs. Gather this number and then store it in the output for the given step. We conditionally run this only on merges into main so that we don’t have messages for in-progress PR work that may not reflect the full state of the merged code.
For Coveralls (Elixir), you can gather this from the output of mix coveralls with steps along these lines: