Before we dive into the crux of this article, let’s first get an understanding of what GitHub Checks is and how it will be useful for you when you use any external Continuous Integration (CI) tool like CircleCI, Heroku CI, or any local tool. The Checks functionality enables integrations to report more than just binary pass or fail to build statuses. Checks integrations can report rich statuses, annotate lines of code with detailed information, and kick off reruns. [Read more in the GitHub Checks Developer Guide.]
In this article, we’ll talk specifically about GitHub Checks integration with Heroku CI, but the same method can be used with any CI tools that expose APIs to fetch CI logs. (Note that some well-known CI tools like CircleCIand Jenkins have built-in or external plugins for integration, so for them you wouldn’t have to follow this method unless you want to customize the Checks output.)
Introduction
This article will help you to set up a Probot app as an integration tool between Heroku CI and GitHub Checks using GitHub APIs. The first thing you might wonder is “Why is this integration needed? Can’t we just use Heroku App to see all the CI logs?”
Well, you can use the Heroku app to see Heroku CI logs — but consider a situation where people are contributing to your GitHub repo who don’t have access to your Heroku Pipeline, such as in the case of an open source project with many contributors. In that case, if an automated test, lint, or other CI process fails for their commits, they will only see the boolean result, not the complete reason. With a GitHub Checks integration, they would be able to see the full result on the GitHub Checks tab, which would be very helpful for remediating the issue.
Architecture
How does it work?
GitHub Webhooks: As per GitHub Docs, “Webhooks allow you to build or set up integrations, such as GitHub Appsor OAuth Apps, which subscribe to certain events on GitHub.com. When one of those events is triggered, an HTTP POST payload is sent to the webhook’s configured URL. Webhooks can be used to update an external issue tracker, trigger CI builds, update a backup mirror, or even deploy to your production server. You’re only limited by your imagination.”
Heroku CI: When a test or build gets triggered for a new git commit, it broadcasts the webhook events for Status. The event gets triggered for different stages like in_progress, success, failure, cancel, etc. GitHub catches those events and updates the pull request page build flag. For our scenario, we intercept statusin our integration app.
DEBUG probot: Webhook received
event: {
"event": "status",
"id": "da6ec6b8-2250-11eb-8843-f0563e0c2987",
"installation": 12779415,
"repository": "<Github-owner>/<Repo-name>"
}
Status (webhook event): When the status of a Git commit changes, the type of activity is specified in the
actionproperty of the payload object.
Probot Integration App: We created a Probot intermediate app that basically intercepts the GitHub webhook events and then executes specific logic as per our requirement. You can catch status events with the following code snippet:
app.on('status', async (context) => {
console.log('Status event update');
}
The “context,” in this case, is an object with details related to issues, pull request, repository, actions, etc. You will also find a related CI URL with context.payload.target_url. You can get the state of CI build / test using context.payload.state.
Once you have the CI target URL, you can get logs directly from it or you can create a test log URL from it that will return a simple text log in response. For example, in Heroku CI, you will get a target URL from the context which will be in the format of
https://dashboard.heroku.com/pipelines/<pipeline-id>/tests/<test-run-number>
The URL will look something like this, with your specifics:
https://dashboard.heroku.com/pipelines/1fa8b9e6-1183-40f6-a66e-701b924b4d90/tests/259
And to get a plain text log, you can convert this URL to a different URL format which will be something like https://api.heroku.com/pipelines/<pipeline-id>/test-runs/<test-run-id>. Here’s an example:
https://api.heroku.com/pipelines/1fa8b9e6-1183-40f6-a66e-701b924b4d90/test-runs/259
If your Probot app receives a Statusevent with state in_progress, it will create a GitHub Checks run using Checks API with the status “In Progress.” You can do that with the following code:
//Configure app to intercept Status event
app.on('status', async (context) => {
// Check the state of
if(context.payload.state === 'pending') {
//Create check runs params
const createCheckParams = {
name: context.payload.commit.sha,
owner: context.repo().owner,
repo: context.repo().repo,
head_sha: context.payload.commit.sha,
status: 'in_progress',
started_at: new Date().toISOString(),
output: {
title: 'Heroku CI Report!',
summary: `Please wait ! Heroku CI is validating the content. This space will be updated soon with report! \n > You can check live status here : [Heroku CI](${context.payload.target_url})`
}
}
// Create new run with Params
await context.github.checks.create(createCheckParams)
}
}
Next, if your Probot app receives a Statusevent with the state failure, successor canceled, it will update the GitHub Checks run using the Checks API with the status “Received.” You will fetch all the checks run for a specific commit and then update the checks run (which has the status “In progress”) with the latest status.
//Configure app to intercept Status event
app.on('status', async (context) => {
// Check the state of
if(context.payload.state === 'canceled' ||
context.payload.state === 'error' ||
context.payload.state === 'success') {
//Flag to ensure if any checks get updated
let checkUpdated = false
const createCheckParams = {
name: commitSha,
owner: context.repo().owner,
repo: context.repo().repo,
head_sha: commitSha,
check_run_id: 0,
conclusion: determineCheckConclusion(context.payload.status),
status: determineCheckStatus(context.payload.status),
completed_at: new Date().toISOString(),
details_url: context.payload.target_url,
output: determineCheckSummaryWithTitleAndSummaryLog(),
actions: []
}
//Fetch all the checks for specific commit
const checkRunList = await getCheckListForCommit(commitSha, context)
if (checkRunList.data.total_count > 0) {
checkRunList.data.check_runs.forEach(async (check: any) => {
if (check.status === 'in_progress') {
createCheckParams.check_run_id = check.id
checkUpdated = true
await context.github.checks.update(params)
}
})
}
// If no in progress check run get updated then we create a new check run
else if (!checkUpdated) {
await context.github.checks.create(params)
}
}
}
EventHandle.ts hosted with ❤ by GitHub
And we are all set.
If you want to improve your user experience, there are a couple of additional things that can be tweaked in our integration implementation:
- Add the annotations object as part of the output in check run params to help to show errors and lint details for each commit file. (See this article for more details: https://developer.github.com/changes/2019-09-06-more-check-annotations-shown-in-files-changed-tab/)
- Display a processed log for better presentation in Checks Tab.
Conclusion
In order to keep all information related to pull requests and commits in one place, it’s good to use various GitHub apps properly. That way, you don’t have to worry whether external collaborators have access to all of your tooling or not, and you don’t have to make your CI tool accessible to everyone. If you’re not using GitHub Actions, but you are using Heroku CI or another CI tool, then this integration with GitHub Checks will prove very useful.
To see the complete code for the integration, visit this git repository: SubratThakur/Heroku-Github-Checks-Integration.
