Skip to main content

Heroku CI and Github Checks Integration

Subrat Thakur
Subrat Thakur
Dec 16 - 5 min read

Before we dive into the crux of this article, let’s first get an understanding of what GitHub Checks is and how it will be useful for you when you use any external Continuous Integration (CI) tool like CircleCIHeroku CI, or any local tool. The Checks functionality enables integrations to report more than just binary pass or fail to build statuses. Checks integrations can report rich statuses, annotate lines of code with detailed information, and kick off reruns. [Read more in the GitHub Checks Developer Guide.]

In this article, we’ll talk specifically about GitHub Checks integration with Heroku CI, but the same method can be used with any CI tools that expose APIs to fetch CI logs. (Note that some well-known CI tools like CircleCIand Jenkins have built-in or external plugins for integration, so for them you wouldn’t have to follow this method unless you want to customize the Checks output.)


This article will help you to set up a Probot app as an integration tool between Heroku CI and GitHub Checks using GitHub APIs. The first thing you might wonder is “Why is this integration needed? Can’t we just use Heroku App to see all the CI logs?

Well, you can use the Heroku app to see Heroku CI logs — but consider a situation where people are contributing to your GitHub repo who don’t have access to your Heroku Pipeline, such as in the case of an open source project with many contributors. In that case, if an automated test, lint, or other CI process fails for their commits, they will only see the boolean result, not the complete reason. With a GitHub Checks integration, they would be able to see the full result on the GitHub Checks tab, which would be very helpful for remediating the issue.


How does it work?

GitHub Webhooks: As per GitHub Docs, “Webhooks allow you to build or set up integrations, such as GitHub Appsor OAuth Apps, which subscribe to certain events on When one of those events is triggered, an HTTP POST payload is sent to the webhook’s configured URL. Webhooks can be used to update an external issue tracker, trigger CI builds, update a backup mirror, or even deploy to your production server. You’re only limited by your imagination.”

Heroku CI: When a test or build gets triggered for a new git commit, it broadcasts the webhook events for Status. The event gets triggered for different stages like in_progress, success, failure, cancel, etc. GitHub catches those events and updates the pull request page build flag. For our scenario, we intercept statusin our integration app.

DEBUG probot: Webhook received
event: {
"event": "status",
"id": "da6ec6b8-2250-11eb-8843-f0563e0c2987",
"installation": 12779415,
"repository": "<Github-owner>/<Repo-name>"

Status (webhook event): When the status of a Git commit changes, the type of activity is specified in the action property of the payload object.

Probot Integration App: We created a Probot intermediate app that basically intercepts the GitHub webhook events and then executes specific logic as per our requirement. You can catch status events with the following code snippet:

app.on('status', async (context) => {
console.log('Status event update');

The “context,” in this case, is an object with details related to issues, pull request, repository, actions, etc. You will also find a related CI URL with context.payload.target_url. You can get the state of CI build / test using context.payload.state.

Once you have the CI target URL, you can get logs directly from it or you can create a test log URL from it that will return a simple text log in response. For example, in Heroku CI, you will get a target URL from the context which will be in the format of<pipeline-id>/tests/<test-run-number>

The URL will look something like this, with your specifics:

And to get a plain text log, you can convert this URL to a different URL format which will be something like<pipeline-id>/test-runs/<test-run-id>. Here’s an example:

If your Probot app receives a Statusevent with state in_progress, it will create a GitHub Checks run using Checks API with the status “In Progress.” You can do that with the following code:

//Configure app to intercept Status event
app.on('status', async (context) => {
    // Check the state of 
    if(context.payload.state === 'pending') {
        //Create check runs params
        const createCheckParams = {
        name: context.payload.commit.sha,
        owner: context.repo().owner,
        repo: context.repo().repo,
        head_sha: context.payload.commit.sha,
        status: 'in_progress',
        started_at: new Date().toISOString(),
        output: {
            title: 'Heroku CI Report!',
            summary: `Please wait ! Heroku CI is validating the content. This space will be updated soon with report! \n > You can check live status here : [Heroku CI](${context.payload.target_url})`
        // Create new run with Params
        await context.github.checks.create(createCheckParams)

status.ts hosted with ❤ by GitHub

Next, if your Probot app receives a Statusevent with the state failuresuccessor canceled, it will update the GitHub Checks run using the Checks API with the status “Received.” You will fetch all the checks run for a specific commit and then update the checks run (which has the status “In progress”) with the latest status.

//Configure app to intercept Status event
app.on('status', async (context) => {
    // Check the state of 
    if(context.payload.state === 'canceled' ||
        context.payload.state === 'error' ||
        context.payload.state === 'success') {
        //Flag to ensure if any checks get updated
        let checkUpdated = false
        const createCheckParams = {
            name: commitSha,
            owner: context.repo().owner,
            repo: context.repo().repo,
            head_sha: commitSha,
            check_run_id: 0,
            conclusion: determineCheckConclusion(context.payload.status),
            status: determineCheckStatus(context.payload.status),
            completed_at: new Date().toISOString(),
            details_url: context.payload.target_url,
            output: determineCheckSummaryWithTitleAndSummaryLog(),
            actions: []
        //Fetch all the checks for specific commit 
        const checkRunList = await getCheckListForCommit(commitSha, context)
        if ( > 0) {
   (check: any) => {
            if (check.status === 'in_progress') {
                createCheckParams.check_run_id =
                checkUpdated = true
                await context.github.checks.update(params)
        // If no in progress check run get updated then we create a new check run
        else if (!checkUpdated) {
            await context.github.checks.create(params)

EventHandle.ts hosted with ❤ by GitHub

And we are all set.

If you want to improve your user experience, there are a couple of additional things that can be tweaked in our integration implementation:


In order to keep all information related to pull requests and commits in one place, it’s good to use various GitHub apps properly. That way, you don’t have to worry whether external collaborators have access to all of your tooling or not, and you don’t have to make your CI tool accessible to everyone. If you’re not using GitHub Actions, but you are using Heroku CI or another CI tool, then this integration with GitHub Checks will prove very useful.

To see the complete code for the integration, visit this git repository: SubratThakur/Heroku-Github-Checks-Integration.


Related DevOps Articles

View all