At Salesforce we regularly use our products and services to scale our own business. One example is Buildpacks, which we created nearly a decade ago and is now a part of Hyperforce. Hyperpacks are an innovative new way of using Cloud Native Buildpacks (CNB) to manage our public cloud infrastructure.
Buildpacks were created to help Heroku users focus on their code instead of the containers and infrastructure that run their apps. That may seem like a distinctly technical solution, but Buildpacks align very closely with one of Salesforce’s original values: our software lets you focus on your business. With Hyperforce, we’re re-architecting the Salesforce platform and we’re using Buildpacks for exactly that reason: so we can focus on our platform transformation. We’re also continuing to innovate by using Buildpacks in a new and surprising way.
Buildpacks are Cloud Native
The goal of the Cloud Native Buildpacks project is to translate source code into container images with a focus on developer productivity, container security, and operating at scale. The project also aims to unify the buildpack ecosystems of the past with a well-defined contract ideal for modern cloud native platforms.
In 2020, the Cloud Native Computing Foundation (CNCF) voted to promote Cloud Native Buildpacks to incubation from the CNCF sandbox. Since joining CNCF in 2018, the Cloud Native Buildpacks project has added dozens of new production users, new committers from more organizations, and defined an open governance process. Buildpacks have now been adopted by an esteemed cadre of end users and implementers including VMware, DigitalOcean, Google, HashiCorp, and Red Hat.
When buildpacks were created they were initially used as a mechanism to enable polyglot programming language support on the Heroku platform. As more and more developers created buildpacks, they grew beyond languages. Today there are buildpacks for Nginx, Meteor, React, and even Minecraft. In Hyperforce, we’re taking buildpacks even further by using them to generate infrastructure-as-code (IAC) and other infrastructure-level configurations.
Using Buildpacks in Hyperforce
Buildpacks are most commonly used to control how a service’s language runtime and dependencies are installed and packaged. But with Hyperforce, Salesforce is also using buildpacks to provision and configure infrastructure. This is an innovative new approach to how we use what has become a mature and well-established technology.
Hyperforce has many moving parts that all need to be configured, provisioned, and managed. In order to encapsulate the complexity inherent in the platform, Salesforce is using Hyperpacks to detect what kinds of infrastructure and configurations a service requires, and generate them on behalf of the service-owner.
The image created by running Hyperpacks differs from other Buildpacks because it’s not an executable representation of the service it’s created from. Instead, the image created by a Hyperpack includes executables for Terraform, Helm, and other tooling needed to apply and validate the service’s infrastructure. That is, instead of an “App Layer,” the final image contains Infrastructure-as-Code layers.
When a new service is created or onboarded to Hyperforce, a Hyperpack runs to detect its shape. Then a build process runs in-line with the normal compile and package phase to generate Helm, Terraform, and other infrastructure configuration manifests (including some proprietary formats). The result is an artifact that contains all the infrastructure-as-code, deployment descriptors, manifests, and other configurations required to instantiate and run a service. In effect, this is a single source of truth for what this service needs in order to run on Hyperforce. This artifact can then be applied via the Continuous Delivery to provision infrastructure and materialize the service.
This model allows us to quickly and consistently instantiate a service in a new environment, even when the environment itself is new. Dependencies and resources that need to be provisioned for the service are defined in the artifact produced by the buildpack.
Left-Shifting Validations
One advantage of using buildpacks to generate IAC is that it provides a great mechanism for encapsulating the validations and security policies we need to apply to every piece of infrastructure before provisioning it. This is essential to maintaining our commitment to trust. At Salesforce, Trust is our number one value. Hyperforce ensures that our customers’ applications are backed by security and compliant infrastructure at every step in the development process.
When Hyperpacks run against a service repo, they apply schema validations, Open Policy Agent (OPA) policies, and other control mechanisms to ensure that the infrastructure being defined is compliant with our strict requirements. In this way, developers can quickly iterate without risking a misconfiguration vulnerability. This allows them to remain agile and productive.
Our use of Hyperpacks goes beyond infrastructure though. We’re also using them to build images that run Integration Tests. Be on the lookout for a future post that discusses these Testpacks.
Try Buildpacks Today
Salesforce is proud of the impactful contribution we’ve made to the open source ecosystem with Cloud Native Buildpacks, as highlighted in the recent public roadmap for Heroku. We hope to continue this commitment by developing a plan for sharing our Hyperpacks with the open source community, but you can already use many of the Heroku Buildpacks and the tools that power them today.
To get started, download the Pack CLI and use it to build your first app. You can choose from the Heroku buildpacks, Paketo Buildpacks, or Google Buildpacks. You can also fork one of the Buildpack Samples, or read up on the implementation specifics laid out in the Buildpack API documentation.
Above all, we hope that you’ll join the conversation on the Buildpacks Slack or Mailing List. You can also follow the project on Twitter or Github.
