Playing games is not only entertaining, but can also be a great way to build and reinforce new skills. Capture the Flag (CTF) competitions are a common game in the security world. In a CTF, individuals or teams compete to see who can solve the most challenging security problems within a time limit. By combining hands-on learning and a little friendly competition, CTFs provide an engaging way to educate users about the latest in security features and threats.
Earlier this year, the Salesforce Security Engagement team led development of a CTF competition to expand the Salesforce user’s knowledge of security features. Built for administrators and developers, the Salesforce CTF aims to teach users how to strengthen their own Salesforce org(s) by providing them with an insecure trial org and challenging them to find and fix the security vulnerabilities.
Build it Fast, Make it Nice
Short on time and to stay respectful of everyone’s day job, we needed to take advantage of tools that would give us the most bang for the buck. First, we needed a system to track the participants, record the flags, and provide a scoring system. The open source CTFd framework met our basic requirements, and allowed for additional functionality based on its permissive open source license.
With CTFd, each challenge has one or more string values, or “flags.” Participants capture a flag by determining the string value and entering that value in the CTFd user interface. For security CTFs, the challenges can be as complex as gaining access to a restricted directory of a server or looking in the contents of an encrypted file to find a flag value.
Since our challenges would also be within the scope of a Salesforce Org, and the participants would be administrators of the org, we knew we had to be extra clever. Flag values hidden within the org would not be very challenging since anyone could look through the data and metadata to find the values. So, we took inspiration from the Salesforce training platform Trailhead, and built out server-side tooling that uses OAuth authentication and delegation to allow an external service to examine the configuration of a Salesforce org using the Salesforce API. The service can then determine if the user successfully completed the challenge in a way that is not visible or accessible to the participant.
Almost any language would work with the Salesforce API, but we built our service in Node.js to take advantage of the NPM ecosystem. Its open source components allowed us to easily pull in building blocks such as Express (for rapid API endpoint development) and JSForce (to work with the Salesforce API).
Additionally, we needed a way to give every participant their own Salesforce trial org. Luckily, Salesforce has the ability to build a template that defines a trial org which includes all of the custom data, code, and branding required for our CTF challenges. We used Trialforce to automatically create custom challenge orgs for each participant as part of their online registration.
Now Make it Easy
Once the basic pieces were in place, we needed to ensure continued engagement, which meant minimizing the number of places users had to visit to complete the challenges (rather than providing a complex set of instructions). This is where we wrote code and built some pretty cool integrations. Since CTFd was the central point of interaction for the participants, we built a number of customizations to support our style of CTF.
First, we added some basic usability extensions using the CTFd plugin model. When a participant registers, they are presented with a custom registration form that captures the additional details required to create a Trialforce org. The registration process also makes the API calls to set up the Challenge org for the participant, and Salesforce handles sending them an email to their very own custom org. A few minutes after registration, participants receive an email informing them that their org is ready.
To limit the browser tabs to CTFd and Salesforce, we built a custom CTFd challenge type (using plugin support) to execute the challenge evaluation APIs without requiring the user to go anywhere else. This Integrated Challenge provides users with a button that calls out to the challenge evaluation code, which then executes the server side API calls to test if the user correctly solved the challenge. Then, the API passes control back to CTFd, which displays the results of the attempt to the user and allows them to claim their flag.
We also built a few other user interface customizations, including customizing the sort order of our challenges (so users don’t have to hunt around to find out what to do next) and adding in a frequently asked questions area (allowing users to quickly find answers to common questions).
Challenges with Challenges
Newly created Salesforce Orgs come with a number of security settings already enabled. This posed a unique challenge for our CTF development team: to not educate users about all of the security features or find a new way to relay the information. So, we employed a carrot, a stick, and some deception. Since the Challenge orgs were secure by default, we needed to make them insecure after they were created. Once they were created, the user was in control of the settings.
We created a new challenge, Initialize, that users had to complete prior to any administrator or developer challenges. Solving this challenge gave the users some CTF points (the carrot), was required in order to unlock the rest of the challenges (the stick), and executed API calls to reduce the security level of the Challenge org — which the users have to re-secure as part of their challenges (the deception).
Can You Add Some Scale?
To ensure growth and engagement, we needed a secure, fast, and scalable infrastructure to support our CTF. We chose Heroku to host the CTFd and API services since it was easy to use, hosted the Docker-based CTFd and Node.js service APIs, and supported our scripted and automated deployment process — allowing us to easily scale up our resource pool as more people joined the game.
We also wanted to easily support multiple events, so we built tooling to perform all parts of challenge authoring (e.g., wording of the challenge, the definition of any hints for the challenge, logic to evaluate the results of the challenge) in the API services and run an export job to create a packaged set of data that is pushed to the CTFd instance during the deployment process. The CTFd container bootstraps and imports the event data from that package so we can support an unlimited number of concurrent and consecutive events from a single codebase; the only human effort is to add the new event identifier to a build definition.
What it Looks Like
In the end, the Salesforce Security team developed a secure, engaging, scalable CTF game with a number of benefits for current and future use:
- An architecture that expresses good separation of concerns, provides for multiple points of scale, and delivers future flexibility to extend the CTF concept to cover other Salesforce products as well as other platforms and technologies (e.g., application developer focused CTF)
- The ability to easily pull from the upstream CTFd repository and incorporate future CTFd enhancements without requiring a labor intensive manual merging of changes due to the use of the existing CTFd plugin capabilities
- An ability to directly contribute back to CTFd in the future with pull requests generated directly from our fork, with no need to manually exclude our customizations because of an upstream-friendly codebase
- The option to open source future modules and further enrich the open source CTFd ecosystem
From TrailheadDX to Dreamforce ’19
In May 2019, we successfully hosted and provided a high performance experience of our CTF at TrailheadDX, Salesforce’s developer conference, with almost 500 participants in two days. With detailed feedback from customers, staff, and management, we greatly enhanced our original customizations and will have an even more robust experience for our customers at Dreamforce ’19. Additionally, we expanded the number and complexity of our challenges, tightened up the integrations, and streamlined the build and deployment process — ensuring an outstanding Dreamforce-scale CTF.
The Future of Capture the Flag
We are hoping to continue to expand our work with CTF as an engaging learning tool to enhance the application security training internally at Salesforce. Having used the work of many others who provided the open source components, we will also look to return as much of our work as possible to the open source community — giving back to the community that helped us build.
We want the Capture the Flag project to help drive the Salesforce culture of innovation, so stay tuned for additional articles as we continue to expand our features and reach.
