When my team started working on our new project we were exactly like these guys on the roof: we had plenty of hands and every one was ready to contribute but available infrastructure to run development, testing, and integration environments was very limited so we weren’t super efficient at the beginning.
Many of us joined the team from a different project where all testing was done on a cloud IaaS. There with a click of a button and a 15 minutes wait, any developer was able to get a new and shiny cluster with all applications they need to test even the craziest ideas.
The idea to have a representative local environment was born. Default build artifacts for services my team owns are containers. So the idea evolved to a next phase: let’s have a representative local environment where all products will run in containers so we can use them for development and testing.
A few weeks later we had all external dependencies we needed running in containers side-by-side with applications we own. A month or so later a security layer was added into the configuration: some connections were protected using kerberos and some connections were protected with mutual TLS authentication. Developers were able to experiment with the environment, debug their applications and try new ideas.
Later we found that our own services together with external dependencies like Apache Kafka, Apache HBase, Apache Storm needed way more resources then we can give them when running locally on our laptops, so we had to explore ways to create additional isolated and shared environments on dedicated servers.
But lessons were learned. This paper was written for my fellow engineers to explain just a bit about containers and how they can benefit from using them as part of their development process and how their applications should be organized so putting them in containers will be easy. I also tried to explain some basics about Docker and docker-compose to save my teammates from reading all of the documentation.
Here is the full series of blogs:
- What is a Container? — Overview of what are containers, what is docker runtime on different systems and an overview of a Dockerfile.
- What is docker-compose? — Overview of the tool, and basic elements of docker-compose.yaml file.
- How to run existing applications in containers — Recommendations for how to run an existing application and all external services it needs in containers.
Hopefully this series helps you learn how to use containers in order to make your team more productive. Let me know if you have any questions!
