Part 2 of 3 of a blog series about Inner Sourcing: What it is, how to go about implementing it and what the benefits and potential gotchas are. This part examines some of the foundational aspects of building a collaborative, inner sourcing culture within the confines of a closed source software development company.
The pillars of an inner source culture
Open, cross-company, collaborative, creative communication
Collaboration, creativity and meaningful innovation depend on good communication. Whether it’s through in-person discussions, real-time chats, distribution lists or forums, it’s important that the mechanisms are discoverable and consistent:
Discoverable: Anyone across the company can easily find discussions that happened in the past.
Consistent: Everyone in the company should use the same mechanism, or depend on integrations that roll their offline team discussions into a common system.
This doesn’t mean that only one method need be employed. Different tools solve different purposes. For example, Chatter is a tool that is used within Salesforce and is effectively used in place of emails. It allows for a much higher level of discoverability than emails as anyone with privileges required to see the content can find conversation threads in search results. It’s also consistent as everyone uses it.
Great thoughts around group communication can be found at communicationtheory.org.
Cross-company, collaborative development
New features are created from feature requests that are entered into a system and prioritized. Eventually they are developed, tested, and released by the project’s team without further assistance from the requesting team. While this process satisfies customer relationship requirements, it doesn’t translate to a collaborative development partnership. All of the companies I’ve been lucky enough to work for are bursting at the seams with incredibly smart polyglots who could contribute to projects in a meaningful way across team boundaries. Enabling and encouraging cross-team contributions benefits the team that owns the project by enriching their product’s feature set. The contributors benefit too as they’re no longer blocked waiting for a feature request to be implemented.
One of the habits I’ve noticed amongst various companies is to label teams that consume their output as customers. This needs to be redefined: The term “customer” implies boundaries. “Customers” should really be regarded as “partners”. “Partner” implies openness and collaborative effort. Everyone within an organization should be considered a partner rather than a customer, a subtle but important distinction.
Implementation & adoption
Unfortunately, implementation and adoption isn’t prescriptive; one shoe doesn’t fit all. However, there are a few key practices that can be adopted to facilitate a more collaborative development environment.
Many contributors, few committers
Typically in the closed source world, teams have a product owner or technical lead. The remaining developers on the team are regarded as contributors. To provide an optimal collaborative culture, the team must evolve. While the product owner or technical lead doesn’t change, short of being exposed to a (potentially) much larger audience, the real change applies to the rest of the team. They become committers rather than contributors. The difference is subtle and yield greater results. Consider the differences:
A technical lead or product owner is responsible for the overall vision, architecture and technical roadmap of the project. They are responsible for providing final sign-off on feature requests and design documents as well as the overall integrity of the project.
A committer is a key stakeholder in the vision, technical roadmap, and architecture of the project. The lead and all committers must all be on the same page in terms of project direction and vision and be able to communicate that direction and vision to other stakeholders. Committers and leads have permissions required to merge their own code as well as contributions from external contributors.
A contributor contributes to the project, whether it’s code, documentation or tests, but does not have access to merge code directly to the project’s canonical repo or main branch.
Automation
The most successful collaborative projects are those that maximize automation. Getting a project up and running consistently between veterans and newcomers alike should be no more difficult than “make build”, “make install”, “make run”, and so on. Nothing further should be required. Projects should also have full automated build and test pipelines in place, giving new contributors confidence in their submissions before code is merged.
Best practices
By having a well-defined, company-wide set of best practices, you promote consistency. Consistency lowers the barrier of entry for contributors to hop from project to project. As much as possible, best practices should be based on open source best practices, which helps ensure that:
- If they have contributed to open source projects and are familiar with the best practices, new hires have a lower barrier of entry.
- The barrier of entry for dependency inspection (debugging, contributions, and so on ) is lowered.
Not everyone will adopt open source community best practices for their team. Use what works for you. Best practices to consider for adoption are specific to:
- Project setup (directory structure, common files, and so on)
- Test and static analysis utilities
- Commonly used libraries
- Shared coding standards
By ensuring consistency across projects and teams in these areas, you help ensure minimal overhead for prospective contributors. A developer working on one project in Python should be able to jump to another Python project owned by another team with little to no overhead.
Marketing and communication
Code is written. Code is deployed. Code rots. If nobody else knows about it, nobody will actively maintain it.
“Build it and they will come” does not apply to code. It’s more “Build it, market it and attempt to build an active community around it” is a little closer to what’s actually required of a successful, collaborative project. Case in point: Docker. Was it the first containerization technology? Nope. Was it the best marketed and most approachable? Absolutely!
A consistent communication method is a fundamental pillar to the success of inner sourcing. Without a consistent communication method that transcends the team barriers, the effectiveness of inner sourcing efforts are inherently limited by a lack of transparency and open communication. When evaluating collaboration tools, it’s important to define what communication problems you’re trying to solve:
- Are you primarily going to be discussing system design and architecture?
- Are you primarily needing channels to facilitate fighting ops fires?
It’s really important to define the actual problems you’re trying to solve as some tools and methods may actually degrade your ability to communicate effectively as a group. For further insight into this, see https://communicationtheory.org/creativity-in-groups/.
VCS != DVCS
VCS (i.e. Perforce, a centralized version control system), typically don’t offer the same out-of-the-box functions that facilitate an inner sourcing model as do DVCS (i.e. git, a distributed version control system). Of course, you can jump through hoops in some VSC to mimic properties of DVCS (for example, setting up code review watches or locking branches), but DVCS workflows are much more suited to flexible development and integration patterns from external contributors (for example, fork and pull request flow using GitHub). Not only that, but out of the box with DVCS, projects are intended to be split into separate repositories, each of which can have a committer list entirely independent of other projects. With this clear separation of administrative controls between projects, it is much easier to adopt external contributors with git than with VCS.
Documentation
High quality documentation should be of paramount importance to the team. It should be a first class citizen, just as important than the shipped code. It effectively makes or breaks your ability to contribute to the project. The more incomplete developer or on-boarding docs are, the higher the barrier of entry is. The more incomplete that user docs and API reference are, the harder it is for folks to integrate the project effectively.
Optimally, each project should have a split between developer and user docs. The the former tailored specifically for developers to the project and the latter enable to folks simply using the project. This split allows users and developers to concentrate solely on what’s most important to them. Documentation should be generated from code and code comments where it makes sense to preserve the integrity of the documentation. Developers are much more likely to update comments inline with code than they are to update external docs. Stale documentation can be just as bad if not worse than an overall lack of documentation as it can lead the reader down an incorrect path.
Generalized solutions
This is a tricky one. It’s a different approach to thinking and realizing opportunities for generalization, modularity, and reusability in system design. Generally, an immediate knee-jerk reaction of a developer is “Hey, I need feature X in Y project. I’ll just dive into code.” Even if a full design is done, it’s generally for a specific project. In either case, you end up with project-specific code that isn’t modular, extensible or reusable.
Generalized solutions take time to design, implement and test. Having the ability to recognize where you’re making assumptions and validating they’re the right ones to make or determining an alternate, assumption-free solution can be a hard thing to do and takes a lot of trial and error. However, in the long run, thinking about problems and developing generalized solutions not only benefits you, but others within your organization. Instead of copy/pasting and refactoring, modular code and libraries can be reused by multiple teams and extended upon.
Important: Not a Prescription!
These are just some of the tools and best practices adopted by open source projects in order to create an open, collaborative development culture. Many more may be uncovered as teams begin to work much more collaboratively. Also, not all implementations of these concepts will be created equally across teams and organizations, nor should they be. What’s important is that the concepts are applied and consistent interfaces are provided across organizations to provide the lowest barrier of entry possible for new contributors and to mitigate duplication of effort as much as possible.
In the next post of this series, we’ll take a look at what the key benefits to an inner sourcing culture are as well as some of the potential gotchas to be aware of.
Follow us on Twitter: @SalesforceEng
Want to work with us? Let us know!
