Part 1 of 3 of a blog series about Inner Sourcing. What it is, how to go about implementing it and what the benefits and potential gotchas are. In this part, we take a look at what inner sourcing is and how it compares to traditional enterprise software development.
Something’s broke. Let’s fix it. Preferably without reinventing the wheel.
Unfortunately, wheel reinvention is a common occurrence within software organizations. The difficulty of the problem is roughly relative to the size and number of teams within an organization. The root cause here typically is a lack of transparency and communication between teams. Because of this, opportunities for collaboration on user-facing projects as well as what-should-be-common tools and libraries is stifled. As a direct result, systems are built with overlapping goals. Non-modular, case-specific code is written to satisfy per-project goals rather than sharing and leveraging generalized solutions. Everyone wants to do more with less. Effort redundancy and system overlap becomes the anti-vision, resulting in general confusion (i.e. “which system do I use?”) and data duplication. More systems result in more full stack maintenance. Duplicate data scattered amongst systems results in an increase in potential exfil targets. Inner sourcing aims to mitigate these and other problems as much as possible through open collaboration and communication.
Inner sourcing: What is it?
Collaborative development isn’t a new concept. However, many organizations often fail to execute on it for any number of reasons: Project timelines, political, technical (programing languages), etc. The term “inner sourcing” was coined by PayPal. Their InnerSource commons initiative seeks to provide tools and best practices to break down barriers and foster open, collaborative development. This effort develops a much more open and collaborative culture, largely modeled after the open source community. Many open source communities have figured out effective ways to collaborate and work successfully in spite of timezone constraints, team distribution and the lack of in-person communication. Many of these same problems are faced by proprietary software organizations as well, regardless of their size. To maximize the collaborative culture within these organizations, experiences and other takeaways can be leveraged from communities that have built out their collaborative methods and processes over a number of years.
During my time working on open source projects and enterprise software, I’ve noticed these philosophical differences occurring repeatedly:
Project scope
Open Source approach: Make as few assumptions as possible. You never know how your users are going to use your code (if only I could count the number of times I had a PR rejected because I wasn’t thinking generally enough)
Result: Typically higher initial development cost due to generalized architecture and testing. Much more extensible, future-proofed implementations that don’t require much refactoring.
Enterprise approach: I have to solve this (sometimes very) specific problem. I’m going to write code for that thing and only that thing.
Result: Leads to redesigned wheels and painful re-factors when reusing/repurposing existing code.
Documentation
Open Source approach: High quality documentation is of paramount importance. Depending on the nature of the project, both user and developer docs will be written.
Result: A good amount of time is spent on writing documentation. However, because of this investment, projects are much more approachable from newcomers and seasoned developers alike.
Enterprise approach: We all know that documentation is important, but we have deadlines that preempt doc development.
Results: Documentation is stale, subpar, haphazard or even non-existent. As a result, on-boarding new hires and including external collaborators proves difficult.
On-boarding
Open Source approach: Ramp-up time must be minimal in order to make the barrier of entry for prospective contributors as low as possible.
Result: Because of the effort in documentation, on-boarding documentation is already established. There should be minimal to no assistance required for someone to get up to speed easily.
Enterprise approach: We don’t get new hires too frequently, so on-boarding docs are considered a “nice to have.”
Result: New hires find it difficult to get up to speed with a new team. New hires also aren’t the only people looking to contribute to your codebase. By increasing ramp-up times, you’re also preventing folks from other teams from contributing to your project.
Incoming contributions
Open Source approach: Anyone can contribute to our codebase.
Result: Anyone can contribute to the codebase. By nature, development is open and collaborative.
Enterprise approach: We only expect contributions from our own team.
Result: An assumption that not only impacts decisions made around system design and work prioritization, but also makes projects unapproachable.
Collaboration
Open Source approach: I want to contribute to other codebases.
Result: I can browse projects and contribute to anything that I find interesting.
Enterprise approach: I’m worried about what political implications my contributions will have.
Result: There is a tendency to not branch out from one’s team’s repo for political reasons, even if there is domain expertise to be shared.
Deployment
Open Source approach: Anyone should be able to execute our test suite with as few barriers as possible. Executing deployments should be fully automated.
Result: Anyone interested in the project can grab the source code and run the test suite easily. Deployments are either push-button or can be simply executed through a command line interface (CLI). The barrier of entry to contribute meaningfully is nearly non-existent.
Enterprise approach: Nobody outside of our team will ever contribute to our code, so it’s okay to have a runbook to stand up a development environment. After all, we have on-boarding mentors to help new folks to the team get started.
Result: Development environments are only understood by the team. As such, the cost of standing one up is high, raising the initial barrier of entry and discouraging folks outside of the immediate team from contributing.
With these differing philosophies, we end up with (generally) sub-optimal results:
- A number of teams that are effectively black boxes to one another
- A large amount of redundant efforts and system overlap
- Code that is written is extremely difficult to repurpose, even by the team that wrote it
- Stale documentation
- High ramp-up times, both for new hires and for collaborative development
- A lack of consistency between projects: Coding standards, tools and utilities, libraries, build pipelines, etc.
- Inadequate cross pollination
We all know that these aren’t positive for organizations, so let’s do something to change it!
As we establish a collaborative development environment across teams within the confines of a closed-source organization, we can avoid these problems by applying the communication techniques, development processes, and best practices employed by many open source communities.
In the next post of this series, we’ll take a look at some of the tools and practices that can be leveraged in order to start building an inner sourcing culture
Follow us on Twitter: @SalesforceEng
Want to work with us? Let us know!
