You Need Source Control Even If You Don't Code
One of the common responses to my last post is a request for more detailed instructions for adopting source control for non-coders. Clearly you’re applying the ideas already! I’m going to try to set a good example by documenting the principles I’m applying, laying them out clearly first, and then offering some specific instructions. In fact, we’re going to do an experiment - I’ll put the instructions on Github, and if this blog post is successful, you should be able to contribute by the time you’re done reading it. So let’s get to it.
The Source and how to Control it
Let’s start with a familiar scenario in a familiar environment - the kitchen.
Like a city in miniature, the kitchen is constantly evolving based on the repeated activity of the people that are there. In some houses only cooking happens in the kitchen, similar to an industrial district, and in other houses everyone eats in the kitchen, similar to a downtown. For our purposes let’s talk about a household where everyone cooks and everyone eats in the kitchen. Let’s also try to assume that everyone is trying to contribute equally, and there’s no expectation that one person is going to take all the responsibility for (i.e. command of) the kitchen.
When you enter a kitchen to cook or to eat, you have certain expectations - that there’s going to be space for the activity you have planned, that you’ll be able to find the things you need or know that they’re out of stock, and that any process that’s already underway, like a kimchi ferment or a slow cooker stew, is going to be something you know how to interact with, even if it’s a label saying “do not open!” All of these outcomes are the result of processes that you and your housemates perform, and if the outcome doesn’t meet your expectations, you immediately understand that something went wrong in the process. Like a city, the process is constantly being revised and adjusted to try to achieve better outcomes, but unlike a city, the instructions are rarely written down.
We’re familiar with certain forms of instructions for kitchens, such as shopping lists and recipes. We’re even familiar with how valuable it is to modify these instructions - for example, think of how valuable all the notes in the margins of a family cookbook can be. And - here’s the important thing - each modification multiplies the value of the instructions. Seeing Grandma’s alterations to a casserole recipe over the years tells a story that goes far beyond the casserole. And if your aunt created her own version, the differences between the two tell a story that neither recipe can tell on its own. It’s the history of changes that help us understand and make better choices. It’s a source of knowledge, context, and power. This becomes more true the more connections are concentrated in the source, and those connections take the form of changes.
That’s the value of source control.
Because we live in a consumer society we tend to think of processes as producing products instead of outcomes. But with that small, powerful shift in our thinking - outcomes instead of products - we can take this storytelling power and expand it beyond the food in the kitchen to the entire kitchen itself. Another way of thinking about recipes is not just a series of steps but a way of doing things that produces a specific outcome. If you use recipe cards instead of cookbooks, you’re selecting specific outcomes that you want.
If you’ve shared a kitchen with nonfamily members, you probably have some experience with instructions that aren’t recipes, most likely cleanup checklists. The usefulness of these instructions compounds as your kitchen becomes more diverse - people with different histories and experiences. Commercial kitchens are overflowing with opportunities for source control - cleaning, stocking, prep, and equipment maintenance are all places where this is useful. If those peoples’ contributions are valued, the instructions are revised to incorporate them. That’s our point of departure for source control.
The Process in a Nutshell, and its pitfalls
Overall, the process I’m describing is something simple, and it begins with differences.
An outcome is either unclear, inadequate or disputed. For example: what does clean mean?
The desired outcome is identified or clarified. What’s desirable, safe, reasonable, achievable? The negotiation looks something like this:
Specific changes are proposed.
The changes are discussed
The changes are accepted, rejected, or revised.
The new process is recorded. Not the result! The process.
The new process is evaluated. If the outcome is acceptable, great! If revision is needed, start over at step 1.
You probably already do this in your kitchen, even with yourself, perhaps without the rigor I’m describing here. If you can bring this process into your organization, even if it’s just a homebrew pen and paper system, great! You will see results, I promise. However, there are sophisticated tools out there to save you the effort of reinventing the wheel. Before we get into the tools themselves, let’s discuss some of the challenges and pitfalls of this process.
Problem: Disagreement. Solution: Not Actually a Problem
Like I said, if you get into a discussion of a process and you find out you have different ideas or expectations, don’t get upset! Disagreement is your starting point. If you already agree on something, you don’t need to write it down. If you find yourself thinking or saying “I shouldn’t have to write this down,” you really need to write it down.
Problem: Agreement. Solution: Put it to the test.
Groups of people, even small ones, can adopt a superstitious approach to new processes. You work out your disagreements, come up with a new process, write it down, put it to the test - and it doesn’t work. “We’re doing the process; why isn’t it working?” This is a problem at the evaluation step. Don’t forget - honest reflection is part of the process. The question to answer here is: how do we know we’re on the same page? Doing source control isn’t a solution, it’s a way to solve problems. Only you can decide when the problem is solved, and if there’s disagreement about that, well, it’s not solved. Write it down.
Problem: Analysis Paralysis. Solution: Divide and Conquer
Sometimes with an enthusiastic group that’s full of ideas, it can be hard to decide where to start, especially when the group is working with limited resources. Reach agreement on priorities and let people assign themselves to working on ideas. As long as there’s some kind of regular check-in, most people are surprisingly good at deciding what to work on from moment to moment. There are some techniques that make this possible to do with very, very large groups, namely forks and branches, which we’ll cover below.
Problem: Unwillingness to Collaborate. Solution: Salvage the Data and Leave
There is one category of problem that source control can’t solve directly, and that is people’s silly power games that they play with group processes. No one wins these games. Don’t play them. The people that like playing them are often in charge because of this. These are always going to exist, but the forms that shut down improvement can’t be salvaged. If you’re not able to leave immediately for some reason, don’t feel bad about gathering as much useful information about what works about the processes you do have and can’t improve - that way you can pick up where you left off in another situation, or help people that are in similar situations.
Even if you’re not able to leave, gathering information about your situation (and publishing it if you can) has cool side effects. You never know how many other people are in your exact situation trying to solve your exact problems. The more that the people that are willing to collaborate do so, the less effect power games can have on the overall improvement process. Remember:
Toys for Organizers - Github
disclosure: I am not paid by or commercially affiliated with Github in any way.
When it comes to using digital tools for source control, you might imagine making copies of folders in a shared drive, maybe naming files different things for different versions, maybe keeping a spreadsheet of changes. Trust me, those will work well, until they don’t, at which point the system will collapse in a mess. If you only have one file to manage, you can use the versioning system in your word processor, but then which file is the correct one? Don’t waste your time reinventing the wheel. That’s what this section is for.
Think of source control as adding a third dimension to your file system. In addition to paths (folders and filenames) and data (the actual information in the files), source control adds a dimension of time. The basic unit of time in a source control system is the commit (or patch, in older systems) - the moment when a person wants to say to the group, “this change is worth recognizing and discussing.” Each commit is signed by its author and references the version of the system they are changing. This is what allows commits to be coordinated and gives you the whole story.
Often a single commit isn’t the whole story - far from it. To take the pressure to get it exactly right on the first time (impossible, by the way), commits are organized into branches. These branches are versions of the entire system that can be developed independently and then merged when they’re ready.
The beauty of this model, and the power of Git, is that each person can work on the entire system independently, with the confidence that they can coordinate their work with others down the road. This empowers individuals to develop systemic changes piece by piece, and that should be attractive to people working on systems other than a network of computers as well.
When you have a team of people all working on different branches, you don’t want their versions to drift too far apart, so you want to make sure they check in frequently. One nice way to do this is to provide a common branch for people to share mostly-complete changes so that they can see how their work works with other people’s work. We call this integration. There are also nice tools to let everybody integrate at their own pace, and we call this continuous integration.
One practice that isn’t technically necessary but is a really good idea is a review process. Just having two or more sets of eyes on a series of changes is a great way to catch mistakes and clarify things up front, to finish discussions or glean insights from the process. A nice way of doing this is with pull requests. A person or team that feels that their branch is ready for integration can make a request for their changes to be “pulled in” to the common branch. Usually someone with a more expert level understanding of the system will perform the review and either give it the OK to proceed or request some changes for housekeeping or consistency. Some parts of this can even be automated, such as ensuring consistent file formatting or even running tests to make sure the new changes don’t break something elsewhere in the system.
To summarize the technical side: source control organizes changes (commits) into related groups (branches) that can then be combined (merged), ideally on a continuous basis. Reviewing a batch of changes when it’s ready to merge (code review/pull requests) is a good way to keep the process tight.
This is the very most basic outline sketch of what source control is. It should be enough to get any non-technical team using some form of it.
Putting it to Work
If that all makes sense, join me over on Github.