Branch-Per-Feature using the Total Integration Total Isolation Principle

I posted this originally as a comment on a Google Plus thread here which has a more complete discussion on ideas (and disagreements) regarding Branch-Per-Feature and Continuous Integration.

The purpose of this entry is to focus on how a principle of Total Integration and Total Isolation shapes the approach taken to branch-per feature.

*****

In this approach, all of the following are givens:

1) releases occur on a regular basis, and development is oriented to the release schedule

2) releases coincide with merge to master

3) the release is tagged, and an empty commit immediately following the release commit is also tagged (as the start of the new cycle)

4) the existing integration ("dev") branch and the qa branch from last cycle are now re-pointed to the new start-of-cycle tag

Key point here: master is ONLY updated once per release, at the point of release--but the integration/dev and qa branches originate from (and are therefore identical to) master

The process now begins, which adheres to a "Total Integration Total Isolation" principle

1) Each developer chooses a ticket and creates a branch with that number (eg. In JIRA, tickets ABC-141, ABC-142, ABC-143, ABC-144)

2) This branch will be short lived (it won't live past the release of the ticket) although the actual commits will be preserved.

3) The developer commits frequently their branch (eg. ABC-143).

4) However, as per some of the heated discussion in this thread, the developer also merges every few hours with the integration/dev branch and checks for conflicts, compile fails, and runs all tests either locally or via the CI server's integraton/dev branch build.

5) Merge conflicts are resolved (and cached for future reuse if the DVCS permits), but outright failures requires the dev to go back to their feature branch and make the fix there, before re-attempting the merge to integration/dev branch.

6) Note that the feature branches never merge FROM the integration branch, because this would violate the isolation side of the Total Integration Total Isolation principle.

7) What about major refactorings or new archicture/scaffolding that one or more features need to share? In that case, a new ticket (eg. Dev task ABC-145) is created to hold that shared work and a branch is created (again, off of the start-of-cycle tag) to hold that major refactoring or scaffolding. The features requiring this branch change their start point/dependency from the start-of-cycle tag to this refactoring/scaffolding branch and they will retain this dependency until the end of the release.

8) Going forward, each new work is commited only to the feature branch, and each feature branch is regularly merged to integration/dev. This results in the Total Integration Total Isolation goal

9) Since every branch now originates from start-of-cycle (or from a shared refactoring/scaffolding branch that originated from start-of-cycle), QA can now safely pick and choose which features to merge onto the qa branch, and ultimately, which to release and merge to master.

10) Any features that were not released can be discarded (if rejected entirely) or rebased onto the next start-of-cycle tag (if to be resumed).

Branch-Per-Feature: Successful Transitions/Cleanup Between Sprints

.

Introduction
Branch-per-feature is the discipline of beginning every feature branch for a given sprint off exactly the same commit (typically, the first commit of the sprint). The strict enforcement of isolation between features quickly reveals the bad habits of dependencies we build between multiple features, and forces us to ask the right questions of how to keep our features independent.

Some benefits of branch-per-feature:

• dev: proper isolation of code changes; breaking bad habits of code dependencies between branches
• dev: embracing granularity of code changes
• dev/QA: (almost) painless merging
• QA: ability to assemble a release made up only of branches that are ready

(For more details, see: Branch-Per-Feature using the Total Integration Total Isolation Principle.)

Successful Transitions (Cleanup) Between Sprints
The focus of this blog post is more narrow: to define the steps involved in successful transitions between sprints/releases when using branch-per-feature to manage the release.

We've been working out the logistics for this recently at work, under the guidance of @martinaatmaa and @adymitruk. What this looks like:

Over the course of a given sprint, each feature is branched off a common commit from the start of the sprint. As each feature reaches a point of stability and completion, it is merged back into an integration branch (named something like projectname-dev). When qa is ready to test, all features to be tested are folded into a qa branch (named something like projectname-qa). Upon release, the projectname-qa branch is merged into master, and tagged as the release branch for that sprint. Now that the code has been released, the feature branches for that sprint are no longer required. They are cleaned up (deleted), although the underlying commits are kept.

Examples and screenshots illustrating this process are shown below.

To begin the next sprint, a new empty commit is created and tagged something like start-sprint2. Each feature is created off that starting commit.

Scaffolding
Early in the sprint, if it is determined that a scaffolding commit is necessary to hold some common architecture to be used by all features, a new scaffolding-only feature is created for that purpose. The scaffolding feature branch becomes the new starting point of the commit, with all features branching off that pre-requisite feature.

Feature Branch Naming
The tools we are using to achieve this are git and JIRA (with the Greenhopper plugin). For feature branch naming, we use the JIRA ticket names. For example, a project whose tickets are named PAYGATE-265, PAYGATE-286 would have corresponding git feature branch names (paygate-265, paygate-286).

Note   If a scaffolding feature is required in the sprint, the scaffolding branch will have a normal branch name (paygate-262) and all subsequent commits can indicate the dependency in their name (paygate-265-d-262, paygate-286-d-262).

Practise Scenario
To practise my skills in branch-per-feature at home, I've been using Ubuntu, rails, git, and a local install of JIRA/Greenhopper. The project used for this practise is the book Ruby on Rails 3 Tutorial. I've broken out the chapter contents into individual features as JIRA tickets. The JIRA project is broken up into very small sprints (1 weekend worth of work per sprint, probably 4-6 hours at most.)

The rest of this blog entry demonstrates branch-per-feature using this simple project, as we fold the features of sprint 1 into an integration branch, then a qa branch, and finally a release branch. We then clean up the old branches and begin work on sprint 2 features, with each one branching off the starting commit of sprint 2.

Here is a screenshot of sprint 1 in JIRA. Each ticket in this sprint has been implemented as a feature branch in git:

To complete the sprint (and its related release), I perform 3 steps:

• merge all the tickets into the integration branch (rails-dev) and test the code
• upon success, merge all passing features into the QA branch (rails-qa) and test the code
• upon success, merge rails-qa into master and tag it as the release. 

Here's the gitk view:

With the sprint completed, its time to go back to JIRA (with Greenhopper plugin) to verify that everything in sprint 1 is closed, and that remaining story points are at 0, followed by setting up and prioritizing sprint 2:

Now that sprint 2 is prepared, its time for the dev team to begin coding. I switch to the JIRA/Greenhopper task board view, and drag my first ticket LRNRAILS-15 to In Progress:

To begin work on the dev tickets, we need to first set up the new sprint 2 in git. I begin by creating an empty commit off the release/master branch. To do this, I checkout the tag release-lrnrails-sprint1 (or master branch). To create the new commit of an empty branch:

git commit --allow-empty


I then tag it with a name such as start-lrnrails-sprint2.

git tag start-lrnrails-sprint2


Since this is the commit that all features will originate from, I will also move my integration (rails-dev) and QA branches (rails-qa) to this commit (by checking out eadch branch and then using git reset --hard start-lrnrais-sprint2 to point them at the starting commit).

Finally, we need to CLEAN UP (remove) all of the sprint 1 feature branches as they are no longer needed. At work we relied on @adymitruk's bash scripts to construct the delete commands that clear out both the local and remote branches. I tried this with my project at home, and got it to work successfully.

For example, given that I want to delete branch jira-lrnrails-5, my local and remote commands would be:

git branch  -D jira-lrnrails-5
git push origin :jira-lrnrails-5

To achieve this for all branch-per-features which were merged into the release, I checkout the release-lrnrails-sprint1 tag (or master branch), and then run the following commands, first as preview (using echo to verify the commands):

git branch --merged | grep lrnrails -i | xargs -i{} echo git branch -D {}


and then the actual execution of the commands:

git branch --merged | grep lrnrails -i | xargs -i{} git branch -D {}


Then, the same for the remote branches:

git branch -r --merged | grep lrnrails -i | cut -d '/' -f 2 | xargs -i{} echo git push origin :{}
git branch -r --merged | grep lrnrails -i | cut -d '/' -f 2 | xargs -i{} git push origin :{}


Now, with all of the branch-per-features deleted for sprint1, the view is much cleaner in gitk:

Now I  am ready to begin work on sprint 2. I checkout the starting point commit for this sprint:

git checkout start-lrnrails-sprint2


and then create my feature branch off that starting commit:
git checkout -b jira-lrnrails-15


I then proceed to write the code for this feature. At a certain point, I will do one-or-more commits for this feature branch:git add . -A
git commit -m "LRNRAILS-15 Adding variables to the views"


With the feature branch commited, I am ready to test my integration branch for the first time. I check out the integration branch (myprojectname-dev) and then do a git merge --no-ff against the new feature branch:

git merge --no-ff jira-lrnrails-15


This gives me my first integration branch merge of sprint 2:

From here, the second (and subsequent) sprints can move forward using branch-per-feature to properly isolate code changes, and to enable QA to assemble release packages based on a specifically chosen subset of verified features.