Upcoming Subversion 1.5 Feature: Changelists

July 5, 2007 C. Michael Pilato

One quality of a typical engineer is a heightened sensitivity to potential problem areas. For software engineers, that often means spotting possible bugs in our peripheral vision while working on some nearby chunk of source code. This is both a blessing and curse. You want your bugs fixed, but unfortunately we sometimes get derailed from our primary task by these little excursions into Perfection.

Another fact of a developer’s life is that there are always more things to work on than time to do so. And everything is of the utmost priority to someone. And so here again, we find ourselves working on multiple things at the same time, making slow-yet-incremental progress on each of them so at least we can say about them all with a clear conscience and a forced smile, "As it turns out, I’m working on that right now."

To my managerial readers, I regretfully admit that I have absolutely no solution for those problems. After all, this is a technical blog. But I can (and will) tell you about an upcoming Subversion feature that might reduce the context-switching costs for those developers: changelists.

Whatever the reason, it is commonplace to find that a developer is working at any given time on multiple different, distinct changes to a particular bit of source code. And often each change isn’t nicely contained in some module, safely separate from the other changes. The groups of changes might overlap, modifying different files in the same module, or even modifying different lines in the same file. Now, there are various work methodologies that developers can employ to keep these tasks organized. Some use separate working copies of the same repository to hold each individual change-in-progress. Others might choose to create feature branches in the repository, and use a single working copy that is constantly switched to point to one such branch or another. Still others use diff and patch to backup and restore uncommitted changes to and from patchfiles associated with each change. Each of these methods has its pros and cons, and to large degree, the details of changes being made dictate the methodology used to distinguish them.

But Subversion 1.5 brings a new feature called "changelists" which adds yet another method to the mix. Changelists are basically arbitrary labels applied to working copy files for the purpose of associating multiple files together. Gmail users are familiar with this concept already. Gmail doesn’t provide the traditional folders-based email organization mechanism. In Gmail, you apply arbitrary labels to emails, and multiple emails can be said to be part of the same group if they happen to share a particular label. Viewing only a group of similarly labeled emails then becomes a simple user interface trick. Many other Web 2.0 sites have similar mechanisms — consider the "tags" used by sites like YouTube and Flickr, "categories" applied to blog entries, and so on. Folks understand today that organization of data is critical, but that how that data is organized needs to be a flexible concept. The old files-and-folders paradigm is too strict for many applications.

So Subversion’s changelist support allows you to:

  • create changelists by applying labels to files you want to be associated with that changelist,
  • remove those labels, and
  • limit the scope of the files on which its subcommands operate to only those bearing a particular label.

For example, say that as part of my vast array of things to accomplish today, I need to fix some minor issues in ViewVC related to diff-viewing operations. My work leads me to change a couple of files.

$ cd projects/viewvc
$ svn status
M      lib/vclib/ccvs/__init__.py
M      lib/vclib/__init__.py
$

But while testing this change, I notice something else that’s not working right in the nearby annotation support code. There are, as previously mentioned, many things I can do at this point in order to switch contexts and fix this secondary issue. We’ll use changelists for the purposes of illustration.

First, I want to create a changelist and associate with it the two files I’ve already changed. These helps me to distinguish those changes from the ones I’m about to make for this secondary issue.

$ svn changelist diff-fixes lib/vclib/ccvs/__init__.py
Path 'lib/vclib/ccvs/__init__.py' is now a member of changelist 'diff-fixes'.
$ svn changelist diff-fixes lib/vclib/__init__.py
Path 'lib/vclib/__init__.py' is now a member of changelist 'diff-fixes'.
$ svn status

--- Changelist 'diff-fixes':
M      lib/vclib/ccvs/__init__.py
M      lib/vclib/__init__.py
$

As you can see, the output of ‘svn status’ reflects this new grouping.

Now, I start fixing the secondary issue, which requires that I change a third file. I’m not yet ready to commit either change, though, so I’ll make a changelist for my second change, too.

$ svn changelist blame-fix lib/vclib/ccvs/blame.py
Path 'lib/vclib/ccvs/blame.py' is now a member of changelist 'blame-fix'.
$ svn status

--- Changelist 'blame-fix':
M      lib/vclib/ccvs/blame.py

--- Changelist 'diff-fixes':
M      lib/vclib/ccvs/__init__.py
M      lib/vclib/__init__.py
$

Now, that visual grouping alone is nice, but not entirely useful. If I wanted to see just the diff-related code changes I’d made, I would still need to explicitly name the files I modified for that change on the ‘svn diff’ command line.

$ svn diff lib/vclib/__init__.py lib/vclib/ccvs/__init__.py

That’s not so bad for two files, but what about for twenty?

Fortunately, Subversion 1.5’s changelist support is more thorough than that. I can use the new --changelist option to limit the scope of many subcommands to just those files associated with a given changelist. The result is a much simpler command line, and a much smaller chance of overlooking one or more of the files I intended to examine.

$ svn diff --changelist diff-fixes
Index: lib/vclib/ccvs/__init__.py
===================================================================
--- lib/vclib/ccvs/__init__.py (revision 1157)
+++ lib/vclib/ccvs/__init__.py (working copy)
@@ -112,8 +112,8 @@
     temp2 = tempfile.mktemp()
     open(temp2, 'wb').write(self.openfile(path_parts2, rev2)[0].getvalue())

-    r1 = self.itemlog(path_parts1, rev1, {})[0]
-    r2 = self.itemlog(path_parts2, rev2, {})[0]
+    r1 = self.itemlog(path_parts1, rev1, {})[-1]
+    r2 = self.itemlog(path_parts2, rev2, {})[-1]

     info1 = (self.rcsfile(path_parts1, root=1, v=0), r1.date, r1.string)
     info2 = (self.rcsfile(path_parts2, root=1, v=0), r2.date, r2.string)
Index: lib/vclib/__init__.py
===================================================================
--- lib/vclib/__init__.py (revision 1157)
+++ lib/vclib/__init__.py (working copy)
@@ -240,7 +241,7 @@
   def readline(self):
     return self.fp.readline()

-  def close():
+  def close(self):
     try:
       if self.fp:
         self.fp.close()
$

As you might expect, I can then provide the same --changelist option to the ‘svn commit’ operation.

$ svn ci -m "Fix some diff-related oopses." --changelist diff-fixes
Sending        lib/vclib/ccvs/__init.py
Sending        lib/vclib/__init.py
Transmitting file data ..
Committed revision 1158.
$

Meanwhile, my other changelist remains undisturbed.

$ svn st

--- Changelist 'blame-fixes':
M      lib/vclib/ccvs/blame.py
$

The new --changelist option is present on many of the Subversion command-line client’s subcommands (commit, diff, info, lock, log, proplist, propget, propset, status, update, …). And, of course, there is also the new ‘svn changelist’ subcommand for adding and removing (via a --remove option) files to and from changelists. You can assign files to changelists before you even modify them. And Subversion doesn’t force you to use changelists universally, or even at all. As in most things, Subversion just gives you the tools, and lets you decide whether and how to use them. All in all, though, this new feature should provide Subversion users — especially those with Perforce experience — an easier way to manage those multiple simultaneous works-in-progress.

There are some limitations of the changelist support, though. First, the feature has path-level granularity. If you find yourself with overlapping changes which touch the same file, you’re going to have to work out a different way to deal with that overlap. Also, a changelist is private to a particular working copy — there’s no way to automatically share your changelist definition with other users of the repository. But given the problem the feature exists to solve, that shouldn’t be any big deal — if you need to share works-in-progress with others, use versioned branches.

Previous Article
From the Question Bin: Subversion Locking

Subversion supports both the copy-modify-merge and lock-modify-unlock version control models of managing ch...

Next Article
Sparse Directories in Subversion 1.5
Sparse Directories in Subversion 1.5

The last few weeks we blogged a lot about the Merge Tracking feature in Subversion 1.5.  Of course there ar...