Fixing Subversion with Command Line Judo

As many of you know, David Bock and I are writing a book about programmer productivity, which includes a fair amount of command line judo. Here's a little piece that I developed for my project.

Occasionally, Subversion breaks with an error that references a path that includes /!svn/ and another line that talks about status 200. (Sorry, I don't have an exact replica right now because it's fixed. I’ll try to capture one in the wild next time I find one.) This seems to happen when someone checks in a binary that is being used by another application: Word document, Excel spreadsheet, or PDB file.

When this happens, you must go through your directory structure to identify the bad file. This is cumbersome (more in a second about this). Once you have found the bad file, remove it from the repository view (i.e., remove it directly from the repository, not your local file system) and do an update. To find the file, you must go to every directory in your local copy of the repository and do an update in that directory. The first directory whose update breaks holds the problem file, which you must find by trial and error. You can do this with Tortoise, but it takes forever.

If you have the foresight to have Cygwin (and therefore the Bash shell) on your machine, you can issue this command (in Bash) from the root directory of your repository:

find . -type d | grep -v "/.svn" | xargs svn up

This command finds every directory, eliminates the ".svn" folders, and pumps the directory name into "svn up". The first directory that breaks when this command executes is your problem folder. Problem solved.

My educated guess on our repository is that it would take 20 minutes to use Tortoise to update every folder, one at a time. It took me about 10 minutes to develop this little bit of command line judo. So, in this case, automating the solution actually took less time than the brute force approach. But, even if the automated approach takes longer to develop, you have a tool for the next time it happens.

Stay tuned for a bunch more stuff like this when we get The Productive Programmer done!

Cheeburger, Cheeburger, cheeps, pepksi

One of my coworkers doesn't have a blog, but he successfully identified another sacred cow that we're converting to cheeburgers on our project. Yet, he's so altruistic, he won't take credit for the idea, saying that it's Marjorie's idea instead. So that he can remain anonymous, I'll just call him Zak T. No, that's too obvious, let's call him Z Tamsen instead.

Zak's sacred cow is the convention in the .NET world of using Pascal casing for namespaces, which is a terrible idea. We've already run into situations where a namespace clashes with a class name, which is annoying. So, we've decided to make all our namespaces all lower case, ala Java, with underscores to separate logical names (very un-ala Java).

Namespaces in .NET are particularly broken, and not just the capitalization convention. This is one of the things that Java got really right, but in a very subtle way. One of the early annoyances for most Java developers is learning the concept of a root directory, and how the package names of your code must match up to the directory structure. Once you grok that concept, though, it makes perfect sense. Only much later do you figure out that this is one of the subtle elegances of the Java language. Because the package structure is tied to the underlying directory structure, it is difficult to create class names that overlap because the OS won't let you create 2 classes by the same name in the same directory. Score one for Java, leveraging the OS to prevent headaches for developers. Of course, with the advent of virtual directory structures in JAR files, you can now create conflicts, but it is thankfully still rare.

Namespaces in .NET have no such useful restrictions. It is trivially easy to create name conflicts because the namespace is just an arbitrary string. Most of the .NET developers I know (especially if they've done any Java) use the useful "namespace must match directory structure" convention (with less restrictions on the actual root folder). In fact, one of my colleagues, Yasser, has created a very useful utility called Namespacer that verifies that your .NET class' namespace matches the directory structure. After some use on our project, he's planning to open source it. Short of fixing namespaces in .NET, at least there is a way to verify the adherence to a useful convention.

Being Productive in Cincinnati

I'm speaking at the Cincinnati Java User's Group (CINJUG) on Monday, June 19th. I'll be presenting my talk The Productive Programmer, based on the book upon which David Bock and I are diligently but slowly working. This has become a pretty popular talk with both user groups and at No Fluff, Just Stuff shows, and I keep finding more stuff to cram into it. Soon, I'm going to have to start pruning the tips or convert it into a Part 1, Part 2 affair. If you are in Cincy, stop in and say "Hi".

Practices of an Agile Developer

I just finished reading Practices of an Agile Developer by Venkat Subramaniam and Andy Hunt. What a great book! I found myself nodding vigorously on virtually every page. For developers new to agility, this is an invaluable resource. And, to those of us who have been in the agile space for a while, it's good to see smart people proselytizing the same stuff I preach all the time. I particularly like the format Venkat and Andy have chosen, presenting common wisdom at the start of each section, then thoroughly debunking it in the body of the section.

Even if you are already a die-hard agilista, this book is a worth read. Highly recommended.

Improving Agile Communication using Old Tools

By popular demand, I'm blogging about a communication tool we're using on our current project. I've discussed this in my Productive Programmer talk at No Fluff, Just Stuff, and I've answered the Expert Panel question of "What is your latest favorite productivity tool" with this answer. Several people have asked me follow-on questions, so I thought I'd blog about it.

One of the difficulties in distributed agile development is keeping the communication link strong between the geographically (and time zone) separated teams. We are trying hard on our current project but still fall well short of the ideal. We do have some bright spots, though. The primary communication medium between the developers is a wiki we set up for the project. For a while, we attempted to type in really comprehensive summaries of each day's development work. However, we eventually realized that we were duplicating effort: we already put detailed comments for our check-ins to Subversion. So, we had one of our temporary resources cook up the following little developer shim.

He created a tool called SVN2WIKI. It uses the SVN post-commit hook to harvest the comment of the code just checked in. It then posts those comments to the Wiki, creating a dated page if one doesn't exist or adding to the page already there if it does. The Wiki we're using (Instiki) offers an RSS feed for all changed pages. So, we installed an RSS Reader (RssBandit) on the developer workstations. Now, when a developer sits down, he or she can get an up-to-the-minute summary of all the stuff that has happened to the code base since the last time he or she looked. Because it's an RSS reader, it keeps track of what you've already read. This is a great way to keep up to date at a really detailed level for what is happening to the code base.

This hasn't eliminated the need to create daily summary pages, but these can be much more terse, and focus on outstanding questions across the ocean. The Wiki contains a living history of the project, told one check-in at a time. For those who say that agile projects don't keep documentation, the Wiki on our project is a living, breathing history of the project at a really detailed level.

Our SVN2WIKI tool is a good example of piecing together a bunch of old and common technologies (SVN, Instiki, RSS) to create a great time saver for developers while improving the toughest part of our project.

The Ajax Experience Recap

I just finished speaking at The First (but certainly not the last) Ajax Experience, in San Francisco. It was held in the beautiful St. Francis hotel in downtown San Francisco, and it had the who's who of the Ajax world there. Ben Galbraith and Dion Almaer, the creators and maintainers of the Ajaxian web site, along with Jay Zimmerman of No Fluff, Just Stuff fame, put on a first-class conference. It was an interesting conference in that Ajax doesn't exist in a vacuum: it must be hosted on top of some other technology. I told one of my friends that it was like a condiments conference: you can't really have Ajax without some medium to present it upon. In any case, it was interesting to see such a diverse crowd rub elbows and get along so well. At one of the expert panels, we had the Lead Program Manager on the IE7 team sitting next to the creator of JavaScript. On another panel, the evangelist for the Microsoft Atlas framework sat next to the creator of Dojo (and 2 seats down from me). I represented pretty much the entire testing track there, showing a packed room how to use Selenium to test Ajax applications.

Because Ajax is at once broad (represented by the number of frameworks) but diverse (because it can be applied to just about any underlying web technology), I wondered if this conference would be a success. I can safely say that it was a resounding success, and it's going to happen again in the fall on the East coast. Kudos to Ben, Dion, and Jay for a great experience.

Buy 2, In Case You Lose the First

The No Fluff, Just Stuff anthology (edited by yours truly) is now orderable (they have gone to the printer, meaning that you can pre-order them from Pragmatic Press or get the PDF version right now). Check out the book page on the Prag's site to see it, order it, and fetishize it. I suggest that you buy at least 2, in case you lose one. And, nothing say lovin' to your spouse like an anthology of technical articles. Great wedding gifts, too.

Spreading the DSL Virus

Everywhere I go now (in a technical context anyway), I'm associated with the idea of Domain Specific Languages. At the No Fluff, Just Stuff expert panel this last weekend in Denver, my friend Scott Davis introduced me (we each introduced the member of the panel sitting on our right), and mostly what he said about me can be paraphrased as "he's the Typhoid Mary of DSL's". When I mentioned DSL's in answering a "Why is Ruby cool?" question, Ted Neward (the moderator) jokingly told me to not talk about it anymore.

But it's spreading further afield. I was at the Microsoft Technology Summit last week, and asked a DSL related question of Don Box when he was giving an Indigo talk. Afterwards, I chatted with him for a while about DSL's. Apparently, I got his attention. This week, he posted a blog entry looking for me to explain what the hell it was that I was talking about at MTS06. His blog entry and my reply is here. I pointed him to a great blog entry from my co-worker Jay Fields to illustrate to Don the power of this technique (found here).

As my regular reader(s) may know, I'm currently working on a book on DSLs for Pragmatic Press. The author team of Joe O'Brien, Jeremy Stell-Smith, Zak Tamsen, and myself are working hard to spread this virus far and wide.

Eating Sacred Hamburger

Software development cults tend to create sacred cows: habits and idioms that might have meant something at one time but only remain as baggage now. I tend to like to kill sacred cows and grill them up, with some nice lettuce, tomato, and a sesame seed bun. On my current project, we're actively killing some sacred cows.

Here are a couple of examples. Thankfully, Hungarian Notation has mostly been banished, except for one lingering, annoying location in the .NET world: the stupid "I" preface on interfaces. In fact, if you understand how interfaces should be used, this is exactly the opposite of what you want. In our application, every important semantic type is represented by an interface. Using interfaces like this makes it easier to do a whole host of things, including mocking out complex dependencies for testing. Why would you destroy the most important names in your application with Hungarian Notation telling you it's an interface? Ironically enough, that your semantic type is an interface is an implementation detail -- exactly the kind of detail you want to keep out of interfaces. I suspect this nasty habit developed in the .NET world because interfaces first came to the Microsoft world as COM (or, back when it started, OLE). It's a stupid cow now, and should be slaughtered.

Another sacred cow we're gleefully grilling up is the rule that all method names must use camel case. We're using this standard convention in our code, but have started using underscores for our test method names. Test methods tend to be long and descriptive, and it's hard to read long camel case names. Consider this test name:

[Test]
public void VerifyEndToEndSecurityConnectivityToInfrastructure()

vs. this version:

[Test]
public void Verify_end_to_end_security_connectivity_to_infrastructure()

Which of these is easier to read? The standard in .NET says that you use camel case, which we do...except in situations where it actually hampers productivity. If a cow gets in my way and slows me down, it's a goner.

In the book Pragmatic Programmer, Dave Thomas and Andy Hunt admonish developers to learn a new programing language every year. Seeing new ways of doing common tasks and learning new idioms is the best defense against sacred cows. Learning new languages helps you focus on how and why things work the way they do, divorced from syntax.

Coming Soon...The Ajax Experience

Jay, the creator of No Fluff, Just Stuff, has started conducting single-topic, destination conferences. The first was last year, The Spring Experience, where he brought together the entire Spring universe in Florida for 3 days.


This year, he's doing it again with The Ajax Experience. This amazing show takes place in San Franciso, May 10 - 12th (the week before JavaOne). It features the entire Who's Who of Ajax luminaries (and some dim lights, like me). I'm going to talk about testing Ajax applications using Selenium. And that's no coincidence. Jay has tried to get the creators of each of the parts of the Ajax world together, and I'm talking about Selenium because it was created by ThoughtWorks. Check out the web site and come to San Francisco. It should be an amazing 3 days.

Walking on the Wrong Side of the Street

The first time I ever went to Sydney, Australia, I couldn't get in sidewalk sync. It seemed like every time I walked down the sidewalk, I was walking head-long into groups of people. It was almost like I was swimming upstream. Then it dawned on me: these folks are used to driving on the left-hand side of the road, so they tend to walk on the left-hand side of the sidewalk too. Suddenly, it made sense, and my sidewalking skills improved overnight.

The same thing occurred to me this morning while I was walking from my hotel to the offices in ThoughtWorks India, and I noticed that everyone here (also being a former British colony) walks and drives on the left-hand side. Which side you tend towards when walking is one of those intrinsic, automatic things that you don't even realize is part of your cultural literacy until you go somewhere where it's the opposite.

The same kind of automatic response happens when developing software. I span the Java and .NET worlds, and I think it's interesting to see the kinds of cultural ticks that programmers from specific platforms develop. For example, in the Java world, you build frameworks for everything. In the .NET world, you organize things around the limitations of namespaces and solution files in Visual Studio, because it's the only real choice when it comes to IDEs. I have an advantage because I live part time in both worlds, so I try to spot cultural eccentricities whenever I sense myself walking on the wrong side of the street. Some things make sense in both cultures (like unit testing), while others are little more than arbitrary cultural baggage.

Dynamic Typing in the Strangest Places

One of my favorite things in college was the accidental synergy that happened between the collection of classes taken in a single quarter. Sure, compiler theory and automata match one another, but it's even more interesting when a history and computer science class find ways to complement one another. I get some of the same type of cross-pollination of ideas when speaking at conferences -- some of the topics end up complementing each other in odd ways.

It happened recently in St. Louis at No Fluff, Just Stuff. Part of the weekend, I was talking about how document centric messaging in SOA avoids endpoint versioning headaches. Later in the weekend, I was talking about the flexibility afforded by dynamic languages and their loose typing. Then, it occurred to me: SOA and the document centric approach is really just another version of dynamic or loose typing. If you free the plumbing from having to know what types you are passing (in this case, just a blob of XML), you create a more flexible system, able to withstand changes more gracefully. Just like with dynamic languages.

I've been sauteing in this idea for a while: we are building the same kind of application right now on the project I'm on. And, even though it's written in .NET 2, we take advantage of loose typing in the transport layer, which ultimately makes for a more robust application. How do we handle catching version conflicts? The way you should handle all such situations: unit testing. Loose typing + unit testing provides the same security you get with strongly typed languages but much more flexibility

Finally, An Accurate Representation of My Graphical Skills

During a lunchtime conversation today, because of the specific context, I came up with a great quote about my abilities as a graphical designer:

Graphical design is like pornography: I know it when I see it, I don't want to participate in it, and you don't want to see me try it either.

'Nuff said.

A Dearth of Fluff, Significant Stuff

As my regular reader(s?) must have noticed, I've been very quiet for a while. No, I haven't given up pontificating; I've been working diligently on a new year of No Fluff, Just Stuff presentations. Every year, I cull some of my old talks from the herd and send them to the virtual glue factory and add some new, spry colts to the fold. This year, I've added about 4 new talks and killed the same number, and I've been spending the last couple of months getting them in shape. This last weekend was the debut of a couple of them (along with 5 of my existing talks).

No Fluff, Just Stuff Gateway Software Symposium occurred in St. Louis over the weekend. I gave a total of 7 talks: clean Up Your Code, Advanced Enterprise Debugging Techniques, SOA & ESB: The Future of Distributed Computing or the Return of the Son of CORBA?, Pragmatic XP, the new Testing with Selenium, the also new The Productive Programmer, and finally Language-oriented Programming and Language Workbenches. Whew! It was a long weekend, but the new talks turned out well (only requiring minor tweaks, not major overhauls). And, someone was nice enough to blog about one of my talks (always good to get feedback).

It's great to get back to No Fluff. Many of the speakers are good friends, so it's good to see them again. One of the speakers made an excellent observation that also applies to me: I have more friends that travel the country every weekend than I do at home! It's good to be back in the saddle again.

Dependency Injection in Geronimo, Part 2

As surely as Spring follows winter, the 2nd part of my depenency injection article on DeveloperWorks is now live. This version builds on the "raw" topic of dependency injection in the 1st part and shows how it applies to Geronimo. Geronimo has a very interesting architecture, and will surprise some people who think that "dependency injection" == "Spring".

My Last Borland Post Ever (I Promise!)

Those of you who follow my blog know that I post an inordinate amount of material about Borland. I have a long-standing love for Borland: my first book was on Delphi (Developing with Delphi: Object-oriented Techniques) and my second was on JBuilder (JBuilder 3 Unleashed). I learned about real software development using Borland tools. When I bought my first computer, I bought just 2 pieces of software: DOS 3.3 and Turbo Pascal 5. And I didn't need any other software for almost a year.

Borland has perpetual financial and directional problems over the years. But, like loyal fans of Saturday Night Live, most of us stuck around even when it sucked (Inprise, anyone?)

Now, though, Borland has taken the last fatal step towards irrelevance: they are in the process of divesting their entire IDE division (check out this eWeek article). Yes, that's right: the company that brought us Turbo Pascal, Turbo C, Turbo Prolog (OK, maybe we didn't need that one), Delphi, and JBuilder will no longer sell a single compiler. I wonder what Anders thinks about this (or if he even noticed).

Borland is reinventing itself as a strictly ALM (Application Lifecycle Management) company, selling tools that have awesome support for Big Design Up Front (BDUF) like CaliberRM, TogetherJ, and the Segue tools they just acquired. What a great day for Waterfall projects (I wonder if they would consider sponsoring the Waterfall 2006 conference)?

For those of us with an irrational (pun intended) love of Borland, this is indeed a dark day. RIP.

Continuous Integration Conference

An interesting (free!) conference is coming to Chicago in early April this year, an Open Space event on Continuous Integration and Automated Testing. Both of these subjects are dear to my heart, so I'm hoping to help get the word out about this conference. They organizers haven't set the date yet because they need to see how many people are interested before they will know how big a space to reserve.

Support CI and AT: contact Jeffrey Fredrick (jtf@agitar.com) or Paul Julius (pj@thoughtworks.com) through the Yahoo group groups.yahoo.com/group/citcon and sign up for the mailing list.


Automation and Testing are Good Things!

Dependency Injection in Geronimo Part 1

IBM DeveloperWorks has posted the first of a two-part series on dependency injection in the Geronimo J2EE container. This article focuses on the mechanisms of DI and uses PicoContainer as the container. I did this to divorce the idea of DI from the particular implementation in Geronimo. One of the problems I see in concepts like DI revolves around how you learn it: the implementation from which you learn influences your perception of if forever. Many developers think that DI and the way Spring works are the same thing. By using the lightest weight container I could find (PicoContainer), I hope to show DI in as context-free an environment as possible. The next part of the article takes the concepts and injects (pun intended) them in to Geronimo.

SOA Under a Giant Arch

I'm flying to St. Louis tonight to speak at the Gateway JUG. I'm doing my SOA: The Future of Distributed Computing or the Return of the Son of CORBA? talk, helping to promote the upcoming No Fluff, Just Stuff Gateway Software Symposium. Because there is so much sound and fury with so little substance in most SOA discussions, I'm trying to take a pragmatic approach to building SOA, which includes the (what I consider essential) focus on agile development techniques. I feel like I can offer some insight on this subject because I've lived through a lot of distributed computing initiatives and seen a lot of failures. This talk is about how to achieve some of the lofty goals espoused by SOA advocates with a healthy dose of skepticism.

D-Cubed

I recently read about a most interesting innovation in software development methodology I've heard in a while: Defect Driven Design (D3, pronounced "D Cubed"). This methodology addresses the weak points of most other development methodologies.

Working in D3 is very simple. At the start of the project, you announce that all development is complete and the application is ready for user acceptance testing. The first thing the user will say is "Hey, I thought there was supposed to be an icon on the desktop to launch the application". That's our first defect. We implement code to put an icon on the desktop and announce again that we are done. The user clicks on the icon and nothing happens: "Where is the application?" That's our next defect. You can imagine what the rest of the development rhythm.

The brilliance in this methodology lies in the estimation power. We can estimate with absolute accuracy: 0 days. The entire lifecycle lies in maintenance, which is more realistic because useful software is never actually complete.