Saturday, August 12, 2006

Search Trumps Hiearchies

I wrote a while back about Pervasive Search, and how it changed the way I find things. I find myself using search more and more versus navigating hierarchies. As developers, we tend to create lots of files, in regular strict hierarchical structure (in fact, I've been blogging about namespaces vs. packages recently as well). File system paths are now too cumbersome to endure. Instead of walking through Explorer or the tree in my IDE, I'm using search.

I use search at 2 levels. Within the IDE, I use the brilliant feature in both IntelliJ and ReSharper to "Find File" (keyboard shortcut: Ctrl-N). This lets you type in the name (or partial) name of a file and open it in the editor. Better yet, it finds patterns of capital letters in names. So, if you are looking for the ShoppingCartMemento class, you could type "SCM", and "Find File" will find it. Highly addictive. And, it works equally well in IntelliJ and Visual Studio with ReSharper (and my Eclipse friends tell me it has made it there as well).

The other place I've been using search a lot is the filesystem, when looking for either a file on which to perform some operation (like Subversion log) or looking for some content within a file. Google Desktop Search has gotten better and better. You can now invoke it with the key chord of hitting Ctrl twice. And, you can download a plug-in that allows you to search through any type of file you want, including program and XML documents. Once you've found the file in question, you can right-click on the search result and open the containing folder. This is the only way to get to some file buried deep in some package or directory structure. My coding pair and I have started using this heavily, and it has sped us up. And, it eliminates annoying repetitive tasks like digging through the rubble of the filesystem looking for a gold nugget.

Thursday, August 03, 2006

Partial Classes

When I first saw that .NET 2 supported partial classes, I groaned. It
looked like a language feature that helps one thing and hurts a dozen
more, once people start abusing it. However, I've come around to appreciate (and dare I say it, like) partial classes. They are obviously useful for code generation (which is why, I suspect) they were added in the first place). However, they are also handy for other problems.

Testing is one place where partial classes offer a better solution than the one offered by Visual Studio.NET 2005. In VS.NET, if you want to use MS-Test to test a private method, the tool uses code generation (without partial classes) to create a public proxy method that turns around and calls the private method for you using reflection. This is not a big surprise; the JUnitX add-ins in Java help you do the same thing. But using code gen for this is a smell: if you change your private method using reflection, the generated code isn't smart enough to change, so you have to do code gen again, potentially overwriting some of the code you've added. Yuck.

Here's a better solution. I should add parenthetically that I don't usually bother testing private methods (especially if I have code coverage) because the public methods will exercise the private ones (otherwise, the private methods shouldn't be there). However, when doing TDD, I sometimes want to test a complext private method. And partial classes work great for this. The example I have here is a console application that does some number factoring (why isn't important in this context). I have a method theFactorsFor() that returns the factors for an integer. Here is the PerfectNumberFinder class, including the method in question:

namespace PerfectNumbers {
internal partial class PerfectNumberFinder {
public void executePerfectNumbers() {
for (int i = 2; i < 500; i++) {
Console.WriteLine(i);
if (isPerfect(i))
Console.WriteLine("{0} is perfect", i);
}
}

private int[] theFactorsFor(int number) {
int sqrt = (int) Math.Sqrt(number) + 1;
List<int> factors = new List<int>(5);
factors.Add(1);
factors.Add(number);
for (int i = 2; i <= sqrt; i++)
if (number % i == 0) {
if (! factors.Contains(i))
factors.Add(i);
if (!factors.Contains(number/i))
factors.Add(number/i);
}
factors.Sort();
return factors.ToArray();
}

private bool isPerfect(int number) {
return number == sumOf(theFactorsFor(number)) - number;
}

private int sumOf(int[] factors) {
int sum = 0;
foreach (int i in factors)
sum += i;
return sum;
}
}
}

Rather than use code gen to test the method, I've made the PerfectNumberFinder class a partial class. The other part of the partial is the NUnit TestFixture, shown here:

namespace PerfectNumbers {
[TestFixture]
internal partial class PerfectNumberFinder {

[Test]
public void Get_factors_for_number() {
int[] actual;
Dictionary<int, int[]> expected =
new Dictionary<int, int[]>();
expected.Add(3, new int[] {1, 3});
expected.Add(6, new int[] {1, 2, 3, 6});
expected.Add(8, new int[] {1, 2, 4, 8});
expected.Add(16, new int[] {1, 2, 4, 8, 16});
expected.Add(24, new int[] {1, 2, 3, 4, 6, 8, 12, 24});

foreach (int f in expected.Keys) {
actual = theFactorsFor(f);
for (int i = 0; i < expected[f].Length; i++)
Assert.AreEqual(expected[f][i], actual[i],
"Expected not equal");
}
}
}
}

I like this because it allows me to test the private method without any messy code generation, reflection, or other smelly work-arounds. Partial classes make great test fixtures because they have access to the internal workings of the class but don't have to reside in the same file. It's dangerous to pile infrastructure on new features like this (especially scaffolding-type infrastructure like classes), but this one seems like a more elegant solution to the problem at hand than stacks of code generation.

Tuesday, August 01, 2006

Pontificating at OSCON

I gave a talk as OSCON last week on Building Internal DSLs in Ruby. Apparently, there is a fair amount of interest in this subject: I was in one of the small rooms, but it was packed to the rafters, with standing room only along the back and side walls. I didn't realize it, but John Lam took a snapshot of me in action and posted it to his blog: .
It's tough to get a good shot while someone is talking, so it shows that John is both a formidable Ruby/.NET guy and a talented photographer!

The Fact of the JMatter

Several years ago, some brilliant designers created Naked Objects, a Java framework that generates applications from domain objects. You supply the POJOs with behavior, point Naked Object at them, and you have a full-blown Swing application that allows you to edit, insert, delete, and browse the objects and their relationships. You could literally create sparse, functional applications in minutes. However, Naked Objects never got much beyond a proof of concept. The automatically generated applications were utilitarian but uninspiring.

Fast forward to now. Eitan Suez, one of my fellow No Fluff, Just Stuff
speakers, has taken the Naked Object idea and run with it. He has created the JMatter framework (found here). It takes the concepts of Naked Objects and updates it to the here and now. JMatter applications still auto-generate from POJOs, but the user interface and interactions are very rich. The sample application that appears on the JMatter web site literally took less than 2 hours to create; written by hand, it equates to developer-weeks worth of effort. It also illustrates a growing trend in development: creating framework and scaffolding code automatically, freeing developers to focus more on producing applications. We've seen this approach done well in Ruby on Rails. JMatter shows that you can apply the same concepts to Swing development. Eitan has released JMatter with a MySQL-style license, so it's worth jumping over to his site to get a preview of the future.

Friday, July 21, 2006

DSLing @ OSCON

I'm off to Portland, Oregon next week (my first ever trip to Oregon, so I can knock that off my travel map at World66), speaking at my first OSCON. I'm doing a talk on Building DSLs in Ruby, based on material that Jeremy, Joe, Zak, and I have produced for the Pragmatic Press book upon which we are working (slowly). I'm also signed up for some pre-conferecnce tutorials, including a 4 hour talk about VIM (I just had to see someone use VIM for 4 hours - I expect it to be quite impressive).

If you are in Portland, look me up. I speak on Thursday, and have some meetings on the other days, but mostly I'll be hanging around. A bunch of my No Fluff friends will also be there, so there may be some Magic games or even some Settlers of Catan.

Tuesday, July 18, 2006

Boy Scout Capabilities

I was having a conversation with a co-worker today whose first name prominently features the letter "Z". Our topic: how does a company like ThoughtWorks, which hires lots of experienced developers, determine at what level that person should be hired. Some candidates are cut and dried: you can tell when you interview them. But what about the developers who fall through cracks? Maybe they are an ace developer in 4 languages, but they've never done agile. Or, a great militant Agilist, but they have never done test-driven development. As a company, we need 2 things: how to categorize these folks upon hiring and, more importantly, how to fill in knowledge gaps after they arrive. After all, the ultimate goal is to create well rounded ThoughtWorkers, who are good at all the things we value highly.

In talking about this subject, I came up with the idea I called the Merit Badge approach. Just like in the Boy Scouts, when a scout moved from one troop to another, you knew their rank instantly because of the acquired merit badges. Each merit badge had deterministic acceptance criteria, and you knew that the scout in question had mastered the badge criteria before moving to the next one. A certain number of badges, covering a certain set of areas, lead to increased rank. If a company like ThoughtWorks wants all Eagle scouts, we must invest in our rookie scouts to enable them to get to that level. We should have technology merit badges. If we get a good candidate that knows everything but TDD, we should send them to a TDD training class or similar until they have mastered that skill. Advancements in the technical ranks becomes an exercise is acquiring useful skills. That keeps the process more objective and allows for clear ascension paths through the technical ranks. The People People can track the merit badges and recommend training and mentoring for the next milestone.

And, we'd all get to wear those cool sashes!

CJUG Redux

I'm speaking at the Chicago Java Users Group tonight (the Downtown one), giving my No Fluff, Just Stuff talk entitled The Productive Programmer, based on material from the book that David Bock and I are (slowly) working on for Pragmatic Press. It's completely technology agnostic, so if any .NET guys want to crash the party, feel free (sure to generate lively conversation). First-time attendees pay no dues or admission, so that makes this a really, really cheap date for you and your significant other.

Sunday, July 16, 2006

Ubiqui-GPS

GPS technology has suddenly gotten really cheap, and I've taken advantage of it in 2 big ways. First, I managed to get a GPS watch from woot.com for a great price, which includes the arm-mounted GPS receiver, for urban running. It's so accurate, it provides miles per hour in real time while you are running. The other cool use of GPS is the updated version of Microsoft's Streets and Trips. This mapping software used to be nice-to-have for road warriors, now it has moved to essential because it includes a small GPS receiver. You arrive in a foreign city with only the hotel address, punch it in, and you have turn-by-turn directions, spoken via your laptop's speakers, with the traced-out route on the screen. Having Streets and Trips on a laptop is better than having one of the little Palm-sized units because a) I'm taking my laptop with me everywhere anyway and b)the screen on the laptop is much bigger and nicer. The only downside is that you've got to be within a couple of hours of your destination or have a car adaptor for your laptop.

GPS has reached the point where it is cheap, available, and plentiful. My friend Scott Davis has a nice keynote presentation at No Fluff, Just Stuff this year where he argues that location based services will be very important in the near term. The combined technologies of cheap GPS, mashup applications that leverage tools like Google maps, and the growing awareness in software of actual location suggests rich applications beyond what we've got now. If we can just get all this down to the phone level, the only thing left will be flying cars.

Thursday, July 13, 2006

EKON X

For the past 8 years, end of September has meant a trip to Frankfurt au Main, speaking at the Entwickler Conference, the premiere developer's conference in Germany. This year it has grown an additional name (EuroDevCon) but it will always be EKON to me. EKON X will kick off the last week of September. This conference and I have sort of grown up together. It used to be primarily a Borland-tool focused conference, but they have expanded the offerings to encompass all different development platforms and tools. I started at this conference way back in 1998, talking about Delphi topics. This year, I talk about SOA, Productivity (based on The Productive Programmer book), and and Agile development in .NET. Over the years, I've gradually migrated from Delphi into Java and .NET stuff.

I've done this conference so many times, it's a natural part of the year. I look forward to this great conference and my good friends in Germany, who I see only once a year. Terry and I will also be running our 5th Berlin Marathon before the conference. The happy conjunction between EKON and Berlin Marathon is great. I'm looking forward to it!

Sunday, July 09, 2006

The Persistent Persistence Question

On my current project, we faced the inevitable, persistent, annoying question of which persistence framework to use. We boiled it down to 2 choices: nHibernate or iBatis. As usual, it was not a cut-and-dried decision, as each had their strengths. nHibernate, being a meta-data mapper, writes all the annoying SQL for you, which is a huge time saver...when it can. However, when talking to complex legacy schemas, nHibernate gets tougher and tougher to configure. iBatis, on the other hand, doesn't try to generate your SQL. It just takes care of the object-relation mapping for you, from SQL you supply. That makes it much better for complex schemas, stored procedures, etc. So, which to use?

In the end, we chose both! We estimated that maintaining separate configurations would take a little time, but it would save us time on both sides: letting nHibernate do its magic when it can, and falling back to iBatis when it made more sense. It has worked out very well. We have a couple of very complex queries being handled gracefully by iBatis, and nHibernate handles all the simple persistence in the application. Sometimes, seemingly mutually exclusive options actually complement one another.

Wednesday, June 28, 2006

Fixing Subversion with Command Line Judo

As many of you know, David Bock and I are writing a book about programmer productivity, which includes a fair amount of command line judo. Here's a little piece that I developed for my project.

Occasionally, Subversion breaks with an error that references a path that includes /!svn/ and another line that talks about status 200. (Sorry, I don't have an exact replica right now because it's fixed. I’ll try to capture one in the wild next time I find one.) This seems to happen when someone checks in a binary that is being used by another application: Word document, Excel spreadsheet, or PDB file.

When this happens, you must go through your directory structure to identify the bad file. This is cumbersome (more in a second about this). Once you have found the bad file, remove it from the repository view (i.e., remove it directly from the repository, not your local file system) and do an update. To find the file, you must go to every directory in your local copy of the repository and do an update in that directory. The first directory whose update breaks holds the problem file, which you must find by trial and error. You can do this with Tortoise, but it takes forever.

If you have the foresight to have Cygwin (and therefore the Bash shell) on your machine, you can issue this command (in Bash) from the root directory of your repository:


find . -type d | grep -v "/.svn" | xargs svn up

This command finds every directory, eliminates the ".svn" folders, and pumps the directory name into "svn up". The first directory that breaks when this command executes is your problem folder. Problem solved.

My educated guess on our repository is that it would take 20 minutes to use Tortoise to update every folder, one at a time. It took me about 10 minutes to develop this little bit of command line judo. So, in this case, automating the solution actually took less time than the brute force approach. But, even if the automated approach takes longer to develop, you have a tool for the next time it happens.

Stay tuned for a bunch more stuff like this when we get The Productive Programmer done!

Wednesday, June 21, 2006

Cheeburger, Cheeburger, cheeps, pepksi

One of my coworkers doesn't have a blog, but he successfully identified another sacred cow that we're converting to cheeburgers on our project. Yet, he's so altruistic, he won't take credit for the idea, saying that it's Marjorie's idea instead. So that he can remain anonymous, I'll just call him Zak T. No, that's too obvious, let's call him Z Tamsen instead.

Zak's sacred cow is the convention in the .NET world of using Pascal casing for namespaces, which is a terrible idea. We've already run into situations where a namespace clashes with a class name, which is annoying. So, we've decided to make all our namespaces all lower case, ala Java, with underscores to separate logical names (very un-ala Java).

Namespaces in .NET are particularly broken, and not just the capitalization convention. This is one of the things that Java got really right, but in a very subtle way. One of the early annoyances for most Java developers is learning the concept of a root directory, and how the package names of your code must match up to the directory structure. Once you grok that concept, though, it makes perfect sense. Only much later do you figure out that this is one of the subtle elegances of the Java language. Because the package structure is tied to the underlying directory structure, it is difficult to create class names that overlap because the OS won't let you create 2 classes by the same name in the same directory. Score one for Java, leveraging the OS to prevent headaches for developers. Of course, with the advent of virtual directory structures in JAR files, you can now create conflicts, but it is thankfully still rare.

Namespaces in .NET have no such useful restrictions. It is trivially easy to create name conflicts because the namespace is just an arbitrary string. Most of the .NET developers I know (especially if they've done any Java) use the useful "namespace must match directory structure" convention (with less restrictions on the actual root folder). In fact, one of my colleagues, Yasser, has created a very useful utility called Namespacer that verifies that your .NET class' namespace matches the directory structure. After some use on our project, he's planning to open source it. Short of fixing namespaces in .NET, at least there is a way to verify the adherence to a useful convention.

Sunday, June 18, 2006

Being Productive in Cincinnati

I'm speaking at the Cincinnati Java User's Group (CINJUG) on Monday, June 19th. I'll be presenting my talk The Productive Programmer, based on the book upon which David Bock and I are diligently but slowly working. This has become a pretty popular talk with both user groups and at No Fluff, Just Stuff shows, and I keep finding more stuff to cram into it. Soon, I'm going to have to start pruning the tips or convert it into a Part 1, Part 2 affair. If you are in Cincy, stop in and say "Hi".

Tuesday, May 30, 2006

Practices of an Agile Developer

I just finished reading Practices of an Agile Developer by Venkat Subramaniam and Andy Hunt. What a great book! I found myself nodding vigorously on virtually every page. For developers new to agility, this is an invaluable resource. And, to those of us who have been in the agile space for a while, it's good to see smart people proselytizing the same stuff I preach all the time. I particularly like the format Venkat and Andy have chosen, presenting common wisdom at the start of each section, then thoroughly debunking it in the body of the section.

Even if you are already a die-hard agilista, this book is a worth read. Highly recommended.

Monday, May 29, 2006

Improving Agile Communication using Old Tools

By popular demand, I'm blogging about a communication tool we're using on our current project. I've discussed this in my Productive Programmer talk at No Fluff, Just Stuff, and I've answered the Expert Panel question of "What is your latest favorite productivity tool" with this answer. Several people have asked me follow-on questions, so I thought I'd blog about it.

One of the difficulties in distributed agile development is keeping the communication link strong between the geographically (and time zone) separated teams. We are trying hard on our current project but still fall well short of the ideal. We do have some bright spots, though. The primary communication medium between the developers is a wiki we set up for the project. For a while, we attempted to type in really comprehensive summaries of each day's development work. However, we eventually realized that we were duplicating effort: we already put detailed comments for our check-ins to Subversion. So, we had one of our temporary resources cook up the following little developer shim.

He created a tool called SVN2WIKI. It uses the SVN post-commit hook to harvest the comment of the code just checked in. It then posts those comments to the Wiki, creating a dated page if one doesn't exist or adding to the page already there if it does. The Wiki we're using (Instiki) offers an RSS feed for all changed pages. So, we installed an RSS Reader (RssBandit) on the developer workstations. Now, when a developer sits down, he or she can get an up-to-the-minute summary of all the stuff that has happened to the code base since the last time he or she looked. Because it's an RSS reader, it keeps track of what you've already read. This is a great way to keep up to date at a really detailed level for what is happening to the code base.

This hasn't eliminated the need to create daily summary pages, but these can be much more terse, and focus on outstanding questions across the ocean. The Wiki contains a living history of the project, told one check-in at a time. For those who say that agile projects don't keep documentation, the Wiki on our project is a living, breathing history of the project at a really detailed level.

Our SVN2WIKI tool is a good example of piecing together a bunch of old and common technologies (SVN, Instiki, RSS) to create a great time saver for developers while improving the toughest part of our project.

Wednesday, May 17, 2006

The Ajax Experience Recap

I just finished speaking at The First (but certainly not the last) Ajax Experience, in San Francisco. It was held in the beautiful St. Francis hotel in downtown San Francisco, and it had the who's who of the Ajax world there. Ben Galbraith and Dion Almaer, the creators and maintainers of the Ajaxian web site, along with Jay Zimmerman of No Fluff, Just Stuff fame, put on a first-class conference. It was an interesting conference in that Ajax doesn't exist in a vacuum: it must be hosted on top of some other technology. I told one of my friends that it was like a condiments conference: you can't really have Ajax without some medium to present it upon. In any case, it was interesting to see such a diverse crowd rub elbows and get along so well. At one of the expert panels, we had the Lead Program Manager on the IE7 team sitting next to the creator of JavaScript. On another panel, the evangelist for the Microsoft Atlas framework sat next to the creator of Dojo (and 2 seats down from me). I represented pretty much the entire testing track there, showing a packed room how to use Selenium to test Ajax applications.

Because Ajax is at once broad (represented by the number of frameworks) but diverse (because it can be applied to just about any underlying web technology), I wondered if this conference would be a success. I can safely say that it was a resounding success, and it's going to happen again in the fall on the East coast. Kudos to Ben, Dion, and Jay for a great experience.

Buy 2, In Case You Lose the First

The No Fluff, Just Stuff anthology (edited by yours truly) is now orderable (they have gone to the printer, meaning that you can pre-order them from Pragmatic Press or get the PDF version right now). Check out the book page on the Prag's site to see it, order it, and fetishize it. I suggest that you buy at least 2, in case you lose one. And, nothing say lovin' to your spouse like an anthology of technical articles. Great wedding gifts, too.

Tuesday, May 09, 2006

Spreading the DSL Virus

Everywhere I go now (in a technical context anyway), I'm associated with the idea of Domain Specific Languages. At the No Fluff, Just Stuff expert panel this last weekend in Denver, my friend Scott Davis introduced me (we each introduced the member of the panel sitting on our right), and mostly what he said about me can be paraphrased as "he's the Typhoid Mary of DSL's". When I mentioned DSL's in answering a "Why is Ruby cool?" question, Ted Neward (the moderator) jokingly told me to not talk about it anymore.

But it's spreading further afield. I was at the Microsoft Technology Summit last week, and asked a DSL related question of Don Box when he was giving an Indigo talk. Afterwards, I chatted with him for a while about DSL's. Apparently, I got his attention. This week, he posted a blog entry looking for me to explain what the hell it was that I was talking about at MTS06. His blog entry and my reply is here. I pointed him to a great blog entry from my co-worker Jay Fields to illustrate to Don the power of this technique (found here).

As my regular reader(s) may know, I'm currently working on a book on DSLs for Pragmatic Press. The author team of Joe O'Brien, Jeremy Stell-Smith, Zak Tamsen, and myself are working hard to spread this virus far and wide.

Monday, April 24, 2006

Eating Sacred Hamburger

Software development cults tend to create sacred cows: habits and idioms that might have meant something at one time but only remain as baggage now. I tend to like to kill sacred cows and grill them up, with some nice lettuce, tomato, and a sesame seed bun. On my current project, we're actively killing some sacred cows.

Here are a couple of examples. Thankfully, Hungarian Notation has mostly been banished, except for one lingering, annoying location in the .NET world: the stupid "I" preface on interfaces. In fact, if you understand how interfaces should be used, this is exactly the opposite of what you want. In our application, every important semantic type is represented by an interface. Using interfaces like this makes it easier to do a whole host of things, including mocking out complex dependencies for testing. Why would you destroy the most important names in your application with Hungarian Notation telling you it's an interface? Ironically enough, that your semantic type is an interface is an implementation detail -- exactly the kind of detail you want to keep out of interfaces. I suspect this nasty habit developed in the .NET world because interfaces first came to the Microsoft world as COM (or, back when it started, OLE). It's a stupid cow now, and should be slaughtered.

Another sacred cow we're gleefully grilling up is the rule that all method names must use camel case. We're using this standard convention in our code, but have started using underscores for our test method names. Test methods tend to be long and descriptive, and it's hard to read long camel case names. Consider this test name:

[Test]
public void VerifyEndToEndSecurityConnectivityToInfrastructure()
vs. this version:

[Test]
public void Verify_end_to_end_security_connectivity_to_infrastructure()

Which of these is easier to read? The standard in .NET says that you use camel case, which we do...except in situations where it actually hampers productivity. If a cow gets in my way and slows me down, it's a goner.

In the book Pragmatic Programmer, Dave Thomas and Andy Hunt admonish developers to learn a new programing language every year. Seeing new ways of doing common tasks and learning new idioms is the best defense against sacred cows. Learning new languages helps you focus on how and why things work the way they do, divorced from syntax.

Wednesday, April 19, 2006

Coming Soon...The Ajax Experience

Jay, the creator of No Fluff, Just Stuff, has started conducting single-topic, destination conferences. The first was last year, The Spring Experience, where he brought together the entire Spring universe in Florida for 3 days.


This year, he's doing it again with The Ajax Experience. This amazing show takes place in San Franciso, May 10 - 12th (the week before JavaOne). It features the entire Who's Who of Ajax luminaries (and some dim lights, like me). I'm going to talk about testing Ajax applications using Selenium. And that's no coincidence. Jay has tried to get the creators of each of the parts of the Ajax world together, and I'm talking about Selenium because it was created by ThoughtWorks. Check out the web site and come to San Francisco. It should be an amazing 3 days.