Static Typing is Communist Bureaucracy

In a conversation that came up at No Fluff, Just Stuff Denver this last weekend, a fundamental realization came to me. We were talking about dynamic languages like Ruby and Groovy, and a lot of the Java developers just couldn't get over the idea of dynamic typing. They feared that utter chaos would reign if we discard typing. First, I asked them about the significance of testing: "Do we all agree that testing is important?" to which they replied "Of course". And, in the course of the expert panel, we had discussed the appropriateness of code coverage, what it tells you, what it cannot tell you, etc. The general consensus was that code coverage is a Good Thing, and that we should all aspire to 100% as much as possible. After all, it's the tests that tell you if your code if correct or not.

Once we had agreed that testing + code coverage is a good thing, I realized: Static typing is Communist Bureaucracy. "To get bread, you must stand in the line over there, but first you must get a blue stamp on your booklet." If you have pervasive testing, static typing == more typing. The static typing is nothing but a requirement to type extraneous code to satisfy a compiler that isn't telling you anything interesting anymore.

I've said it before and I'll say it again: testing is the engineering rigor of software development. Testing destroys the need to static typing. Dynamic typing speeds up development because there is physically less to type. And testing verifies the correctness of your code, not just that it compiles.

Justifying Automation

One of the topics that comes up in my Productive Programmer talk about automation is how to justify it. Sure, everyone knows that automation is a Good Thing. And, if you can demonstrate that you can automate something as fast as you can do it by hand once, that's a total no-brainer. But what about the grayer areas where you aren't sure how long its going to take to automate something, and you aren't sure if you can justify the time it takes to the Powers That Be?

I have a couple of strategies for this that I'm including in The Productive Programmer book. First, make sure you timebox your efforts. Even if the end result of the timebox is that you create another one (because you've made such good progress), that prevents the automation from turning in to a week-long exercise when you thought it would just take 1/2 a day. Timeboxing also cuts down on the inevitable yak shaving that goes on anytime you start experimenting with something you haven't done before.

Automation is all about risk mitigation. For the thing you are thinking about automating, ask yourself the following questions:

• How long does it take to do it by hand now?


• How many times are you going to have to do it before the end of the project?


• What is the consequence of doing it wrong one time (i.e., forgetting a crucial step)?

If you take the product of the first item (how long does it take now) times the second (how many times are you going to have to do it), that tells you what kind of time investment you have now. If that time investment is longer than what it would take you to automate it, no brainer again: automate it. The third item is about risk mitigation: if it's disastrous if you miss a step, that should make you lean towards automating it. For example, I was on a project not long ago where (for bizarre historical reasons) they didn't want to separate the compiled versions of their tests between unit, functional, and integration. Their solution was to create hand-crafted test suites. But, that means that we have to remember to add them by hand every time we created new ones. That's a big risk: I'm never going to remember that! Instead, we automated it by creating a little Groovy script that ran at build time to auto-generate the test suites based on the root directories. The risk of not automating this problem was wasted time: we wrote a test and forgot to include it by hand, meaning that we don't get the benefit of the test and thus wasted the time it took to write it.

I'm curious if others have tried-and-true strategies for justifying automation...

From Architect to Wrangler

One of the nice things about ThoughtWorks is the ability to create your own title, which leads to some interesting business cards (like a title of "Seat Warmer"). But, the restriction is that you only get one set of business cards, so choose wisely. Instead of going nuts on my first set, I chose the ultra-conservative "Application Architect", which is a reasonable approximation for what I do on projects.

But now I've changed. The title of "Architect" for software developers has gotten so diluted that its meaningless anymore. In fact, its almost pejorative because so many "paper" architects give it a bad name. I've gotten "Oh, dude, I'm so sorry" looks from people when I tell them I'm an architect, assuming that I've had a head injury or something and can't do real development work anymore. After long formulating and advice from others, I've now changed my title to the probably more apt "ThoughtWorker/Meme Wrangler". As you've probably noticed, I love the "meme" meme (after all, you are reading this on the "memeagora" blog). While it's a little cutsy, at least I'm not treated as a tragic has-been by coworkers and clients.

The interesting side effect of this happens because I put my title on my presentation slides. In the US, but particularly overseas, people ask me "What does mee-mee wrangler mean?" So, I get to explain it.

But that's way better than the now deprecated "Architect" title. It's a shame that, because we have no real industry-wide certifications, the nominally most advanced title has been co-opted by so many people divorced from reality. I've had to defend decisions made for SOA initiatives in front of "Architectural Review Boards" by people who last wrote code in COBOL. Can they really make good decisions about modern technology if they never touch it?

Git along, now, little neuron...

Keeping Knowledge in Plain Text

The book The Pragmatic Programmer is absolutely worth re-reading every few years, because every page has stuff on it that's still relevant. One of the important lessons is "Keep knowledge in plain text". To paraphrase, you shouldn't keep important information in a proprietary piece of software, because if the software dies (because the company goes out of business) or it relies on a changing protocol (more about this in a second), your knowledge is trapped. If you keep it in plain text, you can read (and manipulate it) using a lowly text editor.

Two things made me realize this in a Big Way recently. The first was my ToDo list. I used to keep all my ToDos in a good (but fatally proprietary) piece of software named LifeBalance. It's the best of that category of software, but using it meant that I had to have a Mac OS X, Windows, and Palm client to access it. It became its own project just to keep it in sync. And, unless LifeBalance works instantaneously, adding items takes time. I was reading the LifeHacker book (distillation of the awesome LifeHacker web site) earlier this year, and one of the first items is an entry about a text-based ToDo list manager called ToDo.txt (which has its own website). It's called "Task tracking for command line lovers". The nice thing about ToDo.txt is that it uses a plain text file for storage. And, the author has written a bunch of bash/batch files to help manage it. That means you can append stuff of your own (using QuickSilver, in my case) or you can use the tools. The command line tools also let you prioritize, archive, and a bunch of other stuff. It's a good marriage between command line and plain text.


The other driving home of the important "Keep knowledge in plain text" lesson concerns my blog. For years, I just wrote my blog entries in a text editor in HTML and uploaded them by hand to Blogger. Last year, I fell for a seductive blog editor/manager called MarsEdit. It works fine, allowing you to create drafts, view your blog, handle multiple blogs, etc. The way it did this was via using one of Blogger's APIs for managing blogs.


Then something terrible happened: Blogger changed their API. Suddenly, MarsEdit didn't work anymore. Posting new entries wasn't a problem because I could go back to doing it by hand, but what amount my 9 months of existing, historical blog entries it held? Trapped! Now, I can't use it, and I can't easily free all my information. I violated the rule!


I just spent a couple of hours fixing this mess, re-syncing my original directory structure by hand (at least I was able to use wget to slurp my whole blog down to my local machine, making it much easier). I've felt the pain of violating the rule, and I promise not to do it again. My last step was removing MarsEdit from my machine (which, BTW, has now updated itself to work with the new API -- unlike our president, I remember the phrase "Fooled me once, shame on you, fooled me twice, shame on me").

Of course, Dave and Andy already knew this, and they told me, but I wasn't listening. Sometimes, you have to suffer for good advice to take hold.

Don't Be a Hater -- Think Different

People still fear what they don't understand or what's not familiar. When I arrived at the SOA conference in Kuala Lumpur a couple of days ago, I was met by the hotel's AV guy to hook up my Mac to the projector. I used the normal elaborate procedure I usually do to sync my Mac with a projector: I plugged the projector into my machine. But, unusually, no signal. I tried Display Preferences, changing the resolution, the number of colors: nothing. He was convinced that my exotic, bizarre machine/OS was the problem, and I kept trying to convince him otherwise. Finally, I persuaded him to go get another cable, which he reluctantly did (I don't know any Malay curse words, but I'm pretty sure I've now heard some). You can probably guess what happened next: new cable, perfect signal, at all my original settings.

Which reminded me of another conversation I had a couple of years ago with a friend in Singapore. We were sitting around talking about computers, and he made an off-hand comment that he didn't like Macs because he thought they were unintuitive, but he thought Windows made perfect sense. So, I asked him "So, you're telling me that the operating system where you click on the Start button to turn it off is the more intuitive on? I think you're just used to it", to which he replied: "You're right, I'll shut up now". I saw him again last night in Singapore, and we were catching up on all sorts of stuff. And you know what he told me? He just got a new machine with Vista on it, used it for 2 weeks, and returned it to Dell. He told me that he can't wait until his laptop finally dies because he's going to get a Mac. The reason for the change? He's now addicted to his iPod, and it's made him realize that Design Matters.

I can't resist beating everyone on the head with the moral: Don't succumb to FUD because something is different. And, in the tortured grammar of the old Apple campaign, Think Different. Because Vista is so different from previous versions, it may be the best thing that ever happened to Apple. If you're contemplating a change anyway, why not look at the entire field so that you end up with the OS equivalent of an iPod, not a Zune.

Pontificating about SOA

What else is there to do about SOA? Back in December, Mark Richards (Enterprise Architect at IBM) and I sat down and recorded a video PodCast about SOA (which we informally named "Taking Back SOA"). The results has now appeared at the No Fluff, Just Stuff web site. Mark and I have a very pragmatic approach to SOA, and we both hate the level of hype that appears in that whole space. He and I had a great time recording this video, which proves once and for all that IBMers and ThoughtWorkers can get along after all!

No, I Won't Stop Talking about IntelliJ!

During my No Fluff, Just Stuff talk on The Productive Programmer, I got a comment at the Minneapolis show to "stop showing us IntelliJ shortcuts -- we all use Eclipse!" And, Venkat Subramaniam blogged about the whole Eclipse vs. IntelliJ debate that came up at the expert panel in that same city. Venkat quite eloquently gives his reasons for choosing IntelliJ: it simply makes him more productive. And the same is true for me. I've used all the major Java IDE's in anger: NetBeans, JBuilder (I used to be considered a JBuilder expert, but I got over it), Eclipse, and IntelliJ. For me, IntelliJ works best. Period. In fact, I think that IntelliJ would land in my top five pieces of software of all time list. I also edited the No Fluff, Just Stuff Anthology chapter on IntelliJ tips and tricks, just out in treeware.

There are actually 2 IDE Tips and Tricks chapters in the anthology. When I sent out call for contributions from the authors and friends for IntelliJ tips, I got a flood of them, all very cool (and a few that I didn't already know about, like the Key Promoter). Ted Neward (who edited the Eclipse chapter) and I sent out the same call for Eclipse tips and tricks, and we got none. We sent out another call, and a few trickled in. And that is reflected in the book. I'm sure we're going to get grief over the IntelliJ chapter having more cool stuff. But that's really the whole point: like the preponderance of Macs on the No Fluff, Just Stuff tour, the people who use IntelliJ are passionate about it, because it exudes excellence. Like Venkat says, many people using Eclipse are in a bad arranged marriage. It's hard to drum up passion for an arranged marriage.

I feel like it is my duty to try to turn people on to things that make their life better. That's part of what being a speaker is about. So, I'm not apologetic about proselytizing IntelliJ. Every-time I find a tool that I think will make developer's lives easier, I'm going to talk about it. And, if I develop a passion about it, so be it.

PodCasting About Groovy

A couple of weeks ago, I recorded a PodCast for AboutGroovy.com, talking about (obviously) Groovy, DSLs, testing, and how to insinuate Groovy into your company's infrastructure. Basically, Scott Davis and I had the kind of conversation we have all the time at No Fluff, Just Stuff. The only difference is that he was recording it. It was fun and the content will come as no surprise to anyone who knows me.

Being a Consultant Makes You Tough (or Stupid)

Since working for ThoughtWorks these 2 years, I've gotten much tougher in the face of being able to work on short or bizarrely interrupted sleep. Case in point: I left Minneapolis (after speaking at the sold-out No Fluff, Just Stuff Twin Cities Software Symposium) on Saturday evening, dashing from my last session at 4:45 straight to the airport. 8 hours later, I was in Amsterdamn, with a 2 hour layover. 12 hours after that, I'm in Kuala Lumpur, arriving at 6:15 AM. I probably got about 8 hour of sleep, but never more than 2 hours at a time. I'm now exactly on the other side of the world from where I live: 12 time-zones away. I cleared customs, got my suitcase, and headed for the car service the conference had arranged. Traffic was awful, and combined with the distance of the airport from downtown KL, I arrived at the hotel at 8:35 AM...and the conference starts at 9 AM with me facilitating. Quick shower, dash upstairs, and I'm ready to go at 9 AM sharp. I then proceeded to talk on and off for 8 hours. I managed to stay up until after 8 PM. It's now the next day, and I feel fine. I'm apparently adjusted to the new time zone (I sleep soundly last night, interrupted only by my phone ringing at 11 PM (or, 11 AM in the states)) which I promptly ignored.

Two years ago, I could have never done this. Just part of the implicit training all the travel and distributed work instills. But there are benefits as well. Here is a picture taken just outside the hotel's meeting rooms.

The Elusive Right Click

Since he got his Mac, Venkat Subramaniam and I have been playing around with pushing the envelope on human computer interaction and productivity as hard as we can. He finds a cool new keyboard shortcut and can't wait to tell me, I learn a new QuickSilver bit of magic and can't wait to tell him. The Holy Grail of this kind of pursuit is finding a way to generate a mouse right-click using only the keyboard. Of course, Mac OS X is pretty friendly to use the track pad, but that's not enough for us: total keyboard control is the goal.

And yesterday I stumbled upon the Holy Grail. I was in the process of shaving a yak, and was waiting for something to finish, so I decided to solve this problem Once and For All. I did a bunch of Googling and experimentation and stumbled upon the solution. Go to Universal Access and turn on Mouse Keys. This the the accessibility option that allows you to drive the mouse strictly from the keyboard, using the embedded keypad. Once you turn that on, you can use the FN-I key to click the mouse, which means that you can use FN-CTRL-I to right click. It's not perfect (it clicks where the mouse current resides, not where the keyboard focus lives) and it precludes using the embedded numeric keypad for doing 10-key entry (which I never do anyway). You can turn it on and off by clicking the OPTION key five times (this is a setting on Universal Access as well).

But we're happy. Venkat got as excited as I've ever seen him when I showed him this morning; I was looking forward to show it to him all day. Ah, simple pleasures!

Don't Misquote Me to Create a Straw Man!

OK, I can't let this pass.

The ServerSide wrote an article, summarizing both mine and Venkat Subramaniam's talks at The ServerSide Symposium last week. It's a reasonable summary, with a few details left out (that's why it's a "summary").

But then, Cedric Beust takes the summary and constructs a agile development straw man to beat up. And, like many straw man arguments, he takes lots of stuff out of context and makes his own conclusions not supported by the original talk.

The first thing he gets wrong is the fault of the summary. Early in the talk, I clearly state that the best way to get good software is to interact directly with the users. And I give an example from my project history of the best project experience I every participated in, where the developers literally sat in the middle of the users. Every time we had a question, we just got up and resolved it directly with the users. It was project heaven.

But most corporate environments won't give you such direct access to the users, because the users are after all there to get work done. Thus, you must have user proxies. And that's where the business analysts come it. Perhaps at ThoughtWorks, our business analysts are different from the "typical" ones (I know our project managers are). The best business analysts are direct proxies for the users. They understand the users concerns and a little about technology. During the talk I also made the point that good business analysts prevent "paving the cowpath", suggesting innovative business solutions to problems rather than "that's the way we've always done it".

When I say that business analysts participate in the estimation process, that's because we can't get the users directly.

Do you know the difference between a good developer and a bad developer? A good developer can give you an estimate of the time needed to complete a task. And the better that developer is, the more accurate their estimate will be. If I ask a developer "How long is it going to take to do this?" and he answers with "It's very complicated", I want him off my team.

Another point I made in my talk is that, regardless of developer, you can't ask developers "how long will this take". Every project is different: in a 2 man project with no meetings and direct access to users, each developer will get more done in less time. In a project with 16 developers and a 2-hour status meeting every day, each developer will be much less effective (this has been known since Brook's The Mythical Man Month). But that is constant is the relative complexity of the story you are trying to complete. Relative to the other stories, not the entire universe. This allows developers to hone their estimation skills because it cuts down on the number of variables they have to hold in their head. This is the closet you can get to an objective measurement of the work required to complete a piece of software.

We don't need agile consultants and book authors to sell us vague and unproven metrics along with rigid guidelines that your team must embrace without any questions.

Ironically enough, we agree wholeheartedly about this, but he doesn't seem to know that this is exactly the point of my talk at The Serverside Symposium. The whole point of the talk was that we need real metrics like passed tests, real reconciliation between estimates and actuals, and binary completion criteria. The article makes that pretty clear, but that doesn't contribute to the weak straw man, so Cedric conveniently left that out.

What's most frustrating about Cedric's rant is the implicit assumption that "agile" is this vague concept that no one seems to be able to define. Yet, at ThoughtWorks, every single project is agile to some degree (and where we control everything (like fixed bid projects), it's pretty close to Extreme Programming). And we have a really high success rate, way above the industry average. I've been around for a while (one of my colleagues once told me that I was old enough to "add diversity to the company"), and I've used a bunch of different methodologies. One of the reasons I'm at ThoughtWorks is because we use proven techniques for building software, and we're damn good at it, making it a much less frustrating place to work.

DSL Podcast on No Fluff, Just Stuff

As my regular reader(s) know, I'm big on Domain Specific Languages, and have been talking and writing about this style of development for a while. During the course of last year, we recorded some PodCasts at some of the No Fluff, Just Stuff shows. One of those shows was Chicago, where I recorded a PodCast about my feeling about DSLs and this approach to building software using this technique. The PodCast has now appeared on the No Fluff, Just Stuff web site, for your listening pleasure.

Anthologization Redux

Well, it's that time of year again: the No Fluff, Just Stuff Anthology, Volume 2 (The 2007 Edition) is at the printers, undergoing the conversion from thought to dead trees.


Just like the blurbage on the back says:

Once again, some Sixteen of the world's best trainers and speakers are writing chapters on things they care passionately about. You'll find topics from the latest conferences including Groovy, JavaScript, Continuations, Web services and REST, JVM Byte Code, and Agility

These essays are a summary of the latest thinking in the industry, and range from the philosophical to the tutorial, covering the topics that the writers felt were the most important for readers today. If you feel like the neatest technology and latest ideas are passing you by, this book can help bring you back you to speed.

It's all good stuff, without any fluffy filler, as these essays are based on presentations given at the incredibly popular "No Fluff, Just Stuff" symposium series. Twenty-six times a year, the symposium visits a city and the speakers and attendees share ideas and perspectives. The speakers are all internationally known experts in their field.

So, to out and buy one as soon as possible (in a book store near you in early April). Or, order it directly from the Prags or at Amazon. In fact, I recommend that you buy two in case you loose the first.

Of course, I was intimately involved in its creation, but this is a really entertaining read (the main reason I like editing the Anthology is because it means I can read the essays as soon as possible).

Enjoy!

Garrulous Software

A particularly annoying characteristic of software these days is excessive chattiness. This occurs at both the user and developer level. In fact, one of the things I prefer on the Mac vs. Windows is the lack of meaningless messages the Mac generates (No, I do NOT want to clean off my desktop right now!). Pestering messages at the OS level distract from your attention and diminish your productivity. In The Humane Interface, Jef Raskin discusses the locus of attention, which is the work you are trying to perform. Anything that moves your locus of attention had better be a) relevant to your work and b) urgent and important. In fact, I think its a good idea to turn off the automatic notifications of email arrival, especially if it interrupts real work. Experts on "flow" maintain that it takes up to 20 minutes to get back to that deep concentration state, and it gets harder as the day goes on to overcome the inertia required to get back to that flow state.

The problem in the development world is much worse. It seems like the norm now to generate tons of meaningless noise. Think about application server startup, scrolling messages into the console faster than you can read it. 99% of that information is just the application server saying, over and over again like an idiot, "I'm OK...I'm OK...I'm OK.............I'm OK" (that last one was an EJB loading). The only thing that allows you to see errors is the massive stack traces that come from errors in the J2EE world. In fact, this problem is so bad that developers have taken to dubious practices to make their own messages to stand out from the flood of useless information. Admit it...you've put a Systen.out.println() in your J2EE code to debug it (a whole other subject). And, to make yours stand out from the noise, you put 30 or 40 "~"'s in front of it. Of course, your coworker is doing the same thing, and they use 30 or 40 "#"'s for theirs. Before long, application servers look like they have Tourette's Syndrome, spewing mostly meaningless noise.

My pair and I encountered this very problem recently. We were working with client code that makes heavy (handed) use of Log4J to spew out messages to the console. Unfortunately, that makes for ugly unit test results: you can't see the interesting stuff for all the immense noise being generated by the code under test. In this case, it wasn't as simple as turning off the logging level (the code under test was creating an instance of something that had its own logging stuff buried deep inside). So, after a little thought, we figured out how to shut it up.

In out test case, we unplug the normal output stream, like so:

protected void setUp() throws Exception {
  super.setUp();
  originalOut = System.out;
  System.setOut(new PrintStream(new DumbOutputStream()));
}

And, in tearDown, we put it back:

protected void tearDown() throws Exception {
  super.tearDown();
  System.setOut(originalOut);
}

Our "DumbOutputStream" isn't stupid, just quiet:

 private class DumbOutputStream extends OutputStream {
   public void write(int b) throws IOException {
     //do nothing, because I'm dumb.
   }
 }

This effectively shut off the fountain of noise, leaving the testing framework's output (which occurs on System.err, not System.out) alone. Sometimes, you have to work hard just to shut something up.

Inside Edition: Star Wars

Warning: this blog post is a complete waste of your time!

On the Relevance site, I saw a link to this fascinating analysis of the connections between the first Star Wars trilogy (episodes IV-VI) and the second one (episodes I-III). As much as I'm loathe to admit it, I've done some thinking along these same lines, especially since 2 things have recently happened: 1) I got a new HDTV and 2) HBO has turned into the "All Star Wars Station" (but only for episodes I and III -- they must not have rights to II). You should probably read the analysis before you go on (or it won't make much sense). I'll wait.

In fact, the only flaw I can find in the analysis is the fact that the Jawas weren't at first going to sell R2D2 to the Skywalkers -- it was only after the other similar droid blew up (maybe R2D2 did something to him in the hold of the Jawas' ship?) But it kills the idea that the reason they ended up at the Skywalkers was some sort of deal with the Jawas (unless they double crossed him). See, I told you this was a waste of time, right?

Actually, I distinctly remember interviews with Lucas back in the late 70's where he talked about a trilogy of trilogies (it was supposed to go episodes IV-VI, I-III, and then VII - IX). He has since retracted that statement (and claims he never made it). Whatever. Given how much work it is, I completely understand.

But I always thought that it would be cool in episode IX for C3P0 and R2D2 to suddenly reveal that they are in fact the keepers of the force, much like some of Asimov's robot stories where the robots turn out to be the guardians of mankind while pretending to serve him. And, if you take C3P0 out of the equation (in light of the analysis above), then R2D2 can be said to be the guardian of the good part of the force. That would be cool!

When I want to waste some more of your time, remind me to tell you about the plot idea I have for the origin of the Borg...

More Sony eReader Pix

By popular demand, I'm posting a few more pictures of my Sony eReader, to convey a better impression of the resolution modes I described in my previous post.

The first shows the smallest eBook resolution:



The second shows the largest eBook size (Mr Magoo Mode):



The third shows a Pragmatic Press PDF (Best of Ruby Quiz), in the best viewing mode (rotated to landscape, zoomed in to text width):



I hope that satisfies everyone's curiosity! If you want to feel and touch one, I saw them on sale at my local Borders in Atlanta last weekend.

Sony eReader

Those who know me know that I'm a gadget nut, and I've gotten a few gadgets lately that fall into the category of essential for my on-the-road lifestyle and of interest to software developers and other geeks as well. One of them is the Sony eReader. This is the first real application of digital ink, which has been The Next Big thing for several years While it looks like a monochrome LCD display (with no back-lighting), digital ink acts more like real ink in that it doesn't consume power while it's being displayed. Most LCDs need a constant (very low level) charge to keep their dark bits aligned. With digital ink, you just zap them once and they stay in place until you zap them again. The obvious use of them in a digital book reader makes perfect sense: it consumes no power while you are actually reading the page, just when you turn the pages.

The Sony eReader was announced at the beginning of 2006, and was originally going to be available in the Spring of last year. Fast forward to November, when they actually shipped (I got mine late November, and I was on the list of "ship it too me right off the assembly line". Here's a picture:



The Good
Reading eBooks on it is an awesome experience, pretty much all I hoped for. The screen is highly readable as long as you have light (it does not have a back light). Essentially, if you have enough light to read a regular book, you can read this one. Even in low light it isn't bad because you have 3 zoom levels for eBooks: small, medium, and large (Mr. Magoo could read the large without his glasses). The page flips are slow because of the digital ink, but it's not really noticeable. I quickly forgot that I was reading something on a new fangled device and just enjoyed it, and have subsequently read more than a dozen books on it. The battery life is stunning: I used it for an entire week in Singapore, reading a couple of hours a day, without recharging it once (and that included 2 marathon reading binges on the 26 hour flights there and back). The eBooks (and even PDFs) take up very little room. It comes with about 100 Mb of storage; I immediately put a 2 Gb Memory Stick in it. I guesstimate that I can probably get 1000 books on this puppy.

The Bad
One of the great appeals for me was the ability to put PDFs on it. I carry (or would like to carry) a fairly sizable collection of reference books with me, in addition to whatever I'm reading at the moment, and lots of books (especially technical ones) exist on PDF. Alas, the reading experience with PDFs leaves something to be desired. Unlike eBooks, the eReader can't reflow PDFs, meaning that you can basically see them in 1 of 2 sizes: regular (the entire page) or zoomed (just the text on the page, eliminating the margins). The text of most PDFs is still too small for portrait mode even in the zoomed state, so I generally flip it to landscape. One of the annoying bugs bites you here: when you flip pages on PDFs in landscape mode, it re-sets itself to regular mode but still thinks it's in zoomed. That means to be able to read the page, you have to flip it back to regular (for real), then flip it back to landscape. That means that every time you flip pages, it's a 3-press operation. Which wouldn't be so bad except for the glacially slow refresh rate for the digital ink (which is hardly noticeable for a single page flip). Once they get this bug fixed, PDF's will be reasonable on this device, but painful for now.

The Ugly
The selection of eBooks is pretty small right now, but growing fast. As you would expect, there are a fair number of science fiction titles, including Neal Stephenson's whole catalog (which I have dully downloaded and and synced onto my eReader). The worst part of that whole experience, though, is the Connect software that comes with it and manages both syncing with the device and shopping at their on-line bookstore. Strike 1: Windows only, so I have to crank up Parallels to access it. Strike 2: Think iTunes written by a high school student who's just learned VB. It's awful and clunky. The should have picked a reasonable starting place (like Eclipse RCP) rather than reinvent a buggy version of an application that needs a file system view, a HTML-centric content pane (or pain, in this case), and standard menu stuff. Strike 3: buggy, crashing at the drop of a hat. One of the few options on the menus is "Update", which I access frequently to try to get a better version. While it works mostly, it also locks up on a regular basis. At least the syncing part pretty much works and is fast. It really shows, though, how much you take something like iTunes + iPod for granted until you see an amateurish version.

The Bottom Line
The above notwithstanding, I love this little thing, especially for eBooks. If they would get the PDF bug mentioned above fixed, it would be Good Enough for PDFs too. Of course, what I would really like is the ability to reflow PDFs, but thats much tougher. If anyone reading this knows of a PDF to eBook converter, please let me know...I've searched in vain. The Sony eReader has some pain points (like the PDF bug) for a version 1.0 product, but it still a great addition to my collection of gadgets.

Why I Hate Christmas Music

Most of my family thinks I'm a curmudgeon because I hate Christmas music. Generally, I don't bother to go into the full details of why I hate it because it is long and drawn out, and chances are slim that they would understand anyway. But here's why.

Music has a special place in my life. Like a lot people in the software industry, I have an intense relationship with music. Ever since I discovered music in a big way when I was in the 7th grade, it has been a constant companion. Literally. I always have some music playing in my head, always in the background. There is no way for me not to have music playing in my head. I wake up with music, spend my whole day with it, and go to sleep at night with it. And therein lies the problem.

I hate catchy tunes because the stick in my head and won't go away. What most people like about pop music is the very thing that makes me dislike it: the hook. The problem is that those little snippets of music get stuck in my head and persist. Hours later, I find the same track playing over and over, an increasingly irritating background noise. After a while, I have almost a sore spot in my conscience, a groove worn deep by the same little track playing over and over. The catchier the tune, the harder it is to exorcise. It's like a cut on the roof of your mouth: constant low level irritation.

This explains several things about my music taste. It explains why I have 1000+ CDs and counting: I must have variety. It also explains why I like so much music that most people find disturbing and discordant. I crave complex music all the time. My wife, Candy, complains that I don't like any "happy" music because I have such a taste for the unusual: the anti-catchy tune.

And that also explains why I don't like Christmas music. By definition, all Christmas music features catchy tunes, designed for humming and singing along. When I was a kid, I enjoyed it as much as the next person. But is has a cumlative effect. Every year, the grooves in my head worn deep from the past get worn deeper. I find myself in a constant bad mood around Christmas because the music is inescapable. This last year, they were playing it on the airplane as they taxied, at exactly the time you are forbidden to use some other music player to drown it out. My head was filled with those tunes for hours afterwards, boring their way through my gray matter. I defensively wore my iPod in every store that I went during the holiday season

Christmas music isn't the only thing that affects me. At the client where I am currently stationed, some fool has his cell phone play "When the Saints Come Marching In" as his ring tone. For the first few weeks I was here, I would wonder why, at the end of the day, my head was full of that irritating tune, playing over and over, drilling its way through my head. It's like inadvertanly stepping in something sticky, and now every step pulls irritatingly on your shoe. I figured that it must be a cell phone somewhere, so I started listening for it. Sure enough, someone in the vast cube farm over my right shoulder gets about 10 calls a day, polluting the air around me with that insidious song. I wondered why coming to work here put me in such a bad mood: now I know. Because I know it's there now, I try to fight it off everytime it rings with some other music that is soothing (to me at least) that I can summon from my mental reserves. It's not cool here to were my iPod or I would, all day, every day. And the worst part is, I can't really complain because I can't find this person (the office is large enough so that I can't home in on the signal before they answer it) and they would think I'm crazy. Just like my family thinks I'm a scrooge.

Technology Snake Oil Part 11: Customizable Rube Goldberg Machines

I've found myself more and more talking to people about SOA because of my background with distributed computing. Over and over, in these discussions with CIO's, CTO's, and CCYAO's, they keep talking (without realizing it) about Customizable Rube Goldberg machines.

Rube Goldberg is a cartoonist whose name made it all the way to the dictionary:

Rube Golderg - n. A comically involved, complicated invention, laboriously contrived to perform a simple operation.


Rube Goldberg was a Pulitzer Prize winning cartoonist, sculptor, and author who lived from 1883 - 1970. He is well remembered for his cartoons (thus the dictionary entry), some of which everyone has seen (maybe without realizing it). Here is Rube Goldberg's "Simplified Pencil Sharpener":



Many SOA architectures look just like this diagram, with similar unintentionally ironic names. As I wrote before in Entropic Software, software tends towards complexity. It seems doubly so that enterprise architecture tends towards (frequently needless) complexity. Every SOA pursuit I've ever seen could stand some significant simplification. Of course, most consultants won't say that; they view monstrously complex software as job security. That is perhaps one of the differences from mercenary consultants who are only in it for the buck (complexity == $$$ + long periods of time == job security). If, on the other hand, you have someone who is trying to implement what you need, the tendency should be towards simpler architecture and systems. This isn't a slam against big consulting firms (I know some guys who work for the biggest who really care about what they do, and they would agree with my assertion).

The Snake Oil here comes from C(x?)O's who believe that Rube Goldberg-like complexity is inevitable, and this myth is perpetrated by mercenary consultants. Like Rube's "How to Tee a Golf Ball without Bending Over":



The key to successful distributed computing lies with extreme simplification. The most scalable distributed architecture in the universe (as far as we know) is based on a very simple protocol: HTTP. The key to successful SOA endeavors is two-fold: simplicity and testing.

You can read more about Rube Goldberg and see a whole gallery of his work at www.rube-goldberg.com.

Writing Assignments

I haven't been blogging a lot lately because I've so tied up with other writing stuff. I have a huge number of writing projects ongoing. Ironically enough, I've been writing more than ever, just on longer-term stuff than blogging (and, for various reasons, my primary blog writing time, which is sitting in airports, has been down lately). Here are the projects:

• I'm editing the No Fluff, Just Stuff Anthology, Volume 2: The 2007 Edition as we speak. Just like last year, this one is filled with contributions from No Fluff, Just Stuff authors (14 of them this year) ranging from Web Services, language stuff, testing, agility, persistence, and a whole bunch more. We've also retained the very popular The Authors Speak chapter, where the authors talk about books, favorite tools, and (new this year) favorite travel horror stories. We've also added a couple of collaborative chapters, one on Eclipse Tips and Tricks and another on IntelliJ Tips and Tricks, with contributions by authors who use those environments. This one is bigger than last year's, weighing in at over 350 pages. It'll be out in March of 2007.
  


• 
  The Productive Programmer, for Pragmatic Press, is the book David Bock and I have been working on for about a year. We keep coming up with good ideas and our writing time keeps getting OBE (Overtaken By Events). But, he and I are really starting to crank out some pages because we really want it out in the first half of 2007. Look for it to pop up on the Pragmatic Press website soon as a beta book.
  


• 
  I'm also involved in a project with 3 other authors (Zak Tamsen, Jeremy Stell Smith, and Joe O'Brien) on writing DSLs in Ruby. Frequent readers of my blog know that this is a big topic for me, and my co-authors have come up with some awesome examples. This book is also for Pragmatic Press, and we've been cranking away at the pages recently, after spending several months discussing and generating examples. This book will blow some people's minds!
  


• 
  I'm also contributing to another anthology which shall go unnamed at the moment. Thankfully, that was just a couple of articles (one solo, one collaboration) that is done and off my plate for now. I wrote about Polyglot Programming, greatly expanding on the ideas presented in my blog entry of the same name.
  

Think I've got enough writing pending? Whew! Hopefully, if it doesn't kill me, I'll have 2 books and 2 anthologies out in 2007.

Finding the balance between short-term, idea fountain type writing in blogs and longer term, more cohesive writing for books is an increasing challenge for those of us who write a lot. Some of my colleagues (like Jay Fields) have essentially written a book, one blog entry at a time. If he published his blog between covers, he'd have one of the influential DSL books on the market. I've been trying to talk him into doing just that. It just goes to show how something as staid as technical publishing is not immune to seismic shifts because of the very topics that appear in their books.

ClearType

My friend and college Karan scolded me the other day when I told him about ClearType, which every user of Windows should use on their laptops. He was scolding me because I hadn't blogged about it, and he and I see each other rarely these days (as I'm no longer in Chicago). So, this is for you, Karan. I'll try to be better in the future when I find stuff like this.

ClearType is one of the great innovations produced by Microsoft R&D. Microsoft has a massive R&D effort world wide. Last year, I attended the Microsoft Tech Summit, and some of the coolest stuff we saw came from the head of R&D's presentation. ClearType is a utility that you download from Microsoft's site (here). It does sophisticated font smoothing, making fonts look much better on LCD displays. I'm not nearly so sensitive to fonts as some of my more aesthetically inclined colleagues (hey, David F., I talkin' about you), who can tell you in minute detail about kerning, anti-aliasing, and a bunch of other arcana. So, it's a big deal that I can tell that ClearType makes a noticeable difference. It makes the fonts look less jagged, and in the latest versions, it allows you to adjust the effect to get the best possible display.

If you are using Windows (still), and especially if you are using a laptop, you should download ClearType and give it a try.

Ruby, Meet the Enterprise

A lot of people (myself included) are convinced that Ruby is going to be the next major language (regardless of the platform it runs upon: native, JRuby, or the CLR). And, consequently, Ruby will have a major impact on enterprise development. In fact, I think that SOAR (SOA + Ruby) will be hugely popular in the coming years. As I've stated before, Ruby is the revenge of the Smalltalk crowd. It has a similar feel to developers but isn't hamstrung (like Smalltalk was) by proprietariness and expensive tools. Where Smalltalk failed, Ruby will succeed. Ruby is perfected positioned to become a major player in the SOA space: loose typing == flexibility, which is what SOA is all about.

Interestingly enough, if you look at the guys who were really into Java in 1995 and 1996, they are almost all heavily involved (and mostly billable now) in the Ruby world. Of course, this isn't an accurate barometer, but there is no good way to tell what is the Next Big Thing. Looking at people who've identified it before is as good a way as any.

Anticipating Ruby's move into the Enterprise world, next year features erubycon, the first Enterprise Ruby Conference. This is highly important to the Ruby community who wants to promote Ruby in the enterprise. Having a conference dedicated to this topic helps validate both Ruby's readiness for the enterprise and vice versa. I predict that, like RubyConf, this conference will be small this year, and in 5 years it'll be massive.

Dynamacism

The Pragmatic Programmer contains the outstanding piece of advice that you should learn a new language every year. It not only gives you more tools to apply to problems, but it encourages you to think in different idioms. Every language has its own set of patterns and styles (the original Kernighan and Ritchie book on C, known affectionately as the "K&R" book) was as much about how C should look as what it did. People still talk about the "K&R style"; they defined the idioms of C along with the language. Learning new idioms and styles makes you a better developer, thus the efficacy of the Prags advice.

Here's a recent example that I perpetrated. After a stint in .NET and Ruby on Rails, I'm back on a Java project for a while (and coding again in my beloved IntelliJ, even if it is on Windows). I ran across some code that caused my refactoring alarm to go off. Here's a snippet:

  
public void record(String toBeRecorded) {
    MessageWriter messageWriter = null;
    try {
        messageWriter = new MessageWriter(_destination);
        messageWriter.write(toBeRecorded);
    } catch (RuntimeException rx) {
        throw rx;
    } finally {
        if (messageWriter != null)
            messageWriter.close();
    }
}
public void record(List<String> listToBeRecorded) {
    MessageWriter messageWriter = null;
    try {
        messageWriter = new MessageWriter(_destination);
        for (String s : listToBeRecorded)
            messageWriter.write(s);
    } catch (RuntimeException rx) {
        throw new RuntimeException("Cannot write list of messages");
    } finally {
        if (messageWriter != null)
            messageWriter.close();
    }
}

This is an overloaded method that does basically the same stuff before and after the important stuff (writing a single message or a list). Because I've been in languages with closures for a while, I refactored it thusly:

public void record(final String toBeRecorded) {
    recordMessage(new Runnable() {
        public void run() {
            _messageWriter.write(toBeRecorded);
        }   
    });                                                   
}
public void record(final List<String> listToBeRecorded) {
    recordMessage(new Runnable() {
        public void run() {
            for (String s : listToBeRecorded)
                _messageWriter.write(s);
        }
    });
}
private void recordMessage(Runnable recordingStrategy) {
    try {
        _messageWriter = new MessageWriter(_destination);
        recordingStrategy.run();
    } catch (RuntimeException rx) {
        throw new RuntimeException("Cannot write list of messages");
    } finally {
        if (_messageWriter != null)
            _messageWriter.close();
    }
}

This is basically just the Command Design Pattern. But before I was using closures on a daily basis, I never would have thought to implement it this way. I would have created some additional classes in their own files, created a formal implementation of the Command pattern, and a lot of other excessive ritual. Of course, if this got much more complex, it might need that ritual but, in this case, I was able to consoliate all that open and close code in a single place (which made things DRYer as well).

Polyglot Programming

My first professional work as a software developer was writing Clipper code. Clipper was a compiler for dBASE code with object-oriented extensions. This was in the days of DOS, and the entire application was written in a single language. We didn't even use SQL. Instead, the data storage was shared DBF files on a new concept, the LAN (I remember reading a PC-Magazine of that era declaring that the current year was the "Year of the LAN").

We are entering a new era of software development. For most of our (short) history, we've primarily written code in a single language. Of course, there are exceptions: most applications now are written with both a general purpose language and SQL. Now, increasingly, we're expanding our horizons. More and more, applications are written with Ajax frameworks (i.e., JavaScript). If you consider the embedded languages we use, it's even broader: XML is used as an embedded configuration language widely in both the Java and .NET worlds.

But I'm beginning to see a time where even the core language (the one that gets translated to byte code) will cease its monoculture. Pretty much any computer you buy has multiple processors in it, so we're going to have to get better writing threading code. Yet, as anyone who has read Java Concurrency in Practice by Brian Goetz (an exceptional book, by the way), writing good multi-threading code is hard. Very hard. So why bother? Why not use a language that handles multiple threads more gracefully? Like a functional language? Functional languages eliminate side effects on variables, making it easier to write thread-safe code. Haskell is such a language, and implementations exist for both Java (Jaskell) and .NET (Haskell.net). Need a nice web-based user interface? Why not use Ruby on Rails via JRuby (which now support RoR).

Applications of the future will take advantage of the polyglot nature of the language world. We have 2 primary platforms for "enterprise" development: .NET and Java. There are now lots of languages that target those platforms. We should embrace this idea. While it will make some chores more difficult (like debugging), it makes others trivially easy (or at least easier). It's all about choosing the right tool for the job and leveraging it correctly. Pervasive testing helps the debugging problem (adamant test-driven development folks spend much less time in the debugger). SQL, Ajax, and XML are just the beginning. Increasingly, as I've written before, we're going to start adding domain specific languages. The times of writing an application in a single general purpose language is over. Polyglot programming is a subject I'm going to speak about a lot next year. Stay tuned...

Enforcing Good Bahavior

I hate tools that force you down a particular path, and fight you all the way if you want to do something different than what they want. Every development environment and framework has their own path of least resistance, and various punishments for those that wander from the path. For example, Visual Studio encourages poorly designed applications by making it trivially easy to drop database components onto a web page, wire everything on that page via properties, and just click run. No real-world application should be written like that: it is a maintenance nightmare. Of course, you can write well structured applications in .NET (we do it all the time), but you have to do it around some of the designers and other affordances.

I really like tools that encourage good behavior and punish bad behavior. For example, Subversion is almost perfect for Agile projects because it strongly encourages you to check in early and often. Because it doesn't do any file locking, any file upon which you are working is subject to change by another developer. If you wait too long to check in, you are punished with Merge Hell, where you have to reconcile the differences between the changed file. The easiest way to avoid Merge Hell is to check in very frequently. Statistically, you are much less likely to bump into Merge conflicts.

A framework that encourages good behavior is Ruby on Rails. It builds layered applications by default. In fact, you would have to fight Rails hard to build a highly coupled application. Similarly, if you don't write good unit and functional tests in Rails, you are in great danger of building a very fragile application.

Both Subversion and RoR have the right priorities: reward the Right Thing and punish the Wrong Thing.

Entropic Software

I've reached a bit of an epiphany lately about software and complexity. "Epiphany" is probably too strong a word because it implies a sudden revelation. My thoughts about software complexity have more crept up than hit me over the head. But no matter how it got here, I'm convinced that software breeds entropy. And I have examples.

If you look at information theory (the mathematics behind information itself, not "information technology", like electronics), you might be startled to discover that the definition of "entropy" and "information" are essentially the same. Both measure the relative complexity of systems. Here's an example. Compare a class of water to a class filled with the makings of a mud pie. Which has more information? Clearly, the mud pie glass does because it is much more difficult to describe exactly. Water is easy: "A glass full of water". But a glass full of mud pie material is much more difficult. You have dirt, which is itself rich in information (composition, density, etc), plus rocks and twigs (what type of rocks, twigs, etc). From an information standpoint, the glass of mud pie is has much more information. The same is true of entropy. More entropic systems have more information density than less entropic ones. If you think of "entropy" as the movement from structure to chaos, you can see that chaotic systems have more information, just as the mud-pie glass has more information. The information density of highly entropic systems is greater than of structured, less chaotic systems.

Given all that, let's talk about software. I've come to the conclusion that software wants to be complex. In other words, it tends towards entropy unless someone takes active measures to stop it. I see examples of this every day, both building and using software. Software wants, needs, strives to be complicated. I don't know if it's something inherent in having an ultimately flexible palate upon which to build things (i.e., general purpose programming languages), something about the nature of engineering, or something about the people who really want to build software. Whatever causes this tendency, it must be assiduously fought at every turn.

Here's a concrete example from the recent past. During the design of Unix, lots of smart guys had observed this tendency towards complexity and fought it down diligently. To design the commands of the operating system, they decided to make everything as simple as they could, and establish simple rules about how different utilities talked to one another: everyone consumes plain text, and everyone produces plain text. While simple, this is a very effective way to create modular little programs that play nicely with a whole host of other simple programs. The utility of this simple idea has spawned many useful applications (by combining simple parts) beyond what the designers anticipated. Another example of the value of simplicity is the HTTP protocol. So simple you can understand it in an afternoon, yet sophisticated enough to create the largest distributed environment in the universe (as far as we know), the Internet.

Here's a counter example. When designing Office and Windows, Microsoft bumped into the same problem: we need to all applications to talk to one another. Recreating the simple mechanism of Unix didn't seem enough: applications in Windows were event driven, graphical, multi-threaded beasts that couldn't be bothered with simple command lines. Thus, DDE was born (Dynamic Data Exchange). DDE was a way for one binary hairball to talk to another binary hairball. Thus, Word and Excel could send information back and forth. But, as it turns out, DDE was fragile. Both applications had to be running, and in the correct mode to be able to talk to one another. DDE was all about sending information, not driving the other application. And thus is was considered not robust enough. So, let's add more complexity. OLE was born (Object Linking and Embedding). This allowed 2 things: embed an application inside another one, so that the user could interact with the spreadsheet embedded in a Word document. This, by the way, is why Office document formats are so obtuse. Each of the Office documents must act as a container for any other OLE object that might be embedded. The other feature of OLE was the ability for one application to drive another through background commands. This aspect of OLE was split off and became COM (and, its distributed cousin, DCOM). That wasn't sufficient for a variety of reasons, so we got COM+. Then .NET Remoting. Which leads us back around to Monad (or whatever Microsoft is calling it now that it's official - Windows Power Shell). Monad is a way for...wait for it...a command line script (or batch file) to make two application interact with one another, through COM+ interfaces. The idea is that you can pump some rows from an Excel spreadsheet into Outlook as email addresses and tell Outlook to send some files to the recipients.

But what is the problem we're trying to solve? Getting applications to talk to one another. I could do the same thing in Unix, with several of its tools, without all the intervening complexity. Building small modular parts with clean interfaces (the Unix way) means that I get to pick and choose what combinations I want. Using the Monad way, the designers of the binary hairballs that I need to get to talk must have anticipated what I want to do before I can use their hairball to do it. In other words, you cannot use Monad in a way unsupported by the huge binary behemoths for which it facilitates communication between.

This is a good example of the way software has of becoming highly entropic. The problem is that I need to have 2 applications send information back and forth. The simple way is the Unix way. The entropic, highly complex, fragile, limited way is to build great complex edifices, with lots of opaque moving parts. If we're ever going to produce really great software, we have to avoid entropic software like the plague that it is.

Post Windows

Professionally, a lot has changed for me over the last few of weeks. I've rolled off the rich client, .NET 2 project to which I've been attached since December and rolled onto a Ruby on Rails project. The projects could hardly be more dissimilar: large (15+ developers, distributed agile desktop application vs. 4 developer, Ruby on Rails project). Add another significant change: I'm now (for the time being, at least), post-Windows. I upgraded my Mac in July to a MacBook Pro, and have been doing .NET development on it using Parallels (which, BTW, works great). On my new project, though, I'm fully Mac: the pairing workstations are Mac Mini's, with 2 keyboards, mice, and monitors. And I've just completed porting all my Java and Ruby conference talks over to the Mac.

This is a big deal for me. For my entire professional life, I've been living with a Microsoft operating system on a daily basis. Starting in DOS 5 back in 1993, then moving to Windows (I've been a power user in all these versions - 3.1, 3.11, 95, NT 4, 2000, XP). Now, though, I'm conducting both my personal and professional lives in OS X. And I'm giddy with joy. I only occasionally need to dip into Windows for 1 of the 2 applications for which I don't have a superior Mac replacement.

Dealing with low-level frustration and annoyance takes a measurable toll on your psyche. I'm not one to be overly religious about tools; I try to learn to use them to their utmost. However, I absolutely believe that my quality of life is better now, in small but subtle ways, mostly having to do with elegance and design. These "OS X rocks, Windows sucks ass" kind of blog entries are generally short on substance, just an inarticulate expression of the intangible. Well, here are some concrete examples.

Windows machines have 2 ways to connect to networks, wired and wireless. On my Dell Latitude 610, when a wireless network is near, it pops up a Windows task tray balloon notifying you that it would like to connect. Yet, when you connect to a wired network, you no longer have a need for the wireless one. Windows still pops up the annoying little balloon, about every 15 seconds, offering to connect you to a network you don't need. When you connect OS X to a wired network, it stops asking you about connecting to a wireless network because it figures out, correctly, that your networking needs are now met.

Another example: power users like to be able to get to the underbelly of all the GUI eye candy to get real work done. I would like access to the Excel command line, in the vain hope that I might be able to open multiple spreadsheets at a time. Yet, in their infinite wisdom, Microsoft has wired Windows to treat Office shortcuts differently, preventing you from getting to the underlying startup command. If you don't believe me, check out this screen shot or check for yourself.



I've done what all power users of Windows ends up doing: I wrote a Ruby script that uses COM automation to open multiple spreadsheets. In fact, my toolbox is full of little scripts and such that get around annoying Windows behavior. Actually, I should be grateful to Microsoft for their annoyances: much of the Productive Programmer book features ways to make programmers more productive in that environment.

Before I get a whole bunch of Spolsky-esque comments about why Windows is the way it is, let me state that I already understand. I know that it's terribly difficult to write an OS that handles all the wide-world of devices that Windows must support because it runs on so much hardware. And, I know that one of Apple's big advantages is their tight coupling of hardware and software. I don't believe that Microsoft is evil or incompetent, and I in fact like some of what they create: .NET has some really nice, elegant parts (and some warts too, like all technologies). But, at the end of the day, as a user of the OS, the little things matter to me. If you cast aside history for the moment, using OS X is much more pleasant and refreshing, regardless of the reasons that got us here.

Technology Snake Oil Part 10: Check-box Parity

A pervasive bit of Snake Oil that's been around a long time is Checkbox Parity. This is the requirement by software companies to add features (or make features up) so that they can create the matrix on the side of the box, showing how their version stands up against the competition via columns of checkboxes. The importance of this marketing scheme should not be underestimated. There is a famous essay I read in a treeware book a long time ago by someone whose name I can't remember (but I remember that it was someone notable as a writer). The essay discussed the writer's trials and tribulations with the first versions of Microsoft Word. To compete against WordPefect (the huge market leader at the time), the first version of Word for Windows needed Checkbox Parity with outlining. The author discusses trying to get this to work in Word (it was after all listed at a feature of the product) and continually being frustrated. Finally, after numerous calls to technical support, he finally got the admission that the feature flat out doesn't work, they knew it didn't work, but they had to include it as a feature to achieve Checkbox Parity. This is not a minor point: part of the reason that Word came to dominate WordPerfect in the market place depended on them appearing essentially equavalent in the nascent days of Word. More honest companies (like Ami) failed in the Darwinian environment for word processors that existed before Office crushed all competitors.

This Checkbox Parity also drove the intense competition in the early days of Java IDEs. JBuilder, in its heyday, released a new version every 8 months (which was disastrous for those of us who wrote books about it). This worked well for Borland, who had a very agile development team for JBuilder. It was disastrous for Visual Cafe, who wasn't so agile. For many managers (and, unfortunately, many technologists who know better), the dreaded checkbox matrix on the side of the box determines purchase. Forget well designed, elegant functionality. If you can hack together something that you can reasonably compare to an elegant solution, you can achieve Checkbox Parity.

This same Checkbox Parity will be used to bludgeon Ruby in the marketplace until Ruby achieves the same types of functionality that Java and .NET already have. The CCYAO of large companies will reject Ruby because it doesn't achieve Checkbox Parity with older technologies, regardless of its suitability for a particular development project. If you are trying to sell Ruby in the enterprise, you need a strong antidote to Checkbox Parity Snake Oil.

Technology Snake Oil Part 9: The CCYAO

There is a corporate title that no one talks about but who is critical in many organizations: the Chief Cover-Your-Ass Officer. He's the C-level executive to whom you must to sell technology choices. He's always skeptical of new technologies because that's his job.

Back in the days when client/server was the norm and PowerBuilder reigned as king of corporate development, the company for which I worked was promoting Delphi as a good alternative for a particular application for a trucking company. Anyone with any technical knowledge could see quickly that Delphi was a better choice. All the technical people at this company clearly acknowledged that they wanted Delphi, and that a PowerBuilder solution for this particular application was doomed to failure. After a series of meetings with the CCYAO officer and others, they told us their choice: PowerBuilder. When asked why: "There is a good chance that this project will not succeed, and frankly we think the only chance it will succeed is is we use Delphi and your solution. However, if it fails, none of us will be fired if we pick the standard that everyone else uses, PowerBuilder. So, we're going with PowerBuilder. Thanks for coming in."

This is the same C-level executive that coined the phrase "No one ever gets fired for choosing IBM", which has been upgraded to "No one ever gets fired for picking Microsoft". No matter what the technical merits of your solution, ultimately, you've got to sell it to the CCYAO officer.

The Condiment Conference Redux

Back in May, I spoke at the first AJAX Experience, and it was a blast. It has been years since I've been to a conference with so much enthusiasm. It is unusual for a conference to focus on what I called a "condiment" technology. You can't write a web application in just Ajax (although TiddlyWiki may prove me wrong on that). Generally, you write the web application in Java, .NET, Ruby, PHP, Python, Perl, or some other "main course" technology. Ajax provides the icing, both visually and via usability polish. Most conferences focus on main courses, but The Ajax Experience focuses on the icing.

This means that this conference has an eclectic mix of developers. Hallway conversations lack the implicit assumptions you can generally make at main course conferences. For example, all Java developers have an implicit context. At The Ajax Experience, you have to throw away your base assumptions, both in sessions and conversations. Just like travel broadens you because you meet people with different contexts and experiences, attending the Ajax Experience does the same for technologists. Instead of the usual low-level animosity that each technology tribe exhibits for the non-tribe members, everyone focuses on common ground. It happens again in October, in Boston. You owe it to yourself to be an ex-patriot for your main course technology and come to the United Nations of web development, The Ajax Experience.

EKON 10

I just completed speaking at my 8th Entwickler Konferenz in Frankfurt (I missed the first two, so this one was EKON 10). Speaking at international conferences is enlightening because you quickly learn that the concerns and priorities of US developers don't apply all over the world. For example, I started speaking about Java at this conference back in 1999. At the time, the most popular tool was Delphi and I couldn't get the time of day from most developers when talking about Java. Germany IT is traditionally very conservative and Java was still an upstart platform. By 2001, Java became the safe choice and suddenly there was not 1 but 2 full conferences devoted to it (JAX and WJAX) that drew bigger crowds than Entwickler (which, by the way, is German for "Developer"). This year, I'm talking to everyone about Ruby on Rails in the hallways and no one has heard of it. That'll change in February, when I'm proposing a RoR talk at the Webinale Konferenz.

An interesting thing happened at breakfast this morning that highlights why I like this conference so much. Terry (my colleague from Atlanta) and I planned to meet another speaker from Amsterdam who we've known for years for breakfast before heading out for bicycling in the German countryside. While we were eating and chatting, one of the conference attendees came over, introduced himself and sat down (drawn by the sound of English and his recognition of one of the 3 of us from our sessions). A little later, another attendee came and sat on the other side. Before too long, we realized that our table had representatives from the US (Atlanta), Amsterdam, Greece, and Nigeria. We had 3 continents covered! Virtually no where else in the world can you spontaneously gather a group like this to talk about technology, programming, and weather. Just like working for an international company, it broadens your perspective on technology and other more important things.

Application Development Isolation from its Environment

Back in my DSW days, we did a fair amount of development in Delphi (a RAD application builder from Borland), and we noticed a trend. We frequently won return business by our customers (the hallmark of a successful consulting company), but we always ran into a major hassle: re-setting up the environment. Delphi, being a component-based development environment, took advantage of a rich ecosystem of third-party components. Why write something from scratch when you can buy it, frequently with source code included? However, using third party components meant that every application development environment is subtly different. Client A uses this widget, but you have to make sure not to use it for client B, because they don't own a license for it. Conceptually (but not actually), these controls were like ActiveX or .NET components in that they were installed on the developer's machine and became part of the operating system (at least as far as the developer tool was concerned). We thought a bit about how to isolate each project from another (some project setups would occupy a week of time, getting the right Delphi version and components installed just the way we left them). The problem was one of isolation: you can't encapsulate the development environment (or the developed application) at any level lower than the operating system.

They we developed a clever solution: start building our applications in VMWare. VMWare had just gotten Really Good at that time, and we realized that we could take a generic Windows ghost and install all the necessary developer tools on a VMWare image and develop on it. The speed hit at the time wasn't terrible, and it allowed us clean-room development for each client. When that phase of the project concluded, we save the VMWare image out to a server. Two years later, when that client came back for enhancements, we started up that application's development environment just like the day we left it. This approach saved us days of downtime, and made developing for multiple clients a breeze. Client A needs some minor tweaks while I'm working on client B's application. No problem, just bounce between virtual machine images.

Why do I bring this up now? Because the exact same scenario is playing out in the .NET development space. Most third-party components either GAC themselves or have stringent licensing requirements. Virtualization has gotten pervasive now, so if you have to do development on a machine that isn't a throw-away pairing machine image, life is easier if you sandbox it into its own virtual machine. I did this out of necessity on my former .NET project because I was developing on a MacBook Pro. However, I think this is wise for any development effort in a platform (like Delphi or .NET) that can't be isolated at any level lower than the entire operating system. This isn't as big a problem with Java or Ruby because they don't irrevocably couple themselves to the operating system. This is one of the prices you pay for that tight integration with Windows that .NET gives you: you can't de-integrate when you need to.

Thinking Different(ly)

I've fully made the switch. Instead of traveling with 2 laptops (a Dell Latitude 610 and a PowerBook G4), I've consolidated to a single machine: a 17-inch, fully loaded MacBook Pro. The tipping point for me? The ability to do real .NET development on the Mac.

Of course, I've seen and heard all the stuff about Parallels and how good it is: many orders of magnitude better than Virtual PC, which must create a virtual set of hardware on which Windows can run. Parallels (and the upcoming VMWare for the Mac) take advantage of virtualization hardware on the Intel chip, so you really do get near native speed when running Parallels inside OS X. Notice: not dual booting, but running Windows in a window inside OS X. But, I'm on a .NET project, and "it almost runs good enough to do .NET development" isn't quite enough. Thus, my hesitation up until this point to take the plunge. Well, I'm hear to say: it works as advertised. Building our project in Parallels on the Mac is essentially as fast as building it on the single processor Dell. The build time is within seconds of one another (for an 8 minute build).

But there are always things that you can't read about in reviews that still cause issues. I've been here before, and know that there are lots of little hidden gotchas. When I decided to move everything over, I reserved some time for glitches. And you know what? I got that time back, because I ran into very few minor ones and no major ones.

Here's an example of something you won't read about but is a huge deal if you are planning to use your Mac for .NET development. For a real .NET project, you must have (of course) the Windows XP operating system, a database server (MS SQL Server), and Visual Studio, including all the 3rd party components required by your project. For our application, you also need Office. How big do you make your virtual disk? This was a very important question in the VMWare days. Like "real" hardware, VMWare virtual disks (at least in the last version I used) cannot be re-sized. Once you create the disk, you are stuck with it. When using VMWare, getting that disk size right is critical. Not in Parallels. Parallels includes a utility that allows you to resize the partition. I started with a ridiculously optimistic 8 Gb drive. I quickly ran out of room. So, I used the Parallels utility to make the drive bigger. But here's the part you can't read about anywhere: once you start the virtual Windows back up, it views that new space as "unpartitioned", meaning that you can't use it for anything yet. But, Windows on Parallels is so Windows that you can run Partition Magic on that newly resized virtual disk and make your main disk bigger. I've done it 3 times now (and am now up to a 20Gb partition for our project).

Here's another illustration of the Window-y-ness of Parallels on OS X. I had some problems with the database setup, and Brian (our DBA) was kind enough to take a look for me. He's in London; I'm in Chicago. I started up Windows, gave him the IP address assigned by DHCP in Chicago, and he VPNed into our network and ran my Windows install via Remote Desktop. He never realized (until I told him later) that he was running Windows on top of OS X.

This represents a watershed event. The MacBook Pro + OS X (and it's siblings) are now the only machines that run every modern operating system. For consultants, that's huge. We can now go into any organization, find out what they are running, and fit in exactly. Your servers are running Ubuntu? No problem, I can create a virtualized version here on my machine. Red Hat, Windows Server 2003, Vista...you name it, I can now run it. The Mac has changed from an artistic, boutiquey machine to the ultimate Swiss-army chain saw for consultants. If I were Dell, I'd be worried. OS X and the wonderfully designed hardware make for a significantly better user experience. And now it's the power users machine of choice. Maybe I should buy some Apple stock...

Pairing Everywhere

The more I pair program, the more I'm convinced that two (compatible) people always produce better results than just one. I know that pair programming is the best way to write code. This started me thinking about other creative artifacts that might benefit from pairing.

There are already some pretty famous pairs. Rogers and Hammerstein come to mind. One of the greatest series of history books, The Story of Civilization, was written by a pair of authors, Will and Ariel Durant. Because they were written in the 1920's, only Will's name appears on the first few, but he eventually acknowledged his wife in the later books as a co-author. Some great authors were essentially pairing with their editors. Numerous examples exist of great writers whose works were made better because of a strong willed editor: Theodore Dreiser, Ernest Hemingway, and on and on.

To this end, my friend and colleague Joe O'Brien and I tried a new trick this year at ThoughtWorks Away Day: pair teaching. He and I used 2 computers, 2 projectors, and one topic (Ruby for ThoughtWorkers Who Don't Know Ruby But Want to Know Why It Rocks: Learning Ruby Through Unit Testing). In the end, the sum was greater than the parts. It was a frantic 1 hour presentation, with something happening constantly. After the smoke cleared, another ThoughtWorker said that he really enjoyed it because his mind only wandered for about 4 minutes total during the entire time, and suggested that if we hire a clown to walk through the audience, juggling, and repeating our key points, that we would have held 100% of his attention. High praise, indeed.

Categorizing Creative Genius

I just read a fascinating article in the July Wired magazine about creative genius. The subject, an economist named David Galenson, has correlated age with perceived value in all sorts of creative fields, and has identified 2 curves. One, which he dubs "Concepualists", tend to peak early in their careers. For example, even though Picasso lived into his 90's, his most cited works in art history and other books were done before he was 30. Mark Rothko, (one of my favorites), did his most cited work the year he died, when he was 59. Galenson calls these guys "Experimentalists". He has done this correlation over painting, fiction, economists, music, and other fields. He believes that 2 distinct flavors of genius exist: one that manifests itself early, with bold, field-changing paradigm shifts (concepualist) and another, slower, accumulated genius (the experimentalist).

This instantly applies to other fields that he hasn't studied, like physics. I've often wondered why so many brilliant, earth shattering discoveries are made by young men Newton, Einstein, and Feynman were quite young when they produced their landmark works). However, if you look at someone like Stephen Hawking, he's still producing significant work. I think this is a great topic, one that resonates with observations I've made but never correlated myself. His book is named Old Masters and Young Geniuses: The Two Lifecycles of Artistic Creativity, and it's jumping to the top of my reading list with a bullet.

Technology Snake Oil Part 8: Service Pack Shell Game

If you could see my face, you would see shock, dumbfoundment, and disgust. It pains me to
even write about something as stupid as this, but it keeps rearing its head. The
majority of my recent clients and someone I talked to casually from another
company recently are relying on one poisonous meme, which seems to be spreading.
The very bad idea: "We never deploy anything until the first service pack is
released".

Let's think about this for a second. If a vendor produced the
most perfect software ever conceived by mankind, there would never be a service
pack, thus none of these companies would ever deploy it. On the other hand, if I
release a really stinky version of some software that requires a service pack
after a week, it now meets this unassailable standard of deployability.

Two factors have led to this smelly idea. The first is just pure laziness on the
part of the decision makers who decide when things get deployed. Regardless of
the service pack level, you should always evaluate software on its merits. A
prescription like the Service Pack Shell Game ignores important factors in
software and tries to find a metric that indicates quality. This is not even
close. When Windows NT Service Pack 1 was released, it was a disaster. Service
Pack 2 basically rolled back all the changes that SP1 wrought. That's why, to
this day, you still see software that requires NT SP3, because that was the
first real service pack that actually fixed anything.

The other reason this is happening is both more subtle and dangerous. Have we really gotten to the
point where we distrust commercial software this much? It's because vendors have
consistently released software that is not ready for prime time and told us that
it's of shipping quality. Companies even apply this selection process to open
source software now. Open source has no marketing department pushing releases
out the door. Generally, open source software ships when it is ready. Thus, most
open source has fewer "service packs" than commercial software. Yet this same
flawed prescription is often applied to it. Software, no matter what the source,
should be vetted based on it's quality, which should be determined by (as much
as possible) objective means. Choosing a random metric like "after the first
service pack" guarantees you'll get hit-and-miss quality software.

ejbKarmaCallback()

When you work with a noxious technology enough, it eventually comes back to bite you. Call it software development karma. While I was at OSCON in Portland, the first hotel room where I was placed had massive problems connecting to the Internet. It was wired access, so there was something related to my room that was causing the problem. I endured several maintenance guys and several phone calls with the actual provider. You all know the drill intimately.

Anyway, at one point, it was declared "Fixed!", and I was instructed to point my faithful browser to the Internet. Lo and behold, Software Karma decreed that it was not to be. I got the following error, captured here in all its public glory.



Gaaaaaah! I now know waaaayyy more about their network infrastructure than I would like. They are using Tomcat and EJB's...to connect me to the Internet???!? I'm sure this is exactly the kind of application the EJB designers had in mind when they birthed this technology. Do we think that maybe this is total overkill? Couldn't the same be done with a simple web application backed by a database. Sigh. That's what I get for dabbling in evil -- sometimes it comes back to haunt you in the strangest places.

Scumbag Spammers

If you have posted a comment to my blog lately, you've noticed that I've turned on the "Word Verification" feature of Blogspot. It's because of the scum of the earth, spammers. They've started posting spam comments (spamments?) to blogs. How clever. How annoying. How I hope they choke on their own vomit as they slide under a gas truck.

Search Trumps Hiearchies

I wrote a while back about Pervasive Search, and how it changed the way I find things. I find myself using search more and more versus navigating hierarchies. As developers, we tend to create lots of files, in regular strict hierarchical structure (in fact, I've been blogging about namespaces vs. packages recently as well). File system paths are now too cumbersome to endure. Instead of walking through Explorer or the tree in my IDE, I'm using search.

I use search at 2 levels. Within the IDE, I use the brilliant feature in both IntelliJ and ReSharper to "Find File" (keyboard shortcut: Ctrl-N). This lets you type in the name (or partial) name of a file and open it in the editor. Better yet, it finds patterns of capital letters in names. So, if you are looking for the ShoppingCartMemento class, you could type "SCM", and "Find File" will find it. Highly addictive. And, it works equally well in IntelliJ and Visual Studio with ReSharper (and my Eclipse friends tell me it has made it there as well).

The other place I've been using search a lot is the filesystem, when looking for either a file on which to perform some operation (like Subversion log) or looking for some content within a file. Google Desktop Search has gotten better and better. You can now invoke it with the key chord of hitting Ctrl twice. And, you can download a plug-in that allows you to search through any type of file you want, including program and XML documents. Once you've found the file in question, you can right-click on the search result and open the containing folder. This is the only way to get to some file buried deep in some package or directory structure. My coding pair and I have started using this heavily, and it has sped us up. And, it eliminates annoying repetitive tasks like digging through the rubble of the filesystem looking for a gold nugget.

Partial Classes

When I first saw that .NET 2 supported partial classes, I groaned. It
looked like a language feature that helps one thing and hurts a dozen
more, once people start abusing it. However, I've come around to appreciate (and dare I say it, like) partial classes. They are obviously useful for code generation (which is why, I suspect) they were added in the first place). However, they are also handy for other problems.

Testing is one place where partial classes offer a better solution than the one offered by Visual Studio.NET 2005. In VS.NET, if you want to use MS-Test to test a private method, the tool uses code generation (without partial classes) to create a public proxy method that turns around and calls the private method for you using reflection. This is not a big surprise; the JUnitX add-ins in Java help you do the same thing. But using code gen for this is a smell: if you change your private method using reflection, the generated code isn't smart enough to change, so you have to do code gen again, potentially overwriting some of the code you've added. Yuck.

Here's a better solution. I should add parenthetically that I don't usually bother testing private methods (especially if I have code coverage) because the public methods will exercise the private ones (otherwise, the private methods shouldn't be there). However, when doing TDD, I sometimes want to test a complext private method. And partial classes work great for this. The example I have here is a console application that does some number factoring (why isn't important in this context). I have a method theFactorsFor() that returns the factors for an integer. Here is the PerfectNumberFinder class, including the method in question:

namespace PerfectNumbers {
   internal partial class PerfectNumberFinder {
       public void executePerfectNumbers() {
           for (int i = 2; i < 500; i++) {
               Console.WriteLine(i);
               if (isPerfect(i))
                   Console.WriteLine("{0} is perfect", i);
           }
       }

       private int[] theFactorsFor(int number) {
           int sqrt = (int) Math.Sqrt(number) + 1;
           List<int> factors = new List<int>(5);
           factors.Add(1);
           factors.Add(number);
           for (int i = 2; i <= sqrt; i++)
               if (number % i == 0) {
                   if (! factors.Contains(i))
                       factors.Add(i);
                   if (!factors.Contains(number/i))
                       factors.Add(number/i);
               }
           factors.Sort();
           return factors.ToArray();
       }

       private bool isPerfect(int number) {
           return number == sumOf(theFactorsFor(number)) - number;
       }

       private int sumOf(int[] factors) {
           int sum = 0;
           foreach (int i in factors)
               sum += i;
           return sum;
       }
   }
}

Rather than use code gen to test the method, I've made the PerfectNumberFinder class a partial class. The other part of the partial is the NUnit TestFixture, shown here:

namespace PerfectNumbers {
   [TestFixture]
   internal partial class PerfectNumberFinder {
      
       [Test]
       public void Get_factors_for_number() {
           int[] actual;
           Dictionary<int, int[]> expected =
               new Dictionary<int, int[]>();
           expected.Add(3, new int[] {1, 3});
           expected.Add(6, new int[] {1, 2, 3, 6});
           expected.Add(8, new int[] {1, 2, 4, 8});
           expected.Add(16, new int[] {1, 2, 4, 8, 16});
           expected.Add(24, new int[] {1, 2, 3, 4, 6, 8, 12, 24});

           foreach (int f in expected.Keys) {
               actual = theFactorsFor(f);
               for (int i = 0; i < expected[f].Length; i++)
                   Assert.AreEqual(expected[f][i], actual[i],
                                   "Expected not equal");
           }
       }
   }
}

I like this because it allows me to test the private method without any messy code generation, reflection, or other smelly work-arounds. Partial classes make great test fixtures because they have access to the internal workings of the class but don't have to reside in the same file. It's dangerous to pile infrastructure on new features like this (especially scaffolding-type infrastructure like classes), but this one seems like a more elegant solution to the problem at hand than stacks of code generation.

Pontificating at OSCON

I gave a talk as OSCON last week on Building Internal DSLs in Ruby. Apparently, there is a fair amount of interest in this subject: I was in one of the small rooms, but it was packed to the rafters, with standing room only along the back and side walls. I didn't realize it, but John Lam took a snapshot of me in action and posted it to his blog: .
It's tough to get a good shot while someone is talking, so it shows that John is both a formidable Ruby/.NET guy and a talented photographer!

The Fact of the JMatter

Several years ago, some brilliant designers created Naked Objects, a Java framework that generates applications from domain objects. You supply the POJOs with behavior, point Naked Object at them, and you have a full-blown Swing application that allows you to edit, insert, delete, and browse the objects and their relationships. You could literally create sparse, functional applications in minutes. However, Naked Objects never got much beyond a proof of concept. The automatically generated applications were utilitarian but uninspiring.

Fast forward to now. Eitan Suez, one of my fellow No Fluff, Just Stuff
speakers, has taken the Naked Object idea and run with it. He has created the JMatter framework (found here). It takes the concepts of Naked Objects and updates it to the here and now. JMatter applications still auto-generate from POJOs, but the user interface and interactions are very rich. The sample application that appears on the JMatter web site literally took less than 2 hours to create; written by hand, it equates to developer-weeks worth of effort. It also illustrates a growing trend in development: creating framework and scaffolding code automatically, freeing developers to focus more on producing applications. We've seen this approach done well in Ruby on Rails. JMatter shows that you can apply the same concepts to Swing development. Eitan has released JMatter with a MySQL-style license, so it's worth jumping over to his site to get a preview of the future.