Wednesday, October 17, 2007

Mocking JRuby

Testing is one of the easy ways to sneak JRuby into your organization because it is easier to write tests in dynamic languages (especially mock object tests) and it isn't code that is deployed, so it eases the minds of the furniture police somewhat. Here is an example, adapted from Martin Fowler's essay on mock object testing.

In Martin's example, he creates Order and Warehouse classes (and a Warehouse interface), then writes tests with them in jMock. Here is one of those tests, rewritten in JRuby, using the Mocha mocking library:

require 'test/unit'
require 'rubygems'
require 'mocha'

require "java"
require "Warehouse.jar"
['OrderImpl', 'Order', 'Warehouse', 'WarehouseImpl'].each { |f|
eval "#{f} = com.nealford.conf.jmock.warehouse.#{f}"
}


class OrderInteractionTest < Test::Unit::TestCase
TALISKER = "Talisker"

def test_filling_removes_inventory_if_in_stock
order = OrderImpl.new(TALISKER, 50)
warehouse = Warehouse.new
warehouse.stubs(:hasInventory).with(TALISKER, 50).returns(true)
warehouse.stubs(:remove).with(TALISKER, 50)

order.fill(warehouse)
assert order.is_filled
end

def test_filling_does_not_remove_if_not_enough_in_stock
order = OrderImpl.new(TALISKER, 51)
warehouse = Warehouse.new
warehouse.stubs(:hasInventory).returns(false)
order.fill(warehouse)

assert !order.is_filled
end

end

The order object is a pure Java object, tested with a mocked out version of the Warehouse interface. Now, this isn't exactly what Martin wrote in in excellent essay about mocks vs. stubs, but the intent is preserved in this example.

Most of the really crufty stuff in mock objects in Java is getting the compiler to agree with you that the mock object type is what you want it to be, and of course that problem goes away in JRuby. Also notice that I can create the mock object directly off the interface. Because JRuby creates proxy objects for the Java classes, you can create a mock from the interface directly by calling the new method on it. The other cool thing about this example is the require "Warehouse.jar" line at the top. JRuby allows you to require a JAR file, which gives you access to the Java classes within it.

Saturday, October 13, 2007

Ruby Matters: Contracts vs. Promises

In teaching Ruby to Java developers, two things seem to shock and annoy developers: open classes and the lack of interfaces. Because Java developers are inundated with interfaces, it's hard to imagine a language whose libraries are strictly defined in this manner.

Interfaces in Java and .NET are exactly like legal contracts (where the compiler is the all-powerful judge). If you implement an interface yet don't meet the terms of the contract, the judge refuses to allow you to proceed.

Mixins in Ruby are more like promises than contracts. When you agree to pick up a friend at the airport, you don't sign a contract, you promise you'll be there. The punishment is purely social: if you break a promise, your friends won't do stuff for you. Here's an example.

In Ruby, a common Mixin is Comparable, which gives you the same behaviors as the Comparator interface in Java. The promise in Comparable is that you will implement the "spaceship" operator (<=>), which returns negative if the left-hand side is less than the right, 0 if they are equal, and positive if the LHS is greater than the RHS (which is exactly like the similar compare() method in Java).

class Employee
include Comparable

attr_accessor :name, :salary

def initialize(name, salary)
@name, @salary = name, salary
end

def <=>(other)
name <=> other.name
end
end

list = Array.new
list << Employee.new("Monty", 10000)
list << Employee.new("Homer", 50000)
list << Employee.new("Bart", 5000)

list.sort!
# Monty vs. Homer
list[0] < list[1] # => true

# Homer vs. Monty
list[0] > list[1] # => false

# Homer is between Bart and Monty?
list[1].between?(list[0], list[2]) # => true

If you violate the promise by mixing in Comparable and yet don't implement the spaceship, nothing bad happens...until you try to ask your friend Comparable to perform a service for you (like compare two things with between? or call the sort! method). Then, you get an error message complaining "undefined method `<=>' for Employee". And this happens at runtime, not at compile time.

This is a fundamentally different mindset than the legalistic strongly contracted languages. It demonstrates one of the reasons that you can't slack off on writing your unit tests in Ruby: you don't find out important things until runtime. It is certainly a different way of thinking about implementing APIs. While lots of Java developers think this leads to chaos in large development teams, I have not found that to be true (and I think that most Ruby developers would agree with my assessment). The trade-off is strict control vs. flexibility. If there is one thing that my experience has taught, it is that flexibility is the most important thing for strong programmers, echoed by many of Paul Graham's essays.

If you must have stronger contracts in Ruby, there is a framework called Handshake that allows you to create pre- and post-conditional invariants ala Eiffel. While I understand the appeal of Handshake, I would be loathe to use it on a project because I prefer the power of flexibility rather than the strictures of constraints.

Monday, October 08, 2007

Ruby Matters: Language Beauty Part 2


A dead chrysanthemum
and yet - isn't there still something
remaining in it?
Takahama, Kyoshi

Let me not to the marriage of true minds
Admit impediments. Love is not love
Which alters when it alteration finds,
Or bends with the remover to remove:
O no! it is an ever-fixed mark
That looks on tempests and is never shaken;
It is the star to every wandering bark,
Whose worth's unknown, although his height be taken.
Love's not Time's fool, though rosy lips and cheeks
Within his bending sickle's compass come:
Love alters not with his brief hours and weeks,
But bears it out even to the edge of doom.
If this be error and upon me proved,
I never writ, nor no man ever loved.
William Shakespeare

Can computer languages be beautiful? Is it possible to create a beautiful COBOL application? I think it is possible, but you can't compare different languages as if they all had the same constraints. For me, these comparisons are like comparing poetry styles. Within a particular poetry style, you can clearly have beauty. But some poetry styles have serious constraints. For example, a Haiku has exactly seventeen syllables, in three lines of five, seven, and five. A sonnet is a poem with fourteen lines, using any formal rhyming scheme, having 10 syllables per line. Shakespeare wrote some stunning poetry within these pretty serious restrictions!

Computer languages are closer to poetry than prose. The syntax of the language imposes order, just as the conventions of different poetry styles constrain what you can create. Within those constrains, though, you can create wonderful things. But the strictures of the poetry style necessarily prevent you from creating more ambitious works. William Faulkner famously said that he wanted to be a poet and failed, then he wanted to be a short story writer and failed, and eventually became a novelist because he couldn't say what he wanted to say within the boundaries of the other styles.

A lot of developers who find themselves "at home" in a particular language can relate to this. For many developers, Java (the sonnet) feels just right. Yes, there are restrictions, but they are well known and seem right. Other developers chafe at the imposed limitations and seek more expressive media, like Perl or Ruby. And if it was just about aesthetics, we could all live happily, each developer writing code in whatever medium he or she liked.

But with computer languages, expressiveness == power. Being able to say what you mean succinctly and effectively matters. The only reader who you must satisfy completely is the execution environment. Thus, when making a language choice, it makes sense to pick the most powerful language you can stand. Of course, different developers rate power differently. For me, meta-programming is extremely valuable. It's one of those features that, once you've used a language that supports it extensively, it's hard to go back to a weaker one.

Which brings me around to Paul Graham again. When I first read Hackers and Painters, I remember thinking that about 80% of what he said was dead on, and the other 20% was either just crazy or intentionally inflammatory. Over the last few years, though, as I've learned more about languages myself, I go back to his writings (most of which are online at paulgraham.com) and realize that he's exactly right about something I used to disagree with him about. Now, I'm down to thinking only 5% is crazy or inflammatory, and that number is shrinking all the time. Anyway, one of those things was his language power scale, which I discounted at first but now believe to be an outstanding yardstick by which to measure language power.

For me, Ruby has a great combination of succinct expressiveness and super powerful meta-programming support. I think that it is currently the most powerful language you can get paid to write. Until Smalltalk makes a come back or Lisp eventually takes over the world, it's the best game in town. But that doesn't mean that I can't still write sonnets in Java and C# from time to time, when the circumstances warrant.

Thursday, October 04, 2007

Ruby Matters: Language Beauty Part 1

Beauty without expression is boring.

Ralph Waldo Emerson


I was talking to a colleague (named Vivek) recently who is originally from India. He was telling me about "Hin-glish", the merging of English an Hindi. I am already familiar with "Singlish", the flavor of English tinged with Malay, Hindi, Chinese, and several other languages spoken in Singapore. Vivek was telling me that it is quite common when talking to someone else who knows both English and Hindi to freely flow back and forth between the languages. That got me to thinking about expressiveness in language. We do this all the time in English without specific foreign influences: we use phrases like "Joie de Vivre" and "Cul de sac" all the time. Expressions in other languages just make more sense; some languages have found a useful, concise label for common human experiences and other languages don't bother creating their own term for it. Phrases like this encompass much more than the literal translation. They manage to encapsulate tons of connotation, not just the denotation of the meaning of the words. (For a supreme distinction between connotation and denotation (and none too shabby language either), check out this Robert Graves poem The Naked and the Nude).

Expressiveness of language makes a big difference. Having concise, nuanced representations of common concepts means less context establishment. This is one of the compelling arguments for domain specific languages: if you encapsulate high-level meaning in code, you spend less time talking about it. This is the vocabulary that describes your business, and everyone talks the same language so that you don't have to start from scratch every time.

Talking about beauty in programming languages sounds silly, because they are after all just ways to drive Turing machines. But people still talk about it. O'Reilly has a popular recent book named "Beautiful Code", which has essays by famous programmers. Reading this book clearly indicates that this opinion is purely subjective. My friend Brian made a great observation about this book: the essays written about other people's code really do illustrate beautiful code, while the ones written by developers about their own code are mostly about cool hacks. Most of those examples I would not consider beautiful by any measure, even if clearly highly functional.

Glenn Vanderburg has a great talk entitled "The Beauty of Ruby", discussing the elegant language design entailed in Ruby. For example, it is necessary in object-oriented languages to distinguish member variables from local variables. Java does this either via scope (which can be ambiguous) or the this keyword. Python does this with self (which is required in some places). Ruby uses the @ symbol. It is unobtrusive, unambiguous, and small. And it conveys exactly the meaning you need it to convey without being too chatty. Similarly, class-level artifacts in Ruby are identified with two @ symbols.

For me, Ruby does in fact have beautiful syntax. It is easy to write, easy to read, just succinct enough to keep you from typing too much but not so succinct to become cryptic. Many Perl developers think that Perl is beautiful (and, Perl was after all developed by a linguist). But I think it goes too far, bordering on cryptic (and I'm not the only one: many people refer to Perl as a "write only" language because it is so hard to read later).

Interestingly enough, even when I was primarily using Java, I never considered it beautiful. Utilitarian, but lacking beauty. But most of the post-Java developers I know still talk about the beauty and elegance of Ruby. I've never heard those phrases so much about other languages.

Language beauty will always be subjective. I explore this idea further in part two of this topic, coming up soon.

Saturday, September 22, 2007

Travel: Biking Adventures Through the Woods Outside Frankfurt

Warning: this post is entirely about travel stuff: there is no software or technology here, so feel free to avoid this if that's all you care about.
Terry Dietzler and I have spoken at the Entwickler conference for 9 years in a row (that's how I always spend my last week of September). On the first trip here, the hotel had bicycles you could borrow for the day, and we did several bike trips through the German countryside (the conference is in a suburb of Frankfurt, named Morfelden, near a national forest). We were hooked. Biking in Europe is a blast: there are bike routes simply everywhere (you can buy maps of just bike routes in bike stores). Our most elaborate trip to date was in 2003, when we biked about 200 miles in 2 days, from Frankfurt (actually, Morfelden) to Metz, France. In 2002, we made a deal with the conference organizer: in lieu of payment for pre-conference tutorials, buy us bikes and keep them in Frankfurt for us, in the basement of their offices. They kindly agreed, so Terry and I have bikes that live year-round in Germany.

This is all prelude to the adventure we had yesterday (we tend to have lots of adventures when we are here in Germany). We took the train (the S-bahn, or regional train) into the city and picked up the bikes. We had thought about bringing them back to Morfelden on the train, but decided instead of ride them back: follow the Main river trail until it gets out of the city, them find our way back to Morfelden. The biggest problem: the Frankfurt airport is between us and Morfelden, but we had maps, so we can find our way. Here is the route we ended up taking (Terry has a GPS, and he turned it on, so we got a record of the entire route):



It was 9 miles (according to the GPS) from where we picked up the bikes to our destination...as the crow flies. We estimated more like 20 miles because of the major obstacle, the airport. About 2 hours should to it; we started out at about 5.

As you can tell, we made good progress along the river, and we got minorly lost a few times as we lost the trail then reacquired it later. Very typical. Then, as we got near the airport, we started having to cross more major autobahns and train tracks. There are abundant bike bridges and crossing, you just have to find them. And, notice the backtracking as we neared the airport. We got on one trail that literally dead-ended at the junction of two sets of train tracks. As we're thrashing around, the sun is setting. Many of the trails are through the woods, so it's getting darker and darker. Our bikes have headlights, but mine was the only one working. So, picture this: we're biking through dense woods in pitch blackness following one headlight. Fortunately, the trails are very good, but it was still spooky and hard to do because the light only reached a little ways ahead of us, and it looked like we were biking into a black wall, which made me claustrophobic.

Fortunately, we've biked around Morfelden a lot, and as we got close, we recognized a landmark from a former trip. We followed the trail through a really dark and increasingly cold section of trail until, to the right, we spotted our hotel. 4 hours after we started, we ended up cycling 23 miles (including all the switch backs). What fun; we're going out again tomorrow.

Monday, September 10, 2007

Ruby Matters: "Design Patterns" in Dynamic Languages

As I mentioned in my last post (Ruby Matters: Meta-programming, Synthesis, and Generation), the Gang of Four design patterns book should have been named "Palliatives for C++". One of the authors finally admitted as much in public at a roast. So, why would design patterns be any different in a dynamic language (like Smalltalk or Ruby)?

In the GoF book, design patterns are 2 things: nomenclature and recipes. The nomenclature part is useful. It's a way of cutting down on repetition of similar code across projects, and it gives developers a way to talk to one another, using shorthand. Instead of saying "I need to create an object that can only be instantiated once", you say "singleton" (we'll leave aside for the moment why Singleton is evil -- the subject of another blog entry). Nomenclature good.

But the fatal flaw in the GoF book was that they included recipes. And many people thought they were the best part. Even now, you see books on Java design patterns that blindly mimic the structure of the examples in the GoF book (even though Java has some better mechanisms, like interfaces vs. pure virtual classes). Recipes bad. Because they suggest more than just a way to name common things. They imply (and put in you face) implementation details.

Because of meta-programming, many of the design patterns in the GoF book (especially the structural ones) have much simpler, cleaner implementations. Yet if you come from a weaker language, your first impulse is to implement solutions just as you would from the recipe.

Here's an example. Let's say you have a simple Employee class, with a few fields (which could be properties in the Java sense, but it makes no difference for this example).


public class Employee {
public String name;
public int hireYear;
public double salary;

public Employee(String name, int hireYear, double salary) {
this.name = name;
this.hireYear = hireYear;
this.salary = salary;
}

public String getName() { return this.name; }
public int getHireYear() { return this.hireYear; }
public double getSalary() { return this.salary; }
}


Now, you need to be able to sort employees by any one of their fields. This is a flavor of the Strategy Design Pattern: extracting an algorithm into separate classes so that you can have different flavors. Java already includes the basic mechanism for this, the Comparator interface. So, here's what the code to be able to compare on any field looks like in Java:


public class EmployeeSorter {
private String _selectionCriteria;

public EmployeeSorter(String selectionCriteria) {
_selectionCriteria = selectionCriteria;
}

public void sort(List<Employee> employees) {
Collections.sort(employees, getComparatorFor(_selectionCriteria));
}

public Comparator<Employee> getComparatorFor(String field) {
if (field.equals("name"))
return new Comparator<Employee>() {
public int compare(Employee p1, Employee p2) {
return p1.name.compareTo(p2.name);
}
};
else if (field.equals("hireYear"))
return new Comparator<Employee>() {
public int compare(Employee p1, Employee p2) {
return p1.hireYear - p2.hireYear;
}
};
else if (field.equals("salary")) {
return new Comparator<Employee>() {
public int compare(Employee p1, Employee p2) {
return (int) (p1.salary - p2.salary);
}
};
}
return null;
}
}


You might protest that this is overly complicated, and that this will do the job:

public Comparator<Employee> getComparatorFor(final String field) {
return new Comparator<Employee>() {
public int compare(Employee p1, Employee p2) {
if (field.equals("name"))
return p1.name.compareTo(p2.name);
else if (field.equals("hireYear"))
return p1.hireYear - p2.hireYear;
else if (field.equals("salary"))
return (int) (p1.salary - p2.salary);
else
// return what? everything is a legal value!
}

}
}

but this one won't work because you must have a return, and returning any value is mis-leading (every possible integer here means something: 1 for greater than, 0 for equal, -1 for less than). So, you are left with the bigger version. I've actually attempted to optimize this several ways (with more generics, reflection, etc. but I always get defeated by the requirement to return a meaningful int from the comparison method. You could find some kind of minor optimization, but you are still building structure to solve the problem: a class per comparison strategy. This is the typical structural approach to solving this problem.

Here's the same solution in Ruby, using a similar Employee class:

class Array
def sort_by_attribute(sym)
sort {|x,y| x.send(sym) <=> y.send(sym) }
end
end

In this code, I'm taking advantage of several Ruby features. First, I'm applying this to the Array class, not a separate ComparatorFactory. Open classes allow me to add methods to built-in collections. Then, I'm taking advantage of the fact that Ruby method calling semantics are message based. A method call in Ruby is basically just sending a message to an object. So, when you see x.send(sym), if sym has a value of age, we're calling x.age, which is Ruby's version of the accessor for that property. I'm also taking advantage of the Ruby "spaceship" operator, which does the same thing that the Java compare method does: return a negative if the left argument is less than the right argument, 0 if they are equal, and positive otherwise. Wow, it's nice to have that as an operator. I should add that to Java...oh, wait, I can't. No operator overloading.

Perhaps it makes you squeamish to add this method to the Array class (open classes seem to terrify Java developers a lot). Instead, in Ruby, you could add this method to a particular instance of array:

employees = []
def employees.sort_by_attribute(sym)
sort {|x, y| x.send(sym) <=> y.send(sym)}
end

Now, the improved employees array (and only this instance of array) has the new method. A place to put your stuff, indeed.

When you have a powerful enough language, many of the design patterns melt away into just a few lines of code. The patterns themselves are still useful as nomenclature, but not as implementation details. And that's the real value. Unfortunately, the implementation seems to be the focus for many developers. Strong languages show you that the implementation details can be so trivial that you hardly even need the nomenclature (but its still useful as a conversation shortener.)

Next, I'll talk about language beauty.

Wednesday, September 05, 2007

Ruby Matters: Meta-programming, Synthesis, and Generation

In my last Ruby Matters blog post, I talked about meta-programming in Ruby, contending that Ruby gives you "places to put your stuff". I always wondered about meta-programming in Smalltalk and how that compares to Ruby, and Where to Put Stuff in Smalltalk. The final piece of the puzzle came after I talked to Glenn Vanderburg (the Chief Scientist of Relevance). I was puzzled as to why the Gang of Four book (which had examples in both C++ and Smalltalk) didn't have more meta-programming. Lots of the design patterns are almost trivially easy to implement with meta-programming, but they didn't do that in the Smalltalk examples. They used the same structural approach as C++. It looks more and more to me that the Design Patterns book was really just a way to solve problems in C++ that should have been easier to solve in a more powerful language like Smalltalk. Which is why I was puzzled about the lack of of more meta-programming solutions in Smalltalk. Glenn enlightened me. One of the overriding characteristics of Smalltalk is the way code is stored, in an image file, which allows for really smart tools. The program and the environment all reside in the binary image file. There are no source files as we know them today, just the image.

A comparison about implementation details is in order, on how Ruby differs from Smalltalk. Let's talk about has_many in Ruby on Rails. Typical Rails code looks like this:

class Order < ActiveRecord::Base
has_many :lineitems
end

For those not familiar with Ruby, this is a method, defined as a class-level initializer (just like an instance code block in Java, a chunk of curly-brace code in the middle of a class definition, which Java picks up and executes as the class is instantiated). So, ultimately, this is the Ruby equivalent of a static method call, which gets called as the class is created.

Let's talk about Smalltalk, which has first-class meta-programming. You could easily build has_many in Smalltalk, implemented as a button you click in the browser which launches a dialog with properties that allow you to set all the characteristics embodied in the Ruby version. When you are done with the dialog, it would go do exactly what Ruby does in Rails: generate a bunch of methods, add them to the class (stuff like the find_* and count_* methods). When you are done, all the methods would be there, as instance methods of your class.

OK, so at this point, the behavior is the same in Smalltalk as in Rails. But there is one key difference: The Smalltalk version using code generation. It's a sophisticated version of a code wizard, generating the code using meta-programming techniques. The Ruby version uses code synthesis: it generates the code at runtime, not build time. Building stuff at runtime means more flexibility. But that is a minor point compared to this one: In the Smalltalk version, you use the dialog and properties to generate all the methods you need. The original impetus for the has_many intent lives only while you are running the dialog. Once you are finished, you are left with lots of imperative code. In the Ruby version, the intent stays right where you put it. When you read the class again, 6 months from now, you can clearly see that you still mean has_many. Smalltalk has the same meta-programming support, but the intent of code synthesis remains forever. That's why it's important to have a place to put your stuff. It isn't accidental in Rails that many of the DSL characteristics appear as class methods rather than code you call in the initialize method. Placing them as part of the class declaration declares intent in a big way, and keeps the code very declarative.

To summarize the similarities and differences:

  1. Both Ruby and Smalltalk give you a place to put your meta-programmed stuff

  2. Both give you a place to put the declaration of intent (the tool in Smalltalk, the shadow meta-class in Ruby)

  3. A time in the lifecycle of the class when things happen. In the Smalltalk version, it's a one-time deal, as you use the tool to generate the code. In Ruby, the synthesis takes place at class load time. This leaves the clean, declarative code right where you put it, rather than generating a bunch of much less clear imperative code.


Glenn made an excellent point here: the Smalltalk version is a great example of accidental complexity, not essential complexity. Software is full of essential complexity: writing software is hard. But we end up subjecting ourselves to lots of accidental complexity in our tools and languages. And it should be stamped out. The Ruby version eliminates accidental complexity by providing a great abbreviation for the intent of has_many. This blog does a great job of illustrating the differences between abbreviations and abstractions.

Smalltalk had (and has) an awesome environment, including incredible tool support. Because the tool is pervasive (literally part of the project itself), Smalltalkers generally shied away from the kind of meta-programming described above because you have to build tool support along with the meta-programmed code. This is a conscious trade off. One of the best things about the Ruby world (up until now) is the lack of tool support, meaning that the tools never constrained (either literally or influentially) what you can do. Fortunately, this level of power in Ruby is pretty ingrained (look at Rails for lots of examples), so even when the tools finally come out, they need to support the incredible things you can do in Ruby.

Next up, "design patterns" in Ruby.

Thanks to Glenn for supplying me Smalltalk information, as a sounding board for this, and all the interesting bits, really.

Monday, September 03, 2007

Ruby Matters: A Place to Put Your Stuff

Why are so many people into Ruby? I get this question a lot because I speak at a lot of Java conferences and talk about Ruby and JRuby. In this and several more blog entries, I'm going to explore this question in depth.


For myself and many of my peers, it was a rite of passage to learn all the intricate tricky stuff in C++. Once I got over my fascination with C++ and got tired of banging my head against all of the arcane nonsense in the language (I used to know the details of all the different ways you could use the word const, but I've fortunately purged that knowledge), I asked one of my colleagues why he continued to love it so much. "Knowing C++ makes me part of the C++ priesthood. When people ask me questions, I'm purposely cagey when I give them answers because I think they should work as hard as I did to figure this stuff out. I don't want to give my advantage away." I shuddered and was glad I wandered to greener pastures.

After a brief visit to the refugee camp of Delphi, I made it to Java and thought I was in heaven. Finally, a language that made some sense. I liked the locked down nature of the language because I thought that Java helped prevent mistakes, especially for large projects. This is before I had embraced unit testing, so that seemed like a good thing. I would gladly give up flexibility for perceived safety. This was when I used to think that one of the most important duties of a computer language was to prevent mistakes first and support building interesting stuff second. Interesting stuff must be hazardous to large projects, and I was an Enterprise Developer. After more than a decade in Java, I started playing with Ruby because I'm a bit of a language geek and I was taking the advice in The Pragmatic Programer to learn a new language every year. But, at the time, Ruby was just a cool little scripting language with nice syntax.

When Ruby on Rails came out, I started looking at it with great interest because it was so vastly different from the Java frameworks with which I was so familiar (this was just after I had written a book comparing them). And the more I looked at Ruby, the more puzzled I got. Wow, there's some funky stuff in there, and I couldn't understand what some of the code meant. So, I dug deeper. And slowly, I gained enlightenment. At first, I thought that it was just more of the same "obfuscated code" syndrome from C++. But I later understood that it wasn't.

The epiphany really came when I understood what some of the Rails code was doing with meta-programming, which made me look closer at the core Ruby stuff. Meta-programming was supposed to be scary stuff that only wild hackers did. But it made problems melt away, in very few lines of code. If you look at how attr_accessor works, it's so simple. I thought for a long time that attr_accessor was a keyword or something infra-structural that I shouldn't bother looking at (before I realized that lots of stuff in Ruby looks like keywords but they aren't).

I have come to understand that meta-programability is an extremely important part of a language. So many things that we solve in languages like Java with complex structures and hierarchies are much more elegantly solved with meta-programming. This realization led me to start looking around at other languages, like Smalltalk and Python.


That's when it hit me: one of the things that makes Ruby so powerful is that it already gives you a place to put your stuff. If you don't think this is any different than Java, check out whytheluckystiff's Seeing Meta-classes Clearly article. I'll wait... Yes, there is a shadow meta-class behind every object in Ruby, giving you the syntax and location to put much of the interesting meta-programming stuff in Ruby. You can do lots of the same meta stuff in, for example, Groovy, but you have to create a Groovy builder object to do the equivalent of eval. And, doing class_eval would take more building a place to put it. I was talking to a Python programmer recently, talking about the same subject, and I got the same response: "Yeah, Python can handle that, but you'll have to build a place to put it." One of the elegant features of Ruby is that it comes pre-defined with places to put your important stuff, rather than forcing you to build a place to put your stuff. Which is why Paul Graham says that Lisp is the most powerful language available; because you are writing code in the abstract syntax tree, you are living in the place where stuff gets put.

In the next post, I'll talk about the special case of Smalltalk, tools, and code generation vs. synthesis.

Friday, August 24, 2007

Martin and Me and DSLs

Back in June of this year, Martin Fowler and I did a keynote at The ServerSide Symposium in Barcelona on a subject for which we both have some interest (and dare I say passion): Building Domain Specific Languages. I saw cameras at the back of the room, but it didn't actually dawn on me that they were filming the whole thing...and that they would put in on the web. But, sure enough, here it is. It took them two months to post it and I think I know why: some poor person had to go and transcribe Martin and me yammering away for an hour. I hope they got paid well!

This keynote pretty much summarizes much of the thinking that I've been doing about DSLs over the past couple of years, and it is nice to see the agreements and disagreements that Martin and I have on some nuances of this topic. Clearly, though, we both think this remains a Big Deal.

Monday, August 20, 2007

Lights, Cameras, Geeks!

At the Agile 2007 conference this year, they asked for submissions for videos featuring agile development themes call the Agile Advert contest. ThoughtWorks submitted a couple (created in ThoughtWorks UK) and won the top prizes. You can see them on YouTube under Developer Abuse and Being Agile is our favourite thing. Hilarious!

Wednesday, August 08, 2007

Pair Programming Spellchecking

My colleague Jay Fields has a nice post about pair programming, when it's good, and when it's not so good. I wanted to add to and amplify his remarks a bit. Basically, anytime you pair two developers of vastly different skill sets (what Jay calls a "spell checking pair"), that's not pair programming: it is mentoring. That doesn't mean that it is wasted time, but it is a different activity.

I find that mentoring is tiring for both the advanced and novice developers: tiring for the advanced developer because they have to go slower than their natural pace and tiring for the novice because they are having to peddle faster to keep up. However, mentoring is a great way to greatly increase the skill level of the novice. If you are going to mentor, it should be for just part of a day. After the mentoring is done, switch pairs with more matched levels.

A lot of companies seem to think that's what pair programming is: take a intern from college and pair them with an ace to build up their skill level. All that does is stress both.

We are generally careful when engaging with clients to pair program with their developers to determine if the project is enablement or delivery. If it is delivery focused, we tell them that the more senior developer may well leave their pair in the dust, and should be reorganized. You can't usually do both enablement and delivery at the same time except in rare circumstances.

Monday, August 06, 2007

Dependency Injection in One Sentence

Jim Weirich issued a challenge to describe dependency injection in once sentence. My friends at Relevance had a good version. Here's mine:


Dependency Injection enables a vitally important but nevertheless weak, limited, syntactically confounding, and dauntingly complex form of one of the kinds of meta-programming that should exist in the language.


This is fun -- can we do annotations(C#)/attributes(Java) next?

No Fluff, Just Stuff eXchange: London 2007

Way back in the late nineties, I was working for The DSW Group, which was a Borland partner. We did Borland training classes worldwide, and that's ultimately how I got into the business of speaking at conferences. At the time, I was all geeked up about Delphi and at the first few BorCons I spoke exclusively about it and C++ (in the form of C++Builder). However, in the late 90s, I stopped having that lovin' feeling for Delphi and started really dedicating my efforts in the rising tide of Java. But what was interesting to me was the adoption level in different countries. I was still traveling to BorCons around the world, talking about JBuilder. And some countries I would have huge audiences. And in others, you could see tumbleweeds blowing through the aisles. I remember the first few Java talks I did at the Entwickler Conference in Frankfurt, no one even knew what "Java" was.

It was interesting because the UK seemed to be one of the last holdouts for Java. Even into the early 2000s, I would go to Borland conferences in London and be in the small room, while the Delphi groups were packed. Of course, these were Borland conferences, so the crowds were self selecting, but it seemed to me like Java was a slow burn in the UK. Then, of course, overnight, it was the cool thing to do.

London is now a first-class Java city, with lots of Java work going on (Java is very popular in the financial sectors, and London sure has lots of that). The only thing missing from the London Java scene finally appears this August: No Fluff, Just Stuff. What better place to make the cross-ocean debut of the premiere Java conference. In the past, people have literally flown from the UK and mainland Europe to see No Fluff, Just Stuff shows in the US, in random cities like Columbus, OH. Now, you can ride the tube to see a No Fluff, Just Stuff show!

The traveling circus that is No Fluff, Just Stuff invades London on August 29th, for 3 days. It's the normal cast of speakers from the US (including Venkat Subramaniam, Ted Neward, Brian Sletten, David Geary, and I've managed to stow away as well) mixed with some great guys from the UK (my ThoughtWorks colleague Erik Doernenburg, Graeme Rocher of Groovy/Grails fame, and some wild cards).

To help pump up the enthusiasm for this event, No Fluff, Just Stuff and Skills Matter have a cool promotion. Every attendee who registers before August 17th with the special code of NFJS-NEF666 will receive a Nintendo Wii. Not "entered into a drawing for a Wii" -- "will receive a Wii". Wow.

Come see us in London -- it should be a blast.

Wednesday, July 11, 2007

Polyglot Programming at erubycon

In less than a week, I'll be doing a special version of my Polyglot Programming keynote at erubycon in Columbus, OH. In it, I talk about how to make Ruby palatable to conservative enterprises, and some arguments against "conventional wisdom".

If you are into Enterprise Ruby, there are still slots available at this conference. This conference will have a great combination of speakers who are actively pushing Ruby and Rails into the Enterprise. Lots of us see bright futures for dynamic languages and their potential in traditional IT; come see a bunch of like minded speakers (and attendees) who are on the cutting edge of making it happen.

Sunday, July 08, 2007

The Rich Web Experience

The web is changing. Just a couple of years ago, we had reconciled ourselves to treating the browsers as a graphical dumb terminal. Ajax and related technologies have changed all that. And anytime there are seismic changes in the technology landscape, developers have to learn the whole new world. The preferred way to do this quickly is to attend a highly focused conference.

That's what The Rich Web Experience is all about: rich application development that running in the browser. This conference is beyond just a single technology's APIs. It also covers security, usability, and other topics. I'm speaking on testing and debugging the web tier, including a bunch of best practices with Selenium.

Come join us in San Jose for what should be a rocking good gathering of cool web based technologies.

Saturday, July 07, 2007

Travel Broadens You

Not too long ago, I was in India to speak at JAX India. The conference was held at a science center in downtown Bangalore, and the speakers were staying in a hotel several miles away. One of the interesting things about India is that you can hire car services for cheap, so the conference organizers had a platoon of cars at our disposal, for trips to and from the conference center, which took about 20 minutes (it would be about 5 minutes if there was no traffic, but apparently that state never exists in Bangalore).

While driving to the conference center a couple of times, we passed a part of town where there were several cows lounging in the road, in amongst the traffic. No one even noticed: the traffic flowed right around them as if they were any old standard barrier in the road. Of course, it struck me as very odd, because I almost never see cows on the main roads of Atlanta.

It made me realize that people can get used to and habituate anything. During the conference, I talked to Java developers about some of the odd things in Java. Java has its share of strange behaviors (generics, anonymous inner class syntax, the recursive Enum definition) that look to strangers as odd as a cow standing in the road. Because Java developers are accustomed to seeing these cows all the time, they don't even notice anymore. But it strikes tourists as very odd and surprising.

Here is a concrete example. Groovy makes it really easy to interact with Java objects, including those that follow the standard JavaBean specification for get and set methods to form properties. In Groovy, when you create a new class, you can create your properties using the get and set style of coding, yet when you call the property, you can leave the get and set off.

class Foo {
def name

def getName() {
return name.toUpperCase()
}

def setName(value) {
name = value
}
}

foo = new Foo()
foo.name = "Homer"
println foo.name

Now, when I refer to what looks like a regular public field (to a tourist), I get the uppercase version because of the automatic mapping to a getter. To a Java developer, this auto-magic mapping from the property methods to actual properties seems as normal as can be. However, to a Java tourist, it looks as strange as a cow in the road.

This suggests 2 things to me. While every language has its own bizarre quirks, it is the seemingly irrational stuff that turns off tourists to the language. Do you really want to go into a history of the JavaBean specification to the poor Python programmer who's vacationing in JavaLand so that they can understand this weird but expected behavior? And will they possibly stay awake for it? Probably not, so they'll just say something like "Wow, Java's broken" and move on. I think this hampers attempts to expand Groovy's meme outside the Java world. I know there are efforts to port Groovy to the .NET platform, but will it possibly attract any .NET developers because of its unabashed Java-ness?

The other thing this suggests to me is the advice in The Pragmatic Programmer to learn a new language every year. Just as physical travel broadens you, traveling to a strange language broadens you too, so that when you see cows in the road back home, you understand that it's not normal for everyone. Stranger in a strange language, anyone?

Thursday, July 05, 2007

Fair and Balance Essays?

An interesting review of the 2007 No Fluff, Just Stuff Anthology appeared on Amazon the other day. Generally, I don't bother to reply to Amazon reviews (because it is after all someone's opinion, and I can't dispute someone's else's opinion), but this one has some interesting points. First and foremost, I would like to thank the reviewer for giving us 4 out of 5 stars, so his complaints should be kept in perspective (he did at the end of the day enjoy the book, and I don't want to gloss over that by way of indicating that we don't appreciate that immensely).

The reviewer mostly takes me to task for not forcing the authors to be more balanced against what he calls "agitation for the new age software movement". If you look closely at the cover of the book, you'll notice I'm not listed as the editor, but as the compiler (as in "Compiled by Neal Ford"). This distinction is important and planned. I did not edit this book in the normal sense (of vetting what the authors want to write), I merely compiled the essays. Frankly, it would be an awful job trying to get this group of authors to bend to my will! All the authors of this anthology are quite passionate about their subjects (which is what makes the book interesting, in my opinion).

The reviewer takes Brian to task for not providing the WS-* case against REST, but I don't think that was the purpose of the essay. Where in the vast ocean of information about WS-* do you even see a mention of REST? Unless you are specifically comparing two technologies, you frequently don't, well, compare them. I notice that David Geary (whose excellent essay was praised by the reviewer) didn't compare and contrast it with Struts, which is the de-facto market leader.

But the more interesting issue for me is the clear disdain the reviewer has for what he calls "new age dogma": REST vs. SOAP, dynamic languages, and Agile development. It's no secret that many of the No Fluff, Just Stuff speakers do prefer agile development and looser contracts. While not suitable for all applications, this is the cutting edge of software development right now. It is contingent on "thought leaders" to point out the latest trends in software development, so that the attendees and readers know what's on the horizon. Software and software development continues to evolve at a furious pace. While no one can predict the future (except maybe Bruce Tate), it is interesting to see where the people who were really into Java in 1996 are spending their time now (a hint: mostly with loose contracty type stuff).

Oh, and the reason there is more information about IntelliJ than Eclipse? Because most of the authors use IntelliJ (because we think it's the best tool available), I got inundated with cool IntelliJ tips and tricks (and ended up cutting a bunch of them). We had to ask over and over to even get tips for Eclipse, which is not to say that Eclipse is bad, it just doesn't generate as much passion than IntelliJ. There's that word again: passion. All the writers of this volume are passionate about technology (which is extraordinary in and of itself), and want to write about it for very little remuneration.

Given the amount of other open source in the book, why wouldn't we prefer Eclipse over IntelliJ it they were essentially the same? As a group, we tend to choose things that we think are best of breed, whether web framework or IDE. Hopefully, that's at least some of the appeal of both the anthology and the No Fluff, Just Stuff tour.

Monday, July 02, 2007

PragMactic-OSXer

New Mac OS X users are like recently reformed smokers: they can't stand to see it when someone indulges in their old vice (in this case, Windows). I made the total plunge about 1 1/2 years ago, and more and more of my friends are doing it too. People who attend No Fluff, Just Stuff often note that most of the speakers are using Macs: the answer I frequently give is "because we can". It is crystal clear to me that I am simply more productive on the combination of Mac hardware and Mac OS X. I'm paid for how much I can produce in a given amount of time, so it makes sense for me (even if I have to buy it myself) to pick the sharpest tool I can find.

However, just picking up a Mac won't instantly make you more productive. In fact, if you just scratch the surface, you have no idea that Mac OS X is a deep ocean. There is always a learning curve when you switch something as important as an operating system. To that end, several of the No Fluff, Just Stuff speakers and myself have started a blog devoted to making the best use of the Mac. It is called "PragMactic-OSXer". If you are switching, or are just curious to see how the other half lives, come over for a visit (or attach your blog reader to it.)

Thursday, June 28, 2007

TSS DSL Keynote Recap

Martin Fowler and I just finished delivering the keynote on Language-Oriented Programming at The ServerSide Symposium in Barcelona, Spain. He and I have never presented together before, but we are both passionate about this subject, so it sounded like fun. And it was. I created a skeleton slide show in Keynote, to which he added and made suggestions, and we paired for about 1/2 hour the night before to make sure that it represented not only a consensus view but also good leaping off points for the presentation. We approached it like a public conversation: we had pretty minimal slides, and we took turns talking about the subject (along with minor disagreements) along the way.

There was a little bit of a frenzy just before it started. For some reason, some projector switching boxes just don't seem to like the Mac. My laptop worked fine in all the other breakout rooms (that only had one projector). In the main room, my machine could see that it was attached to an external monitor (I could see the resolution, the refresh rate, and every other detail about the external projectors), but for some reason they refused to show the image. I worked on this with an increasingly frantic IT guy from the hotel. We tried a Dell laptop and it worked fine. He kept insisting that there was something wrong with my machine, and I kept demonstrating to him that I could see his equipment just fine. I've encountered this problem before (see this entry, and it always turns out to be the switching box for the projectors, which for some bizarre reason has an affinity for Windows). Finally, as a last resort (time was running out), I exported the presentation from Keynote to a PDF file, copied it to a thumb drive, and moved it to the Windows laptop. I lost all the animations and other effects, but the slides looked OK. By this time, it was time to start speaking, so I opened the slide show full screen on the laptop. Unbeknownst to me, someone had set the machine to operate in kiosk mode when Acrobat is full screen, so the slide show starting playing without me doing anything. Finally, rather than try to figure out how to turn that off in front of 300 people, I just left it with the Acrobat Reader border showing and launched into the talk. As much as I love the Mac, it seems that the rest of the world still harbors ill-will for anything non-Windows. And it's not major stuff anymore -- it's the strange little things like switching boxes for projectors.

For anyone who saw the presentation, that was why it was in non-full screen Acrobat Reader view. Despite this hiccough, I thought the talk went reasonably well. Martin and I both have lots to say on this topic. When we started, I feared that we wouldn't have enough material, but he and I are both prodigious talkers, so we ended up having to rush a little to get to the end. I feel like we presented a pretty good summary view of how we both feel about this important subject.

Saturday, June 23, 2007

Speaking at TSS Barcelona Next Week

I spoke at the The ServerSide Symposium in Las Vegas earlier this year. Next week, I'm traveling to Barcelona (for the first time) to speak at the European edition. Martin Fowler and I are pairing on the keynote, about Domain Specific Languages (a topic for which he and I share considerable enthusiasm). I'm also doing a regular talk on Selenium.

Friday, June 08, 2007

Coalmine Canary Tests

I was pairing the other day and we started talking about the scope of tests. I'm a huge advocate of writing the simplest, just slightly more than trivial tests when doing test-driven development, especially in a dynamic language (we were using Ruby). My pair, Kristy, and I got into a discussion of around 2 areas: just how simple should they be and should you keep them around after you've written the code? Lots of developers use these ultra simple tests, then make the test more robust once they've gotten the first incarnation to pass. I'm a fan of what I call "Coalmine Canary" tests. Back in the old days, coal miners used to take caged canaries down in the mines with them. The birds were much more sensitive to the gas that can build up in mines; if the bird died or starting struggling for breath, the coal miners knew it was time to high-tail it out of the mine. In the same way, very simple tests help you judge the safety and semantics of the code under test. We created a very simple test (basically, verify a method that included a semi-complicated ActiveRecord query returned a non-zero number of rows). We decided to keep it around, because it verified that we were calling the infrastructure correctly (in other words, that the semantics of the method were correct). Sure enough, as we started adding more code to the class, we introduced bugs. The first test we ran after making changes was the Canary test. If it failed, we knew that we had a fundamental problem with our infrastructure and didn't even bother looking at the more complex, nuanced behavior we have added until the Canary test passed again.

Having (and keeping) the simple Coalmine Canary test is a great sanity check on your code. If it fails, you've got serious problems, and you'd better fix those before you do any more mining in your code.

Tuesday, June 05, 2007

Neal and David Talkin' 'Bout Agile

As a sneak peak to the workshop David and I are doing at The Agile Experience, No Fluff, Just Stuff caught us late last year in a video podcast about agility, how companies react to it, and it's future. It's a pretty accurate representation of how I feel about agility (minus the faux pas of saying that the Agile Manifesto was signed in Colorado, not Utah).

Tuesday, May 29, 2007

Are Open Classes Evil?

When doing my JRuby talk at No Fluff, Just Stuff, one of the consistent questions I get is the morality question about open classes: are they evil? Open classes in dynamic languages allow you to crack open a class and add your own methods to it. In Groovy, it's done with either a Category or the Expando Meta-class (which I think is a great name). JRuby allows you to do this to Java classes as well. For example, you can add methods to the ArrayList class thusly:
require "java"
include_class "java.util.ArrayList"
list = ArrayList.new
%w(Red Green Blue).each { |color| list.add(color) }

# Add "first" method to proxy of Java ArrayList class.
class ArrayList
def first
size == 0 ? nil : get(0)
end
end
puts "first item is #{list.first}"

Here, I just crack open the ArrayList class and add a first method (which probably should have been there anyway, no?). When you define a class, Ruby checks its classpath to see if another class of the same name has already been loaded. If it has, it adds the new behavior to the class.

It it's too frightening to add a method to the entire class, Ruby gives you the option of adding it to an object instance instead. Consider this:
# Add "last" method only to the list object ... a singleton method.
def list.last
size == 0 ? nil : get(size - 1)
end
puts "last item is #{list.last}"

Here, I add the last method just to the instance of the list class (i.e., this list object). That way, you don't add it to the entire class.

Many Java and C# developers are shuddering in horror right now. The consensus seems to be that this is just too dangerous. And, like all advanced language features, it can be abused. But here is a counter argument. How many Java and C# developers have a StringUtils class in their project? Pretty much everyone. Why? Because the language designers for both languages no only won't allow you to add your own methods to String, they won't even allow you to subclass it to create your own String class. Thus, you are forced by the language design to switch from object-oriented coding to procedural coding, passing Strings around like you are writing in C again.

Open classes allow you to make your code much better, when used responsibly. One common argument against this feature from paranoid developers is that they don't trust "junior" developers with this kind of power. So, tell the junior developers on the project not to do it! And, if you really hate the additions that have been made, you can also reopen a class and remove methods at runtime, using either remove_method (which removes it from a class) or undef_method (which removes it from the entire inheritance tree).

You can add new methods to classes in Java if you want to use Aspects, but:
  • uses an entirely different syntax from Java

  • you give up tool support in your IDE for the new methods (in fact, it probably won't even compile it properly without some plugins and such)

  • it's so much trouble that you just suffer through the ugly, non-OOP StringUtils class


In JRuby, you can add methods to existing classes:
  • using the natural syntax of the language

  • it is supported as much as anything else (we don't have IntelliJ for Ruby...yet), but nothing chokes either

  • it is so easy that it is natural


I think this level of power makes developers used to non-dynamic languages queasy. But in the right hands, it can make you code much more expressive, and not pollute your namespaces with lots of made up class names with Util and Helper tagged on the end. Like all advanced features, when used correctly, it makes your code much better. Are open classes evil? No more evil than any advanced language feature.

Monday, May 21, 2007

Cruising

This is old news to some, but Ruby now has a great continuous integration story: cruisecontrol.rb. Written by a bunch of Really Smart Guys, it has some innovative features (convention over configuration, no XML, DSL-style tasks, and many more). I mistakenly said in my aboutGroovy Podcast that cruisecontrol.rb supplants the other versions of CruiseControl, but that was clearly wrong: Enterprise CruiseControl was announced recently, which provides consulting, 24x7 support, and new features.

CruiseControl is an example of the best kind of tool: one extracted from real need on a project. The original CruiseControl was developed because developers on a project were feeling build pain and came up with a great solution. Tools are always better when they are derived from real-world experience. Another great example of this is Ruby on Rails. DHH didn't wander over to an ivory tower and think about what a framework should look like. He built some web applications and extracted a framework. Getting developers together to invent frameworks and tools never works as well as taking working stuff and distilling frameworks and tools from it. The classic easy target of how not to do it is EJB 1 and 2: get a bunch of vendors in a room and figure out what developers need (or, more likely, what do we have to sell that we can convince developers they need).

CruiseControl is an ongoing illustration of what happens when you get clever people together who can not only spot a problem but come up with a top notch solution to fix it, then distill the results.

Tuesday, May 15, 2007

The Importance of Language

Language is important.

One of the purposes of my last two blog posts was to use hyperbole to make a point. And, judging from the hue and cry, I was successful in irritating a lot of people. To do that, I used 2 purposefully inflammatory terms: "communist" and "bureaucracy". As many people pointed out, communism as a form of government is clearly a bad thing, as proven in very expensive social experiments in the last century. Bureaucracy, though, merely has a negative connotation because it tends to be misused. In very large groups of people, some level of bureaucracy is necessary to preserve order. It is in the arbitrary application of bureaucracy that earns it a negative connotation.

When you read the first sentence of this post, you probably though I was referring to computer languages, not English. That's a perfect example of how spoken languages are sometimes ambiguous in useful ways. In fact, poetry is an art form that takes good advantage of the imprecise nature of our spoken languages. It is in fact hard to create a useful human language with very strict, unambiguous rules (as Esperanto showed). I'm sure lawyers would love to have a language where nothing is ambiguous; it seems like a large part of contract negotiations concerns removing ambiguities and trying to craft real, objective meaning from our incredibly malleable language.

Computer languages are important in just the same way as spoken languages. When creating a bureaucratic, legalistic system, strong contracts and unambiguous language matters. Very large systems benefit from bureaucracy. Using Java, C#, or some other statically typed language for building incredibly large applications takes advantage of the useful bureaucracy inherent in the language. I used to think that this serious hampered the ability to build large, complex systems in dynamically typed languages. You necessarily remove some of the useful ambiguity from such systems.

Some of the richness of spoken language hinges on purposeful vagueness. This means that syntax matters (in both spoken and computer languages). Ultimately, computer languages help us operate Turing machines, which means that any Turing-complete language will allow you to do anything you want, contingent on effort. As Paul Graham aptly states, you could build all of the features of Lisp using C, but the shortest route would be to build a Lisp interpreter in C, then write your Lisp code on top of that.

That suggests that the choice of language depends on the useful tension between inherent capabilities, expressiveness, rules, and the productivity because of the interplay between these factors. This helps explain why there are thousands of computer languages, each finding their own combination of those factors. Paul Graham has a great essay about languages, power, and suitability called Revenge of the Nerds.

Computer languages are just tools, and I try not to get religious about tools (my inflammatory posts for effect to the contrary). I personally have changed my views over the last few years about the efficacy of dynamically typed languages like Ruby, even for large enterprisey applications, as long as you have strong testing to back you up. The flexibility of the meta-programming in Ruby allows you to solve hard problems in elegant ways that are either too cumbersome or just flat impossible in a more limited language (even if those limitations are on purpose). Rails is a great example of a framework builder pushing the really powerful features of the language and in the process creating a framework that you don't need to know all those things to operate. Limiting the power of the language hurts your best programmers the most. The interesting question is: given the productivity difference between the best and worst developers, how much to you want to limit your best developers at the expense of "protecting" the weaker ones? And, before you think that bureaucratic languages make bad developers safe, consider the great quote by Glenn Vanderburg (based on his experiences in the real world) : "Bad developers will move heaven and earth to do the wrong thing."

Thursday, May 10, 2007

Static Typing & Bureaucracy Redux

As I expected, there is serious dissension in the ranks over my post claiming that "static typing is communist bureaucracy". Some of the areas worth rebutting:

  • Static typing helps tools

    Yes, it does (although dynamic typing doesn't preclude tool support -- Smalltalk had the one of the first refactoring tools built in, and NetBeans is adding some pretty good support for JRuby). This is a problem that will be solved. However, I reverse this question: tools make static languages merely palatable. Iterating over collections in Java or C# is ridiculously painful. Using the each method with a closure in Ruby or Groovy is a joy. Tool support in languages cover up for deficiencies in languages and libraries. Graeme Rocher (creator of the Grails framework) stated this quite elegantly in the GroovyOne panel I was on, stating that you don't need as much tool support in well designed languages because they tend to be more consistent (what Martin Fowler calls "Humane Interfaces").

    Granted, tooling makes things easier: no question there. But, are you willing to give up the flexibility of a dynamically typed language just to get better tool support? In my aboutGroovy podcast, I said (and meant it) that I would rather write Groovy in VI than Java in IntelliJ (and I think my position on IntelliJ is pretty clear).

  • Static analysis tools are useful.

    Yes, indeed, anything you can find with an automated tool is low-hanging fruit. But, again, it's not a pure trade-off. You can do some of the same kinds of things in dynamic languages (although clearly not as much), but I would still rather have the flexibility of the dynamic language plus tests to verify my code. In the perfect world, I'd have the flexibility and power of the dynamic language and the tooling possible and available for static ones. This is one of the places that Groovy comes in handy: if your static analysis tool works off byte code (like FindBugs), you can run it on a Groovy codebase.

  • Static typing creates better contracts and calling semantics for libraries.

    Dynamic languages tend to eschew strong contracts like interfaces. What I've found it that it forces developers to create much better method and parameter names, because that becomes the primary way that you consume methods. When done right, this is an effective way to communicate. Granted, this is not universally the case even in the Ruby libraries, but removing props sometimes forces people to make better use of what's left. The best example is the town of Drachten, in the Netherlands, who removed all their traffic signs and lights, forcing people to interact in the intersections. It made their streets much safer because they removed signage.

Sure, in a perfect world, all the facilities that we lean on in statically typed languages would be available in dynamic languages. But they're not. And, having worked extensively in both static and dynamic languages, I prefer the flexibility and expressiveness of the dynamic languages currently extant to the static ones, even with all the tool support. Still, testing is the most important verifier of code. I think it is professionally irresponsible to write code that isn't tested. If you are going to write that level of testing anyway (especially if you do it test-first), a dynamic language allows you to get more done in less time with a lot less typing (a lot of which is required to make static typing work).

Tuesday, May 08, 2007

Static Typing is Communist Bureaucracy

In a conversation that came up at No Fluff, Just Stuff Denver this last weekend, a fundamental realization came to me. We were talking about dynamic languages like Ruby and Groovy, and a lot of the Java developers just couldn't get over the idea of dynamic typing. They feared that utter chaos would reign if we discard typing. First, I asked them about the significance of testing: "Do we all agree that testing is important?" to which they replied "Of course". And, in the course of the expert panel, we had discussed the appropriateness of code coverage, what it tells you, what it cannot tell you, etc. The general consensus was that code coverage is a Good Thing, and that we should all aspire to 100% as much as possible. After all, it's the tests that tell you if your code if correct or not.

Once we had agreed that testing + code coverage is a good thing, I realized: Static typing is Communist Bureaucracy. "To get bread, you must stand in the line over there, but first you must get a blue stamp on your booklet." If you have pervasive testing, static typing == more typing. The static typing is nothing but a requirement to type extraneous code to satisfy a compiler that isn't telling you anything interesting anymore.

I've said it before and I'll say it again: testing is the engineering rigor of software development. Testing destroys the need to static typing. Dynamic typing speeds up development because there is physically less to type. And testing verifies the correctness of your code, not just that it compiles.

Sunday, April 29, 2007

Justifying Automation

One of the topics that comes up in my Productive Programmer talk about automation is how to justify it. Sure, everyone knows that automation is a Good Thing. And, if you can demonstrate that you can automate something as fast as you can do it by hand once, that's a total no-brainer. But what about the grayer areas where you aren't sure how long its going to take to automate something, and you aren't sure if you can justify the time it takes to the Powers That Be?

I have a couple of strategies for this that I'm including in The Productive Programmer book. First, make sure you timebox your efforts. Even if the end result of the timebox is that you create another one (because you've made such good progress), that prevents the automation from turning in to a week-long exercise when you thought it would just take 1/2 a day. Timeboxing also cuts down on the inevitable yak shaving that goes on anytime you start experimenting with something you haven't done before.

Automation is all about risk mitigation. For the thing you are thinking about automating, ask yourself the following questions:

  • How long does it take to do it by hand now?

  • How many times are you going to have to do it before the end of the project?

  • What is the consequence of doing it wrong one time (i.e., forgetting a crucial step)?


If you take the product of the first item (how long does it take now) times the second (how many times are you going to have to do it), that tells you what kind of time investment you have now. If that time investment is longer than what it would take you to automate it, no brainer again: automate it. The third item is about risk mitigation: if it's disastrous if you miss a step, that should make you lean towards automating it. For example, I was on a project not long ago where (for bizarre historical reasons) they didn't want to separate the compiled versions of their tests between unit, functional, and integration. Their solution was to create hand-crafted test suites. But, that means that we have to remember to add them by hand every time we created new ones. That's a big risk: I'm never going to remember that! Instead, we automated it by creating a little Groovy script that ran at build time to auto-generate the test suites based on the root directories. The risk of not automating this problem was wasted time: we wrote a test and forgot to include it by hand, meaning that we don't get the benefit of the test and thus wasted the time it took to write it.

I'm curious if others have tried-and-true strategies for justifying automation...

Friday, April 27, 2007

From Architect to Wrangler

One of the nice things about ThoughtWorks is the ability to create your own title, which leads to some interesting business cards (like a title of "Seat Warmer"). But, the restriction is that you only get one set of business cards, so choose wisely. Instead of going nuts on my first set, I chose the ultra-conservative "Application Architect", which is a reasonable approximation for what I do on projects.

But now I've changed. The title of "Architect" for software developers has gotten so diluted that its meaningless anymore. In fact, its almost pejorative because so many "paper" architects give it a bad name. I've gotten "Oh, dude, I'm so sorry" looks from people when I tell them I'm an architect, assuming that I've had a head injury or something and can't do real development work anymore. After long formulating and advice from others, I've now changed my title to the probably more apt "ThoughtWorker/Meme Wrangler". As you've probably noticed, I love the "meme" meme (after all, you are reading this on the "memeagora" blog). While it's a little cutsy, at least I'm not treated as a tragic has-been by coworkers and clients.

The interesting side effect of this happens because I put my title on my presentation slides. In the US, but particularly overseas, people ask me "What does mee-mee wrangler mean?" So, I get to explain it.

But that's way better than the now deprecated "Architect" title. It's a shame that, because we have no real industry-wide certifications, the nominally most advanced title has been co-opted by so many people divorced from reality. I've had to defend decisions made for SOA initiatives in front of "Architectural Review Boards" by people who last wrote code in COBOL. Can they really make good decisions about modern technology if they never touch it?

Git along, now, little neuron...

Monday, April 23, 2007

Keeping Knowledge in Plain Text

The book The Pragmatic Programmer is absolutely worth re-reading every few years, because every page has stuff on it that's still relevant. One of the important lessons is "Keep knowledge in plain text". To paraphrase, you shouldn't keep important information in a proprietary piece of software, because if the software dies (because the company goes out of business) or it relies on a changing protocol (more about this in a second), your knowledge is trapped. If you keep it in plain text, you can read (and manipulate it) using a lowly text editor.

Two things made me realize this in a Big Way recently. The first was my ToDo list. I used to keep all my ToDos in a good (but fatally proprietary) piece of software named LifeBalance. It's the best of that category of software, but using it meant that I had to have a Mac OS X, Windows, and Palm client to access it. It became its own project just to keep it in sync. And, unless LifeBalance works instantaneously, adding items takes time. I was reading the LifeHacker book (distillation of the awesome LifeHacker web site) earlier this year, and one of the first items is an entry about a text-based ToDo list manager called ToDo.txt (which has its own website). It's called "Task tracking for command line lovers". The nice thing about ToDo.txt is that it uses a plain text file for storage. And, the author has written a bunch of bash/batch files to help manage it. That means you can append stuff of your own (using QuickSilver, in my case) or you can use the tools. The command line tools also let you prioritize, archive, and a bunch of other stuff. It's a good marriage between command line and plain text.


The other driving home of the important "Keep knowledge in plain text" lesson concerns my blog. For years, I just wrote my blog entries in a text editor in HTML and uploaded them by hand to Blogger. Last year, I fell for a seductive blog editor/manager called MarsEdit. It works fine, allowing you to create drafts, view your blog, handle multiple blogs, etc. The way it did this was via using one of Blogger's APIs for managing blogs.


Then something terrible happened: Blogger changed their API. Suddenly, MarsEdit didn't work anymore. Posting new entries wasn't a problem because I could go back to doing it by hand, but what amount my 9 months of existing, historical blog entries it held? Trapped! Now, I can't use it, and I can't easily free all my information. I violated the rule!


I just spent a couple of hours fixing this mess, re-syncing my original directory structure by hand (at least I was able to use wget to slurp my whole blog down to my local machine, making it much easier). I've felt the pain of violating the rule, and I promise not to do it again. My last step was removing MarsEdit from my machine (which, BTW, has now updated itself to work with the new API -- unlike our president, I remember the phrase "Fooled me once, shame on you, fooled me twice, shame on me").

Of course, Dave and Andy already knew this, and they told me, but I wasn't listening. Sometimes, you have to suffer for good advice to take hold.

Friday, April 20, 2007

Don't Be a Hater -- Think Different

People still fear what they don't understand or what's not familiar. When I arrived at the SOA conference in Kuala Lumpur a couple of days ago, I was met by the hotel's AV guy to hook up my Mac to the projector. I used the normal elaborate procedure I usually do to sync my Mac with a projector: I plugged the projector into my machine. But, unusually, no signal. I tried Display Preferences, changing the resolution, the number of colors: nothing. He was convinced that my exotic, bizarre machine/OS was the problem, and I kept trying to convince him otherwise. Finally, I persuaded him to go get another cable, which he reluctantly did (I don't know any Malay curse words, but I'm pretty sure I've now heard some). You can probably guess what happened next: new cable, perfect signal, at all my original settings.

Which reminded me of another conversation I had a couple of years ago with a friend in Singapore. We were sitting around talking about computers, and he made an off-hand comment that he didn't like Macs because he thought they were unintuitive, but he thought Windows made perfect sense. So, I asked him "So, you're telling me that the operating system where you click on the Start button to turn it off is the more intuitive on? I think you're just used to it", to which he replied: "You're right, I'll shut up now". I saw him again last night in Singapore, and we were catching up on all sorts of stuff. And you know what he told me? He just got a new machine with Vista on it, used it for 2 weeks, and returned it to Dell. He told me that he can't wait until his laptop finally dies because he's going to get a Mac. The reason for the change? He's now addicted to his iPod, and it's made him realize that Design Matters.

I can't resist beating everyone on the head with the moral: Don't succumb to FUD because something is different. And, in the tortured grammar of the old Apple campaign, Think Different. Because Vista is so different from previous versions, it may be the best thing that ever happened to Apple. If you're contemplating a change anyway, why not look at the entire field so that you end up with the OS equivalent of an iPod, not a Zune.

Wednesday, April 18, 2007

Pontificating about SOA

What else is there to do about SOA? Back in December, Mark Richards (Enterprise Architect at IBM) and I sat down and recorded a video PodCast about SOA (which we informally named "Taking Back SOA"). The results has now appeared at the No Fluff, Just Stuff web site. Mark and I have a very pragmatic approach to SOA, and we both hate the level of hype that appears in that whole space. He and I had a great time recording this video, which proves once and for all that IBMers and ThoughtWorkers can get along after all!

Tuesday, April 17, 2007

No, I Won't Stop Talking about IntelliJ!

During my No Fluff, Just Stuff talk on The Productive Programmer, I got a comment at the Minneapolis show to "stop showing us IntelliJ shortcuts -- we all use Eclipse!" And, Venkat Subramaniam blogged about the whole Eclipse vs. IntelliJ debate that came up at the expert panel in that same city. Venkat quite eloquently gives his reasons for choosing IntelliJ: it simply makes him more productive. And the same is true for me. I've used all the major Java IDE's in anger: NetBeans, JBuilder (I used to be considered a JBuilder expert, but I got over it), Eclipse, and IntelliJ. For me, IntelliJ works best. Period. In fact, I think that IntelliJ would land in my top five pieces of software of all time list. I also edited the No Fluff, Just Stuff Anthology chapter on IntelliJ tips and tricks, just out in treeware.

There are actually 2 IDE Tips and Tricks chapters in the anthology. When I sent out call for contributions from the authors and friends for IntelliJ tips, I got a flood of them, all very cool (and a few that I didn't already know about, like the Key Promoter). Ted Neward (who edited the Eclipse chapter) and I sent out the same call for Eclipse tips and tricks, and we got none. We sent out another call, and a few trickled in. And that is reflected in the book. I'm sure we're going to get grief over the IntelliJ chapter having more cool stuff. But that's really the whole point: like the preponderance of Macs on the No Fluff, Just Stuff tour, the people who use IntelliJ are passionate about it, because it exudes excellence. Like Venkat says, many people using Eclipse are in a bad arranged marriage. It's hard to drum up passion for an arranged marriage.

I feel like it is my duty to try to turn people on to things that make their life better. That's part of what being a speaker is about. So, I'm not apologetic about proselytizing IntelliJ. Every-time I find a tool that I think will make developer's lives easier, I'm going to talk about it. And, if I develop a passion about it, so be it.

PodCasting About Groovy

A couple of weeks ago, I recorded a PodCast for AboutGroovy.com, talking about (obviously) Groovy, DSLs, testing, and how to insinuate Groovy into your company's infrastructure. Basically, Scott Davis and I had the kind of conversation we have all the time at No Fluff, Just Stuff. The only difference is that he was recording it. It was fun and the content will come as no surprise to anyone who knows me.

Monday, April 16, 2007

Being a Consultant Makes You Tough (or Stupid)

Since working for ThoughtWorks these 2 years, I've gotten much tougher in the face of being able to work on short or bizarrely interrupted sleep. Case in point: I left Minneapolis (after speaking at the sold-out No Fluff, Just Stuff Twin Cities Software Symposium) on Saturday evening, dashing from my last session at 4:45 straight to the airport. 8 hours later, I was in Amsterdamn, with a 2 hour layover. 12 hours after that, I'm in Kuala Lumpur, arriving at 6:15 AM. I probably got about 8 hour of sleep, but never more than 2 hours at a time. I'm now exactly on the other side of the world from where I live: 12 time-zones away. I cleared customs, got my suitcase, and headed for the car service the conference had arranged. Traffic was awful, and combined with the distance of the airport from downtown KL, I arrived at the hotel at 8:35 AM...and the conference starts at 9 AM with me facilitating. Quick shower, dash upstairs, and I'm ready to go at 9 AM sharp. I then proceeded to talk on and off for 8 hours. I managed to stay up until after 8 PM. It's now the next day, and I feel fine. I'm apparently adjusted to the new time zone (I sleep soundly last night, interrupted only by my phone ringing at 11 PM (or, 11 AM in the states)) which I promptly ignored.

Two years ago, I could have never done this. Just part of the implicit training all the travel and distributed work instills. But there are benefits as well. Here is a picture taken just outside the hotel's meeting rooms.

Friday, April 13, 2007

The Elusive Right Click

Since he got his Mac, Venkat Subramaniam and I have been playing around with pushing the envelope on human computer interaction and productivity as hard as we can. He finds a cool new keyboard shortcut and can't wait to tell me, I learn a new QuickSilver bit of magic and can't wait to tell him. The Holy Grail of this kind of pursuit is finding a way to generate a mouse right-click using only the keyboard. Of course, Mac OS X is pretty friendly to use the track pad, but that's not enough for us: total keyboard control is the goal.

And yesterday I stumbled upon the Holy Grail. I was in the process of shaving a yak, and was waiting for something to finish, so I decided to solve this problem Once and For All. I did a bunch of Googling and experimentation and stumbled upon the solution. Go to Universal Access and turn on Mouse Keys. This the the accessibility option that allows you to drive the mouse strictly from the keyboard, using the embedded keypad. Once you turn that on, you can use the FN-I key to click the mouse, which means that you can use FN-CTRL-I to right click. It's not perfect (it clicks where the mouse current resides, not where the keyboard focus lives) and it precludes using the embedded numeric keypad for doing 10-key entry (which I never do anyway). You can turn it on and off by clicking the OPTION key five times (this is a setting on Universal Access as well).

But we're happy. Venkat got as excited as I've ever seen him when I showed him this morning; I was looking forward to show it to him all day. Ah, simple pleasures!

Tuesday, March 27, 2007

Don't Misquote Me to Create a Straw Man!

OK, I can't let this pass.

The ServerSide wrote an article, summarizing both mine and Venkat Subramaniam's talks at The ServerSide Symposium last week. It's a reasonable summary, with a few details left out (that's why it's a "summary").

But then, Cedric Beust takes the summary and constructs a agile development straw man to beat up. And, like many straw man arguments, he takes lots of stuff out of context and makes his own conclusions not supported by the original talk.

The first thing he gets wrong is the fault of the summary. Early in the talk, I clearly state that the best way to get good software is to interact directly with the users. And I give an example from my project history of the best project experience I every participated in, where the developers literally sat in the middle of the users. Every time we had a question, we just got up and resolved it directly with the users. It was project heaven.

But most corporate environments won't give you such direct access to the users, because the users are after all there to get work done. Thus, you must have user proxies. And that's where the business analysts come it. Perhaps at ThoughtWorks, our business analysts are different from the "typical" ones (I know our project managers are). The best business analysts are direct proxies for the users. They understand the users concerns and a little about technology. During the talk I also made the point that good business analysts prevent "paving the cowpath", suggesting innovative business solutions to problems rather than "that's the way we've always done it".

When I say that business analysts participate in the estimation process, that's because we can't get the users directly.

Do you know the difference between a good developer and a bad developer? A good developer can give you an estimate of the time needed to complete a task. And the better that developer is, the more accurate their estimate will be. If I ask a developer "How long is it going to take to do this?" and he answers with "It's very complicated", I want him off my team.


Another point I made in my talk is that, regardless of developer, you can't ask developers "how long will this take". Every project is different: in a 2 man project with no meetings and direct access to users, each developer will get more done in less time. In a project with 16 developers and a 2-hour status meeting every day, each developer will be much less effective (this has been known since Brook's The Mythical Man Month). But that is constant is the relative complexity of the story you are trying to complete. Relative to the other stories, not the entire universe. This allows developers to hone their estimation skills because it cuts down on the number of variables they have to hold in their head. This is the closet you can get to an objective measurement of the work required to complete a piece of software.

We don't need agile consultants and book authors to sell us vague and unproven metrics along with rigid guidelines that your team must embrace without any questions.


Ironically enough, we agree wholeheartedly about this, but he doesn't seem to know that this is exactly the point of my talk at The Serverside Symposium. The whole point of the talk was that we need real metrics like passed tests, real reconciliation between estimates and actuals, and binary completion criteria. The article makes that pretty clear, but that doesn't contribute to the weak straw man, so Cedric conveniently left that out.

What's most frustrating about Cedric's rant is the implicit assumption that "agile" is this vague concept that no one seems to be able to define. Yet, at ThoughtWorks, every single project is agile to some degree (and where we control everything (like fixed bid projects), it's pretty close to Extreme Programming). And we have a really high success rate, way above the industry average. I've been around for a while (one of my colleagues once told me that I was old enough to "add diversity to the company"), and I've used a bunch of different methodologies. One of the reasons I'm at ThoughtWorks is because we use proven techniques for building software, and we're damn good at it, making it a much less frustrating place to work.

Wednesday, March 21, 2007

DSL Podcast on No Fluff, Just Stuff

As my regular reader(s) know, I'm big on Domain Specific Languages, and have been talking and writing about this style of development for a while. During the course of last year, we recorded some PodCasts at some of the No Fluff, Just Stuff shows. One of those shows was Chicago, where I recorded a PodCast about my feeling about DSLs and this approach to building software using this technique. The PodCast has now appeared on the No Fluff, Just Stuff web site, for your listening pleasure.

Saturday, March 17, 2007

Anthologization Redux

Well, it's that time of year again: the No Fluff, Just Stuff Anthology, Volume 2 (The 2007 Edition) is at the printers, undergoing the conversion from thought to dead trees.

NFJS Book cover
Just like the blurbage on the back says:

Once again, some Sixteen of the world's best trainers and speakers are writing chapters on things they care passionately about. You'll find topics from the latest conferences including Groovy, JavaScript, Continuations, Web services and REST, JVM Byte Code, and Agility

These essays are a summary of the latest thinking in the industry, and range from the philosophical to the tutorial, covering the topics that the writers felt were the most important for readers today. If you feel like the neatest technology and latest ideas are passing you by, this book can help bring you back you to speed.

It's all good stuff, without any fluffy filler, as these essays are based on presentations given at the incredibly popular "No Fluff, Just Stuff" symposium series. Twenty-six times a year, the symposium visits a city and the speakers and attendees share ideas and perspectives. The speakers are all internationally known experts in their field.

So, to out and buy one as soon as possible (in a book store near you in early April). Or, order it directly from the Prags or at Amazon. In fact, I recommend that you buy two in case you loose the first.

Of course, I was intimately involved in its creation, but this is a really entertaining read (the main reason I like editing the Anthology is because it means I can read the essays as soon as possible).

Enjoy!