Saturday, September 22, 2007

Travel: Biking Adventures Through the Woods Outside Frankfurt

Warning: this post is entirely about travel stuff: there is no software or technology here, so feel free to avoid this if that's all you care about.
Terry Dietzler and I have spoken at the Entwickler conference for 9 years in a row (that's how I always spend my last week of September). On the first trip here, the hotel had bicycles you could borrow for the day, and we did several bike trips through the German countryside (the conference is in a suburb of Frankfurt, named Morfelden, near a national forest). We were hooked. Biking in Europe is a blast: there are bike routes simply everywhere (you can buy maps of just bike routes in bike stores). Our most elaborate trip to date was in 2003, when we biked about 200 miles in 2 days, from Frankfurt (actually, Morfelden) to Metz, France. In 2002, we made a deal with the conference organizer: in lieu of payment for pre-conference tutorials, buy us bikes and keep them in Frankfurt for us, in the basement of their offices. They kindly agreed, so Terry and I have bikes that live year-round in Germany.

This is all prelude to the adventure we had yesterday (we tend to have lots of adventures when we are here in Germany). We took the train (the S-bahn, or regional train) into the city and picked up the bikes. We had thought about bringing them back to Morfelden on the train, but decided instead of ride them back: follow the Main river trail until it gets out of the city, them find our way back to Morfelden. The biggest problem: the Frankfurt airport is between us and Morfelden, but we had maps, so we can find our way. Here is the route we ended up taking (Terry has a GPS, and he turned it on, so we got a record of the entire route):



It was 9 miles (according to the GPS) from where we picked up the bikes to our destination...as the crow flies. We estimated more like 20 miles because of the major obstacle, the airport. About 2 hours should to it; we started out at about 5.

As you can tell, we made good progress along the river, and we got minorly lost a few times as we lost the trail then reacquired it later. Very typical. Then, as we got near the airport, we started having to cross more major autobahns and train tracks. There are abundant bike bridges and crossing, you just have to find them. And, notice the backtracking as we neared the airport. We got on one trail that literally dead-ended at the junction of two sets of train tracks. As we're thrashing around, the sun is setting. Many of the trails are through the woods, so it's getting darker and darker. Our bikes have headlights, but mine was the only one working. So, picture this: we're biking through dense woods in pitch blackness following one headlight. Fortunately, the trails are very good, but it was still spooky and hard to do because the light only reached a little ways ahead of us, and it looked like we were biking into a black wall, which made me claustrophobic.

Fortunately, we've biked around Morfelden a lot, and as we got close, we recognized a landmark from a former trip. We followed the trail through a really dark and increasingly cold section of trail until, to the right, we spotted our hotel. 4 hours after we started, we ended up cycling 23 miles (including all the switch backs). What fun; we're going out again tomorrow.

Monday, September 10, 2007

Ruby Matters: "Design Patterns" in Dynamic Languages

As I mentioned in my last post (Ruby Matters: Meta-programming, Synthesis, and Generation), the Gang of Four design patterns book should have been named "Palliatives for C++". One of the authors finally admitted as much in public at a roast. So, why would design patterns be any different in a dynamic language (like Smalltalk or Ruby)?

In the GoF book, design patterns are 2 things: nomenclature and recipes. The nomenclature part is useful. It's a way of cutting down on repetition of similar code across projects, and it gives developers a way to talk to one another, using shorthand. Instead of saying "I need to create an object that can only be instantiated once", you say "singleton" (we'll leave aside for the moment why Singleton is evil -- the subject of another blog entry). Nomenclature good.

But the fatal flaw in the GoF book was that they included recipes. And many people thought they were the best part. Even now, you see books on Java design patterns that blindly mimic the structure of the examples in the GoF book (even though Java has some better mechanisms, like interfaces vs. pure virtual classes). Recipes bad. Because they suggest more than just a way to name common things. They imply (and put in you face) implementation details.

Because of meta-programming, many of the design patterns in the GoF book (especially the structural ones) have much simpler, cleaner implementations. Yet if you come from a weaker language, your first impulse is to implement solutions just as you would from the recipe.

Here's an example. Let's say you have a simple Employee class, with a few fields (which could be properties in the Java sense, but it makes no difference for this example).


public class Employee {
public String name;
public int hireYear;
public double salary;

public Employee(String name, int hireYear, double salary) {
this.name = name;
this.hireYear = hireYear;
this.salary = salary;
}

public String getName() { return this.name; }
public int getHireYear() { return this.hireYear; }
public double getSalary() { return this.salary; }
}


Now, you need to be able to sort employees by any one of their fields. This is a flavor of the Strategy Design Pattern: extracting an algorithm into separate classes so that you can have different flavors. Java already includes the basic mechanism for this, the Comparator interface. So, here's what the code to be able to compare on any field looks like in Java:


public class EmployeeSorter {
private String _selectionCriteria;

public EmployeeSorter(String selectionCriteria) {
_selectionCriteria = selectionCriteria;
}

public void sort(List<Employee> employees) {
Collections.sort(employees, getComparatorFor(_selectionCriteria));
}

public Comparator<Employee> getComparatorFor(String field) {
if (field.equals("name"))
return new Comparator<Employee>() {
public int compare(Employee p1, Employee p2) {
return p1.name.compareTo(p2.name);
}
};
else if (field.equals("hireYear"))
return new Comparator<Employee>() {
public int compare(Employee p1, Employee p2) {
return p1.hireYear - p2.hireYear;
}
};
else if (field.equals("salary")) {
return new Comparator<Employee>() {
public int compare(Employee p1, Employee p2) {
return (int) (p1.salary - p2.salary);
}
};
}
return null;
}
}


You might protest that this is overly complicated, and that this will do the job:

public Comparator<Employee> getComparatorFor(final String field) {
return new Comparator<Employee>() {
public int compare(Employee p1, Employee p2) {
if (field.equals("name"))
return p1.name.compareTo(p2.name);
else if (field.equals("hireYear"))
return p1.hireYear - p2.hireYear;
else if (field.equals("salary"))
return (int) (p1.salary - p2.salary);
else
// return what? everything is a legal value!
}

}
}

but this one won't work because you must have a return, and returning any value is mis-leading (every possible integer here means something: 1 for greater than, 0 for equal, -1 for less than). So, you are left with the bigger version. I've actually attempted to optimize this several ways (with more generics, reflection, etc. but I always get defeated by the requirement to return a meaningful int from the comparison method. You could find some kind of minor optimization, but you are still building structure to solve the problem: a class per comparison strategy. This is the typical structural approach to solving this problem.

Here's the same solution in Ruby, using a similar Employee class:

class Array
def sort_by_attribute(sym)
sort {|x,y| x.send(sym) <=> y.send(sym) }
end
end

In this code, I'm taking advantage of several Ruby features. First, I'm applying this to the Array class, not a separate ComparatorFactory. Open classes allow me to add methods to built-in collections. Then, I'm taking advantage of the fact that Ruby method calling semantics are message based. A method call in Ruby is basically just sending a message to an object. So, when you see x.send(sym), if sym has a value of age, we're calling x.age, which is Ruby's version of the accessor for that property. I'm also taking advantage of the Ruby "spaceship" operator, which does the same thing that the Java compare method does: return a negative if the left argument is less than the right argument, 0 if they are equal, and positive otherwise. Wow, it's nice to have that as an operator. I should add that to Java...oh, wait, I can't. No operator overloading.

Perhaps it makes you squeamish to add this method to the Array class (open classes seem to terrify Java developers a lot). Instead, in Ruby, you could add this method to a particular instance of array:

employees = []
def employees.sort_by_attribute(sym)
sort {|x, y| x.send(sym) <=> y.send(sym)}
end

Now, the improved employees array (and only this instance of array) has the new method. A place to put your stuff, indeed.

When you have a powerful enough language, many of the design patterns melt away into just a few lines of code. The patterns themselves are still useful as nomenclature, but not as implementation details. And that's the real value. Unfortunately, the implementation seems to be the focus for many developers. Strong languages show you that the implementation details can be so trivial that you hardly even need the nomenclature (but its still useful as a conversation shortener.)

Next, I'll talk about language beauty.

Wednesday, September 05, 2007

Ruby Matters: Meta-programming, Synthesis, and Generation

In my last Ruby Matters blog post, I talked about meta-programming in Ruby, contending that Ruby gives you "places to put your stuff". I always wondered about meta-programming in Smalltalk and how that compares to Ruby, and Where to Put Stuff in Smalltalk. The final piece of the puzzle came after I talked to Glenn Vanderburg (the Chief Scientist of Relevance). I was puzzled as to why the Gang of Four book (which had examples in both C++ and Smalltalk) didn't have more meta-programming. Lots of the design patterns are almost trivially easy to implement with meta-programming, but they didn't do that in the Smalltalk examples. They used the same structural approach as C++. It looks more and more to me that the Design Patterns book was really just a way to solve problems in C++ that should have been easier to solve in a more powerful language like Smalltalk. Which is why I was puzzled about the lack of of more meta-programming solutions in Smalltalk. Glenn enlightened me. One of the overriding characteristics of Smalltalk is the way code is stored, in an image file, which allows for really smart tools. The program and the environment all reside in the binary image file. There are no source files as we know them today, just the image.

A comparison about implementation details is in order, on how Ruby differs from Smalltalk. Let's talk about has_many in Ruby on Rails. Typical Rails code looks like this:

class Order < ActiveRecord::Base
has_many :lineitems
end

For those not familiar with Ruby, this is a method, defined as a class-level initializer (just like an instance code block in Java, a chunk of curly-brace code in the middle of a class definition, which Java picks up and executes as the class is instantiated). So, ultimately, this is the Ruby equivalent of a static method call, which gets called as the class is created.

Let's talk about Smalltalk, which has first-class meta-programming. You could easily build has_many in Smalltalk, implemented as a button you click in the browser which launches a dialog with properties that allow you to set all the characteristics embodied in the Ruby version. When you are done with the dialog, it would go do exactly what Ruby does in Rails: generate a bunch of methods, add them to the class (stuff like the find_* and count_* methods). When you are done, all the methods would be there, as instance methods of your class.

OK, so at this point, the behavior is the same in Smalltalk as in Rails. But there is one key difference: The Smalltalk version using code generation. It's a sophisticated version of a code wizard, generating the code using meta-programming techniques. The Ruby version uses code synthesis: it generates the code at runtime, not build time. Building stuff at runtime means more flexibility. But that is a minor point compared to this one: In the Smalltalk version, you use the dialog and properties to generate all the methods you need. The original impetus for the has_many intent lives only while you are running the dialog. Once you are finished, you are left with lots of imperative code. In the Ruby version, the intent stays right where you put it. When you read the class again, 6 months from now, you can clearly see that you still mean has_many. Smalltalk has the same meta-programming support, but the intent of code synthesis remains forever. That's why it's important to have a place to put your stuff. It isn't accidental in Rails that many of the DSL characteristics appear as class methods rather than code you call in the initialize method. Placing them as part of the class declaration declares intent in a big way, and keeps the code very declarative.

To summarize the similarities and differences:

  1. Both Ruby and Smalltalk give you a place to put your meta-programmed stuff

  2. Both give you a place to put the declaration of intent (the tool in Smalltalk, the shadow meta-class in Ruby)

  3. A time in the lifecycle of the class when things happen. In the Smalltalk version, it's a one-time deal, as you use the tool to generate the code. In Ruby, the synthesis takes place at class load time. This leaves the clean, declarative code right where you put it, rather than generating a bunch of much less clear imperative code.


Glenn made an excellent point here: the Smalltalk version is a great example of accidental complexity, not essential complexity. Software is full of essential complexity: writing software is hard. But we end up subjecting ourselves to lots of accidental complexity in our tools and languages. And it should be stamped out. The Ruby version eliminates accidental complexity by providing a great abbreviation for the intent of has_many. This blog does a great job of illustrating the differences between abbreviations and abstractions.

Smalltalk had (and has) an awesome environment, including incredible tool support. Because the tool is pervasive (literally part of the project itself), Smalltalkers generally shied away from the kind of meta-programming described above because you have to build tool support along with the meta-programmed code. This is a conscious trade off. One of the best things about the Ruby world (up until now) is the lack of tool support, meaning that the tools never constrained (either literally or influentially) what you can do. Fortunately, this level of power in Ruby is pretty ingrained (look at Rails for lots of examples), so even when the tools finally come out, they need to support the incredible things you can do in Ruby.

Next up, "design patterns" in Ruby.

Thanks to Glenn for supplying me Smalltalk information, as a sounding board for this, and all the interesting bits, really.

Monday, September 03, 2007

Ruby Matters: A Place to Put Your Stuff

Why are so many people into Ruby? I get this question a lot because I speak at a lot of Java conferences and talk about Ruby and JRuby. In this and several more blog entries, I'm going to explore this question in depth.


For myself and many of my peers, it was a rite of passage to learn all the intricate tricky stuff in C++. Once I got over my fascination with C++ and got tired of banging my head against all of the arcane nonsense in the language (I used to know the details of all the different ways you could use the word const, but I've fortunately purged that knowledge), I asked one of my colleagues why he continued to love it so much. "Knowing C++ makes me part of the C++ priesthood. When people ask me questions, I'm purposely cagey when I give them answers because I think they should work as hard as I did to figure this stuff out. I don't want to give my advantage away." I shuddered and was glad I wandered to greener pastures.

After a brief visit to the refugee camp of Delphi, I made it to Java and thought I was in heaven. Finally, a language that made some sense. I liked the locked down nature of the language because I thought that Java helped prevent mistakes, especially for large projects. This is before I had embraced unit testing, so that seemed like a good thing. I would gladly give up flexibility for perceived safety. This was when I used to think that one of the most important duties of a computer language was to prevent mistakes first and support building interesting stuff second. Interesting stuff must be hazardous to large projects, and I was an Enterprise Developer. After more than a decade in Java, I started playing with Ruby because I'm a bit of a language geek and I was taking the advice in The Pragmatic Programer to learn a new language every year. But, at the time, Ruby was just a cool little scripting language with nice syntax.

When Ruby on Rails came out, I started looking at it with great interest because it was so vastly different from the Java frameworks with which I was so familiar (this was just after I had written a book comparing them). And the more I looked at Ruby, the more puzzled I got. Wow, there's some funky stuff in there, and I couldn't understand what some of the code meant. So, I dug deeper. And slowly, I gained enlightenment. At first, I thought that it was just more of the same "obfuscated code" syndrome from C++. But I later understood that it wasn't.

The epiphany really came when I understood what some of the Rails code was doing with meta-programming, which made me look closer at the core Ruby stuff. Meta-programming was supposed to be scary stuff that only wild hackers did. But it made problems melt away, in very few lines of code. If you look at how attr_accessor works, it's so simple. I thought for a long time that attr_accessor was a keyword or something infra-structural that I shouldn't bother looking at (before I realized that lots of stuff in Ruby looks like keywords but they aren't).

I have come to understand that meta-programability is an extremely important part of a language. So many things that we solve in languages like Java with complex structures and hierarchies are much more elegantly solved with meta-programming. This realization led me to start looking around at other languages, like Smalltalk and Python.


That's when it hit me: one of the things that makes Ruby so powerful is that it already gives you a place to put your stuff. If you don't think this is any different than Java, check out whytheluckystiff's Seeing Meta-classes Clearly article. I'll wait... Yes, there is a shadow meta-class behind every object in Ruby, giving you the syntax and location to put much of the interesting meta-programming stuff in Ruby. You can do lots of the same meta stuff in, for example, Groovy, but you have to create a Groovy builder object to do the equivalent of eval. And, doing class_eval would take more building a place to put it. I was talking to a Python programmer recently, talking about the same subject, and I got the same response: "Yeah, Python can handle that, but you'll have to build a place to put it." One of the elegant features of Ruby is that it comes pre-defined with places to put your important stuff, rather than forcing you to build a place to put your stuff. Which is why Paul Graham says that Lisp is the most powerful language available; because you are writing code in the abstract syntax tree, you are living in the place where stuff gets put.

In the next post, I'll talk about the special case of Smalltalk, tools, and code generation vs. synthesis.