Ruby for Java Developers

Ruby for Java Developers

My article on the Ruby Developer's Kit has just been published by the IBM Developer Works site. The RDT is not a full-blown IDE for Ruby yet, but it does have excellent support for debugging Ruby code, which is a huge win for me. I've debugged some pretty hairy Ruby code in it (including code that does automation in Windows) and have been very happy.

See, I'm trying to find a way to bind the Java and Ruby communities together. Can't we all just get along?

Missing the Point

One more post from the Entwickler Conference, where the Borland faithful gather in Germany. I sat though the technical keynotes, and they preach two things: Rapid Application Development tools and Big Design Up Front with requirements gathering tools (CaliberRM) and UML (Together). I know a lot of vendors are still beating these horses, and Borland is no different. Delphi now includes a EJB-like framework for persistence and workflow called ECO, which makes heavy use of tooling and inheritance. It doesn't use any of the interesting techniques used widely in the Java world (like dependency injection). Looking at ECO gives me Java flashbacks from 3 or 4 years ago. Anyway, the big finale of the technical keynote (which showed that they have created very good products in their categories), David I (Borland evangelist) showed a video of a 5th grader using Delphi and ECO to build a Teachers -> Classes -> Grades application with persistent and a generated front end in 9 minutes. The obvious message: you can build simple things Really Fast with our tools. Or maybe it was "Build Crappy Software Faster!"

Is that the right message to send out to the world? Is raw developer productivity for mindlessly simple applications buying us anything? The only people impressed by this type of demo-ware are non-technical managers, who have budgets. "Wow, if a 9 year old can build an application in 9 minutes, just think our developers will be building our entire corporate infrastructure! We must buy this tool!". I've posted my thoughts on RAD before. It is disappointing to see a company with so many sharp people pushing 2 aspects of software development that I think counter productive.

Not everyone in Borland feels this way. I'm corresponding with several Borlanders about how they can support agility in their Application Lifecycle Management (ALM) tools. The tide is turning. I am not naive enough to think that they should just ditch their product base because of what is still just opinion. I just hate to see what I consider poisonous messages to developers and their managers.

Dexter

As I stated in my last post, I spent the last week at the Entwickler Conference with the Delphi crowd. One of the highlights of the conference was the technical keynote where they showed the next version of Delphi, code-named Dexter. First, a little background about Delphi. From version 1 thought 7, Delphi was written in itself. The IDE is an Object Pascal project, written using the tool itself (including the components available in the tool. This is nothing new, building a tool in itself, but it was an eloquent argument for the capabilities of the language and libraries. Starting in version 8, Borland re-wrote their IDE base, creating Galileo. Galileo was the base host IDE for both Delphi 8 and C#Builder, their competitor to Visual Studio. One of the key benefits of Delphi 1 through 7 disappeared overnight, because they had to start from scratch. Delphi 8 had a different look and a small sub-set of the capabilities of the version 7 IDE. It was slow, buggy, and had serious re-drawing issues (watching it try to launch an application caused so much flickering that it could induce seizures). This, by the way, was richly ironic to the JBuilder crowd, which had been suffering taunts about slowness and ugliness for years from the Delphi crowd. Flash forward a couple of years. C#Builder became a non-product, and was rolled into Delphi 2005, which hosted both languages. It was a little better but still only about 30% as capable of a really good IDE like IntelliJ.

This has changed somewhat with Dexter. I was pretty impressed with what I saw. As a development tool, it is approximately on par with Visual Studio now in all but stability (beta testers tell me it is still pretty scary). Microsoft has the luxury of beta testing VS for 2 years; Borland cannot afford even a fraction of that. Still, the environment looks pretty good. It finally has a reasonable number of refactorings available (Delphi 2005 had 4, and from the reaction of the Delphi faithful last year, you would have thought that Borland had invented cold fusion). In my estimation, it is about 70% of a real IDE now. It is still hosted on Galileo, but it has greatly improved over the last year or so. The same IDE supports both Win32 and .NET development and has all the Borland Secret Sauce components. The next release, due out mid-next year, will even support .NET 2.0. Still not compelling enough to get anyone to consider it as an alternative to Visual Studio for new C# development (unless you are deeply immersed in Borland's Application Lifecycle Management integration strategy) but at least it isn't an embarrassment now.

How Important is Syntax?

I spent the last week at Kool-aid central for Borland's Delphi RAD
development tool. I was speaking at the Entwickler Conference
("Entwickler" is German for "developer") in Frankfurt, talking about
Java and .NET. This conference is ostensibly a general developer's
conference, but Borland is one of the main sponsors and it has
traditionally been the European gathering place for Delphi developers.
The faithful only get one Delphi focused conference per major land
mass, so they all come. The Java and .NET guys can get information
anywhere. Probably 70% of the talks at this conference either focus on
Delphi or overlap significantly (for example, I did an "Updated
Design Patterns in .NET " in C#, but the material also applies to
Delphi.NET).

The interesting part of my trip were the discussions between sessions,
at dinner, and in the halls. Even though Delphi is a niche development
tool with a shrinking market, the people at this conference are
unusually passionate about their tool. I sought to understand this
blazing exuberance. I traverse several different languages and tools
pretty regularly (Java, .NET, and Ruby), and to me they are just tools
-- each has its own strengths and weaknesses. I also used Delphi
pretty much exclusively for about 4 years (and wrote one of the first
Delphi books), so I understand the tool and language. But I've never
gotten so vein-bulging-in-the-forehead excited when someone told me
they thought something else was better.

This crowd has its back against the wall: Borland's clear future
direction with Delphi is to make it Yet Another .NET Language
(YA.NL?). The last couple of versions of Delphi have supported both
Win32 and .NET capabilities (and most of the developers in Germany are
still doing Win32 applications). In my opinion, this marginalizes
Delphi to also-ran status with C# and Visual Studio. Sure, if you are
building in Delphi now and have a bunch of code, you'll stick to Delphi
(maybe). But if you are starting a new project, there is no way you
would pick Delphi over C#. The IDE isn't any better (and it is less
stable) than Visual Studio, and it will not have a version that
supports .NET 2 until mid-next year.

Tiptoeing around the fanatics, I asked a few people if the above
assessment is true. "No! The next version of Delphi will win converts in
droves!". At the end of the day, it boils down to which syntax do you
like (if the tools are at about parity and the framework is the same,
you are down to begin...end pairs vs. curly braces). I've always had an
easy time switching syntaxes, both between similar (Java and C#)
languages and very different ones (Java and Ruby). I asked some of
these folks: "Is the syntax of the language really that important to
you?" I got a resounding "Yes!" from several people. They are willing
to use a third-world development tool just so they get to type Pascal
all day instead of C#. Just like all marginalized peoples, they
overcompensate anytime someone like me desecrates the object of their
devotion.

You see some of the same fervency in the Ruby and Lisp crowds, but they
are genuinely different from their peers. The Delphi vs. C# crowd are
supporting the same platform, with different syntaxes. Would you be
willing to jeopardize your job over the syntax of a programming
language? I still do not get it.

Swiming Up the Waterfall

Sometimes, you write something that you have to keep a lid on for a little while, because you don't want to offend someone who might be reading. This is one of those cases.

A fellow co-worker and I found ourselves in a client site who was a big believer in my most favorite of dysfunctional of methodologies, Waterfall. We were hired to help them design an advanced new system (that, of course, interacted with their still very active Mainframe development) with innovative technology. My coworker and I showed up with great enthusiasm (because we didn't know what was going on yet).

Here is what we were tasked to create: First, we need a Functional Design document (a technical-ish set of requirements) to pass off to the business unit to get approval for the project (which was already approved -- that's why we were there). It took man-weeks of effort to produce this utterly useless document, which was created by emailing Word documents around, because they don't put design documents in version control (What? Did I hear that right? Why not? "We just don't" “No, why?” “If it’s in version control, others can see it”). OK.

Next comes the Technical Design Document, another multi-man-week project, where we take the Functional Design and produce a Technical Design, meaning that we finally get to reveal all the technical decisions we made while writing the Functional Design but couldn't put in that document. And we can't produce "prototypes", because that's a bad word -- business people want to see prototypes run. "Can we call them spikes? Sure, no one knows that that means". A few times, I thought my coworker's head was actually going to explode. We set up our own secret version control system and starting writing our design documents (oh, sorry, Design Documents) using XML so that we could diff them, and converted them to Word documents when the big flurry of Consolidating the Documents happened.

Then we made a decision. Given the amount of time that we had to create the Technical Design Document, we could just write the code, using XP techniques (test-first coding, iterative design, etc.) and produce the design document based on the working, tested code. And that's what we did with part of the system. We kept this under wraps (after all, we're supposed to be designing, not coding) and made good progress. To show the quality of the code, I implemented code coverage using Cobertura so that we could demonstrate that the unit tests were actually exercising the code. With just a couple of days left in the Technical Design phase, we were asked to provide estimtes as to how long it would take to implement our design. That's when we let the cat out of the bag: "Do you want us to tell you how long it would take, or how long we have left? We can give you really good estimates because it's already done". We showed them the implementation and the unit tests, along with code coverage (96% code coverage, 100% branch coverage) . The Technical Design document that was already based on working code.

They weren't upset, but weren't particularly happy either -- just kind of stunned. This way of writing software is so foreign that they have no perspective on it. They've never seen code with unit tests, or code coverage, nor any code that was produced in an iterative way. You are supposed to spend months on design documents, then write the code, right? How is it that you can not do all that? Instead of the elation that we expected, we just got a weak "Good job, guys", and they took their working code and incorporated it where it was supposed to go. We were finished with our tasks, so we left.

But not without planting some seeds. You see, we involved some of their other programmers in what we were doing. We didn't code with them, but explained how we were going about it. And we showed them unit tests (which they had heard of) and code coverage (which was new). The other developers clearly saw the benefit of what we had done. Sometimes, you can't dig a deep hole all at once because you have to move too much dirt. But a river can dig a deep hole a little at a time, eroding away grime and stone bit by bit. Even if we didn't dig a deep hole, maybe we created a rivulet.

Technology Snake Oil, Part 5: The Integration Myth

Integration is a tricky thing. When it works flawlessly, it is a huge productivity boon. But when you trust it and it lets you down, it is a huge time sink. Some examples: I used to work with someone who had a genuine fetish for tool integration. He wasted more time trying to get tools to work together than he did performing useful work. At the time, we were using JBuilder and Visual Source Safe (for my opinions on this piece of technology, check out VSS: Unsafe at Any Speed). The integration between these two tools was spotty at best. I never even tried – it was easy enough to keep the VSS explorer open alongside JBuilder and just bounce between them. But not Mr. Integration. He tried a variety of supposed integration plug-ins, each with serious shortcomings. Finally, exasperated, I told him that I had the perfect integration strategy. “I’ve fixed your JBuilder/VSS integration. First, open both of them. Second, I’ve installed a special hot-key in your copy of JBuilder, Alt-Tab. When you hit it, the VSS explorer pops up, and it works just like the explorer that VSS itself uses – it’s indistinguishable!”

Recently, another example popped up from an unexpected place. It turns out that there is a bug in my beloved IntelliJ and WebLogic, but only when you are doing distributed transactions and messaging. My colleague spent the better part of a day trying to resolve that little nasty bug.

The snake oil hiding in the shadows here is the promise of integration sold by tool vendors. IDE’s try to encompass more and more. Visual Studio has always been successful with this (especially if you use all Microsoft technologies) without achieving a truly great code experience. Visual Studio.NET 2003 is a third world country compared the scarily intuitive IntelliJ, which is the Rolls Royce of IDEs. VS.NET 2005 is better, but still not up to IntelliJ’s standard. Borland tried to take this to the extreme conclusion with its Software Delivery Optimization suite, which tried to bind together requirements gathering, version control, coding, deployment, and monitoring into a single environment. This vision yields an awesome productivity gain as long as the integration is flawless. However, integration at that level is never flawless, meaning that you spend as much time (and considerably more frustration) trying to figure out why something that should be working isn’t, only to find out that it actually is working but the integration is obscuring the results.

Some integration is good – that’s why we have integrated development environments. However, there is a fine line where vendors try to go too far and end up lessening productivity rather than enhancing it. The problem is that tool vendors are trying to create monolithic environments. I suspect that Visual Studio in all its incarnations may be the culprit here – everyone is trying to replicate that environment. Finding the fine line between integration vs. stand-alone tools is tough and going to get tougher, as vendors produce more and more immersive environments to get you to buy into their integration strategy.

Someone has done this exactly right. In the next Technology Snake Oil (Drowning in Tools), I’ll talk about that.

Technology Snake Oil, Part 4b: Prescriptive Approaches in the Large

I wrote recently about Prescriptive Approaches in the Small. More pervasive and dangerous, though, are Prescriptive Approaches in the Large. Everyone wants a repeatable pattern, a guaranteed way of doing things. Recently, at No Fluff, Just Stuff Columbus, OH, one of the questions for the expert panel was: “Our company is looking for a road map, a sure-fire prescription of what technologies, methodologies, and approaches we should take – can you provide this?” Uhhhh, no. Most of the panelists nearly crawled out of their skin to get in line to answer this question (I thought Prag Dave was going to have a heart attack – if you know him, you know this is anathema to him!). There is no prescription for software development done right in your company – every place is different. You cannot come out and say that this approach (whatever it is) is right in every circumstance.

Following this line of reasoning, I think that the PetStore was one of the worst evils inflicted on the young, innocent, fawnlike Java Enterprise Edition. It was supposed to be a catalog – “Look, here are all the things this shiny new technology can do” – and people took it as a prescription. Rather than sit back and determine for what EJB was suitable, developers dove right into using them in their projects because the prescription said so. EJB was (and is) suitable for only very specialized types of applications. If you were writing a real pet store application (even back when PetStore came out), you would be crazy to use EJB!

The pattern repeats over and over, with technology (“Everyone is using Struts, so we should be too”) to methodologies to products. You can bet that the SOA vendors are salivating over the prospect of creating the prescription that says “Everything must be decoupled, document centric messages – here, we have an expensive product that just happens to do this!” As much as it hurts, you have to make determinations for yourself on which technology (methodology, framework, tools, etc.) are best for the situation you are in right now. There is no prescription, no matter what ersatz technology doctors tell you.

Joel is Plain Wrong about XP

Joel Spolsky writes in a recent blog entry:

As I worked through the screens that would be needed to allow either party to initiate the process, I realized that Aardvark would be just as useful, and radically simpler, if the helper was required to start the whole process. Making this change in the spec took an hour or two. If we had made this change in code, it would have added weeks to the schedule. I can’t tell you how strongly I believe in Big Design Up Front, which the proponents of Extreme Programming consider anathema. I have consistently saved time and made better products by using BDUF and I’m proud to use it, no matter what the XP fanatics claim. They’re just wrong on this point and I can’t be any clearer than that.

Joel has an interesting perspective on this subject and has written quite eloquently on this and other topics (I highly recommend his books and blog). However, I think he’s just wrong about this. He has a skewed view on software development, having worked at product companies his whole career. Building products is a lot different from building business applications (which is where my experience lies). First, for products, the requirements and design are pretty much made up of a few people’s vision on what the product needs to do. The market place determines if they were right or not. Second, you generally have to do this in secret, especially if you have competitors who will cherry pick your design. Third, you are pretty much required to do a “big bang” deployment, when you ship to customers.

Most business applications aren’t like this. The requirements are very fluid (quite often, the business stakeholders don’t know what they want even well into the process). When done correctly, the process is collaborative across business units and developers, making for a stronger product. Last, you can roll out iterations (even if just to QA and/or acceptance testing) without the big bang.

Joel is using the straw man arguments against XP. Many misinformed naysayers against XP assume that we never think about what we’re doing up front. Of course we do. This same group assumes that there is no coherent architecture for complex XP projects. Just because we’re doing agile development doesn’t mean that we just sit down and hack out code with no planning or forethought. Believe me – I’ve had lots of experience building business software, and Big Design Up Front just plain doesn’t’ work, even when it is done well (I’m fighting it yet again even as we speak). XP delivers better quality software, in shorter times, closer to the ultimate target, which has a fighting chance of actually usefulness.

Taking this topic to the logical conclusion, I wonder what would happen if a product company did a full-on agile approach? Building the product based on custom feedback from the early iterations. You would need a patient customer base, willing to comment on the work in progress. I can’t help but believe that, just as in the business software world, you would end up with a better product. How many features could you eliminate in something like Word if it had been developed incrementally? So much useless cruft could be removed from commercial software if product teams listened to their customers while developing the software rather than the complaints after it is done.

Some companies are tiptoeing in this direction. Check out JetBrains Early Access Program for innovative tools like Fabrique and MPS (The Meta Programming System). I think these products will be extraordinary if they can get their target audience to give them feedback. Agile product development – what a concept!

My Blog, in Korean

To satisfy overwhelming demand (OK, there is no such demand), Trustin Lee is translating some of my blog entries into Korean and posting them here. To quote:

I'm an Apache Committer who is still on the way of studying English. I'm trying to level-up my English skill translating great blog entries from great authors.

I think that's more than a bit generous, but I'm gladly supporting this effort because, hey, it isn't costing me anything and it spreads my meaningless opinions all over the world. What an unbeatable combination. Seriously, I appreciate the hard work Trusten is doing for this, and I hope that my dribbly writing is worth translating.

Now, I’m really interested to see if he translates this blog entry, creating a kind of recursive blog referencing stack overflow.

Technology Snake Oil

Technology Snake Oil, Part 4a: Prescriptive Approaches in the Small

The world is a much simpler place if you can just coast through it, without having to expend precious mental energy. In fact, it seems widely encouraged in many organizations: if we can prevent our employees from thinking too much, they will stay out of trouble. Unfortunately, to build insanely great software requires thought on many different levels, which doesn’t go well with prescriptive approaches.

You see all sorts of prescriptive approaches in software development, both large and small (this entry covers small, the next Snake Oil piece covers large). Small prescriptive manna floats down in many different guises, mostly in a good context like design patterns and best practices. Many developers take these nuggets out of context and start believing that this Prescription for building all types of things represents the One True Way (without bothering to apply additional thought). Don’t think that I’m denigrating design patterns – they are clearly a good thing, as a way to catalog common patterns. The problem comes in when they are used without thinking.

Here is a classic example that appears all over the Java code I see every day. Java doesn’t really have a struct-type thing, which is just a holder for data. In Java, everything is a class, which is OK. Nevertheless, I see developers who just need to pass composite data around creating a Java class with accessors and mutators that won’t every be used for encapsulation. It has become a common prescriptive practice in Java to create the class, create private member variables, and generate getters and setters for them without remembering why you create accessors and mutators. If this class is never going to have any behavior attached to it, if it is a simple container for information, you don’t need the accessor/mutator pair or the private scoping – make them public. If you really need to add behavior later and/or encapsulation, you can refactor the class into a real class, not just a composite data type.

The real sermon here is to think about why you create things, don’t just blindly create artifacts. Yes, encapsulation is generally a good thing, but isn’t required in every single thing that looks like a class – sometimes it’s really just a struct. You must always be diligent about why you are creating what you create.

Fighting for Your Life

Fighting for Your Life

I was almost voted off the island recently.

Because of some scheduling issues involving me rolling off a project, another one starting, and the slated person still a week away of rolling of his project, I was in the unfortunate position of going to a client's site for just 1 week, at the start of a new project. The first day, I arrived late because of travel details, and we spent the day going over the technical details of what it was we were going to be doing. All fine and good. In fact, one of the overviewers was kind enough to identify something he thought I could do in just a week, something orthogonal to the rest of the work.

The next day I get to work and the client supervisor asked me into his office. "Neal, we just don't think the one week thing is going to do us any good, we like you and all, but you should just go ahead and leave." He was trying to kick me off the island! The last thing he said was "Unless you can think of a good reason we should keep you..." That was my chance. I said "Well, of course, it is your decision, but I have found something I can get done in a week", and went on to tell him about the one-off sub-project we had identified the day before in technical detail. He listened to my case, then called the technical lead into his office and had me repeat it. The tech lead said "Well, we do need that piece, and I think he can get it done in a week". The supervisor turned to me and said "Alright, get back in your office and get to work!". Saved!

And it turns out that I was able to get the work done in a week, and the client requested that I stay on instead of the guy originally slated for the gig. How is that for defending your life?

An Actual, Real, Everyday Use for XSLT

I have played with XSL and XSLT for a while, and even toyed with the idea of basing my whole website on an XML+stylesheet vision. Because XSLT can get complicated, that idea died before its time. However, I recently found a really good use of XSL on a development project on which I was working. This project included the ubiquitous Log4J library, including a largish logging.xml document. My job was to make sure this document is still relevant. The problem with trying to analyze such a document lies in the way that items reference each other. Instead of pulling my hair out bouncing around in the XML file, I created an XSLT stylesheet to format the whole thing into HTML, with links from loggers to appenders and appenders to loggers. The end result looks like this (broken into 2 pieces for space savings):

The stylesheet that generates this info from the XML document is not long or complex:

<?xml version="1.0"?>
<!--  XSL Stylesheet to make it easier to determine the relationshps between -->
<!-- the loggers and appenders used in the application.                      -->
<!-- This stylesheet is automatically linked into the logger.xml document.   -->
<!-- If you are using any reasonable browser, this stylesheet should be      -->
<!-- automatically applied to this file.                                     -->


<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
   xmlns:log4j='http://jakarta.apache.org/log4j/'>

   <xsl:template match="/">
         <html>
           <head>
             <title>
                 Loggers
             </title>
           </head>
           <body>

             <xsl:apply-templates select="log4j:configuration" />
           </body>
         </html>
   </xsl:template>

   <xsl:template match="log4j:configuration">
       <h1>Project Loggers</h1>
       <h2>Loggers</h2>
       <table border="1">
           <tr>
               <th>Logger</th>
               <th>Priority</th>
               <th>Appender(s)</th>
           </tr>

           <xsl:for-each select="category">
               <xsl:sort select="@name" />
               <tr>
                   <td><xsl:value-of select="@name" /></td>
                   <td><xsl:for-each select="priority" >
                       <xsl:value-of select="@value" />
                   </xsl:for-each></td>

                   <xsl:for-each select="appender-ref">
                       <td><xsl:value-of select="@ref" /></td>

                   </xsl:for-each>
               </tr>
           </xsl:for-each>
       </table>
       <h2>Appender to Logger Relationship</h2>
           <ul>
           <xsl:for-each select="appender">
               <xsl:sort select="@name" />
               <tr>
                   <li><xsl:value-of select="@name" /></li>
                   <xsl:for-each select="param">
                       <xsl:if test="@name = 'File'">
                           [ <xsl:value-of select="substring-after(@value, '${logging.home}/')" /> ]
                       </xsl:if>

                   </xsl:for-each>

                   <ul>
                   <xsl:call-template name="show-categories">
                       <xsl:with-param name="appenderName"><xsl:value-of select="@name" /></xsl:with-param>
                   </xsl:call-template>
                   </ul>
               </tr>
           </xsl:for-each>
           </ul>
   </xsl:template>

   <xsl:template name="show-categories">
       <xsl:param name="appenderName" />
       <xsl:for-each select="//category">
           <xsl:for-each select="appender-ref">
               <xsl:if test="@ref=$appenderName">
                   <li>
                   <xsl:value-of select="../@name" />
                   </li>
               </xsl:if>
           </xsl:for-each>
       </xsl:for-each>

   </xsl:template>

</xsl:stylesheet>

I also added a reference to this stylesheet in the logger.xml document itself:

<?xml-stylesheet href="ShowLoggerRelationships.xsl" type="text/xsl" ?>

The beauty of this is that any reasonable modern browser will automagically apply this stylesheet anytime you double-click on the XML file. This is the ultimate form of documentation -- generated directly from the source in real time. Any changes to the relationships in the logger.xml document are instantly shown when you look at the file. If you still want to see the raw XML, you right-click and "Open With..." instead. This makes the browser view of the XML document useful without hiding or obscuring the functional elements. And is a good example of the DRY principle (Don't Repeat Yourself) from the Pragmatic Programming applied to documentation.

Technology Snake Oil, Part 3: MDA

Model Driven Architecture (or "Cartoons to Code") is the ultimate culmination of the vision of UML. Let's draw the software with diagrams, including decorations to represent logic, then click a button to generate the code. A laudable goal, but far short of reality.

The only real success stories with MDA are for very limited problem domains, like embedded software. Here's why: consider the cash register at McDonalds. The number of items is very limited, and the number of things you can do with them are very limited. This is well suited to a pictorial (in fact, an ideogrammatic) representation. That's because the concepts on display are retricted to morphemes, which in linguistic terms represents the smallest language unit that carries a semantic interpretation. Thus, it's easy to create a McDonalds cash register using just pictures and make it a quite effective solution to the food ordering problem.

For complex problems, you can't break down the problem to simple morphemes -- imagine the semantic complexities of a concept like "loan" or "payment". For a problem domain this rich, you need the richer representation of a full-blown language (which the Egyptians also discovered -- they were the last MDA success story, and they eventually abandoned it!). Trying to cram rich semantics into a pictorial representation so that you can create software that abstracts those semantics is absurd.

In fact, trying to cram the entire world into a strict tree-shaped hierarchy isn't much better (see Programming, Object-oriented). At least OOP has more flexible semantics, and we keep forcing solutions to this hierarchy problem with inventions like aspects, which cut across the tree-shaped world we've created (because stuff in the real world cuts across trees). So, where do we go from here?

Let's solve it the same way mankind (including the Egyptians) did -- create language! Language is ultimately the only way to handle the rich semantics of the world around us because that's what the people stating the problem domain use to state the problem. Which is why I'm so geeked about Language Oriented Programming -- building domain specific languages that are as close as possible to the problem we're trying to solve. LOP done right encapsulates the low-level details of a language like Java, allowing developers and others to work at a higher level of abstraction. We've been mired in curly-brace languages too long -- we need to upgrade our abstraction.

MDA is the wrong direction for abstraction upgrade. It tries to further restrict our expressiveness, believing that the entire world can be graphed and you can create real software this way. Imagine for a moment the MDA diagrams (including Object Constraint Logic) for your average J2EE enterprise application. You can build it in code, with the crude semantics of curly-brace Java in 1/3 the time you could model it to the point where you can click the "And Then a Miracle Occurs" button to generate the application. MDA might work for simple, semantically unique domains, but no where near the real world.

Happpy with What You Have to be Happy With

Recently, I did a 3-hour tutorial workshop at the SOA conference in Singapore entitled "Agile Development and SOA", where I talked about aspects of Agile development as it applies to SOA type projects. Mostly, it was an excuse to talk about Agility in a foreign land.

I asked the attendees what type of methodology they were currently using. The three replies I got were 1. None (what's a methodology?), 2. Waterfall (although she didn't know that name for it), and 3. SDLC (the Software Development Lifecycle (another variant of Waterfall, check out this interesting Wikipedia image of SDLC )). So, basically, everyone is using some form of Waterfall. What's interesting is that I asked to most vocal of the crowd (meaning that she would actually answer without me jumping up and down on the table in front of her -- she was the SDLC victim) if her company was happy with that methodology. She quickly replied "Oh, yes, it works very well for us". Needless to say, I was surprised. So I asked her a few follow-up questions:

• Do you ever miss deadlines? Oh, yes


• Are your user's happy with what you produce? Well, no


• Are your developers frustrated by vague requirements, which leads to user dissatisfaction? Yes


• Do you ever accidentally break code in your "big bang" deployments, which takes a long time to fix? Yes

I asked a few more, but you get the gist. She thought they were very happy with SDLC because they didn't know that something better existed. Perfect crowd for an agility talk, right? It was an uphill battle. No one could believe that this crazy idea could ever work -- waste all that time writing tests? Include the users as part of the development team? Pair Program?!? That cuts your productivity in half! Sigh. I waged a good battle, but don't know how well the war is going. At the end of the talk, I had either convinced them to tip-toe towards Agility (because these were developers -- can you imagine what their boss will say?) or they were just tired of me talking about it and said anything to get me to shut up.

I've blogged about this phenomenon before -- why is it that developers and managers believe that all forward progress in methodology stopped in the mid-70's and all language development froze about the time Objects appeared? Is it just a non-technical person's fatigue at trying to keep up, so an arbitrary milestone was created beyond which "I won't think about it anymore?" Are developers so overwhelmed that they just shut down too, and keep working on what's in front of them even though better things abound?

Technology Snake Oil, Part 2: RAD

Back in the mid-nineties, Rapid Application Development was the answer to all developer woes. RAD tools came out in droves and everyone touted the huge productivity gains made available by drag and drop user interface design and component based development. RAD environments made for great demo-ware -- you could stand in front of a group of Pointy Haired Boss types and build a working application right before their eyes while you were talking about it (I know because I did some of this -- and I still have trouble sleeping sometimes). Of course, the message was clear: buy This Tool, and you too can have developers that can point and click their way to huge productivity gains. And bought it they did.

Then a funny thing happened to the developers on the way to this astounding productivity. They learned the tools and dutifully dragged and dropped their way to building applications...and it was good. Until, that is, they figured out that building the application is the short, easy part -- now they have to maintain the monstrosity they've created. RAD development doesn't scale well, and when you build large and very large applications with it, maintaining those applications is a nightmare of searching for the scattered crumbs of code scattered hither and yon. Working for a while in this environment is what made the Design Patterns book really hit home -- the kind of code we were creating was entirely anti-pattern, and more about "how fast can it be created"? The maintenance nightmares led me, as CTO of a consulting and development company, to figure out new ways to use the RAD tools we had to use (because of our clients) that did follow good design principles. What we found is that the tools fight you when you try to do the right thing. RAD Development tools have a path of least resistance, which makes things very easy while you stay on the path (and use their wizards and other support structures). Stray from the path, and you find obstacles at every turn. But, it's worth fighting through them to get to a place where you can create maintainable code -- and you find that you are no longer doing Rapid Application Development, just Application Development.

RAD is now rightfully deprecated in most serious software development efforts. Even serious developers using RADish tools like Visual Studio are avoiding the RAD pitfalls. Some developers still fight with RAD tools, and think that maintenance is just always extremely difficult in big projects. I'm still shocked by the developers who have still never heard of Design Patterns and who still work on Sisyphean development chores.

Which is why I think the RAD aspect of Ruby on Rails misses the point of what makes Rails interesting. RAD doesn't work for large, complex projects, no matter what technology. The interesting part of Ruby on Rails is the effective use of Ruby to build a domain specific language on top of the underlying Ruby language that makes it easier to do web applications with persistence. It is more advanced than the RAD tools of old because you can drop to the underlying Ruby language and get underneath the framework when you must -- and that's a powerful idea. But, building complex applications in it still requires real thought. Maybe the breathless amazement of the Rails community reflects the absence of effective RAD tools in this space before. Don't get me wrong -- I think that Rails is cool, just not for the reasons touted by so many Rails-flavored Kool-aid carriers.

Technology Snake Oil, Part 1: UML

I was told by someone recently at ThoughtWorks that I was so old that I "added to the company's diversity". Well, I've decided to take this sage old-timers wisdom and put it to some good use. In the development world, we keep getting sold Snake Oil in one guise or another, and I'm going to start pointing out some snake oil. Starting with the most venerated of crappy technology: UML.

Some of us craggy old-timers remember the time before UML, when we had Booch, Rumbaugh, and several other competing diagramming notations. The Holy Grail was going to be the Unified Modeling Language, which would solve all our problems and finally give us software developers a universal language, for everything from use cases through structure to deployment. And that's what we got. A classic example of "be careful what you ask for". UML feels like a compromise between a group of people, and it's so general that it's almost useless. The standards bodies keep bolting on new stuff to try to get it to the point of basic usefulness, but it just makes it worse. Does anyone really use Object Constraint Language to decorate their diagrams so that they can generate code that Just Works? I have never seen a case where you couldn't get it done faster by writing the code, then reverse engineering it into one of the expensive tools. Of course, it's good to design things before you code them, but a white board is much better. Combine the nature of UML with the fact that you can't ever show its diagrams to business analysts or end users because it is too obtuse, and I end up doing most of my architecture and design work in Visio (and I'm no fan of it either). It creates diagrams you can show everyone that capture the essence of what you are doing just before you implement it.

I'm not just a tourist in the UML and RUP world -- the company I worked for in the dim and distant past used RUP, and we foisted Heavy Duty Object-oriented Analysis and Design on clients, and used it ourselves. Why do you think I'm so militantly agile now? It just flat doesn't work for 90% of the types of projects that real developers write. And it's Bastard Child Model Driven Architecture is featured in an upcoming Snake Oil entry...

SOA (Service Oriented Aberration)

While talking to some of the other speakers at the SOA conference in Singapore, it occurred to me that SOA has an interesting side effect that has nothing to do with technology. To make SOA really work, a business has to strictly define what it is they do, to the point that a developer can create services to represent it. That's the rub -- most businesses don't understand their own business to that level of detail. Time and time again, I've noticed that the analysts and developers on a project actually define a client's business processes to a much finer level of detail than the business ever has. To write code for something, you have to really understand it. Some of the clients for which I've worked even commented on the fact that, because of the development process, they understand their business better.

Of course, the opposite is also true (and, alas, more likely) -- companies with poorly defined business processes will try to adapt SOA and fail miserably, creating a case study of SOA failure, when the technology had nothing to do with it. Worse yet, tool vendors of expensive development tools like Enterprise Service Buses will sell their tools to unsuspecting clients as a way to "fix" their business process. Putting a dress on a pig doesn't make the pig any prettier, and it just annoys the pig. SOA, like so many other similar technological breakthroughs (see Programming, Object-oriented and Development, Component-based), won't solve the dysfunction of dysfunctional businesses.

Conspicuously Caucasian

In both trips this year to Singapore, I had the eerie sense of sticking out like a sore thumb -- I was conspicuously Not From Around Here. In most of the other places I've traveled, I could, with a little effort, blend in with the natives. In Europe or Australia, if I dress like the natives, I look like a native. Of course, as soon as I open my mouth, I'm a goner, but I can quietly blend. Not in Singapore! I am obviously not from the neighborhood.

I don't mind being different (I certainly was in high school, and I was from the neighborhood). However, in Singapore, I'm a target for all sorts of commerce attempts. I can't walk past a store front, even in the tony parts of town, without someone trying to sell me something. I'm not only conspicuously Caucasian, I'm American, and therefore both gullible and rich! I can't get the locals price around here for anything. Just another example of how travel broadens you, and why I like travel so much because it provides perspective.

Battle of the Desktop Search Giants

Back in the dim and distant past, I blogged about desktop search in general and Google Desktop Search specifically. In that post, I lamented the search tools of yore that went away for one reason or another. A commenter kindly pointed out that my all time favorite (Magellan) now lives on as X1..

Lo and behold, Yahoo has entered the fray with their own desktop search tool, based on X1. In fact, it is a free version of X1 that doesn't include the enterprise features. It has a slick Windows-based UI and does a lot of cool tricks. So, I displaced Google and installed Yahoo Desktop Search (PCMagazine also gave it their editor's choice).

Well, after using it for a while, I'm un-installing it and going back to Google. Both are in beta, but Google is much more stable IMO. Yahoo was constantly doing odd things, and seemed to be belly up when I needed it the most. While Google has a sparser UI and doesn't handle as many file types, it is (at least for now, for me) more stable. And, for infrastructure software like desktop search, stable wins the day for me. I'm writing this while un-installing Yahoo and re-installing Google. Maybe once Yahoo turns 1.0, I'll re-evaluate. But for now, I'm switching back to Google.

Passion Part 2

As if on cue from my last post, an aphorism from Robert Fripp:

Life is too short to spend time on activities where our passion is not engaged.

I know a lot of people who fundamentally don't enjoy their work. While earning a living is a laudable goal, I think those of us who do really enjoy their work are lucky. Most of my friends who don't enjoy their work envy the passion with which I attack my work. Some of them accuse me of not really working, just enjoying my hobby: that's not true -- it's still work. For my hobby, I would create an entirely different set of software. It still tires me out at the end of the day. But, for the most part, I take joy in the job that I do. I'm lucky.