Emergent Design & Evolutionary Architecture at DeveloperWorks

For the last few months, I've been toiling away on an article series for IBM DeveloperWorks, and it's rolling out today! From the abstract for the series opener:

This series aims to provide a fresh perspective on the often-discussed but elusive concepts of software architecture and design. Through concrete examples, Neal Ford gives you a solid grounding in the agile practices of evolutionary architecture and emergent design. By deferring important architectural and design decisions until the last responsible moment, you can prevent unnecessary complexity from undermining your software projects.

The first two articles in the series appeared today:

• Evolutionary architecture and emergent design: Investigating architecture and design


• Test-driven Design, Part 1

I plan to use this series to start a conversation about something that we all do everyday that we can't really describe well, even to other technical people (much less our grand parents). I don't supposed to know the answers (I'm not even sure I know all the questions), but at some point we have to talk about it. In the first installment, I provide some working definitions and some overarching concerns. Let me know what you think about it.

Speaking at the IT Architect Regional Conference in Atlanta

At the end of February (the 25th - 27th), I'll be making a rare Atlanta conference appearance at the IT Architect Regional Conference, hosted by the International Association of Software Architects (IASA). This is the first in a series of regional conferences focused on an important but generally neglected segment of the developer population, software architects. What that title actually means is some matter of debate (hey, maybe this conference will help define that term), but the topics covered certainly trod some important ground. I'm doing a short version of my Smithying in the 21st Century keynote (overviewed here). My ThoughtWorks colleague Steven "Doc" List will also be there, imported from the west coast, convening an open space called Beyond Fight or Flight: Meetings Don’t Have to be Gladiatorial Combat, which sounds quite interesting. It's still not too late to sign up for it; hope to see you there.

Tools & Anti-Behavior (SOA & the Tarpit of Irrelevancy)

This is the third in a series of blog posts where I discuss what I see wrong with SOA (Service Oriented Architecture) in the way that it's being sold by vendors. For those who are behind, you can read the first installment and second installments.

While rank and file developers go to conferences to soak in deep technical content, their peripherally technical managers (the ones who wrote some rockin' good Cobol code back in the day, but now they make decisions about modern enterprise architecture) go to different conferences in Palm Springs. At those conferences, they have a 2-hour morning session, run by a big tool vendor, then play golf for the balance of the afternoon. And what the vendors show them is poison.

Mostly what they see these days are tools that support SOA and ESBs. And in particular, the favorite demo-ware application is their BPEL (Business Process Execution Language) designer. This designer allows you to wire together services by drawing lines between boxes. The lines can include transformations and other sexiness. And it demos great. "Look, just draw a couple of lines here and here, click on the Run button and voila! Instant SOA".

Then, the manager brings it back home and notifies the developers that this is the new tool of choice. When developers start using it, they realize the awful truth: they've been sold a hairball generator. Tools like this work great on really small scales, when it's easy to see all the connections between things. But, as things get complicated, they start suffering from the hairball effect: all the lines start running together, and you can't create a diagram that makes sense to anyone anymore. Perhaps maybe you can fight through this, by creating workflows in chunks, and zooming in and out.

Then reality arrives. Because you create workflows using these tools, you are coding, in the worst possible language (a graphical representation). Thus, you are defining behavior, just like you do when you write source code. But the behavior you define lacks all the benefits you get from writing it in code.

• reuse: you can't really reuse portions of your workflow because their is no method or subroutine functionality (you might get lucky with a sub-workflow). Mostly, people achieve "reuse" by copy and pasting, which you never do in code.

• refactoring: no refactoring, making it harder to identify common workflow chunks for reuse. When you don't have refactoring, you don't watch for opportunities for refactoring as much.

• limited programmability: you don't get if statements and for loops, you get whatever this particular BPEL designer supports. You get flow-chartly looking stand-ins for real decision statements, but they are much more brittle than the facilities offered in modern languages.

• testing: you can't write unit, functional, or integration tests for these workflows. The only real testing option is user acceptance, meaning that the entire universe must be up and running. If you have no unit testing, you also don't have mock objects or other testing techniques common in code.

• hard to diff: lets say you fought the beast and get a non-trivial workflow up and running, and everything is great. In six months, you change it in non-trivial ways, and all is good. Then it comes time to see what's different. BPEL tools don't have diff facilities, so you can either visually diff the diagrams (yuck) or diff 2 10,000 line XML documents (double yuck). BPEL relies on either heavy-weight diagramming tools or raw XML, and nothing in between.

Tools like this fall into the category one of my colleagues identified as doodleware tools. They let you create pretty pictures but collapse under scale. And they don't support all the common facilities offered by good old fashioned code. Is it really worth giving up robust reuse, refactoring, testing, programmability, and versioning/diffing just to see the pretty picture? Ironically, it's pretty easy to generate a similar picture from code, using tools like GraphViz.

I am a strong believer in the mantra

Keep behavior in code.

We have great tools for code (including ways to generate doodles) -- why would you want to abandon what works for something new and shiny? Except, of course, that code won't take you out for a golf outing in Scotland if you choose it.

On the Lam from the Furniture Police at Agile Atlanta

I have two keynotes this year that I'm presenting at one conference or another. The first out of the gate was my talk entitled On the Lam from the Furniture Police at the Code Freeze conference. This talk has multiple parent ideas. The first derives from the book The Productive Programmer. It turns out that when you write a book about something, you get really immersed in the subject matter. That's obvious. What's less obvious is the inability to turn that overwhelming interest off after the book is done. That can either be good or bad. Once I had finished Art of Java Web Development, I pretty much never wanted to see another web framework! But the Productive Programmer was different, I guess because it's such a broad and unsolved problem. For whatever reason, I'm still soaking up Productive Programmer stuff.

The other parent of this talk involves the intersection of agile software development and productivity. While there's a little of that in The Productive Programmer, I wanted to keep the focus mostly on developer productivity in the book. But that's where On The Lam from the Furniture Police fits. This talk discusses how to be productive in corporate environments, including broader topics like agility. Here's the abstract:

When you were hired by your current employer, you may think it's because of your winning personality, your dazzling smile, or your encyclopedic knowledge of [insert technology here]. But it's not. You were hired for your ability to sit and concentrate for long periods of time to solve problems, then placed in an environment where it's utterly impossible to do that! Who decides that, despite overwhelming evidence that it's bad for productivity and people hate it, that you must sit in a cubicle? The furniture police! This keynote describes the frustrations of modern knowledge workers in their quest to actually get some work done, and solutions for how to gird yourself against all those distractions. I talk about environments, coding, acceleration, automation, and avoiding repetition as ways to defeat the mid-guided attempts to sap your ability to produce good work. And I give you ways to go on the lam from the furniture police and ammunition to fight back!

Every time I make changes to this talk, it's to enhance it's agile focus. So it's perhaps not surprising that I'll be giving a preview of this keynote at Agile Atlanta on Tuesday evening, February 3rd. If you're in Atlanta and interested in how productivity and agility intersect, stop by. And if you can't make it there, the next scheduled appearance is in Las Vegas, as the opening keynote of The ServerSide Symposium.