Tuesday, December 05, 2006

Polyglot Programming

My first professional work as a software developer was writing Clipper code. Clipper was a compiler for dBASE code with object-oriented extensions. This was in the days of DOS, and the entire application was written in a single language. We didn't even use SQL. Instead, the data storage was shared DBF files on a new concept, the LAN (I remember reading a PC-Magazine of that era declaring that the current year was the "Year of the LAN").

We are entering a new era of software development. For most of our (short) history, we've primarily written code in a single language. Of course, there are exceptions: most applications now are written with both a general purpose language and SQL. Now, increasingly, we're expanding our horizons. More and more, applications are written with Ajax frameworks (i.e., JavaScript). If you consider the embedded languages we use, it's even broader: XML is used as an embedded configuration language widely in both the Java and .NET worlds.

But I'm beginning to see a time where even the core language (the one that gets translated to byte code) will cease its monoculture. Pretty much any computer you buy has multiple processors in it, so we're going to have to get better writing threading code. Yet, as anyone who has read Java Concurrency in Practice by Brian Goetz (an exceptional book, by the way), writing good multi-threading code is hard. Very hard. So why bother? Why not use a language that handles multiple threads more gracefully? Like a functional language? Functional languages eliminate side effects on variables, making it easier to write thread-safe code. Haskell is such a language, and implementations exist for both Java (Jaskell) and .NET (Haskell.net). Need a nice web-based user interface? Why not use Ruby on Rails via JRuby (which now support RoR).

Applications of the future will take advantage of the polyglot nature of the language world. We have 2 primary platforms for "enterprise" development: .NET and Java. There are now lots of languages that target those platforms. We should embrace this idea. While it will make some chores more difficult (like debugging), it makes others trivially easy (or at least easier). It's all about choosing the right tool for the job and leveraging it correctly. Pervasive testing helps the debugging problem (adamant test-driven development folks spend much less time in the debugger). SQL, Ajax, and XML are just the beginning. Increasingly, as I've written before, we're going to start adding domain specific languages. The times of writing an application in a single general purpose language is over. Polyglot programming is a subject I'm going to speak about a lot next year. Stay tuned...


Anonymous said...

The biggest challenge IMHO would be to get younger programmers to become polyglots. Too many developers I know have only worked with one language (usually Java) so far - there's no practical reason to learn more other than the love of programming, and surprisingly few share that trait.

Simon said...

To produce polyglots you need environments where polyglot-ism is encouraged or required. At work they don't seem to need polyglots, they prefer to be able to call on "the [INSERT PROGRAMMING/ SCRIPTING LANGUAGE] guy"

chubbsondubs said...

I think what is needed is more of a blue print of what a productive polygot environ looks like. What problems are other languages solving better, and what is my main stay language handling well. I think finding useful problems where say Java or .Net don't really work, and showing how a new language can help is helpful. The first I can think of is JSP. Ditch it in favor of a scripting language whether it's velocity, ruby, or javascript on server. There is just so much more you can do with a general purpose template system over JSP. The other I think is automated testing. I think if we can illustrate problems like those and demonstrate productive ways to integrate them into your projects you'd have more polygots. Game devleopment is always a great polygot environment. Most game logic is implemented in a scripting language that directs a super fast 3D engine underneath. What if we built rich UIs like that?

Unknown said...

Some corners of the programming world have always been polyglot. Ever seen a shell script with bits of sed and awk mixed in? How about a shell script that wraps a C program? And it would be silly to implement a program that parses a grammar without using lex and yacc. DSLs have been around since the beginning of time - well computer time, anyway.

Unknown said...

The best programmers are polyglots in my experienec. Even better, they may create their own task specific languages as needed. Interpreters are not hard to write, a simple case/switch statement or state machine is all that's needed for simple languages.

I salute the Polyglot idea!

More in my blog:




SharpProgrammer.com said...

Really glad I found your blog.Please let me know if I could add a link to your site on my blog.

Advanced Programming Tips Tricks

JP said...

Agree 100%


a) We know current limitation of our mindset,creative power etc...
b) We have understood enough of language A to jump to language B to come to some comparision and very important is what is that we learned?Always learning a tool/language may not be sufficient.We need to think how might be the implementation of the code/compiler etc would have been done???
....... etc

I think If we won't follow any of these it will always benefit to company but may not benefit in the same ratio to us.


Paul W. Homer said...

This sort of discussion always reminds me of the intro to the first Lord of the Rings film where Galadriel says "and some things that should not have been forgotten were lost ..."

In a typical UNIX/C project last decade, or was it the one before, we always used multiple languages: C, SQL, sh, csh, awk, sed, tcl/expect, vi|emacs(lisp), make and occasionally perl. That was normal. The 'single language' programmers were generally PC-based or people dependent on IDEs for their automation or pure interface programmers. If you want to properly automate your packaging, you have to go outside of an application language for those capabilities.

The ugly part about tying two or more application languages together is that you tend to get more technical problems where they overlap. The early Macs were good examples as the OS was written in Pascal and most of the apps were in C, it was awful. Consistency in your core code is worth some small coding or performance hits.


Mike Schinkel said...

Great post Neil! As a former Clipper programmer myself (and someone who shares a former company/employer with you!) I've often thought the same thing; glad to see you have given it a name.

One thing I'd like to add regarding your mention of the "two" leading platforms of Java & .NET. Yes, those are the two enterprise platforms of the moment but by describing them in that manner I think you give the implicit impression that everything of interest is on those two platforms when in reality upcoming open-source dynamic languages that exist outside those two platforms are rapidly evolving. Given their very nature I expect they will do a surprising end-run on Java & .NET over the next decade. I speak of, to name a few, PHP and its various frameworks and CMSes; Ruby-on-Rails and other Ruby frameworks; and Python with Django and TurboGears etc.

Anyway, like I said, great post; it's nice to read references to "the good old days" from a fellow former Clipperista.

Mihai Fonoage said...

I agree. And the best way to achieve this is by working at a higher abstraction layer. Instead of concentrating on the programming language, why not work in a modeling language and use a tool that can generate whatever code you like/need.

Neal Ford said...

Mihai -

I agree with the part about higher abstractions, but I don't think modelling tools are the way to achieve it. Modelling tools are too abstract, which I think accounts for the broad movement away from things like MDA, towards more text-based DSL techniques.

Mihai Fonoage said...


Most of the tools adhere to the modeling language specification, but I do agree that there is always room for improvements. Regarding polyglot programming, I honestly cannot think of a better way of dealing with it; if your work environment does not offer variety, then between work and family, there is little time left for diving into a new language, especially when you need to stay up-to-date with the one that you (mostly) work with.