OOPSLA is the ACM conference that most broadly and widely addresses the sorts of questions I tried to explore in DREAMING IN CODE. I found the two events that I’ve attended, in 2004 and 2006, both highly rewarding. I couldn’t make the 2007 edition in Montreal, but I was delighted to find this page of podcast recordings of many of the conference’s highlights.

I’ve only had time to dip my toes in this stuff so far; there are talks by David Parnas, Fred Brooks, John McCarthy, Patti Maes, Guy Steele, Richard Gabriel, Gregor Kiczales and many others.

This is the thirteenth edition of Code Reads, a series of discussions of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s the full Code Reads archive.

This month’s paper, Daniel Berry’s “The Inevitable Pain of Software Development, Including of Extreme Programming, Caused by Requirements Volatility,” is a sort of update and latter-day restatement of Frederick Brooks’s classic “No Silver Bullet” argument — that the traits of software development work that make it difficult are inherent in the enterprise and extremely unlikely to be vanquished by some breakthrough innovation.

Berry is, as he admits, not the first to locate the source of software’s essential difficulty in “requirements volatility” — unpredictable fluctuations in the list of things that the software being built is expected to be able to do (including variations in user behavior scenarios, data types and all the other factors that a working piece of software must take into account). Read any development manual, listen in on any software team’s gripe session and you will hear curses directed at “changing requirements.”

Every new approach to improving the software development process includes a proposed method for taming this beast. These methods all fail, Berry maintains, leaving software development just as much of a “painful” exercise as it was before their application.

In each case, Berry locates this failure in some aspect of or practice dictated by a particular method that programmers find to be too much of a pain to actually perform.

Every time a new method that is intended to be a silver bullet is introduced, it does make many part of the accidents easier. However, as soon as the method needs to deal with the essence or something affecting or affected by the essence, suddenly one part of the method becomes painful, distasteful, and difficult, so much so that this part of the method gets postponed, avoided and skipped….

Each method, if followed religiously, works… However, each method has a catch, a fatal flaw, at least one step that is a real pain to do, that people put off. People put off this painful step in their haste to get the software done and shipped out or to do more interesting things, like write more new code.

So that, for instance, the method of “requirements engineering” (exhaustively “anticipate all possible requirements and contingencies” before coding) offers many benefits, but “people seem to find haggling over requirements a royal pain.” Also, it demands that “people discover requirements by clairvoyance rather than by prototyping.”

Similarly, Extreme Programming (XP) depends on programmers writing test cases first. That’s a step that in itself seems to be painful for many developers. When requirements change, XP calls for frequent refactoring of existing code. “Refactoring itself is painful,” Berry notes. “Furthermore, it may mean throwing out perfectly good code whose only fault is that it no longer matches the architecture, something that is very painful to the authors of the code that is changed. Consequently, in the rush to get the next release out on time or early, refactoring is postponed and postponed, frequently to the point that it gets harder and harder.”

Berry goes right down the list and confirms that the pain he diagnoses is a condition universal in the field.

The situation with software engineering methods is not unlike that stubborn chest of drawers in the old slapstick movies; a shlimazel pushes in one drawer and out pops another one, usually right smack dab on the poor shlimazel’s knees or shins. If you find a new method that eliminates an old method’s pain, the new method will be found to have its own source of pain.

Berry’s paper concludes with an alternative version of the principle that in Dreaming in Code I dubbed, tongue-in-cheek, Rosenberg’s Law:

To the extent that we can know a domain so well that production of software for it becomes almost rote, as for compiler production these days, we can go the engineering route for that domain, to make building software for it as systematic as building a bridge or a building. However, for any new problem, where the excitement of innovation is, there is no hope of avoiding relentless change as we learn about the domain, the need for artistry, and the pain.

Berry writes about the creation of software as much from the vantage of psychology as from that of engineering, and that gives his observations a dimension of bracing realism. In “The Inevitable Pain of Software Development” I found a willingness to examine the actual behavior of working programmers that’s rare in the software-methodology literature.

Too many authors are all too eager to pronounce what developers should do without considering the odds that any particular developer actually will do these things. Berry is a realist, and he keeps asking us to consider the cascade of consequences that flows from each method’s weak spots.

His case against “pain” seems not to be a naive attitude of trying to take a process that’s fundamentally difficult and somehow conjure the hardness right out of it. Instead, he asks us to note carefully the location of the “pain points” in any particular approach to software creation — because, given human nature, these are its most likely points of failure.
[tags]code reads, software development, software methodologies, daniel berry[/tags]

Joel Spolsky’s latest essay, “Strategy Letter VI,” offers a smart analogy between the desktop software wars of the 1980s — when companies like Lotus bet on producing code that could run on the slow, small-memory machines of the present, only to lose as PCs got faster, quick — and the Web-based software wars of today.

I think the following passage about Web-app development today could even be read as a (partial, qualified) endorsement of Big Ball of Mud:

The developers who put a lot of effort into optimizing things and making them tight and fast will wake up to discover that effort was, more or less, wasted, or, at the very least, you could say that it “conferred no long term competitive advantage,” if you’re the kind of person who talks like an economist.

The developers who ignored performance and blasted ahead adding cool features to their applications will, in the long run, have better applications.

[tags]joel spolsky, web development[/tags]

This is the twelfth edition of Code Reads, a series of discussions of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s the full Code Reads archive.

“Big Ball of Mud,” a 1999 paper by Brian Foote and Joseph Yoder (pdf), sets out to anatomize what it calls “the enduring popularity” of the pattern of software construction named in its title, “this most frequently deployed of software architectures,” “the de-facto standard software architecture,” “the architecture that actually predominates in practice”: a “haphazardly structured, sprawling, sloppy, duct-tape and baling wire, spaghetti-code jungle.”

This is dire stuff, and when I first glanced at “Big Ball of Mud” I thought I was in for an amusing satire — perhaps a parody of the “software patterns” school. Instead — and what I found most fascinating about the paper — the authors actually walk a fine and narrow line between a Swiftian embrace of the mud-splat school of programming and the sort of “we know better than all those idiots” arrogance that’s found in a lot of the software literature.

Despite the best efforts of “best practices” advocates and methodology gurus, mud is everywhere you look in the software field. This cannot be a coincidence or represent mere laziness. The authors ask, “What are the people who are building [Big Balls of Mud] doing right?”

Their answer: “People build big balls of mud because they work. In many domains, they are the only things that have been shown to work.”
[Read more…]

This is the eleventh edition of Code Reads, a series of discussions of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s the full Code Reads archive.

Playfulness in writing about programming isn’t always so easy to find. But here it is again. In our last Code Reads we got to enjoy Guy Steele’s words-of-one-syllable language game; this time around, we’re in the hands of two writers who are playing games with the entire corpus of software history.

James Noble and Robert Biddle are colleagues at Victoria University in Wellington, New Zealand (Biddle is now at Carleton University in Ottawa). In Dreaming in Code I summarized a bit of their work (along with a group of colleagues) on “Scale-free geometry in OO programs” — a study which found that software objects are “scale-free, like fractals, and unlike Lego bricks.”

Earlier this decade Noble and Biddle presented a series of papers at the OOPSLA conference on the theme of “Postmodern Programming.” The first, “Notes on Postmodern Programming” (2002), opens with a tongue-in-cheek transposition of the Bauhaus manifesto into the computing realm. From there, it jumps into a sometimes line-by-line rewrite of Edsger Dijkstra’s “Notes on Structured Programming” (which we looked at back in Code Reads #4). So we’re on notice that this paper will draw heavily on the postmodern aesthetic of cobbling together scraps, references, tributes and parodies.

But those bits and pieces form a serious argument, too.
[Read more…]