Wordyard

Hand-forged posts since 2002

Archives

About

Greatest hits

Big software: time to give up?

by

On today’s New York Times op-ed page, Nicholas Carr of “Does IT Matter?” puts the FBI software meltdown in the context of other recent enterprise-scale software train wrecks like McDonald’s Innovate and Ford’s Everest (he could have dragged in the IRS, too). As everyone does who addresses this topic, he references the Standish Group “Chaos” report from 1994, with its dire statistics about software failure.

As with his previous arguments on the topic, Carr gets things just about half right: Of course the record of large-scale software projects, particularly those meant to replace existing systems that are functional but graying, has been awful, and the complexity of these systems remains daunting. Carr concludes that the complexity is so overwhelming we should give up on innovating in software and just concentrate on doing the same things we currently do more efficiently: “When it comes to developing software today, innovation should be a last resort, not a first instinct.”

He’s forgetting that, in the world of software, innovation is the primary way to add value. We move existing “off-line” systems and processes into software not only to make them more efficient, but to give them capabilities the physical world can’t provide. Thus online publishing isn’t just about delivering text and images more cheaply; it’s about connecting publisher and information consumer in new ways that change the whole relationship. Manufacturers implement inventory control systems not just to save money but to transform their businesses so they can build products when customers ask for them, rather than trying to guess what the market needs. If software isn’t providing new capabilities, why bother? It seems obvious that we’re a long way away from exhausting the possible new wrinkles software can offer business, government and society.

Carr is right that large institutions get into trouble when they try to replace big old systems and introduce complex new features at the same time. But his advice — give up on those new features, be happy with what you’ve got — is needlessly ostrich-like. The answer is not to abandon change but to structure change so that it’s not a big bang but an evolutionary process. The failures in so many of these software disasters don’t stem from ambition but from impatience and bad planning.

1968: the year of the blog

by

Having heard Alan Kay’s inspiring talk at the 2003 Emerging Technology conference, I already knew how much of modern computing Douglas Engelbart‘s famous 1968 demo of the NLS (“oNLine System”) contained within it — and how far we still have to go to match the stuff Engelbart showed off then, not only in its individual elements (graphical interface, hypertext, advanced input devices, distance collaboration, and so on) but in their total integration.

What I didn’t know was that the NLS appears even to have had a kind of blog capability as one of its many tricks.

I’ve been watching the amazing videos (shot by Stewart Brand) of Engelbart’s demo — all available online, here. If you take a look at this one, you’ll see Jeff Rulifson explaining that the NLS programmers — who, in true bootstrapping fashion, seem to have maintained all their code within the NLS itself — kept a kind of bug log. Since NLS tracked who was using it and what everyone did when, each entry in the bug log has a little subscript line, flush right, with the name of the person who posted it and the time it was posted.

Sure looks like a weblog! And if you were logged into NLS you could even add comments. (I’d clip a still from the Real stream but haven’t been able to do a screen capture — perhaps part of the Real format’s DRM, or I’m too much of a klutz. Anyway, the video clip is under a minute.)

Semper stickies

by

Today I confronted the sheer vastness of the topic I have chosen to write my book about. I indulged my vertigo for about 15 minutes. Then I borrowed a page from some of the people I’m writing about.

A while back, the team at OSAF, in an effort to wrestle the schedule of their project to the ground, temporarily moved their planning process off the wiki and onto a whiteboard. They broke their project down into roughly equal chunks of work and wrote the name of each chunk on a simple yellow sticky note. Instantly, the outlines of a schedule became easier to discern.

Stickies (a k a “Post-It” notes)! I’d seen play- and screen-writing friends do the same for their projects. I’m a devotee of outlining software, and I’m using a venerable outliner to organize my research. But I needed a different approach to get beyond the sense of “Oh crap, how do I find a way out of this swamp and onto that mountain range?” Somehow, laying all the pieces out in an open-ended, non-hierarchical way on a two-dimensional plane just helped: Something about being able to take in all the pieces in a map-like overview rather than peering in through the keyhole of screen real estate.

My stickies are now marshalled out on a 3′ x 4′ foamcore board and looming over my desk. Over the next few months I will add to them, rearrange and reorganize them, then remove them from the board one by one as they pass from concept into actual pieces of writing.

One day the board will be empty. And I’ll be done with a first draft.

Spolsky in Salon

by

I’ve been an admirer of Joel Spolsky’s writing on software since I started reading it several years ago. Last month when I was in New York I sat down with Joel and had a good long talk about software development, partly for the purpose of my book research and partly because I knew he’d be entertaining and thoughtful. Today’s Salon features a write-up of the interview, pegged in part to the publication of a book collection of Spolsky’s essays.

Book break

by

Earlier this year I wrote about the book project I’ve been working on. For the past six months I’ve been splitting my time between Salon and work on the book. That’s been great but, as you might imagine, at times I’ve felt my world to be a little…bifurcated. (Since I’m a father of twins, this is not an unfamiliar sensation.) And there’s no way to write a book — none that I know of, anyway — without putting your back into it, 100 percent.

So beginning this week, I’m stepping aside from my job at Salon. It’s a highly orderly transition: I’ve planned it for some time, my colleagues and friends at Salon have been great about giving me the freedom to do it, and I expect to return when I’m done — but for now, the book is my work. After nine years (I left the San Francisco Examiner for Salon at the beginning of October, 1995, and we went live on the Web on Nov. 13 of that year), I’m ready for a creative sabbatical.

This blog will continue pretty much in its current form, with the usual spasms and lapses in posting, but the mix of entries may change a bit — in this post-election period, I’ll probably be posting less on politics (hey, the War Room is still on the case!) and more on the subject of the book: the nature of computer programming and software development. Why it’s still so hard to build the stuff that runs our world. And what interesting ideas are out there to make things better.

I won’t be writing the book in public here on the blog. I’m in awe of those people who seem able to blog full-time about the subjects of their books-in-progress, and I admire experiments in wiki-style open editing like J.D. Lasica’s. But I’m still a linear sort of guy at heart; if I’m able to do what I plan on doing, this book will be something more than the sum of its parts, and I can’t imagine how to roll it out piecemeal without altering its nature. I also don’t see how I’ll ever get the writing done if I put too much of my energy into blogging about it. But I certainly expect to be opening up some of the topics as I tackle them. And I know I’m gathering for more material than I will ever be able to include in the final text.

I’m profoundly lucky to be exploring this subject at a moment in history when throngs of thoughtful programmers have adopted the Web as a public space to talk about their work. It makes my work almost too easy.

Except that some time soon I have to stop collecting great notes and URLs and interviews and start, uh, writing.

Vancouver bound

by

I’m off Tuesday through Thursday this week to Vancouver for the wonderfully named ACM OOPSLA event (the acronym stands for “Object-Oriented Programming, Systems, Languages & Applications”) — a venerable (this is the 19th year) conclave of people who are thinking about the future of programming and how to improve things. This is part of my continuing book research. I’ll be blogging from there as time (and Net access) permit.

Software rot

by

Nicholas Carr’s article (and now book) “Does IT Matter?” caused a stir when it was first published. Some aspects of Carr’s argument — that information technology is a more mature industry than it once was — made sense; other points — that somehow innovation is dead, there’s nothing new under the sun, and all the technology industry faces today is an unending vista of cost-cutting and cutthroat commoditization — were at best unprovable and more likely dead wrong. (Chad Dickerson had some good commentary on Carr here and here.)

Carr wrote a perfectly reasonable op-ed for the New York Times last week about Microsoft’s humongous dividend give-back as an indication of the company’s middle-age. I agree with much of the piece, but one passage caused my jaw to drop ground-ward:

  Software never decays. Machinery breaks down, parts wear out, supplies get depleted. But software code remains unchanged by time or use. In stark contrast to other industrial products, software has no natural repurchase cycle.

Software never decays? Carr is a Harvard Business Review veteran, and I assume he works with computer software every day, as most of us do — but I can’t imagine such a sentence being written by anyone who uses a personal computer or runs a software-dependent business (which means virtually any business today) for any extended period of time.

In the abstract, of course software doesn’t decay the way a pair of garden shears grows dull from use or an automobile engine loses compression over time. Abstract code shows no frictional wear. But the notion of “decay-free software” is as divorced from everyday reality as the notion of a “friction-free market”: Both exist only in the vacuum-space of the professional economist.

In truth, while well-written software can often lead an extraordinarily long and fruitful life (I am storing the fruits of two years of book research in a 2 megabyte Ecco Pro file, in outlines composed in a program that has not been upgraded or modified since around 1997), most software today begins to rot from the moment of first use.

And the most notorious piece of decay-prone software is the one Microsoft’s billions are founded on. Windows begins to accumulate barnacles of cruft in the registry the moment you first crank it up and try to use it to do anything. If you are a typical user, after two or three years of regular use your operating system will be grinding to a halt, crushed by the weight of the junk your various applications and Windows have together conspired to scatter across your directories. I know plenty of people who choose to buy a new computer not because they necessarily need some new hardware feature or upgrade but because they have given up on trying to save an ailing Windows installation — and reinstalling Windows is enough to send most people screaming toward the nearest Dell ad.

So while Carr may be technically correct — that software code does not “decay” the way a blade dulls — he is, by any pragmatic view, dead wrong. From the user’s perspective, software almost always decays — it stops doing what you want it to do, or you try to do something it is supposed to do and find that you can’t. The more you use it, the more likely it is to break, because the more likely you, as a cantankerous and unpredictable human being, are likely to do something the programmers haven’t imagined you would do.

Tim O’Reilly’s writings about software as a service outline this basic truth: Most software today “has people inside.” Software decay is so universal that every piece of software needs a corps of developers to keep the rot in check — whether you’re talking about a Web service like Amazon or Google, where the programmers deliver upgrades through the Web site; or a custom business application, where the programmers work for the software company or the client company; or a consumer application, where the programmers provide users with a constant stream of patches and upgrades to keep the bugs at bay.

This is a bad thing if you are dreaming of a world of perfect software. But it certainly keeps a lot of programmers employed. And it’s a more natural model for a world in which we don’t expect perfection, but hope for steady improvement.

The robot heart of software

by

Isaac Asimov was one of the science fiction authors whose works I avidly consumed when I was in my early adolescence, and though even then I could tell that his writing lacked a certain level of nuance and style, I loved it for its cleverness and its imagination. Standing at the podium at science fiction conventions, expounding on any subject under the sun, he was like a polymath Woody Allen with the neurosis circuits disabled, and his optimistic rationalism — even in the 1970s, an era during which optimism was hard to make credible — was infectious. (Read Cory Doctorow’s appreciation of Asimov in Wired for more.)

So I don’t think I’ll be able to bear going to see the new movie “inspired by” his “I, Robot” stories — those inventive chestnuts about what happens when robots programmed with “the three laws of robotics” tangle with the chaos of human affairs. (Chris Suellentrop in Slate offers an overview of how the movie betrays Asimov that makes me feel my decision is completely logical.) But I was glad to read this editorial in the Sunday New York Times, which thoughtfully nailed exactly what made these stories such fun:

  Each of the stories in “I, Robot” works out a problem in the application of these laws, usually caused by an unforeseen implication or contradiction. Asimov’s robots are perfectly logical, and therefore all the real problems are caused by humans, who are shockingly unaware of the way their intentions and emotions run counter to logic. What look like manufacturing flaws in the robots nearly always turn out to be faults in the way a command was articulated. Humans, it turns out, are mainly good at bossing other humans around. Our computers remind us of this every day.

The “I, Robot” stories, in other words, are exercises in logical debugging that happen to take the form of miniature mysteries.

Saying “the real problems are caused by humans” is, of course, awfully close to saying, “It’s the user’s fault!” — an excuse that conscientious software developers and designers shun. Yet, as I dig deeper into work on my book about software, I’m learning a lot more about exactly how hard it is to make the absolute logic of computing serve the messy ambiguities of human desire, when all the pressure of the undertaking is to make things work the other way — to force us human beings to conform to the rigorous precision of machines. Asimov’s wonderful stories pre-imagined this dilemma for us. Maybe someday he’ll find a filmmaker who can do his particular imagination justice.

Back to BASIC

by

David Ahl’s BASIC Computer Games (1978): I actually have a copy of this book in paper, dug up in some used book store pile ages ago, but now, you can revel in its full glories online.

Before there were sprites and polygons and first-person-shooters, before there were CRTs on every desk to splash graphics in our eyes, there were simply teletypes chattering out lines of text. And there were paper tapes for you to store your work. Right around the time Richard Nixon was being kicked out of office, I was learning BASIC by reading the code to some of these games. (We didn’t get them from the book — they were just floating around on the minicomputer we timeshared.) “Hammurabi,” a sort of primitive, text-only SimCity, was the one I and my circle of friends latched onto — and proceeded to amend. Because all these programs were free and, in the manner of their time, open source. [Thanks, Boing Boing and Oblomovka]