Wordyard

Hand-forged posts since 2002

Archives

About

Greatest hits

An interview and a profile

by

A few weeks ago I had the pleasure of being interviewed by Ed Cone about Dreaming in Code. I first met Cone years ago when he was organizing the panel I spoke on at the first BloggerCon. I’ve always enjoyed his work; like me, he’s someone who is equally interested in politics and technology, and blogs about both of them.

The Cone interview is now up at CIO Insight. It was fun to talk about the issues in the book for a relatively expert readership, where I could skip over some of the basics and jump right to the harder questions. Cone did a great job of drawing me out and then trimming the verbal excess from my responses.

CIO Insight: Are we just being impatient with a branch of knowledge that is still fairly new? Or is there something inherent to software development that makes it so weird and vexing?

Rosenberg: You get one perspective that says, hey, we now have a computer on every desk that does things that were unimaginable 20 years ago, and they’re all connected in this network that gives us instant answers and instant connections. These are miraculous things. And then you find other people who say, you know what? We’re still writing code basically by picking out characters one at a time, we still have programs that are laid low when a single bug creeps in, we still have projects that take ten times longer than they should, we need to rethink everything from the ground up.

I don’t have an answer between them. My personal temperament is more towards the optimistic. In the end, what you’ve got is this industry that’s been conditioned by Moore’s Law, and by its own fantastic financial success, to assume that the curve is always an upward curve, that everything gets better at an exponential pace. That’s the experience of the technology industry. You have that smacking up against the reality of human experience, of creativity, of people working in teams. We have these basic human factors, psychology, the limits of the conceptual capacity of the human brain—things that do not move at an exponential pace. They simply don’t. They tend to move linearly, if they are improving at all. People in the technology industry are loath to accept that.

This theme is also at the heart of another piece that occupied me for a considerable part of the fall — a profile of Charles Simonyi that is on the cover of the new issue of Technology Review. I covered Simonyi and his Intentional Software project just a little bit in Dreaming in Code, and I’m grateful to Jason Pontin at TR for giving me the chance to look at him, and it, more fully.

The first part of the profile, “Anything You Can Do, I Can Do Meta,” is up at the TR Web site now; the second part is slated to go up tomorrow. Since the piece was written as one integral whole, you might want to wait till you can read it all at once — I’ll post the link. It was fun to be writing for print again, and Technology Review is looking very spiffy these days, so this is one that you just might be better off reading on paper.
[tags]charles simonyi, ed cone, technology review[/tags]

First responses to Dreaming in Code

by

I spent many, many years writing reviews — of plays, movies and books. Over the next few weeks, it’s my turn to be on the receiving end. I have vowed to savor the praise, to engage with the honest feedback, and to avoid tiresome quarrels with any pans. Hold me to that if I slip, okay?

I got some warm advance comments from Joel Spolsky earlier this week, and that was cool — since Joel’s reliably entertaining commentary was one of the factors that persuaded me it’s possible to write about programming without putting readers to sleep. Today marked my first mainstream review, in BusinessWeek. They liked the book, calling it “a fascinating look inside one software-development project” and saying that I “know my subject” and “its scenes are vivid.” I’m grateful for the praise.

BusinessWeek also said that it was “frustrating” that, at the end of the book, Chandler still isn’t done: “Under deadline pressure from his publisher, [Rosenberg] sat down to write the book even though the project had not been completed.”

In fact, the decision to wrap up when we did was mine. No one at Crown, my publisher, pressured me. No publisher wants to wait forever for a manuscript, I guess. But I’m sure if I’d gone to Crown and said, “It will be a better book if we wait another six months,” they’d have approved.

If it had looked like the product was going to be complete (or reach some critical milestone) in only a few more months, I’d have just pushed back my writing schedule. But at the end of 2005, which is where the book’s saga leaves off, there was still no way to predict when the Chandler story would end.

More important, I felt that I’d already unearthed more answers to the questions I’d set out with — why is making software so hard? why does it take so long? what can we learn by observing the intricacies of a real-world project? — than I could possibly fit in one book.

One of the themes at the heart of Dreaming in Code is the strange nature of what I call “software time.” Working on software often means entering a sort of twilight zone in which the normal timeline of the calendar becomes a bottomless black hole.

If the Chandler story had provided a slam-bang finale, that’s how I’d have ended the book. Instead I tried to give Dreaming in Code a conclusion that’s peculiarly true to the material, in a way that I hope readers will find pleasing.

At book’s end, the Chandler team was just beginning to use their own program in “dogfood” fashion. I’ll be posting more soon about what’s happened with Chandler since that point; 2006 saw considerable further development, with a new focus on a “ship-it mindset,” and a fully usable “preview” edition is now scheduled for an April 2007 release. (Katie Parlante, one of the Chandler team’s key managers, has posted an update over at the OSAF blog with more details.)
[tags]osaf, chandler, book reviews, dreaming in code, businessweek, joel spolsky[/tags]

Code Reads #7: Parnas’s “Star Wars” paper

by

Code ReadsThis is the seventh edition of Code Reads, an aspiring-to-weekly discussion of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s the full Code Reads archive.

In the early 1980s, president Ronald Reagan proposed the missile-defense program known formally as the Strategic Defense Initiative (SDI) — and informally as “Star Wars.” Then the Pentagon and its contractors began trying to build the thing. They’re still at it today.

In 1985, David Lorge Parnas — a well-reputed computer scientist best known for elucidating the principle of “information hiding” that underlies modern object-oriented techniques — publicly resigned from a government computing panel that had been convened to advise the Star Wars effort. Star Wars couldn’t be built according to the government’s requirements, Parnas declared, because the software simply couldn’t be built. Unlike many who weighed in on Star Wars, Parnas didn’t express opposition on moral or political grounds; he was, he said, a longtime toiler in the field of defense software engineering, and had no “political or policy judgments” to offer. He just thought that SDI’s engineering aspirations lay impossibly far outside the bounds of real-world capabilities.

In support of this judgment Parnas composed a series of eight short essays that the American Scientist — and later the Communications of the ACM — published together under the title, “Software Aspects of Strategic Defense Systems.” It’s a magnificently forthright piece of writing — it encapsulates the best of the engineering mindset in prose. And it explains some fairly complicated notions in terms anyone can follow.

For instance, it wasn’t until I read Parnas’s paper that I fully understood why digital systems are so much harder to prove reliable than analog ones. [Read more…]

Building a book site with WordPress

by

It probably says something about me that, after completing years of work on my book, one of the things I most looked forward to was…building the Web site for the book. Aside from my work at Salon and my launching of various blogs, it had been years since I built a Web site on my own, and I relished the chance to look around at all the new tools for content management that have emerged in the interim.

But pretty quickly I realized that learning any new tool takes time, and I didn’t have a lot of time. And then I also realized that I’d spent a lot of time last summer learning to use WordPress as the new platform for my blog, and there was no reason under the sun I couldn’t use WordPress to build the book site for Dreaming in Code. WordPress lets you publish static pages; all I had to do was customize some templates, and voila! — blog software without the blog.

I am, as I admit at the start of Dreaming in Code, barely a programmer myself. But I can find my way around a template, I can borrow snippets of code and mess with a system that already works just enough to get it to do what I need it to do. This approach was, once upon a time, called “end-user programming”: the idea of enabling sophisticated users to extend and adapt a powerful piece of software without their needing to master complex programming languages. Spreadsheets depend on end-user programming; back in its day, Hypercard did, too. (Bonnie Nardi’s 1993 book A Small Matter of Programming is a good outline of the concept, focusing on spreadsheets and CAD systems.) A good blogging tool like WordPress is an invitation to end-user programming. And I admit it: I had fun!

The part that wasn’t fun was wrestling with CSS. I know that CSS achieved a Good Thing in helping Web designers clean up HTML and separate content from presentation and all that. But making a Web site look exactly the way you want it to look was a hell of a lot easier back in the days of tables and simple HTML than it is today. There are still a few elements of my site that aren’t aligned exactly the way I want them; I gave up trying to figure out why — life is too short! CSS was a step forward for designers’ control of their Web work but a step backward for the end-user programming that made the Web what it is today.

A tale of two book covers

by

My book Dreaming in Code — which I first conceived in 2002, started researching and proposal-writing in 2003, got a contract for in 2004, wrote mostly in 2005, and saw through the editing and production process in 2006 — will arrive in bookstores in about two weeks. I am torn between elation and exhaustion. Mostly, I’m deeply happy that — in a world where writers’ work is too often run through a gantlet of crass attitudes, careless handling and fickle judgments — I got the opportunity to write exactly the book I wanted to.

For the next several weeks, this blog will become heavily focused on Dreaming in Code, its reception and surrounding events. Ideally, that will interest you; if not, apologies, but you’ve been warned.

Let’s start with the saga of the book’s jacket. When my editor at Crown, Rachel Klayman, and I first discussed the cover, we agreed that it might be fun to aim for a classical design, one that ran counter to some cliches of the tech-books field (no mouse on the cover, please!) and that signaled, in a light-hearted way, one theme of the book — the aspiration toward elegance shared by the software developers at the story’s center.

Old cover for Dreaming in CodeSo the wizard designer at Crown came up with this, and we were pretty happy with it. This cover actually went out to the first readers of the book, appeared in early catalog pages, and was transmitted seemingly instantaneously onto Amazon’s page for the book, which first appeared online scarily soon after I’d completed the manuscript.

My agent, however, got concerned, raised some questions, and got the rest of the book’s team equally concerned. For one thing, given that the book tells a story about computer programming, mightn’t it be a good idea for that to be, er, a little more evident from the cover? And then there was the slight problem that some other book that had sold a few copies and also, coincidentally, features the word “Code” prominently in its title uses a similar crimson-and-gold color scheme.

Dreaming in Code, final coverSo it was back to the drawing board for our designer, and we wound up with the splendid cover you see here — its embossed electric-green title mysteriously undulating in a blue-black void.

What I learned from this process is that my own default preference for being subtle and ironic is probably not the best guide for selecting a jacket design. After all, people really do judge a book by its cover. There may only be a nanosecond’s chance to catch someone’s eye. In that instant, you might as well take the direct approach.
[tags]dreaming in code, publishing[/tags]

Code Reads #6: Mitch Kapor’s Software Design Manifesto

by

Code ReadsThis is the sixth edition of Code Reads, an aspiring-to-weekly discussion of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s thefull Code Reads archive.

Mitch Kapor first presented his Software Design Manifesto at PC Forum in 1990. It got published in Dr. Dobbs’ Journal in 1991, and later appeared as the first chapter of Terry Winograd’s book Bringing Design to Software (much of the book is available in online form now). At a point roughly midway between the start of the personal-computing revolution in the mid-’70s and the present, Kapor threw down a gauntlet: Computers were too hard to work with. “Everyone I know (including me) feels the urge to throw that infuriating machine through the window at least once a week,” Kapor wrote.

The PC revolution’s promises of “personal empowerment” remained unfulfilled because the software that drove the miraculous new machines was simply not well-enough shaped. The people who made it weren’t thinking clearly enough about how their products would be used. Somebody had to do that work; Kapor argued that programmers probably weren’t the best people to take it on, and proposed the creation of a new discipline of “Software Designer” to wear this mantle.
[Read more…]

Code Reads #5: Knuth’s “Structured Programming with go to Statements”

by

Code ReadsThis is the fifth edition of Code Reads, a weekly discussion of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s the full Code Reads archive.

I have felt for a long time that a talent for programming consists largely of the ability to switch readily from microscopic to macroscopic views of things, i.e., to change levels of abstraction fluently.
— Donald Knuth, “Structured Programming with go to Statements”

We’ve been looking at Edsger Dijkstra’s principles of structured programming for some time now. Today we’ll conclude that phase of this series with a look at Donald Knuth’s “Structured Programming with go to Statements” (1974). Since “Go To Statement Considered Harmful” was the most famous declaration of the structured programming ethos, Knuth’s title itself was a kind of provocative joke — like naming a paper “Judaism with Idols” or “Vegetarianism with Pork Chops.” And that combination of playfulness and attentive humor extends throughout the article, from the epigraphic references to Bob Dylan (and a laxative advertisement!) on. In the middle of a complex analysis of an algorithm in ALGOL, in which Knuth is reviewing ideas for new ways to notate a particular kind of loop, he interjects:

Readers who remember 1967 will also appreciate [this] second suggestion,
turn on begin S; when B drop out; T; end.

The levity’s a good thing, because, I confess, as a non-mathematician and only a rudimentary programmer I approached Knuth’s formula-laden text with trepidation. And there remain parts of “Structured Programming with go to Statements” that are beyond me. But for the most part, this text bears out Knuth’s reputation for brilliant clarity — it’s surprisingly accessible.

[Read more…]

Code Reads #4: Dijkstra’s “Notes on Structured Programming”

by

Code ReadsThis is the fourth edition of Code Reads, a weekly discussion of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s the full Code Reads archive.

The art of programming is the art of organizing complexity, of mastering multitude and avoiding its bastard chaos as effectively as possible.”
— Edsger Dijkstra, “Notes on Structured Programming”

“Notes on Structured Programming” (1970) is the third and, for now, final Edsger Dijkstra work this series will look at. The paper fills out the terse observations in “Go To Statement Considered Harmful” and the historical overview of “The Humble Programmer” — and offers some detailed examples of the disciplined sort of approach to programming Dijkstra was advocating at the time.

In his 2001 “What led to ‘Notes on Structured Programming'” Dijkstra recalled that he wrote the original “Notes” while emerging from a depression; he felt that his previous work on ALGOL and other systems had “only been agility exercises” and he now faced tackling “the real problem of How to Do Difficult Things.”

[Read more…]

Crash in Iraq; Grove’s “shift left”

by

Sometimes when I tell people I’ve written a book about how difficult it is to make software well, I get a blank stare, as if to say, “What could be further from mattering to me?” And I repeat my now well-rehearsed remarks about the way our lives increasingly move through a vast web of programming products — way beyond what we encounter when we’re online. Sometimes the stares even resolve into interest.

So when I read items like this in the news I’m reminded of why these questions still matter. This is from a longer New York Times piece on how the U.S. seems to have lost track of hundreds of millions of dollars worth of materiel in Iraq:

Mr. Bowen found that the American military was not able to say how many Iraqi logistics personnel it had trained — in this case because, the military told the inspector general, a computer network crash erased records. Those problems have occurred even though the United States has spent $133 million on the weapons program and $666 million on Iraqi logistics capabilities.

What — no backup? No paper records? How about, you know, asking the people whose job it was to train those Iraqi logistics personnel? Now it could be that this is a “dog ate my homework” sort of excuse, and that in fact the fruits of $666 million did not disappear in a network crash but rather into various people’s pockets, on the U.S. or the Iraqi side. But even at face value, that’s a pretty expensive crash.

(Peter Neumann’s Risks Digest tracks endless amounts of stuff like this on a regular basis.)

It’s hard to analyze this particular disaster without more detail. But military software tends to be big, complex, sometimes bloated. I thought of that, reading today’s fascinating Journal column by Lee Gomes about Intel’s Andy Grove and his latest cause — improving the health care system and its record-keeping. Grove is advocating a simple approach — plain text.

To explain “Shift left,” Mr. Grove describes the bottom axis of a scale in which products and services grow more full-featured, complicated and expensive as you move to the right. To “Shift left” on this scale is to, in effect, “Keep it simple, stupid.”

…Rather than designing an elaborate and technically sophisticated medical-database system, something practically every tech company is now trying to do, Mr. Grove suggests the exact opposite. Shift left; keep the record of a patient’s visit in, for example, a generic but Web-accessible word-processing file.

Just like the early personal computer, it will be far from ideal, but it will be a start, and it can get better over time. The alternative, he says, is to wait endlessly for a perfect technology.

That last sentence should be etched onto the monitors of CTOs and development managers around the globe.

Fallows on Dreaming in Code

by

In his technology column in the new Atlantic, James Fallows had some extremely nice things to say about my book. In a sidebar to a piece that reviews lots of different information-organizer programs, he names Dreaming in Code as “this month’s tech-literature pick”:

The book is the first true successor to Tracy Kidder’s Soul of a New Machine, and is written with a combination of technical sophistication and narrative skill not seen in many years. Read it to understand what all these software wizards actually do.

I’m grateful for the advance enthusiasm. (The book won’t be in stores till January, but there’s always Amazon pre-order…) To be associated favorably with The Soul of a New Machine in the pages of the magazine that served as its author’s home is an honor. I hope I can live up to it!

I’ve always been a little cautious about connecting Dreaming in Code with The Soul of a New Machine. Kidder’s book is a non-fiction classic that I’ve always admired. But the comparison sets a high bar and raises expectations to daunting levels. If people end up feeling that my book is one-quarter as good as Soul of a New Machine, I’ll take it as a compliment.

Kidder’s book — exactly a quarter century old this year — introduced a whole generation to the romance and the nightmare of building computers. It didn’t matter that he was writing about refrigerator-sized minicomputers just as the IBM PC was bringing the “microcomputer” into the spotlight and ushering in the computer-on-every-desk era. The book’s great achievement was its glimpse into the world of Route 128 engineers and managers — the intense, focused way they lived their work and worked their minds.

That world has become a widely familiar one in our era of “knowledge work” and startup-company culture. The opportunity for a writer today lies not in exploring this realm for the first time, but instead in trying to fathom some of its enduring mysteries. I’ve always been fascinated by the minds and work of programmers, and I wanted my book to tell a story that would capture some of their pleasures and terrors — and tell us something about why, 50 years into the computer era and 25 since The Soul of a New Machine came out, writing the software that runs our world remains a singularly unpredictable undertaking.
[tags]james fallows, dreaming in code[/tags]