I took a quick trip to the DC area at the end of last week to talk about Dreaming in Code to the folks at The Motley Fool. It was a blast. The Fool, as it’s known, has been around online from the earliest days of the Web; like Salon, it’s had its ups and downs, weathered many storms, and is now in a growth phase again.

The engagement happened because a developer who worked for the company happened to read the book, liked it, and thought it would be a good topic for the rest of the company to learn more about it. What I heard from the “Fools”, as they call themselves, is the same thing I’ve heard from lots of other software developers: they feel that the book captures the elusive nature of their work, and they want their colleagues to read it to learn, in greater depth, about the difficulties of creating software.

I’m home for a few days, then heading up to Seattle for Gnomedex. It’s an event I’ve heard about for years, but never made it up there before. If you’re going too, let me know!

Dreaming in Code has sold particularly well on Amazon.com, which does not surprise me. Given the subject matter, the book was bound to appeal to buyers who shop online, and Amazon is the dominant player in the online bookstore market.

I’ve also been pleased to see the profusion of customer reviews on Amazon. As of about three weeks ago, we had 33 reviews posted. Most were positive, a handful were negative; either way, each one meant that some reader cared enough to take the time to post their reactions, and that meant a lot to me.

Then something weird happened about ten days ago. Suddenly, Amazon showed only 10 reviews. Two dozen reviews posted between mid-February and the end of June had simply disappeared. In the time since then, a couple of new reviews have joined the total, but the missing reviews have not reappeared.

I’ve been building Web sites long enough, and worked with software long enough, to imagine a variety of different scenarios for what might be causing this. Whatever happened, this is something that Amazon ought to be concerned about — these glitches are rarely limited to a single page; there’s likely sporadic data loss in multiple places. Amazon runs a gigantic Web service that a lot of people depend on. It has even recently gotten into the business of offering back-end data storage services (Amazon S3) to other Web companies and individuals. So I trust they’ll be pursuing this issue. They ought to have this data somewhere from which it can be restored.

I’ve asked my publisher to look into the matter. I also contacted Amazon through their bottom-of-the-page feedback mechanism. The good news is, I actually got a response; the bad news is, it was feeble — I think the customer-service rep. simply looked up the page, saw there were a dozen reviews, and reported such back to me. I could do that from the comfort of my home, thank you!

Amazon was one of the very first businesses to understand the value of what the Net industry now calls “user-generated content.” Customer reviews are the heart of its operation. The most basic compact between a Web service and its users is, “If you contribute something of value, we promise not to lose it.”

UPDATE Mid-afternoon Wednesday: The reviews appear to be back. Thanks, Amazon.

This is the eleventh edition of Code Reads, a series of discussions of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s the full Code Reads archive.

Playfulness in writing about programming isn’t always so easy to find. But here it is again. In our last Code Reads we got to enjoy Guy Steele’s words-of-one-syllable language game; this time around, we’re in the hands of two writers who are playing games with the entire corpus of software history.

James Noble and Robert Biddle are colleagues at Victoria University in Wellington, New Zealand (Biddle is now at Carleton University in Ottawa). In Dreaming in Code I summarized a bit of their work (along with a group of colleagues) on “Scale-free geometry in OO programs” — a study which found that software objects are “scale-free, like fractals, and unlike Lego bricks.”

Earlier this decade Noble and Biddle presented a series of papers at the OOPSLA conference on the theme of “Postmodern Programming.” The first, “Notes on Postmodern Programming” (2002), opens with a tongue-in-cheek transposition of the Bauhaus manifesto into the computing realm. From there, it jumps into a sometimes line-by-line rewrite of Edsger Dijkstra’s “Notes on Structured Programming” (which we looked at back in Code Reads #4). So we’re on notice that this paper will draw heavily on the postmodern aesthetic of cobbling together scraps, references, tributes and parodies.

But those bits and pieces form a serious argument, too.
(more…)

Near the end of Dreaming in Code I took a chapter to look at some of the more visionary efforts today to reform the troubled world of software development. One key portrait was of Jaron Lanier. (It was a delightful coincidence that, well after I’d settled on my book title, I discovered that Lanier had once told an interviewer, “I used to dream in code at night when I was in the middle of some big project.”)

This month in his column in Discover, Lanier uses my book as a jumping off point to discuss some of the same questions I set out with:

Why do some software projects sail to completion while so many others seem cursed? Why must software development be so difficult to plan?

These questions should concern everyone interested in science, not just programmers, because computer code is increasingly the language we use to describe and explore the dynamic aspects of reality that are too complicated to solve with equations. A comprehensive model of a biological cell, for instance, could lead to major new insights in biology and drug design. But how will we ever make such a model if the engineering of a straightforward thing like a personal productivity package confounds us?

In the heart of the piece, Lanier explains, more fully, his big idea — “phenotropics”: a software system, inspired by biology and robotics, in which surfaces “read” each other using fuzzy pattern recognition, allowing for systems that are better able to handle small variations from the norm without crashing.

Suppose software could be made of modules that were responsible for identifying each other with pattern recognition. Then, perhaps, you could build a large software system that wouldn’t be vulnerable to endless unpredictable logic errors.

He mentions Web 2.0-style mashups as one fledgling step in this direction, and also provides an anecdotal account of a project from the 1980s that he collaborated on with Andy Hertzfeld (another central figure in Dreaming in Code) called Embrace.

It’s a mind-expanding read, like so much of Jaron’s stuff. Embrace surfaces! Find patterns!

This is the tenth edition of Code Reads, a series of discussions of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s the full Code Reads archive.

You don’t have to read too far into Guy Steele’s “Growing a Language” to figure out what he’s up to. In this 1998 talk on the nature of programming-language design, Steele writes under a set of constraints: he may use any word of one syllable, but if he wishes to use longer words, he must define them first — using only words of one syllable.

Steele aimed to illustrate how awkward and confining it can be to try to express yourself when you don’t have access to an extensive vocabulary: “I wanted to show you want it is like to use a language that is much too small.” (By this point in the talk, he has already defined “language.”) His larger purpose is to advocate a philosophy of programming-language development that aims for neither a small nor a large language, but rather for one that is “designed to grow.”

Such growth is to be propelled not by a closed cadre of gurus but by the language’s users, who have been given an appropriate set of tools to extend the language. The shape of the language is such that these user-generated additions do not look or behave any differently from the “original” elements of the language provided by its initial creators. Nearly a decade later, this argument — which parallels similar ideas that have become popular in the open-source software community as well as in the “user-generated” enthusiasms of Web 2.0 — sounds neither arcane nor controversial.
(more…)

In the context of Chandler’s effort to escape the confinement of tree-based models of information organization, I quoted Ted Nelson’s famous line about “intertwingularity” in Dreaming in Code:

People keep pretending they can make things deeply hierarchical, categorizable and sequential when they can’t. Everything is deeply intertwingled.

This quotation can be found on many Web pages, bbut when I went to source it for the endnotes, I couldn’t locate the authoritative original statement. I found the same quote in David Weinberger’s Everything is Miscellaneous, and — knowing that Weinberger is a scrupulous attributor — I asked him if he knew its original source. He, too, found it hard to pin down, but now reports a definitive answer (courtesy Frank Frank Hecker — more details on Weinberger’s errata page).

It turns out it’s exactly where I first looked for it, in Nelson’s book Dream Machines, the 1987 revision, on page 31. (Nelson only used “deeply” once, the second time; the word doesn’t appear before “hierarchical” in the original — it must have crept in across multiple reuses.)

I spent hours hunting through Nelson’s volume — a reissue of both Dream Machines and Computer Lib in one book, with one starting from the front and the other from the “back”, upside down. I guess I should have looked harder.

On the other hand, my failure to locate the quote might also have been the result of thei books’ unconventional format. These books are true miscellanies, examples of the havoc a fertile mind and a page-layout program (or an X-acto knife) could wreak on the conventions of book design in those heady early days of desktop publishing. (Think of the old Whole Earth Catalog, if you’ve seen that.) Chunks of text are scattered in different typefaces, mixed up with graphs and hand-drawn flowcharts and ALL-CAPS EMPHASIZED TEXT. It is a format designed to frustrate the simple linear quest to attribute a quotation.

This whole tiny story is, in its way, a tribute to the “intertwingularity” of Nelson’s work itself. You have to give the man credit for finding a form that matched the content of his digital-liberation ideas — even when he was stuck using paper.

Since my decision to wrap up Dreaming in Code without a conclusive ending to the Chandler saga left me dealing with complaints (unfair! not true!) that I’d “bailed out,” I have become inordinately obsessed with tracking examples of successful creative works that lack a traditional conclusion.

Students of the epic understand that the classic form of that tradition begins in medias res — and ends that way, too.

Mitch Kapor and Tony Soprano could never be tagged as “separated at birth.” But I’m proud that my little book now has something in common with America’s highest-regarded TV series.

UPDATE: On the other hand, the Sopranos ending left Dave Winer with Post-Traumatic Sopranos Stress Disorder.

This is the ninth edition of Code Reads, a series of discussions of some of the central essays, documents and texts in the history of software. You can go straight to the comments and post something if you like. Here’s the full Code Reads archive.

The annals of programming include a small but significant number of radical thinkers who propose total overhauls of our basic techniques for making computers perform useful tasks. One of them is the late John Backus. Backus was the central figure in the creation of FORTRAN — the archetypal first-generation “higher level” programming language — and the devising of the BNF (”Backus Naur Form” or “Backus Normal Form”), a formal method for specifying programming-language syntax.

In his 1977 Turing Award lecture, however, Backus stepped away from the world he’d helped shape and asked, “Can Programming Be Liberated from the Von Neumann Style?” It’s a question that gets at fundamentals, at breaking free of founding constraints. It’s like asking whether psychology could be liberated from Freud, or physics from Einstein.

Backus describes the prevalent Von Neumann paradigm of computing:

In its simplest form a von Neumann computer has three parts: a central processing unit (or CPU), a store, and a connecting tube that can tramsit a single word between the CPU and the store (and send an address to the store). I propose to call this tube the von Neumann bottleneck. The task of a program is to change the contents of the store in some major way; when one considers that this task must be accomplished entirely by pumping single words back and forth through the von Neumann bottleneck, the reason for its name becomes clear….

Surely there must be a less primitive way of making big changes in the store than by pushing vast numbers of words nback and forth through the von Neumann bottleneck. Not only is this tube a literal bottleneck for the data traffic of a problem, but, more importantly, it is an intellectual bottleneck that has kept us tied to word-at-a-time thinking instead of encouraging us to think in terms of the larger conceptual units of the task at hand.

(I will pause here to note the presence of that word “tube.” Perhaps Senator Ted Stevens, far from being a techno-rube, was actually a student of Backus.)
(more…)

In Dreaming in Code I wrote a lot about the difference between ambiguity in programming, where it is often a source of trouble, and in “natural” languages (i.e., English or French or Mandarin), where it is often a source of value.

Jason Kottke points to this fascinating essay by Perl creator Larry Wall, in which Wall distinguished between “local” and “distant” ambiguities:

People thrive on ambiguity, as long as it is quickly resolved. Generally, within a natural language, ambiguity is resolved rapidly using recently spoken words and topics. Pronouns like “it” refer to things that are close by, syntactically speaking. Perl is full of little ambiguities that people never even notice because they’re resolved so rapidly. For instance, many terms and operators in Perl begin with identical characters. Perl resolves them based on whether it’s expecting to see a term or an operator, just as a person would. If you say 1 & 2, it knows that the & is a bitwise AND, but if you say &foo, it knows that you’re calling subroutine “foo”.

In contrast, many strongly typed languages have “distant” ambiguity. C++ is one of the worst in this respect, because you can look at a + b and have no idea at all what the + is doing, let alone where it’s defined. We send people to graduate school to learn to resolve distant ambiguities.

Apple recently announced that it had to delay the release of the next version of Mac OS X, Leopard, by a few months — too many developers had to be tossed into the effort to get the new iPhone out the door by its June release. Commenting on the delay, Paul Kedrosky wrote, “Guess what? People apparently just rediscovered that writing software is hard.”

In researching Dreaming in Code, I spent years compiling examples of people making that rediscovery. I’m less obsessive about it these days, but stories like this one still cause a little alarm to ring in my brain. They tend to come in clumps: Recently, there was the Blackberry blackout, caused by a buggy software upgrade; or the Mars Global Surveyor, given up for lost in January, which, the LA Times recently reported, was doomed by a cascading failure started by a single bad command.

Kedrosky suggests the possibility of a Brooks’s Law-style problem on Apple’s hands, if the company has tried to speed up a late iPhone software schedule by redeploying legions of OS X developers onto the project. If that’s the case, then we’d likely see even further slippage from the iPhone project, which would then cause further delays for Leopard.

This is the sort of thing that always seemed to happen at Apple in the early and mid-’90s, and has rarely happened in Steve Jobs Era II. I write “rarely,” not “never,” because I recall this saga of “a Mythical Man Month disaster” on the Aperture team. If the tale is accurate, Apple threw 130 developers at a till-then-20-person team, with predictable painful results. We’ll maintain a Brooks’ Law Watch on Apple as the news continues to unfold.

UPDATE: Welcome, Daring Fireball and Reddit readers! And to respond to one consistent criticism: Sure, iPhone isn’t late yet, but Apple is explicitly saying it needed to add more developers to the project to meet its original deadline. If that all works out dandy, then the Brooks’s Law alarm will turn out to have been unwarranted. Most likely, given Apple’s discipline, the company will ship iPhone, with its software, when it says it will. What we won’t and can’t know is whether, and if so how much, the shipping product has been scaled back. And sure, of course this is all conjecture. Conjecture is what we have, given Apple’s locked-down secrecy.