Wordyard

Hand-forged posts since 2002

Archives

About

Greatest hits

Berkeley Cybersalon

by

This coming Sunday, March 4, come on down to the Hillside Club, where, from 5 to 7 p.m., at the Berkeley Cybersalon, we’ll be talking about why software is (so often) so hard.

For this event, practically in my backyard and as part of what must be the longest-running technology salon around (I started going over a decade ago), I didn’t want to just give another spiel about Dreaming in Code.

Instead, it’s a panel discussion, with Eric Allman of Sendmail (program and company), Chad Dickerson of Yahoo’s Developer Network, and Lisa Dusseault of CommerceNet (and formerly of the Open Source Applications Foundation).

It was originally scheduled for yesterday, 2/25, but we realized that counter-programming against Oscar Night was foolhardy — even for an event as geekish as this.

Stealth fighter trips over dateline

by

Back when my job as Salon managing editor involved overseeing our daily production, I noticed that, every spring and fall, almost without fail, our publishing system would experience a glitch of some kind on the weekend that the clocks got moved forward or back — nothing serious, mind you, but enough to throw a wrench in the works of our site updates. It wasn’t a single bug, but some sequence of related bugs, so we’d fix one and then six months later something else would happen. Eventually we got in the habit of just making sure that one of the developers kept a close eye on things when that weekend rolled around. It was prudent.

I thought of that as I read these accounts that are filtering out about the F-22 Raptors that, the speculation is, lost their bearings when they crossed the International Date Line. (Further speculation is that this was somehow connected to a software patch/upgrade related to the coming change in the date of Daylight Savings Time onset, but that’s harder to source.) The planes, en route to Japan, limped back to Hawaii instead

The F-22 costs $125 million or so and its operating system is written in 1.5 million lines of Ada code. It appears that, for all its “stealth” prowess and advanced weaponry, its soft underbelly may lie in the realm of the abstract.

It seems that this is one of the unexpected consequences of living in a world operated by software: new danger zones lie where human abstractions — borders, measurements, languages — change or conflict or fail to behave as expected. Clocks and calendars and maps are no longer just assists for human understanding; they are symbols at the heart of systems upon whose performance lives depend. I suppose this started with the first railway schedule, but with the dateline-addled F-22 it has entered a whole new realm of disconcert.
[tags]f-22, bugs, software[/tags]

Dreaming on the Well

by

The Well is where I first experienced the addictive power of online conversation, almost two decades ago. So I couldn’t be happier that the Well is now hosting an ongoing Q&A with me about Dreaming in Code. Since this is taking place in the Well’s Inkwell forum it’s readable to all (not just Well members), and open to questions from non-members, too (via e-mail — see the discussion for details).

Check it out — Christian Crumlish is hosting the interview, and we’ve already talked about how Dreaming in Code is (or isn’t) like Moby-Dick, and ways in which the Chandler project is (and isn’t) like the Iraq war.

Beginning to see the light

by

I don’t read the East Bay Express as much as I used to since it lost its old-style Berkeley individuality and got swallowed up by the big alt-weekly chain that is now known as Village Voice Media. But I stumbled on this very funny interview in it today, in which a hapless music writer quizzes Lou Reed about his soundtrack for a new Tai Chi DVD.

I was in the audience last fall at the Web 2.0 conference when Reed’s iceberg-like self-possession collided with the tanker of the Web industry elite’s smug self-regard — a fiasco set up through the offices of then-AOL honcho Jonathan Miller, who explained that he and Reed met because they study with the same Tai Chi master. So Reed’s martial-arts enthusiasm didn’t come as the surprise to me that it seems to have for the Express writer.

What do you say to the people who can’t reconcile your classic Velvet Underground druggy image with this healthy New Age one?

“That was forty years ago!” he implores. “This is 2006! 2007! My God! I can’t worry about things like that. If I did, I wouldn’t do anything! I can’t live in 1967 for people. That’s crazy. I have a broader palette. Everything I do, I’ve always tried to do the best that I could as honestly as I could from wherever space I was viewing things at the time. I can’t satisfy everyone, and I’m not trying to.”

[tags]Lou Reed, martial arts, tai chi, dvds, east bay express[/tags]

Teraflop software?

by

Of the many “laws” I encountered in the course of writing Dreaming in Code, I think Wirth’s law (by the software pioneer Niklaus Wirth) is my favorite: Software gets slower faster than hardware gets faster.

Here is a contemporary instance. All right, it’s not exactly parallel; but it’s an example of the very common situation we encounter as hardware improves exponentially while software improves on only a linear basis.

This is from John Markoff’s recent piece about Intel’s demo of a prototype of a new chip-making technique that packs 80 processor cores on a single chip (the “Teraflop Chip”):

The shift toward systems with hundreds or even thousands of computing cores is both an opportunity and a potential crisis, computer scientists said, because no one has proved how to program such chips for many applications.

“If we can figure out how to program thousands of cores on a chip, the future looks rosy,” said David A. Patterson, a University of California, Berkeley computer scientist who is a co-author of one of the standard textbooks on microprocessor design. “If we can’t figure it out, then things look dark.”

Mr. Patterson is one of a group of Berkeley computer scientists who recently issued a challenge to the chip industry, demanding that companies like Intel begin designing processors with thousands of cores per chip.

In a white paper published last December, the scientists said that without a software breakthrough to take advantage of hundreds of cores, the industry, which is now pursuing a more incremental approach of increasing the number of cores on a computer chip, is likely to hit a wall of diminishing returns — where adding more cores does not offer a significant increase in performance.

I wrote about this “multicore competency” issue a couple of years ago. Looks like it’s not going away.

UPDATE: Corrected to fix a (happily) mistaken suggestion that Wirth had passed away.

Damien Cave in Baghdad

by

I worked with Damien Cave for years at Salon, where he did great work for our technology section. Several years ago he decamped for New York and ended up at the New York Times. He’s now reporting from Baghdad.

I’ve been catching up on reading some old papers that I neglected during the frenzy of my book launch. This morning I read his two-week-old piece “‘Man Down’: When One Bullet Alters Everything.” It’s a remarkable bit of eyewitness reporting from Haifa Street in central Baghdad, just outside the Green Zone, where Cave accompanied an American platoon on a sweep. It’s about the difficult choices facing U.S. forces trying to coordinate with Iraqis who are ostensibly leading the mission. It’s about the terrors and horrors facing Iraqi residents of the torn city. But mostly it’s about the choices and emotions encountered by the young American soldiers when one of their sergeants is struck down by a sniper. It is entirely sympathetic to the embattled Americans at the same time it illuminates how futile their effort is.

Perhaps the next time President Bush calls a press conference, the White House correspondents could collectively agree to stop wasting their time asking questions of a leader who will not give truthful answers — and instead, each read a sentence from this article, telling the president a story about what his mistakes have wrought.

Dreaming in Code: sound and vision

by

Today Mark Frauenfelder chatted with me for a while on Skype and then whipped our interview around into a fine little podcast (here’s the direct MP3 link), part of his “Get Illuminated” series. (My interview succeeds talks with Douglas Rushkoff, Rudy Rucker, Loren Coleman and Steven Levy — extraordinary company!)

Mark described Dreaming in Code as “addictively good reading” — a compliment that bears special weight from the founder of the blog that, more than any other, has defined the addictions of the mid-2000s Internet. More important to me, I’m happy to get word about Dreaming in Code out to the devotees of what we used to call “digital culture” (I was calling it that in the San Francisco Examiner way back in the early ’90s, when the term tended to elicit strange looks). It’s as much a book about the culture around software development as it is about programming itself, and that aspect so far has been a little overlooked.

Since we’re talking multimedia manifestations, this is probably a fine place to post links to videos of two of my talks that are now available. Here’s my talk at Microsoft Research, and here’s my talk at Google. Please note that these are very similar 35-40 minute presentations, though the Q&A sessions — and the camera angles — are different. (The Microsoft link, alas, won’t work except in IE, because they’ve done something proprietary in how they’ve hooked up the presentation video and the slides.)

My talks describe how I came to write the book and discuss its themes. They make reference to Ibsen’s Master Builder and the Sumer game, and Ellen Ullman and Link Wray, and various other things. And they explore a couple of points in depth that really aren’t in the book at all: Does Web-based software invalidate the “software is hard” problems? (My answer: it solves many of the previous generation’s problems, but opens us up to a bunch of new ones.) And what would happen if software developers treated the bugs they face daily as opportunities? What if “It’s not a bug, it’s a feature” weren’t a sarcastic joke, but rather a kind of wisdom? (If you don’t want to hear me blab about the book but are interested in these points, you’ll find the Web-based software argument in the Google video at around 17:50, and the discussion about bugs and creativity at 30:00.)

Blog responses to Dreaming in Code

by

Bloggers love Dreaming in Code! Not all of them, of course. But enough to make me proud. Here’s some of the most enthusiastic reaction. (I’m keeping a full index of them in Delicious under the “coderesponse” tag at this page.)

One of my goals in writing the book was to create an account of the work that software developers do that was accurate and entertaining enough for programmers to hand to uncomprehending relatives and friends and say, “Here — here’s what I do all day.” Based on the comments below, for at least some readers, I succeeded. That has made my month.

From JP Rangaswami:
If you’re interested in software development, you should read the book.

From Dan Rabin at Information in Rotation:
Scott Rosenberg’s Dreaming in Code is the best journalistic portrayal of software development that I’ve ever read.

The romantic cliche of the lone introverted genius shaping masterpieces through many midnights of unfathomable incantations is mercifully absent. … We see the process as it actually is: as a highly social undertaking in which people pass through the project, and the project passes through people’s lives. The developers have families, pets, outside interests; they also have passions (often conflicting) about technology and the process of creation.

Dreaming in Code is much more than a simple chronicle: Rosenberg delves deeply into the history of software development and the frustration it causes for its participants and customers as the results never seem to improve even as the underlying hardware undergoes the most rapid progress of any technology ever.

Issues of data representation, storage, and synchronization are front and center in Dreaming in Code, all carefully explained by the author in terms that make sense to the non-practitioner while remaining recognizable to us professionals (he’s really, really good at this).

I might give this book to my mom to read.

From Titus Brown:
…a ripping good tale, and it’s definitely one of the best books on software engineering that I’ve read lately…. I can unabashedly recommend it to anyone who likes a good yarn. Yes, it’s about software development, and you’ll need a fair bit of technical exposure — not experience, just exposure — to navigate the references. But anyone who is reading this, including my not-so-technical friends, should be able to understand it, enjoy it, and even learn from it. Rosenberg’s descriptions of the projects, the people, the technical decisions, the thought processes, and above all the social component of software development are spot on…. I’m seriously thinking of trying to use it as the cornerstone of a software testing course at MSU. The problems encountered by the developers of Chandler, and the narrative that Rosenberg builds around them, could be used to neatly demonstrate step by step just how much a “test-driven” development technique can buy you.

From Reuven M. Lerner at Altneuland:
I only wish that I could make all of my non-programmer clients, friends, and family members read this book; it’s a fantastic introduction to the world of software, and why it’s so hard to do a good job of writing it. I’ve long enjoyed Rosenberg’s writing in Salon, but it was never clear to me just how well he understood the software industry. I think that he has now demonstrated a deep understanding of the difficulties that programmers and technical managers face. In particular, he understands that despite many different languages and methods software engineers have created over the years, it’s still slow, difficult, and expensive to write high-quality software…. I’m delighted that Scott Rosenberg has written such a wonderful book, and hope that many people will have a better understanding of the software industry as a result.

From Kevin at LeanLeft:
Rosenberg uses the individual experiences of the Chandler people to illuminate a given set of theories, weaving their personal frustrations and triumphs as a touchstone to a more academic discussion of the history and qualities of software engineering methodologies. It is a very effective tactic, the difference between telling you that Picasso could paint and showing you a print. This is not just a book for programmers… The soul of Rosenberg’s book is the struggle of the Chandler team members to take what happens in their heads and turn it into software. Understanding that struggle is one of the best ways to come to terms with the failures, compromises, and limitations of the software that runs your life.

Vista’s successor, Longhorn deja vu?

by

Word, in the form of this Infoworld report, is beginning to trickle out from Redmond about the next Microsoft operating-system cycle. In two-and-a-half years or so, Microsoft expects to unveil Vista’s successor (code-named, apparently, “Vienna” — or maybe something else).

The report provides eerie echoes of the early days of Longhorn, as Vista was originally known. At the start of the Longhorn process in 2002 the promise was a similar 2 1/2 year delivery (see for instance this now amusing report from the WINHEC hardware conference in 2002, headlined “Longhorn slips to Late 2004”). In summer of 2003 Bill Gates and other Microsoft spokespeople began telling us about all the cool stuff Longhorn would provide. At the same time, Microsoft buckled down for what would turn out to be a year-long, all-hands-on-deck effort to tighten the security holes in Windows XP; this process resulted in XP’s “Service Pack 2” release in 2004. In 2004 Microsoft realized that the original Longhorn vision was hopelessly out of reach, and it now admits that it “rebooted” the entire development process at this point.

So that’s when Microsoft exec Ben Fathi starts the clock in looking at how long it really takes Microsoft to prepare a new edition of Windows:

Vista shipped about two-and-a-half years after XP SP 2, and Vista’s follow-up is expected to take about the same amount of time, according to Fathi. “You can think roughly two, two-and-a-half years is a reasonable time frame that our partners can depend on and can work with,” he said. “That’s a good timeframe for refresh.”

All well and good. Only when Fathi starts talking about what new stuff Vienna will have to offer users, it all sounds remarkably like what Microsoft had to say in Longhorn’s early days:

So what will be the coolest new feature in Vienna? According to Fathi, that’s still being worked out. “We’re going to look at a fundamental piece of enabling technology. Maybe its hypervisors, I don’t know what it is,” he said. “Maybe it’s a new user interface paradigm for consumers.”

Over on Engadget they’re interpreting this talk to mean stuff like “full virtualization and a radical new user interface” and “a break in compatibility with older applications.”

But to me it all sounds like: “We’re going to do big, big things, but we don’t know exactly what they are yet.” And that is precisely what Longhorn’s leaders were saying as they marched their troops down the roads that would swallow the project’s first 2-3 years.

Lesson learned?
[tags]microsoft, software, windows, vista, vienna[/tags]

Bloggers, Edwards, and transparency

by

Back when blogging was young, one idea its early enthusiasts shared was that blogs would cut through the fog of corporatespeak and give the players in business and politics and other hierarchically organized fields a chance to communicate honestly, openly and directly. Blogs were a means to route around the PR pros and the media intermediaries.

And they do still sometimes achieve that: Look at how Steve Jobs issued his challenge to the music companies to drop their counter-productive stance on “digital rights management.” (It wasn’t a blog posting, but same principle.) As Dave Winer points out, Jobs didn’t hand this as a scoop to a New York Times or Wall Street Journal reporter; he just posted it on his Web site.

All this makes it exceedingly strange to be reading, today in 2007, about the little dustup in the Edwards campaign, where, as you may have heard, two bloggers who’d been hired by the campaign found themselves targeted by the right-wing media for stuff they’d written on their own blogs. Salon reported they’d been fired, but now it seems (see the Salon follow-up) that, after a day of turmoil, the campaign is keeping them on.

What’s strange is that we’re talking about two bloggers here and a campaign that has its own blog; and yet, as far as I can tell, none of their blogs actually tells much of the story of what’s actually happened. There’s a couple of ritual apologies from the two bloggers whose opening paragraphs read like they were written by committee, and an official statement from Edwards that’s similarly impersonal. Were they actually fired? No? What really happened between them and Edwards? Isn’t this precisely the sort of thing a blogger might tell us?

I don’t spend a lot of time in the political blogosphere that Amanda Marcotte and Melissa McEwan hail from. I don’t doubt that they’re basically victims of a witch-hunt. I just find it strange to be reading stories like those in Salon, full of quotes from unnamed “sources close to the campaign” telling us of an apparent inside story about a firing and then rehiring, while the bloggers at the center of the tale — who are, presumably, torchbearers of transparency — don’t give their own readers the scoop on what’s happened.

I suppose some of this is inevitable when the practice of blogging meets the crucible of presidential campaigns. I just wonder what the point of bringing bloggers into the political machinery is unless you let them be bloggers. (This is a variation on the old debate about blogging from inside big companies, which I was pessimistic about several years ago, because I figured it would face similar hurdles.)

No doubt we’re entering a long period of time in which people are going to be forced to ritually abase themselves and disown anything controversial they’ve written on a blog or elsewhere online before they are allowed to participate in the councils of power. And then, down the line — just as we now have presidential candidates who freely admit that, once upon a time, they inhaled — all of this will become somewhat quaint.