Wordyard

Hand-forged posts since 2002

Archives

About

Greatest hits

Buggy BART

by

People ask me what my book (Dreaming in Code) is about, and I usually answer, “It’s about software…” And, if their eyes don’t glaze over immediately, I’ll add, “…and why software is still so hard. Why it’s always late and it’s always breaking. Why we’re 50 years into the computer age and we still don’t know how to make it reliably.”

By this point, one of two things will have happened: either listeners will have nodded and smiled and said, “I know exactly what you mean!” Or their eyes will have eventually glazed over, after all, and they’ll look at me a little quizzically, as if to say, oh really? And why does this matter? What do I care?

I thought about those people as I passed through the BART turnstiles this morning, a little glazed-eyed myself. There, neatly by the attendant’s booth, lay piles of orange flyers under a “BART BULLETIN” letterhead. I grabbed one and read it on the escalator-ride down.

It was an apology for the screwed-up state of BART yesterday morning — which had seen half-hour delays and incorrect train-destination signs. How considerate! A mass transit system that apologizes to you! In my many youthful years of New York City straphanging, I can’t say I ever had that experience.

But this is the paragraph that caught my eye:

  BART technicians believe the delays were caused by new computer software that was installed over the weekend. The new software has been removed and the software that was previously in use has been re-installed. Although the new software was repeatedly tested before installation, it failed in the demanding real-world environment of a weekday morning commute.

BART had botched a software upgrade. It had plenty of company in that experience, of course.

As C++ creator Bjarne Stroustrup puts it: “Our civilization runs on software.” BART does, too. And understanding why software remains so balky — a topic I happen to find fascinating in the abstract — also has some everyday, pragmatic interest.

UPDATE: And how. I just tried to get on BART for my ride home this evening but could tell from the milling crowds outside the Embarcadero station something was radically wrong. Walked down the stairs to catch a garbled announcement on the PA: “We have closed the gates… no trains are moving… computer problems…”

I’m grateful for the consideration in illustrating my point, but I’d really rather just be on my way home!

Windows Vista: no escape from software time

by

Last September the Wall Street Journal ran a fascinating lead article about Microsoft’s Vista development effort. Robert Guth chronicled how the Vista project had initially ballooned as Bill Gates and others piled on their dream features, like the advanced, metadata-rich WinFS file system. When Vista hit trouble, Windows czar Jim Allchin brought in two software development experts, Brian Valentine and Amitabh Srivastava, to whip the project into shape by introducing rigorous new testing methodologies.

Still, by mid-2004 the whole project was in danger of collapsing. Microsoft decided to postpone Vista till “the second half of 2006” and cut back lots of promised features (including WinFS).

As Guth’s article had it, the result, finally, was a development process Microsoft could begin to be proud of:

On July 27 [2005], Microsoft shipped the beta of Longhorn — now named Windows Vista — to 500,000 customers for testing. Experience had told the Windows team to expect tens of thousands of reported problems from customers. Instead, there were a couple thousand problem reports, says Mr. Rana, the team member.

When I read the article at the time, I took it as a kind of victory-lap valedictory for Allchin, who’d announced he was retiring once Vista was done. I also read that many people have already begun checking out Direct Components Xilinx fpga price list for a faster software running process, but there are still some companies which are reliant on the non-developmental softwares as it saves their initial and current capital. Unless you’re certain of prevailing, though, victory laps are dangerous (just think of the phrase “Mission Accomplished”). With this week’s news of a another slip in the Vista schedule — the software won’t be out until January 2007, after the crucial holiday buying season — we’re left wondering, what happened to that vaunted new process?

Certainly, this widely linked story that claims Microsoft is now going to rewrite 60 percent of the operating system between now and release seems hard to credit (something tells me rewriting that much code would take a lot more than 8 months). But between this embarrassing delay and the recently announced “reorg” of Windows leadership, it’s clear that this turn of the Windows cycle is going to be no smoother or predictable than any of its predecessors.

My book, Dreaming in Code, is all about what I call “software time” — the peculiar spell that software projects so often cast on the people involved, turning schedules into Mobius strips and stretching time like taffy. I imagine that, as Valentine and Srivastava described the beauty of their testing systems to Guth last year, they honestly believed that they’d meet their deadlines. They thought they’d cheated software time. That confidence doesn’t look too smart today.

UPDATE: Steve Gillmor wonders whether maybe there really is 60 percent of the Vista code that needs a rewrite — and much more. Adam Barr, on the other hand, offers some reasons why that notion might be far off-base.

[tags]Dreaming in Code, Microsoft, Windows, Vista[/tags]

Horn tooting

by

Somehow I missed the fact that this humble blog made CNet’s “Top 100” list some time ago, in the “Tech Business” category. Thanks, CNet! I’ll try to live up to the billing. I’ve generally adopted the “blog is a mirror of the blogger” philosophy here; rather than trying to niche-ify the blog as a product (everything you always wanted to know about ROUTERS!), which is certainly a good business strategy, I’ve allowed the blog to reflect my own interests across tech, politics and culture. I can fairly well guarantee you won’t find much sports or celebrity news here — but beyond that, almost anything goes.

I have some new ideas and approaches to the blog that I intend to start experimenting with when work on my book, Dreaming in Code, is done. The handful of you who have been paying attention may recall that I completed a first draft of my manuscript before the holidays. Well, we’re still in the middle of editing, and making progress. I hope to be done soon, and when we are, I’ll be posting a lot more about the book and related subjects (basically, software development and its discontents). For those of you who’ve inquired in e-mail, wondering as to my health, or the health of the project: sorry for the slowness. Publishing, like software, has its own rhythms.

In other news on the “self” front, I am proud to see this site’s homely name over at the Electronic Frontier Foundation on a short list of blogs that helped that group with a membership drive last November. It’s fine company to be in. (And if you didn’t join back then, you can still join now.)

Programmers and cuisine

by

I’ve spent a lot of time attempting to fathom the mindset and thought patterns of software engineers in the course of my labors on my book. Two sites that reflect the collision of programmers and cuisine are fascinating in their own right: First, there’s Cooking for Engineers, which applies an analytical mindset to techniques for cooking bacon or making Rice Krispies treats. (Shades of the Twinkies Project!) After indulging in such fare, you can have a look at the Hacker’s Diet, in which Autodesk founder John Walker explains the essential similarities between computing systems and the human body (garbage in, garbage out; calories in, calories out).

Time flies when your writing’s fun

by

Well, I’ve researched and researched, and written and written, and now I must revise and revise. When I actually have a manuscript turned in to my editor — before Thanksgiving, for sure! — I will exhale and write a bit here about the process, and how things have turned out. I should also be blogging a bit more henceforth. Thanks for bearing with me through this hiatus.

In the meantime, I should mention that Salon is celebrating its 10th anniversary this week. It’s hard for me to believe that ten years ago I was a disaffected ex-theater critic, technology columnist and fledgling HTML adept making the jump from print to online. I knew it was risky, and the fact that our money-making plans were strictly theoretical made me think, okay, this will be fun for a year or two, then Salon’s likely to go down the tubes and I’ll make a living freelancing.

After a fun year became two, then three, then four, I moved over from my job as technology editor to managing editor, taking on a lot of responsibility for things like budgets and site management. And I began to think, hey, this thing might actually last! Whoops — cue the bursting of the dot-com bubble. The moment I started getting confident, suddenly it really did look for a while like Salon might go down.

But by then I was hooked. I was determined to see it through, as was everyone else who stuck it out, and I put my back into a lot of difficult and creatively unfulfilling work to do my part to help Salon survive. It was only last year that I felt comfortable enough about the company’s stability that I could think about taking a book leave this year without feeling like I was abandoning ship. Things are definitely on the upswing, but, you know, I’m a little wary of feeling too confident. Old wounds and all.

By now you may have seen Gary Kamiya’s history of the site — it’s hilarious, deftly captures much of the heart and soul of Salon through the years, and brought back a torrent of memories for me. My angle on some of the history might have been different: Gary’s less focused than I’d have been on Salon’s place in the evolution of the nascent field of Web publishing; he’s more immersed in consideration of Salon’s place in the general milieu of political and literary journalism. I always worked one step closer than him to the business side of things, and many steps closer to the technical side of things. My version of those aspects of the story may be a little less Stranger in a Strange Land and a little more In the Belly of the Beast.

When I’m a little less written out I may trot out some of those tales. Or maybe I’ll just wait till Salon’s 20th.

For now, as I near my own milestone of a finished book, I toast all my current and former Salon colleagues, and look forward to rejoining them in January. More on that, soon, too.

Alan Kay: “Generate enormous dissatisfaction”

by

I am entering the final sprint of completing a first draft of my book between now and Thanksgiving or so, so pardon my general bloggy sluggishness. My plan is to resume somewhat more active blogging in December and return in full blast by January.

In the meantime, here’s something that caught my eye:

One of the computing pioneers whose work I’ve had the pleasure of digging into for my book is Alan Kay. In the course of my research I had occasion to read Kay’s epic account of The Early History of Smalltalk. Smalltalk is the object-oriented programming language Kay created in the early 1970s at Xerox PARC (while he was also inventing much of the rest of modern computing). The paper is full of interesting stuff, but this observation near the end, about how to motivate yourself to tackle difficult challenges, jumped out at me:

  A twentieth century problem is that technology has become too “easy”. When it was hard to do anything whether good or bad, enough time was taken so that the result was usually good. Now we can make things almost trivially, especially in software, but most of the designs are trivial as well. This is inverse vandalism: the making of things because you can. Couple this to even less sophisticated buyers and you have generated an exploitation marketplace similar to that set up for teenagers. A counter to this is to generate enormous disatisfaction with one’s designs using the entire history of human art as a standard and goal. Then the trick is to decouple the disatisfaction from self worth — otherwise it is either too depressing or one stops too soon with trivial results.

“Generate enormous dissatisfaction” with one’s work — well, gee, that’s something most ambitious people know how to do, one way or another. But such dissatisfaction quickly blossoms into neurotic self-doubt. Ergo Kay’s careful recommendation to “decouple the dissatisfaction from self-worth”: that’s genius. And, I might add, really, really helpful to anyone laboring over a big project like, say, a book.

Of course, this means that you have to figure out other bases for self-worth than the work one has generated enormous dissatisfaction with!

Google’s Windows-only world

by

Jason Kottke’s intriguing review of the current status of the Web-as-platform question (are Web apps now good enough to threaten the primacy of a certain desktop operating system monopoly? will they ever be?) is only the latest in a long line of musings on this theme that stretch all the way back to Netscape’s heyday. The dream of rendering individual users’ choice of desktop operating system irrelevant by getting them to move all their significant work into the browser was what fueled all those death-march development cycles during the browser wars.

Microsoft cut off Netscape’s air supply — with plenty of help from its victim’s own asphyxiating mistakes — before the browser company could complete building all the parts of this new computing world. Java was supposed to be an alternate road to the same destination; it turned out to be good for some other things, but not for that.

So we lost a few years there.

More recently, the Web-app universe has come roaring back, as GMail, GoogleMaps, Flickr and other Ajax-based Web interfaces have provided users with something speedier and more interesting than the old, slow, click-and-wait world of Web computing. It is possible, today, to begin moving more and more of one’s work and data into browser-accessible stores and programs. This is all great, and it’s unfolding with a kind of inevitability.

For a while there, during the downturn years, it seemed like the Web-based future might arrive without any one company driving it. The new structure of our technology would simply be built by a swarm of lilliputian enterprises that would gradually overwhelm the Gulliver of Redmond.

Suddenly, though, it looks like we’re back in the land of corporate showdown. In a wave of media reports, Google is being cast as the new Netscape — reluctantly, to be sure, since Netscape showed how dangerous it is to say to a company with an effectively bottomless warchest, “Bring it on!” Rather prematurely, I think, a lot of people quoted by Gary Rivlin in this morning’s Times suggest that Google is already the new Microsoft — that the company with the “don’t be evil” motto has morphed into a new evil empire.

Wherever you place Google on this spectrum, there’s no other way to read Google’s latest moves than as part of a broad effort to bring users onto Google’s platform so that, one day, they can be moved off Microsoft’s. That day is doubtless far off. But not unimaginable.

Google’s decision to raise $4 billion more on Wall Street, timed almost certainly not coincidentally to coincide with its release of two new software products (a new desktop application and a new “Google Talk” IM and voice communicator), reinforces the message first sent by GMail: that, when Google defines its mission as “organizing the world’s information and making it universally accessible and useful,” “the world’s information” very much includes your own personal information.

Which leads us to the paradox here. There is one little weakness in the theory that Google is setting out to challenge Microsoft. For some reason, each time Google releases any software that is not browser-based — whether it’s Google Desktop, or Picasa, or the new Google Talk — it has offered only a Windows version of the product. No Mac versions, no Linux versions.

Maybe Google feels that the Mac already offers a rich software environment for geeks (with good desktop search already built into the latest OSX) and Linux isn’t a big enough desktop market. Maybe they just target Windows because, to paraphrase the old bank-robber line, “that’s where the users are.” Or maybe they’re targeting Windows users precisely because they want to woo Microsoft addicts on their own turf.

No doubt, it would take a lot of extra work to release editions of Google software for non-Windows platforms. Cross-platform development is enormously difficult: that’s a fact of software life. (Browser-based software is so attractive because you don’t have to worry about writing different versions for different operating systems; the browser makers have already done that heavy lifting for you.) I always understood this intellectually, but now, after several years of following the work over at OSAF for my book, I feel it in my bones.

But Google has assembled a vast reserve of computer-science horsepower. It is, if Rivlin’s story is to be believed, sucking Silicon Valley’s software brains dry. Surely, with all that coding prowess, Google could set aside some cycles to offer non-Windows users equal access to the cool toys it is providing. If the Googleplexniks are serious about that phrase “the world’s information,” they need to look beyond the realm of Windows. The world doesn’t stop where the “Start” menu ends.

A Web birthday

by

Dave Winer just turned 50. Here’s a happy birthday to send his way.

I first met Dave over a decade ago at one of Sylvia Paull‘s digital salon gatherings. I didn’t know what to make of him but he was clearly smart and creative, as well as argumentative. I crossed paths with him again in the days of the San Francisco Free Press, when he pitched in with some code to help us post our stories during the newspaper strike. I joined his DaveNet mailing list and read his Wired column. In 1999, after I wrote about blogs (before we called them that), he came to visit us at Salon and urged us to start using his new Manila software, but we’d just built our own content management system and couldn’t face trying to integrate a whole different platform. Then Dave started writing about something called RSS, and at first I didn’t understand what the fuss was all about. Later I did.

I think I’ve been reading Dave’s stuff longer than I’ve read anyone else online, watching as he developed new ideas in full public glare, made mistakes and made history. At first, there was something unique about a software developer stepping forward and saying, “I’m tired of journalists messing up my story — I’m going to tell it my way.” Now there are thousands of programmers doing the same thing; the Web hums with their conversations. As I work on my book I’m tuning in to a lot of them, amazed at how much understanding is unfolding in this abstruse and hitherto cloistered field — and how great an example the programmers are setting for other groups going down the same road. I don’t think these conversations would be happening in quite the same way without Winer’s difficult, challenging, inspired example.

Long time no blog

by

Apologies — I’ve been digging into the work of actually writing some chapters of my book, and something had to give. Since the kids still expect to be tended and entertained, the blog had to be left neglected and bored. I could hear its whimpers from the corner, but I had to harden my heart. (The kids scream louder, anyway.)

Conveniently, I no longer need to explain the deep psychological reasons behind my choice not to blog a whole lot of the work of my book, because Steven Johnson has said it all. Johnson has a new book coming out next month, Everything Bad is Good For You, a defense of pop culture that I am looking forward to reading as a reward to myself once I finally finish the half-dozen books on software disasters I am working my way through.

Pattern precognition

by

David Weinberger is posting interesting notes from a high-powered TTI Vanguard conference here in SF.

This post on Eric Bonabeau’s presentation caught my eye:

  Eric Bonabeau of Icosystem says that we tend to believe that the future is present in the mass of data and we just need to find it. But we use existing patterns to search the data, which can’t turn up new patterns. Humans are amazing pattern-detecting machines but we’re terrible at exploring alternatives. Eric suggests selective breeding: make new combinations, look at the results, pick the most interesting results, recombine them, etc. He does a live demo that discovers a “hidden bagel” in a 50-dimensional financial data set. (I have no idea what that sentence means once it gets past the word “bagel.”) It helps not to know what you’re looking for.

I’ve heard Bonabeau, at the O’Reilly Emerging Technology conference (coming up again soon), give a talk on emergent behavior (among ants and in other places) and he’s always fascinating. This notion — that, since “we use existing patterns to search the data”, we’re blind to new patterns — has echoes and analogues all over the map, all the way from the little copy-editing exercise that Dave Winer periodically challenges us with to the disastrous failure of American policy in the lead-up to the Iraq war (a mistaken pattern so potent that much of the population still believes in a patently untrue scenario). Our minds fall into existing patterns like wheels in a rut; we “see” words spelled right even when they’re misspelled, and we “see” events unfold according to the sequence we expect, even when the information parading across our eyeballs tells us otherwise.

What interests me is applying this insight to journalism. We all know the hack’s drill: First you decide what your story is, then you go out and find the facts and quotes to fit. The sad truth today, however, is that this approach isn’t just for hacks: Most reporters in most newsrooms don’t have the time, the freedom or the resources to cover most stories in any other way. This is one of the big causes of the much-vaunted credibility crisis we hear so much about in the blogosphere and all of its attendant conferences. For an astonishingly high percentage of professional journalists, the news they recognize is the news that fits the pattern they have already selected as the template for their coverage. This keeps working until real news starts dancing in front of their eyes — and they miss it. (For instance, the story of the ringer reporter from Talon News was sitting right under the eyes of the entire Washington press corps. But it took a swarm of Web-based sleuths operating collaboratively across multiple blogs to piece the strange saga together and discredit President Bush’s friendly plant in the White House press room.)

“It helps not to know what you’re looking for” ought to be the proud banner of the journalistic generalist — the writer who can step into any situation, ask the right questions and get out with clear explanations for the rest of us. But reporters have tight deadlines and editors telling them what they want and, often, instincts dulled by years of repetition — and all but the very best and most creative end up coming out with what they went in for, rather than something unexpected.

One of the pacts I’ve made with myself as I’ve worked on my book is: Don’t go in with the story. Give myself the freedom not to know what I’m looking for. It’s less efficient, everything takes longer, there are blind alleys and extra interviews. But, if I’m persistent and lucky, maybe I’ll end up with something other than the same old patterns.

Newspapers and magazines and Web sites aren’t, can’t be, books, of course. They’ll always have deadlines; they’ll never have enough bodies or money or time. And more and more, the news that they miss is being covered by amateurs and solo operations on the Net. So forward-looking people in the business are all abuzz about new approaches to “citizen journalism” and “we media” and other ideas for harnessing the Internet’s many-to-many dynamics. Books are being written (congratulations to Jay Rosen, who has announced one that I will look forward to); new ventures planned.

Will some hybridization of old media and the new swarm dynamic help more hidebound publications find their souls again? I’m optimistic about the creative possibilities, skeptical about the business realities. But I don’t doubt that journalists could learn a lot from Bonabeau’s ants.