Wordyard

Hand-forged posts since 2002

Archives

About

Greatest hits

Meanwhile, back at OSAF…

by

Anyone who reads Dreaming in Code through to the end is going to want to know what happened at the Chandler project in the time since the conclusion of the book’s narrative (it ends at the end of 2005, with Chandler at version 0.6, ready to begin some limited “dogfooding,” or use by inhouse early adopters).

Some of the early reactions to my book have presented Chandler as a total bust and proceeded on the assumption that the project is dead. That’s not at all the case. For the moment, Chandler remains a program that most people aren’t going to download and use, and it’s still not going to break any speed records. But it plainly has made steady progress over the past year. OSAF is now planning what it’s calling a “preview” release in April.

A lot more of the project’s big-picture features are now at least partially implemented — particularly the Dashboard, a sort of universal “inbox” for sorting tasks and calendar events and email according to “Getting Things Done”-style principles.

I sat down with Katie Parlante, Sheila Mooney and Mitch Kapor right before Christmas to get an update on what had happened at OSAF with Chandler during 2006.

[Read more…]

Notes from the actual road

by

In the past several days I’ve been talking nonstop about Dreaming in Code to radio hosts and bookstore crowds and a roomful of people at Microsoft Research in Redmond.

I am continually delighted by the interest people are showing in the book and its subject. Most wonderfully, I am finding that the radio folks are actually reading the book (or, you know, some of it, anyway!). I know how crazed their work schedules are, so that tells me that either (a) there are a lot of closet programming geeks out there in the radio world or (b) maybe I achieved some of my goal of taking this dauntingly arcane subject and making it approachable for people outside the field.

Tomorrow it’s Google for me, then I’ve got a long-planned family vacation over an extended weekend. But I’ve also got a backlog of stuff to post — so I’ll try to get to it all soon.

Notes from the quasi-road

by

Thanks to all the people who showed up at Kepler’s Thursday night to hear me talk about Dreaming in Code. It was my very first event as a published author, and it was great to see so many interested faces and hear so many intelligent responses. I’m even getting the hang of signing books as a left-handed person (binding gets in the way, ink smudges, etc.). In the Murphy’s Law category, my car battery died on University Ave. in Palo Alto where I’d stopped for coffee beforehand; fortunately, I knew my way around well enough to hike to Kepler’s in time to make an only slightly sweaty appearance. (Triple A took care of getting the car started for the trip home to Berkeley.)

Yesterday I had a similarly invigorating event at Yahoo — what seemed like more than 100 Yahoo folks brought in their lunches and heard my talk. I’ve got several former colleagues and longtime blogger acquaintances at Yahoo, and I was glad to have the chance to talk about the book, and software, to such a knowledgeable, and attentive, crowd.

Tomorrow (Sunday) I’m out at the Pleasanton Library at 2 p.m. in an event sponsored by Towne Center Books, so if you’re in the (farther) East Bay and interested, come on by!

Ed to Amanda to George

by

First, Ed Cone was reading my book so he could interview me. Then, Amanda Congdon was dropping in on Ed to record a promo for the ConvergeSouth conference. Congdon was thumbing through Dreaming in Code at the start of the promo, so they worked in a little reference to the book. I found it amusing and posted it on my blog.

Now George Coates and his dramatic crew at BetterBadNews have taken this brief video clip and deconstructed it in a bizarrely funny way. “‘Dreaming in Code’ is probably one of those rare works of literature of the sort that you really have to read to enjoy,” the deadpan announcer begins. By the time the commentators have done picking the clip apart — “A dog? A guy without a head?” — we’re in David Lynch-land.

This tickles me in multiple ways, partly because I know Coates’s work from many years of covering the multimedia extravaganzas his theater company used to present, but mostly because I love the process by which this little meme has propagated — and now, mutated. (Thanks to Dave Winer for the link.)

Lessons from MySpace: Success is a bug

by

Apropos of my previous post on YouTube and MySpace, today I read this fascinating case-study from Baseline magazine about the saga of MySpace’s understandably overtaxed systems.

MySpace’s exploding popularity has basically forced its infrastructure through a continuous cycle of upgrades, refactorings and revampings. Its managers have never had the luxury of sitting back and calmly planning upgrades; they’ve had to perform their engine surgeries on a careening vehicle.

This is what Web 2.0 is like from the back end, and it ain’t pretty. Outside of the real masters of this stuff — the Googles and Yahoos that know how to deploy, manage and maintain vast online services — it’s a big mess. This is another little-understood dynamic of the Web 2.0 startup world: There are financial reasons a successful small service might want to be acquired, but there are even more pressing operational reasons. And the more success a service finds, the more likely it’s going to risk systems flameout.

It’s not at all clear from the Baseline piece that MySpace has yet achieved a level of stability that a more mature company might desire. MySpace, of course, was acquired not by a technology company but by a media outfit, so — unlike other popular companies that were acquired by Yahoo or Google — they’re still somewhat on their own.

The Baseline piece offers two other fascinating tidbits. In the first, a normal phenomenon for a successful site — massive surges of traffic — was interpreted as a bug by the Microsoft server platform MySpace uses:

Last summer, MySpace’s Windows 2003 servers shut down unexpectedly on multiple occasions. The culprit turned out to be a built-in feature of the operating system designed to prevent distributed denial of service attacks—a hacker tactic in which a Web site is subjected to so many connection requests from so many client computers that it crashes. MySpace is subject to those attacks just like many other top Web sites, but it defends against them at the network level rather than relying on this feature of Windows—which in this case was being triggered by hordes of legitimate connections from MySpace users.

“We were scratching our heads for about a month trying to figure out why our Windows 2003 servers kept shutting themselves off,” Benedetto says. Finally, with help from Microsoft, his team figured out how to tell the server to “ignore distributed denial of service; this is friendly fire.”

Second, it seems that MySpace didn’t actually originally intend to allow the level of customization that has made it so popular; its engineers just never got around to filtering out the user-customized formatting.

That feature was really “kind of a mistake,” says Duc Chau, one of the social networking site’s original developers. In other words, he neglected to write a routine that would strip Web coding tags from user postings– standard feature on most Web sites that allow user contributions.

The Web site’s managers belatedly debated whether to continue allowing users to post code “because it was making the page load slow, making some pages look ugly, and exposing security holes,” recalls Jason Feffer, former MySpace vice president of operations. “Ultimately we said, users come first, and this is what they want. We decided to allow the users to do what they wanted to do, and we would deal with the headaches.”

Here we have the state of Web development today: Your site’s massive success gets treated as a bug by your server; and the feature your users love best is something your programmers forgot to block.
[tags]baseline, myspace, web 2.0, software development[/tags]

Appearing now, on dead trees!

by

Today, Dreaming in Code should be arriving in stores. Amazon has officially switched it from “available for preorder” to “in stock now.” And I’ve swapped out the “coming soon” banner on the book’s Web site for a “now available” sign.

For a writer whose career has arced from weekly newspapers to daily newspapers to round-the-clock Web sites, the transition to writing an old-fashioned book has been one big exercise in delayed gratification. By the time I started writing about the project on my blog, I’d already been working on it (researching and preparing a proposal) for almost two years. From that point to first draft was another year and half, and then more than one year further to get from draft to finished-book-in-your-hand. In this business, impatience doesn’t pay.

The extended timeline does provide many opportunities for reflection, and one of the things I’ve kept returning to is how utterly essential to my work the blogosphere has been. Of course I interviewed lots of people the old-fashioned way. But the fact that so many software developers now use the Web as an open notebook allowed me to explore the subject far more deeply and more widely than if I’d needed to track down and talk to each one of those programmers in person.

So thanks to every developer who’s posted thoughts on his or her work — you’ve made my work easier, and better. And thanks to all of you here who’ve followed along the escargot-paced progress of this project. During its multi-year course, writing here — and knowing that a bunch of great, smart people were reading and responding — helped keep me sane.

Now it gets fun. I’m not doing a world tour, a fact for which I and my family are grateful. But I’ll be making a number of public appearances here in the Bay Area, as well as some in-house events at some companies. Here’s the list (they’re also on Upcoming and Eventful):

Thursday, Jan. 18, 7:30 p.m.: Reading at Kepler’s Books, Menlo Park

Sunday, Jan. 21, 2 p.m.: Pleasanton Library (cosponsored by Towne Center Books), Pleasanton

Wednesday, Jan. 24, 12:30 p.m.: Reading at Stacey’s Books, San Francisco

I’m also doing presentations at Yahoo, Microsoft (Redmond) and Google over the next couple of weeks. If you work at one of those places and want more info, just let me know.

We’ll see what I learn from the people at those events, who undoubtedly will know more than I do, and I’ll try to bring some of it back to the Berkeley Cyber Salon on Feb. 25, where I’ll be hosting a panel on the themes of Dreaming in Code.

Does Web 2.0 invalidate Rosenberg’s Law?

by

A book should stand on its own, reviews should stand on their own, and in general, it little profiteth a writer to reply directly to criticism. But at the end of his review of Dreaming in Code in the Journal (which is now available for free at this link), Paul Boutin asked me a direct question. So it doesn’t seem out of line to answer.

The author reluctantly condenses what he’s learned into Rosenberg’s Law: “Software is easy to make, except when you want it to do something new.” Cute, but MySpace and YouTube went from half-baked ideas to billion-dollar businesses while Mr. Rosenberg was writing his book. Is he saying they were hard, or that they do nothing new?

There’s no question that creating software using the Web as a platform provides multiple advantages, and has helped lots of companies, including those Boutin names, roll out products fast and improve them quickly. Web apps can be upgraded and patched without troubling the user and they come with a tight built-in user-feedback loop, so they lend themselves perfectly to an incremental improvement process, which is the smartest way to avoid the software-disaster swamp. They also solve cross-platform issues and provide sharing and collaborative features as an organic part of their environment.

This change in the software landscape is a running theme in Dreaming in Code, and one that the Chandler developers were acutely conscious of. But in today’s Cambrian explosion of Web-based applications, I can’t really think of each service as a discrete piece of software. The teeming startups of Web 2.0 are really each adding a feature or two — photo-sharing! backup! social bookmarking! word-processing! etc. — to the vast application that is the Web itself. (That’s why they make such fine acquisition-bait for the larger Web-platform companies who are competing with arsenals of such features.)

I do not intend any denigration of the huge accomplishments of the creators of both MySpace and YouTube, and I have no idea how hard or easy their development efforts were. But I think of these companies’ achievement as more in the realm of community-building than software invention. Of course they had to develop some software along the way. But their software platforms — for enabling the hang-out-together experience (at MySpace) or the sharing of short videos (on YouTube) — were not the centerpiece of their achievement. Each company refined and elaborated functions already being performed by other services. Each of them triumphed not by inventing novel extensions to the world of stuff you could do on the Web but by making things people were already doing online much easier, particularly for non-geeks.

As for their creation of “billion-dollar businesses”: remember that these are businesses that haven’t seen anything like billion-dollar revenues; they were valued by acquirers at half a billion dollars (in MySpace’s purchase by Fox) and $1.5 billion (in YouTube’s acquisition by Google)– and much of these prices was paid in stock, not cash.

Still, Boutin’s point is that both companies produced significant value quickly, and I’m not going to argue with that. But in each case what the acquisitors valued was less the code than the customers. You can be idealistic and call that “community,” or be cynical and call it “eyeballs”; either way, it’s what Google and Fox were buying.

In any case, my tongue-in-cheek Rosenberg’s Law made no claim to address the relative ease or difficulty of making money in the software business. I was trying to say something about the correlation between the ambition to innovate and the likelihood of ending up with your wheels in a ditch. MySpace wasn’t the first online hangout, and YouTube wasn’t the first video-sharing service. Instead, each of these companies went to school on their predecessors over the last decade of social media on the Web. And good for them! I imagine that what they did was “hard” in its own way — just not in the specific way that Dreaming in Code explores.

Some early returns

by

Dreaming in Code isn’t in stores till Tuesday, but there’s already been quite a bit of coverage.

Today saw Paul Boutin’s review in the Wall Street Journal (subscriber-only content, I’m afraid). I’m very happy the Journal chose to assign my book to a writer with Boutin’s experience and expertise in the software world. Also pleased with the review’s length and prominence. And I loved the illustration — a Sisyphus shouldering a boulder of code.

One thing Boutin noted made me smile as I read it: “His goal seems to be to teach non-programming managers not how to fix late projects, but how to accept them.” I can just see myself hanging out the guru shingle and peddling the Zen of project management!

In truth, my goal wasn’t to teach anyone anything. I neither promise nor deliver bullet-points of how-to advice. For me, the best non-fiction provides readers with an initiation into the complexity and fascination of a world that they barely even knew existed. I’m happiest when I hear from a reader who feels that I accomplished at least some of that.

But certainly I came to the conclusion, based on the evidence, that late software projects are — maybe not forever, but probably in our lifetimes — a fact of life. They are like quarreling among children or rebelliousness in adolescents. These are things we can mitigate, certainly, and learn to cope with better, or even sometimes turn to our advantage. But we are foolish if we think we can “fix” them or eliminate them. Sanity dictates some sort of accommodation with this kind of reality.

Boutin also concluded the review with a question to me about the principle I jokingly dub “Rosenberg’s Law” in the book. I actually have a lot to say about in response. But I must leave that for a post of its own!

Scott Berkun, who unlike me really does have a lot to teach people about how to avoid the pitfalls of bad project management, has also written up a measured and well-reasoned response to the book. And Rick Kleffel has some entertaining thoughts on the topic even before reading the book:

One of the most interesting aspects of my interview with Vernor Vinge was the point where we talked about what might suggest that we were or were not going to experience the Singularity in a manner described by most science fiction writers. Vinge suggested that the failure or success of large-scale software development projects would be a fine way to measure our progress towards the Infocalypse. Judging by the events described in ‘Dreaming in Code’, well, it appears that the Singularity Is Not Quite So Near as one might hope, if one were to hope It Is Near.

Lulu forecasts, Amanda approves

by

Lulu, the innovator in online self-publishing of printed materials, has an amusing little tool up at their site: you plug in a title and it tells you that title’s likelihood of becoming a bestseller. Is it an algorithm? Is it magic? Or is it just a silly online trick? I don’t know, but here’s what I learned about my prospects of bestseller-dom — far better than I’d expect for a book about software development and programming!

While we’re goofing around, here is a very brief clip recorded by celeb videoblogger Amanda Congdon. After she left the Rocketboom gig but before she graduated to the major leagues, she stopped in to hang out with Ed Cone, who had my book on his desk because he was preparing to interview me. Ms. Congdon picked up the book, and here’s what she had to say. (The full-length original clip that contains this excerpt is here. YouTube embedding seems funky in some browsers so: here’s the direct link to the clip on the YouTube site.)

[tags]dreaming in code, amanda congdon, lulu, ed cone[/tags]