Wordyard

Hand-forged posts since 2002

Archives

About

Greatest hits

Berkeley Cybersalon

by

This coming Sunday, March 4, come on down to the Hillside Club, where, from 5 to 7 p.m., at the Berkeley Cybersalon, we’ll be talking about why software is (so often) so hard.

For this event, practically in my backyard and as part of what must be the longest-running technology salon around (I started going over a decade ago), I didn’t want to just give another spiel about Dreaming in Code.

Instead, it’s a panel discussion, with Eric Allman of Sendmail (program and company), Chad Dickerson of Yahoo’s Developer Network, and Lisa Dusseault of CommerceNet (and formerly of the Open Source Applications Foundation).

It was originally scheduled for yesterday, 2/25, but we realized that counter-programming against Oscar Night was foolhardy — even for an event as geekish as this.

Dreaming on the Well

by

The Well is where I first experienced the addictive power of online conversation, almost two decades ago. So I couldn’t be happier that the Well is now hosting an ongoing Q&A with me about Dreaming in Code. Since this is taking place in the Well’s Inkwell forum it’s readable to all (not just Well members), and open to questions from non-members, too (via e-mail — see the discussion for details).

Check it out — Christian Crumlish is hosting the interview, and we’ve already talked about how Dreaming in Code is (or isn’t) like Moby-Dick, and ways in which the Chandler project is (and isn’t) like the Iraq war.

Teraflop software?

by

Of the many “laws” I encountered in the course of writing Dreaming in Code, I think Wirth’s law (by the software pioneer Niklaus Wirth) is my favorite: Software gets slower faster than hardware gets faster.

Here is a contemporary instance. All right, it’s not exactly parallel; but it’s an example of the very common situation we encounter as hardware improves exponentially while software improves on only a linear basis.

This is from John Markoff’s recent piece about Intel’s demo of a prototype of a new chip-making technique that packs 80 processor cores on a single chip (the “Teraflop Chip”):

The shift toward systems with hundreds or even thousands of computing cores is both an opportunity and a potential crisis, computer scientists said, because no one has proved how to program such chips for many applications.

“If we can figure out how to program thousands of cores on a chip, the future looks rosy,” said David A. Patterson, a University of California, Berkeley computer scientist who is a co-author of one of the standard textbooks on microprocessor design. “If we can’t figure it out, then things look dark.”

Mr. Patterson is one of a group of Berkeley computer scientists who recently issued a challenge to the chip industry, demanding that companies like Intel begin designing processors with thousands of cores per chip.

In a white paper published last December, the scientists said that without a software breakthrough to take advantage of hundreds of cores, the industry, which is now pursuing a more incremental approach of increasing the number of cores on a computer chip, is likely to hit a wall of diminishing returns — where adding more cores does not offer a significant increase in performance.

I wrote about this “multicore competency” issue a couple of years ago. Looks like it’s not going away.

UPDATE: Corrected to fix a (happily) mistaken suggestion that Wirth had passed away.

Dreaming in Code: sound and vision

by

Today Mark Frauenfelder chatted with me for a while on Skype and then whipped our interview around into a fine little podcast (here’s the direct MP3 link), part of his “Get Illuminated” series. (My interview succeeds talks with Douglas Rushkoff, Rudy Rucker, Loren Coleman and Steven Levy — extraordinary company!)

Mark described Dreaming in Code as “addictively good reading” — a compliment that bears special weight from the founder of the blog that, more than any other, has defined the addictions of the mid-2000s Internet. More important to me, I’m happy to get word about Dreaming in Code out to the devotees of what we used to call “digital culture” (I was calling it that in the San Francisco Examiner way back in the early ’90s, when the term tended to elicit strange looks). It’s as much a book about the culture around software development as it is about programming itself, and that aspect so far has been a little overlooked.

Since we’re talking multimedia manifestations, this is probably a fine place to post links to videos of two of my talks that are now available. Here’s my talk at Microsoft Research, and here’s my talk at Google. Please note that these are very similar 35-40 minute presentations, though the Q&A sessions — and the camera angles — are different. (The Microsoft link, alas, won’t work except in IE, because they’ve done something proprietary in how they’ve hooked up the presentation video and the slides.)

My talks describe how I came to write the book and discuss its themes. They make reference to Ibsen’s Master Builder and the Sumer game, and Ellen Ullman and Link Wray, and various other things. And they explore a couple of points in depth that really aren’t in the book at all: Does Web-based software invalidate the “software is hard” problems? (My answer: it solves many of the previous generation’s problems, but opens us up to a bunch of new ones.) And what would happen if software developers treated the bugs they face daily as opportunities? What if “It’s not a bug, it’s a feature” weren’t a sarcastic joke, but rather a kind of wisdom? (If you don’t want to hear me blab about the book but are interested in these points, you’ll find the Web-based software argument in the Google video at around 17:50, and the discussion about bugs and creativity at 30:00.)

Blog responses to Dreaming in Code

by

Bloggers love Dreaming in Code! Not all of them, of course. But enough to make me proud. Here’s some of the most enthusiastic reaction. (I’m keeping a full index of them in Delicious under the “coderesponse” tag at this page.)

One of my goals in writing the book was to create an account of the work that software developers do that was accurate and entertaining enough for programmers to hand to uncomprehending relatives and friends and say, “Here — here’s what I do all day.” Based on the comments below, for at least some readers, I succeeded. That has made my month.

From JP Rangaswami:
If you’re interested in software development, you should read the book.

From Dan Rabin at Information in Rotation:
Scott Rosenberg’s Dreaming in Code is the best journalistic portrayal of software development that I’ve ever read.

The romantic cliche of the lone introverted genius shaping masterpieces through many midnights of unfathomable incantations is mercifully absent. … We see the process as it actually is: as a highly social undertaking in which people pass through the project, and the project passes through people’s lives. The developers have families, pets, outside interests; they also have passions (often conflicting) about technology and the process of creation.

Dreaming in Code is much more than a simple chronicle: Rosenberg delves deeply into the history of software development and the frustration it causes for its participants and customers as the results never seem to improve even as the underlying hardware undergoes the most rapid progress of any technology ever.

Issues of data representation, storage, and synchronization are front and center in Dreaming in Code, all carefully explained by the author in terms that make sense to the non-practitioner while remaining recognizable to us professionals (he’s really, really good at this).

I might give this book to my mom to read.

From Titus Brown:
…a ripping good tale, and it’s definitely one of the best books on software engineering that I’ve read lately…. I can unabashedly recommend it to anyone who likes a good yarn. Yes, it’s about software development, and you’ll need a fair bit of technical exposure — not experience, just exposure — to navigate the references. But anyone who is reading this, including my not-so-technical friends, should be able to understand it, enjoy it, and even learn from it. Rosenberg’s descriptions of the projects, the people, the technical decisions, the thought processes, and above all the social component of software development are spot on…. I’m seriously thinking of trying to use it as the cornerstone of a software testing course at MSU. The problems encountered by the developers of Chandler, and the narrative that Rosenberg builds around them, could be used to neatly demonstrate step by step just how much a “test-driven” development technique can buy you.

From Reuven M. Lerner at Altneuland:
I only wish that I could make all of my non-programmer clients, friends, and family members read this book; it’s a fantastic introduction to the world of software, and why it’s so hard to do a good job of writing it. I’ve long enjoyed Rosenberg’s writing in Salon, but it was never clear to me just how well he understood the software industry. I think that he has now demonstrated a deep understanding of the difficulties that programmers and technical managers face. In particular, he understands that despite many different languages and methods software engineers have created over the years, it’s still slow, difficult, and expensive to write high-quality software…. I’m delighted that Scott Rosenberg has written such a wonderful book, and hope that many people will have a better understanding of the software industry as a result.

From Kevin at LeanLeft:
Rosenberg uses the individual experiences of the Chandler people to illuminate a given set of theories, weaving their personal frustrations and triumphs as a touchstone to a more academic discussion of the history and qualities of software engineering methodologies. It is a very effective tactic, the difference between telling you that Picasso could paint and showing you a print. This is not just a book for programmers… The soul of Rosenberg’s book is the struggle of the Chandler team members to take what happens in their heads and turn it into software. Understanding that struggle is one of the best ways to come to terms with the failures, compromises, and limitations of the software that runs your life.

WordPress’s Pyethon trap

by

One of the most drudge-like tasks of writing a book is assembling the endnotes, but if there is anything even more tedious, it is converting those endnotes into HTML. But I promised readers of Dreaming in Code that I would do so — so many of the references are Web-based, it makes a lot of sense to provide them with working hyperlinks.

I’d hoped to have this done before the book was published, but I’ve only just finished. Ed Yourdon wanted me to do it; how could I let him down?

But in posting this material, I discovered something very odd. Sometimes WordPress simply did not want to upload a page (I’m using WordPress’s static page feature for the whole Dreaming in Code site). At first I thought I’d hit some undocumented limit on the number of characters, or maybe number of links, on the page; or, I thought, there’s so many links on these pages, maybe it’s overloading the cool WordPress pinging/trackback stuff, so I just turned all that off.

No go. Certain pages just would not save properly, the browser would just hang. By removing chunks of text I slowly zeroed in on the problem: three notes that contained the word “Python,” as in the programming language, were causing the trouble. If I removed them, no problem!

I just tried to post one of them here as an example and this instance of WordPress won’t allow it either — it generates a “file not found” message, oddly. A quick hunt through WordPress’s documentation “codex” offered no clue. Anyone have an idea? Is WordPress’s dedication to php so intense that it will not even allow a mention of the competition?

I couldn’t even use the word “Python” in the headline of this post — it caused the same error! I had to misspell the forbidden name in order to get this post to publish properly. Very odd.

UPDATE: Thanks to the commenters who pointed me to apache’s mod_security, which is plainly to blame, and not wordpress itself. A little creative escaping of the “P” in “Python” and I now have restored the previously unpostable endnotes. Live and learn…

Salon coverage, Portland anecdotage

by

Salon is where Dreaming in Code started, and so it’s fitting — and wonderful — that my colleagues are featuring the book there now. There’s a Q&A with me by Andrew Leonard (with beautiful art by Mignon Khargie, another one of Salon’s founding gang) and a brief excerpt from the book as well.

Tonight I returned from Portland from the final trip of my two-and-a-half-week quasi-pseudo book tour. I moderated a panel on “new media” at the American Booksellers’ Association Winter Institute, which gathers the people who run independent bookstores and tries to equip them better to deal with increasingly volatile times in their biz.

As I told the crowd, I found it funny that we’re still calling this stuff “new media” — a term that first came into vogue 15 years ago, when it typically referred to CD-ROMs. It was great to try to peer into the crystal ball with the other panelists — Amanda Edmonds of Google, Madeline McIntosh of Random House and C.J. Rayhill of O’Reilly.

I was to be introduced by a gentleman who is the CIO of Ingram Book Co., the mega book distributor based in Tennessee who supplies many of those book stores with their stock. He quite nicely asked me if I had a copy of Dreaming in Code that he could hold up to the crowd; foolishly, I was not carrying one. Unwilling to sacrifice this tiny opportunity for promoting the book, I dashed across the street to a humongous mall that contained a branch of a certain giant chain bookstore — and there, somewhere behind giant stacks of Bob Woodward bestsellers and the “Cosmo Kama Sutra,” I found one precious copy of my book, which I purchased and carried triumphantly back to the hotel.

When I handed the volume to the Ingram exec, he noticed the reference to The Soul of a New Machine among the blurbs on the back. “That’s one of my favorite books!,” he declared, and asked me if he could keep Dreaming in Code. The coals-to-Newcastlishness of the proposition struck me as amusing — Ingram is, as its site says, “the world’s largest wholesale distributor of book product” — yet what could I say but “Of course”?

More soon. Tonight, I sleep.

Mace’s podcast, Kedrosky’s plug

by

It is now two weeks to the day from Dreaming in Code’s in-store arrival, and I’m still trying to catch my breath. I’m going to write some fuller posts responding to some of the questions and criticisms I’ve received both in person and online. And I promise I’ll get back to Code Reads soon! Here, first, are a couple of quick pointers to more recent coverage and interesting stuff:

Scott Mace, whose Calendar Swamp blog has been an invaluable resource, interviewed me early on during my media marathon for the Open Source Conversations podcast, and the interview is now available. This was one of the deepest and fullest (and longest!) interviews I’ve done to date; many thanks to Scott for his enthusiasm for the book and his willingness to dig into some of the details.

Paul Kedrosky is both extremely smart and, I tend to think, at the harder-nosed end of the tech-pundit spectrum, so I was delighted to read his post calling Dreaming in Code “excellent.” “Rosenberg’s new book,” he wrote, “is sobering and required reading for anyone naive enough to still expect miracles from large-scale software development.” Thanks, Paul.

“Code” on Marketplace, in Washington Post

by

While I was stuck with my family at an airport for hours today waiting for the airline to figure out how to start the plane’s engines (no kidding), elves were at work extending my national sway. Or at least, I could say, work I had previously finished was making its way toward the public.

The Marketplace interview is online now, here. And I’ve got an op-ed in tomorrow’s Washington Post. Both of these are pegged to the occasion of Windows Vista finally heaving itself across the finish line.

Dreaming in Code on Marketplace

by

If all goes as planned, today’s edition of Marketplace, the Public Radio business show, will include an interview with me about the book, Windows Vista, and the challenges of making software. I had an enjoyable talk with Kai Ryssdal, the host, last week. I’ve been traveling with my family this weekend, but I’ll be posting more soon…