Code mining

I wrote Dreaming in Code because I believed that, as Bjarne Stroustrup says, “our civilization is built on software.” I noticed that creating software remains stubbornly difficult in certain ways, and, despite its centrality to our civilization, our understanding of that difficulty remains deficient. But I also wanted to create a journalistic record of the day-to-day experience of the software developer at the start of the 21st century — to tell a story about the act of programming itself.

I’m grateful that a good number of the book’s readers who’ve posted their thoughts feel that I achieved that goal. Others don’t think I did, and some days I agree with the criticism. Writing about the act of programming itself is as difficult as writing about any act of writing: the subject is an essentially interior process between the mind and the page (or screen), and it’s highly resistant to illumination.

Consider the difference when the topic of an essay is a rough physical act — like, say, digging coal out of the ground. I read a lot of George Orwell early in my career but I’d forgotten this passage, which Brad DeLong hoisted into the light of blog last year:

Our civilization, pace Chesterton, is founded on coal, more completely than one realizes until one stops to think about it. The machines that keep us alive, and the machines that make machines, are all directly or indirectly dependent upon coal. In the metabolism of the Western world the coal-miner is second in importance only to the man who ploughs the soil. He is a sort of caryatid upon whose shoulders nearly everything that is not grimy is supported. For this reason the actual process by which coal is extracted is well worth watching, if you get the chance and are willing to take the trouble.

When you go down a coal-mine it is important to try and get to the coal face when the ‘fillers’ are at work. This is not easy, because when the mine is working visitors are a nuisance and are not encouraged, but if you go at any other time, it is possible to come away with a totally wrong impression. On a Sunday, for instance, a mine seems almost peaceful. The time to go there is when the machines are roaring and the air is black with coal dust, and when you can actually see what the miners have to do. At those times the place is like hell, or at any rate like my own mental picture of hell. Most of the things one imagines in hell are if there — heat, noise, confusion, darkness, foul air, and, above all, unbearably cramped space. Everything except the fire, for there is no fire down there except the feeble beams of Davy lamps and electric torches which scarcely penetrate the clouds of coal dust.

It should go without saying that a wide gulf separates the strenuous and perilous experience of the “grimy” miners that Orwell depicted and the abstract, cerebral work of programming. The parallels are less obvious — but they jump out at me, too.

Both activities are essential to industry and highly profitable to those at the top of the economic pyramid they support. Both require the exploitation of long hours put in by young workers. In each, the treasures society values are struggled for in dim places and retrieved into the daylight after obscure labors to which their beneficiaries are oblivious.

Writing about a software project couldn’t have been more physically different from descending into a mine. But there were times, during my three years of research, when I felt like I was in an underground labyrinth, hunting for nuggets of insight in the dark.

It was, in any case, striking to find the “Our civilization is founded on…” construction that kicks off Dreaming in Code in Orwell’s penetrating lead. I’ve been trying to trace the Chesterton passage Orwell refers to as his antecedent, but so far no luck. Anyone have a clue?