Mark Penn’s fuzzy pro-blogging stats

I did a lot of digging around in the numbers around blogging for my book, so I’m on alert when I read a piece like Mark Penn’s look at pro blogging in the Wall Street Journal, which is getting lots of attention this morning. A little skepticism is definitely in order.

Here’s the nub of hard numbers in Penn’s piece:

The best studies we can find say we are a nation of over 20 million bloggers, with 1.7 million profiting from the work, and 452,000 of those using blogging as their primary source of income. That’s almost 2 million Americans getting paid by the word, the post, or the click — whether on their site or someone else’s.

Where do these numbers come from?

“20 million bloggers” links to a 2008 report from Emarketer that costs $695 if you actually want to know how they got their numbers (I confess I haven’t made the investment).

“1.7 million profiting” links to a promotional page for BlogWorld Expo that cites no source at all for its data.

“452,000 of those using blogging as their primary source of income” is drawn from a Mediabistro rewrite of numbers from Technorati’s State of the Blogosphere reports. Technorati’s are the longest-running and most valuable, and consistent, series of blogging studies over time, but like any study’s numbers, they can be easily misrepresented: here, Penn relies on them for the datum that bloggers who reach 100,000 uniques a month can earn $75K a year. But if you read the source, you find this:

The average income was $75,000 for those who had 100,000 or more unique visitors per month (some of whom had more than one million visitors each month). The median annual income for this group is significantly lower — $22,000.

In other words, the $75K average is skewed by a handful of outlier successes, but the great majority of bloggers who get 100,000 uniques/month earn more like $22,000. Here, the median is far more relevant than the average. Penn, of all people, knows this.

Later on, Penn’s piece cites other sources, including a Pew study and this iLibrarian post which references a 2008 study by an outfit called BIGResearch. The BIGResearch study particularly flummoxed me as I was researching my book, and in email correspondence with a company representative I got to the root of the oddness of their numbers: Their study defined “blogger” as, basically, anyone who writes or reads a blog. That’s one way to muddy the waters!

The methodology of Penn’s piece seems to be: gather as many numbers as you can and don’t worry about the fact that they are from many different sources at different times using different methodologies and even differing definitions of what it means to “be a blogger” — just toss them all together and start drawing conclusions. Those conclusions, in turn, seem to be based on a misapprehension that bloggers are by definition opinion writers. Many are, to be sure; but many others — particularly in the “pro blog” world Penn focuses on — concentrate on becoming expert sources in a particular area, or informational services, or link reviews.

My suggestion to Penn (who — full disclosure — I briefly worked for, decades ago, during my college years, when he was starting his company): You should commission a real study of blogging, using real sampling techniques, and share the results with the world. No one has done this yet that I’m aware of. You know how to do it! And we’d get a lot better information than this crazy-quilt pastiche of mix-‘n’-match stats.

UPDATE: Penn has posted an addition to his column that goes into more detail about the numbers. “I was surprised at how few studies there are on this,” he writes, “and I believe there definitely should be more. So perhaps in the future I will do some original research, but for this piece we took the best we could find and referenced every number so people would know where they came from.”