Aaron Shapiro

No Vacancies in the Long Tail

Originally published in Notes on Digital, September 21, 2010

Go Google something. Most likely at least one of first three listings—the prime real estate that gets about 63% of the clicks—link to content produced by Yahoo’s Associated Content, Wikipedia, AOL’s Seed, Demand Media and/or About.com. These content factories are churning out gobs of text and video on niche topics that span to the tail end of the long tail. Nearly anything that two people on earth would desire to search for has content to match produced by these companies. Add in the volumes of search-optimized content from local news endeavors like AOL’s Patch and Yahoo’s San Francisco webpaper effort, traditional media properties like Reader’s Digest and History.com, or informational archives like Cancer.org and NIH.gov, and the world of search optimization is starting to be a very cluttered place.

Other than Wikipedia—the non-profit of the bunch—and most famously Demand Media, the content farms produce long-tail content based on popular search terms and the keyword market in order to reap rewards from the long tail of advertising. For example, Yahoo’s news blog, The Upshot, relies on Yahoo’s search query data to guide its content, The New York Times explained. To help create content for the blog, a team of people analyze search patterns and pass along the findings to Yahoo’s news staff.

The money in these types of enterprises comes from the traffic generated when a link then lands at the top of the search results. Dan Roth of Wired wrote about Demand, “The company’s ambitions are so enormous as to be almost surreal: to predict any question anyone might ask and generate an answer that will show up at the top of Google’s search results.” This summer, Demand filed for an IPO. In its discussion of risk, it pointed out its dependence on search engine optimization and advertising revenue.

  • “We have an extensive relationship with Google and a significant portion of our revenue is derived from cost-per-click performance-based advertising provided by Google. For the year ended December 31, 2009 and the six months ended June 30, 2010, we derived approximately 18% and 26%, respective, of our total revenue from our various advertising arrangements with Google.”
  • “SEO involves developing websites to rank well in search engine results…Our failure to successfully manage our SEO strategy could result in a substantial decrease in traffic…which would result in substantial decreases in conversion rates and repeat business, as well as increased costs if we were to replace free traffic with paid traffic.”

The aggressive attack on populating search results threatens to undermine the open fabric of the web. The more search-optimized content, which factories like Demand produce, the harder it will be to dislodge them from their perch. Search results get defined in part by how many people link to the item. Once an article by one of these properties is in the top, it’s hard to knock it loose because the content networks so heavily cross-link to each other. The bigger the cross-linked network gets, the tighter the web holding the top listings in place. That’s why you see so many Wikipedia, eHow and About.com articles on the top of search results, even though similar—if not higher quality—content is available elsewhere on the web.

We’re moving to a future of consistent, unchangeable top-line search results—at least for instructional or evergreen, encyclopedic topics. We’re literally running out of search queries that return meaningful traffic that hasn’t already been taken over by the SEO production machines. Much of this content is of the low-cost production kind, semi-professional journalists paid low wages. It’s not the kind of high-quality, thoughtful pieces produced by professional (paid) journalists that isn’t meant to so algorithmically compete on SEO. A sad state for a technology valued for its democratization of content and content producers.

The problem for users and the opportunity for the content factories lie in the fact that there are only a few ways to reach the long tail. Everyone has to go through the Google or Bing funnel to ultimately reach their point of interest on the everlasting asymptote. Take the top spots for a query and that’s what that particular spot on the long tail becomes.

If search engines are interested in re-opening the web, they may have to adjust their algorithms in a way that upends the SEO real estate moguls. But until then, AOL and Yahoo’s exploration into search-optimized content and local news might be a good idea. If they can’t own search, they might as well try to own the search results.