Google And The Need For Speed

David A. Utter, Staff Writer, webpronews:

If the scientists who claim news content has a half-life of 36 hours online before it falls off the radar of Internet users are correct, Google's successor online might match up freshness and relevance.

I remember overhearing a conversation between a bank teller and a manager one time, where the teller asked if the manager wanted something done fast or wanted it done well. The manager, of course, said "both."

After posting a story about articles being unseen by a prospective audience about 36 hours after they have been placed online, I received an IM from our publisher, Rich Ord.

There's nothing like seeing an instant message popup from one's publisher at 7:15 in the morning. It does the work of a Jolt Cola and six Pixy-Stix when it comes to waking me up.

Rich commented on the story, noting how the shorter life span of articles could be a disadvantage when it came to indexing them. Even with the use of Google Sitemaps, Google News still updates a lot more rapidly that the main Google index.

That Bigdaddy of an index is a lot larger than Google News, since it has a lot more sources to spider and more being added all of the time. Google does a good job of keeping pace, but what happens if the next search engine to come along can reasonably mimic Google's algorithms and grab content faster?

Blog search engines like Technorati, Sphere, and even Google's own Blogsearch all pick up new content at a brisk pace. It could even be argued that the rapidly growing blogosphere presents more of a challenge than conventional websites do when it comes to indexing.

To mate speed with performance would require a lot of servers and a lot of bandwidth. With its new headquarters under construction, the purchase of a switching facility in New York City, and its ongoing acquisition of dark fiber throughout the United States, one company could pull this off.

That would be Google.