The Evolution of Web Search

By David M. Ewalt
Forbes.com

As we've become completely immersed in the Internet era, Google has become a verb, librarians are increasingly lonely, and most of us have mastered the basics of Boolean logic--without even knowing exactly what it is. We've become a society of information managers, navigating huge amounts of data with ease and expertly tracking down obscure facts and figures.

But as far as we've come, all we've really done is become good at finding needles in haystacks. There's no sophistication, no wisdom involved, and it's largely because our search tools are pretty dumb.

Imagine you were suffering from a bad case of tennis elbow and wanted to find a doctor who could see you on Saturday. A simple Google (NasdaqNM:GOOG - News) search for "doctors" would find some referral services, but it would also produce pages of doctor jokes and medical associations. More significantly, you'd miss all kinds of "physicians" and "therapists" who might be able to help, simply because you didn't choose that word. Search on "tennis elbow" and you're not going to find help for "athletic injuries." And searching for offices that are "open Saturdays" won't help you find the ones with "weekend hours."

To solve that problem, we need a search system that doesn't just process and parse our language, but understands it; programs that don't just match your search terms but intuitively recognize context to deliver what you're really looking for. Fortunately, engineers and researchers around the world are already at work to bring about this system, and they call it the semantic Web.

Conceived by Tim Berners-Lee, a computer scientist generally considered the father of the World Wide Web, the semantic Web isn't an entirely new network. It's a vision of a world where "tags," or code, is hidden inside Web pages to help computers understand meaning. Individual terms like "doctor" would be tagged with identifying code allowing a program reading the document to refer back to a central dictionary and learn that a "doctor" is the same as a "physician."

But the semantic Web isn't just a fancy thesaurus. It also defines the relationships between words, allowing a program to understand that "price" is measured in "dollars," which can be converted into "yen," and that both of those words refer to different kinds of "money."

The resulting system bears the same relationship to today's Web as a pile of books does to a well-cataloged library. "The World Wide Web, as we know it today, is mostly unstructured content," says Burton Group analyst Peter O'Kelly. "The general idea is to infuse more meaning, try to provide more of a sense of structure about the world."

At its most basic level, a semantic Web would allow search engines to act more intelligently, making it easier to find specific things. That's good news for Web surfers, as well as for the companies who develop search engines, like Google, Yahoo! (NasdaqNM:YHOO - News) and Microsoft (NasdaqNM:MSFT - News).