Wolfram|Alpha
'Seeking Authority: Taming the Wild, Wild Web'

The recent launch of Wolfram|Alpha could be a signal that we are entering the next stage in the development of the Internet - moving from a Wild West frontier mentality toward a more orderly society.

It’s been more than 15 years since the web emerged as a powerful new medium, transformative in its impact.  Two activities have fascinated most of us since then: What can I find on the Internet, and how can I find it? Much of the pleasure and value of the Internet has been in exploring information as part of a planned excursion or more serendipitously.

The early search engines, like Excite where I once worked, tried to provide some order amidst the chaos by employing editors to organize the content of the Internet and present to users the best-of-breed. Yahoo! was not even a search engine at all in its early days but rather a hierarchical directory of websites. But editors and directory organisers could not keep up with the volume of data being added to the Internet, and employing people was expensive. Anyway, who really cared anymore what an editor thought they should read when individuals could decide for themselves?

And they could also contribute for themselves. User-generated content has become incredibly significant, as each of us gains an ability to participate in the dialogue and add to the amalgamation of knowledge now available. Based on this principle, Wikipedia has become the de-facto tool for cataloguing knowledge, enabling anyone to submit or edit entries. The trend toward user-generated content has culminated in the Twitter phenomenon, which has made it simple for anyone to jump in and out of the global, real-time data stream.

Google has emerged as the tool of choice for finding the needle of value in the haystack of human communication now at our fingertips. Recognizing the futility of using humans to catalogue the entire web, Google uses a set of computational algorithms and assumptions to try to bring us results that match our requests. It’s up to us to sift those results to find the items of real value to us.

This works much of the time, but it leads to great frustration also as the stream of information is polluted by websites that add little of value but merely position themselves as knowledgeable. These charlatan sites hide behind a thin mask of relevance to rank highly in Google results, but actually seek only to intercept the pursuit of knowledge so that they can extract some coin from it.

Even Wikipedia, supervised as it is by an army of well-intentioned volunteers, is only as good as the expertise of its contributors. All its supervisors can do is point out the flaws and gaps in each of the entries, or roll back any attempts at self-serving edits or outright vandalism.

But what’s been lacking in these days of the Wild West Web has been an authoritative source that can deliver knowledge that’s been vetted, reviewed and checked – so that by the time we consume it we know it’s completely reliable and accurate.

Wolfram|Alpha may fill this gap. Its aim – to take sets of quantifiable data and make it easy to search and manipulate the data in different ways – is much different from that of Google, Wikipedia or many of the other organizers of knowledge on the web.

Wolfram|Alpha understands that amidst the sea of information we can now access and contribute to, there is still to be found data that is authoritative, empirical and absolute. Much of it is locked away in databases accessible only to those with the right passwords or tools – and some of this will be surely unlocked over time. But much of it is simply buried under the tons of information of varying degrees of quality that we must all sift through every day. It’s like panning for gold nuggets in a muddy creek.

Google provided us with the pan. If Wolfram|Alpha works as intended, it may be the one to provide us with the gold bars. There is so much information out there, but little authority. Sometimes we need someone to just give us the answer.

Google says, “Take a look at these results I came up with and see if they’re any good.” Wikipedia says, “Here’s the information we were able to assemble on this subject, as accurate as we could get it.”

And that’s fine in many cases. As my old colleague Peter Bengelsdorf, formerly of Newsday, recently pointed out, “Wikipedia is great for the everyday-life questions that don't require particularly authoritative answers. Like, who is that rock star I never heard of?”

But for answers that must be exact, whom do we go to? Wolfram|Alpha is trying to say, “Here is exactly what you wanted to know.”

To be sure, Wolfram|Alpha has a big challenge ahead. Their aim “to collect and curate all objective data; implement every known model, method, and algorithm; and make it possible to compute whatever can be computed about anything” is nothing if not lacking in ambition. Yet finding the expertise needed to understand and organize highly specialized data sets will be tricky and costly. They’re going to have to do it a little bit at a time.

But at least they’re trying. Thanks to Wolfram|Alpha, the Wild, Wild Web may finally have a sheriff. Or at least a librarian.