I gave a workshop Tuesday on "emerging search technologies," by which I meant roughly searching for just about anything on or via the Internet using means that went past just the text-based websearching of Google or Yahoo. Thus I mentioned sites like Grokker and Hakia and a new (to me) tool, Chunkit. We spent a lot of time on so-called "social search," ranging from Wikia and Mahalo to Worio and Sidestripe (which work with Facebook Connect) to Aardvark, a tool so new and hip I’m not even allowed to use it yet.
It’s enjoyable to see what some obviously very clever people are developing every day, and I had a great time researching the workshop, but I was also struck by how much wasted effort there is in any innovation. I’m not sure I have enough of a social network to take advantage of things like Sidestripe or Aardvark, though they look pretty nifty. But who really needs Stumpedia? "Human powered search"? What were they thinking? Is this going to be any improvement over the old but still useful concept of a web directory? I don’t see how. Or Truevert, the "green" search engine? Is anyone really likely to find more informtion about composting toilet systems on this than on Google or Yahoo? Or Delver, with the dubious claim that "your friends know best." With all due respect to my friends who are reading this, I’m not sure you really know best about any topic I’m likely to be searching the Internet for, and I’m almost positive that I don’t know best about whatever you’re searching (unless you happen to be searching for stuff about me, in which case I probably do–but what are the chances?).
Even sites I kind of like pitch themselves as solving a problem I don’t have. Consider Rollyo, which allows you to create customized search engines a lot more easily than Google Coop. Rollyo asks, "Are you tired of wading though thousands of irrelevant search results to get to the information you want?" To which I’d have to answer, nope. Google does a pretty good job of giving me relevant sites on the top page. Or, "ever wish you could narrow your search to sites you already know and trust?" Almost never, to be honest. Those sites I just go to directly, usually using Google Bookmarks.
Obviously I’m not the target audience for search engines and sites that claim to solve the problem of too many irrelevant resources, but I do wonder how many people really are these days. If people can’t get relevant search results on Google or Yahoo, how likely is it they’re going to do much better asking their friends for help with Aardvark or Sidestripe? Chances are that my social network, such that it is, contains a lot of Internet savvy people, and if I actually had a question, someone might be able to point me to something I hadn’t discovered. But it’s at least possible that the people who are the least Internet savvy are going to have an entire network of unsavvy friends, none of whom can help them.
I’m not even sure how much the people using Rollyo can be trusted. I searched one specialized search engine for guitar tablature, and noticed that it doesn’t have Chordie, which is far better at finding guitar tabs than the Rollyo engine and any of the websites it searches.
I was thinking about the waste because I remember reading a few months ago about some controversy regarding libraries building innovative search tools to rival Google, and wondered how much of our effort we might waste doing things like that. It’s not because I think that out of this waste good things won’t emerge, because I believe they will. It’s just that there are so many people out there wasting a lot of time and money and effort to come up with the next new thing that it seems hard enough for most of us just to keep up with what’s already going on. There absolutely has to be wasted effort to produce useful innovations. I guess I’m just glad there are a lot of clever working on things like this so I don’t have to.
it is essential that libraries build reference tools. these days, that includes search tools. but the search tools we build will not be the ones they build, because we have different goals.
for example, amazon’s recommendation engine suggests books you might like based on your past buying and browsing habits. it is focused on selling you stuff and is a neat idea, but it doesn’t play for us – we need that recommendation engine that helps you explore a topic thoroughly, and the one that helps you make the best choice to become a better citizen. libraries’ work on these problems is not wasteful, it speaks to our mission.
commercial search tools are focused on sales – some sell the results themselves, some sell ads, but all of them have something to sell. most of them are following google’s lead of ‘popular means better’. libraries make no such claim.
there’s a reason we put those damn cards in alphabetical order and i think it is important not to forget it: the meaning of any given information query is based as much on context as it is text.
we are not going to out-google them, but i’ll be damned if i let them out-library us.
(also i am not cool enough for aardvark either)
I’m not necessarily disagreeing with you, Caleb, but I’m not sure that the profit motive of these companies means they aren’t trying to do some of the same things. Grokker and other clustering engines are trying to lead people more effectively through a maze of resources, and these same techniques are now showing up in the faceted search results of some OPACs and library databases. Google Coop and Rollyo have definitely library applications. Hakia is trying to develop search based on recommendations of librarians as well as some sort of quasi-semantic search. A big part of our jobs these days is to sort through the tools out there and pick up what we think will work well for our own purposes.
Good article!
My company, http://www.deepwebtech.com/, has been working hard on this problem. We are still trying to figure out what the right visualization tools are, to help people find contextual-based information from the deep web.
The clustering mechanism appears to be quite helpful, but there are limitations, especially when you begin to introduce highly specialized topics or foreign languages.
One area we’re beginning to explore, is combining the features introduced by social networking with the analytical tools you see with Amazon and other shopping websites. For example, giving researchers the ability to rate and provide comments on specific articles and sources of information. To provide helpful suggestions like, “People who viewed this, also viewed …,” with the appropriate filters and sorts to limit scope.
We’re not there yet, but working on it diligently. We have a few public websites we’re using to evaluate user patterns and satisfaction, to continually evolve our product. If you’re interested, check out http://www.science.gov/ , http://www.mednar.com/ , and http://www.biznar.com/ .
Larry.
Waste certainly leads to innovation. And disruptive innovators are the visionaries, the starry-eyed fools who believe when no one else will.