I’ve seen lots of criticism of Google Books, but I find Google Scholar to be more frustrating. Google Scholar tends to be something of a last resort for me. It’s where I go when I’ve tried everything else and hope that the keyword searching will pull up something with at least some relevance that might have been missed in standard indexes. Usually I’m disappointed. For example, I was looking for scholarly information around a controversy within the International Churches of Christ, specifically regarding a controversial letter criticizing the organization and the aftermath. Here’s the Google Scholar search.
There are eight results, only half of which might count as scholarly. The book about God and karate could be considered scholarly. Another is a 4-page article from Leaven: the Journal of Campus Ministry, which I wouldn’t consider scholarly in the way that, say, the Journal of Religion is scholarly, and it has no references, but it’s sort of scholarly. Another is a link to a PDF of “Discipling Sisters” at the University of Georgia’s institutional repository, which wasn’t working at the time. By searching their OPAC and following links, I discovered it was a 2007 dissertation. Finally, success! Except that the only mention of the guy I was looking for uses a Wikipedia article as the source of information. Failure! The only other link that is at all scholarly is to a master’s thesis in the digital commons at McMaster University. That’s scholarly, but a master’s thesis is pretty low down the food chain for scholarly secondary resources. On the other hand, no Wikipedia articles are cited. One actual book, one questionable article, and two theses. Half the search results were sort of relevant.
The other links are not. Two are links to the same article from two different websites, spirtualpornograpy.com and reveal.org, both of which are anti-ICOC websites, so there’s some obvious bias and article is definitely not scholarly. There’s another link to a lecture housed at douglasjacoby.com, which is a Christian ministry site. How did they end up there? The only thing I can think of is that they’re all in PDF format. Does Google assume that anyone who can save a document to the web in PDF format is a scholar? Finally, there is a link to a Christianity Today article, only it’s to a Russian website instead of to the Christianity Today. Not scholarly, and possibly bootleg. Three non-scholarly websites and a bootleg news article. Half the results weren’t remotely relevant.
A broader search for ICOC alone brings more results, and with more results, there is a larger number of actual scholarly sources. However, buried in those results are numerous questionable sources, like PDFs from icocinvestigation.org, whose subtitle is “exposing the International Churches of Christ.” At least their bias is obvious. There’s also gospelpreaching.com, willofthelord.com (both linking to the same non-scholarly article), starringjesus.com (which doesn’t exist anymore), and regainnetwork.org, whose “mission is to outreach, unite and support those touched or adversely affected by the Legion of Christ and Regnum Christi Movement.” These might all be great websites, but there’s nothing remotely scholarly about them.
It’s like Google Scholar is deliberately putting in non- or quasischolarly material just to make us have to evaluate the information more. Instead of filtering out the nonscholarly stuff littering the Internet, which is what I thought Google Scholar was supposed to do, it clutters up the results with dubious sources based on a questionable search algorithm. The best I can figure is that if a source on the Internet is in Google Books, is in PDF format, or has any citations, Scholar seems to consider it scholarly.
On the other hand, it’s a good exercise to discuss Scholar with students who want a quick fix when searching for scholarly sources. Do a search and start evaluating the source with even the most cursory criteria for scholarship and it’s pretty easy to show what is and what is not scholarly and why. That’s typically necessary when searching the open web, but it’s the sort of thing I wish one didn’t have to do with something like Google Scholar. There are no royal roads to research.