Alternatives to Deception

A couple of posts ago I took a stance that was apparently controversial. That’s not like me. I usually save my controversial opinions for lunchtime conversation after making sure I’m not being recorded surreptitiously. After I criticized lies and deception in fake reference, someone very rightly asked if I meant just the particular type of deception that particular library school student tried to use on me, which had nothing to do with assessment as such, or did I instead mean to question the value of all so-called unobtrusive reference assessment that makes use of such deception. Just to clarify, I am definitely questioning the value of such assessment, and indeed do not believe that the end (producing a research article that might or might not be useful) justifies the means (lying to and deceiving people). I believe such practices are ethically suspect, as should be clear by now.

The commenter, Steven Chabot, rightly notes that “unobtrusive evaluation of reference services is a generally accepted methodology when investigating questions of the quality of reference service. Are we then to say that all of these useful studies completed by actual librarians and scholars in the field are wasting librarians’ time?”

Such deception is indeed a generally accepted methodology, but I think it should not be. Fraud is fraud, and I don’t see how the means justifies the end here. If the end is vitally important and can be achieved by no other means, then just maybe, but such is not the case here. Such lies and deception are ethically unsound and are unnecessary to boot.

And yes, they are a waste of librarians’ time, which is why it doesn’t surprise me that every one of these unobtrusive studies that I’ve read has been conducted by non-librarians. Perhaps we should have librarians posing as fake students in library school courses evaluating the teaching effectiveness and feedback on assignments. Then we can all have a discussion on the ethics and effectiveness of deception.

He apparently had a similar assignment in library school, and “had to cite relevant other unobtrusive studies, such as the classic by Hernon and McClure (1986) which posited the whole ’55 percent rule’: that only 55% of transactions are satisfying to the user. How are we to improve that statistic without precise measurement of it first?”

Here we get into tricky ground, indeed. I have to disagree on so many levels. Perhaps this is heresy among librarians, but I will boldly state first, that I don’t think the so-called “55% Rule” tells us much about the state of reference in any given library; second, that I don’t think such studies in general provide a “precise measurement” of anything useful; and third, that there are ways to assess reference without resorting to lies and deception.

What follows is primarily an excerpt from an annotated bibliography I wrote on reference assessment a couple of years ago. If you want to read the whole thing, you can find it here:

“Best of the Literature: Reference Assessment.” Public Services Quarterly. 2: 2/3 (July 2006), 215-220.

Part of my opinion of the 55% Rule, which I never completely trusted, was formed by the following article:

Hubbertz, Andrew. “The Design and Interpretation of Unobtrusive Evaluations.” Reference & User Services Quarterly 44, no. 4 (Summer 2005): 327-35.

Hubbertz provides an excellent, sustained critique of the normal methods of unobtrusive evaluation of reference services, arguing that for the evaluations to be useful and meaningful the subjects need to be given uniform tests, that the results need to be interpreted to provide a comparison rather than an overall assessment of reference service quality, and that the one area in which such observations may be useful is to evaluate the ways libraries organize their collections and deliver services. His analysis of various published studies of unobtrusive evaluations shows them to be inconsistent and “for practical purposes, nearly worthless.” Not administering uniform tests “may be a principal culprit for these perplexing and disappointing results.” He criticizes in particular the domination of the “55 percent rule,” arguing clearly that such evaluations are designed specifically to generate middle range results, and in fact test reference questions that almost no one or almost everyone answers are excluded from the evaluations. Thus, the evaluations are designed to generate something like a 55% success rate. Hubbertz amusingly shows how we can design the tests to improve the rate of reference success. While middle range results may be useful for comparing the services of different libraries or different ways of providing reference service in the same library, they are useless for determining the overall quality of reference service. He concludes that in the future unobtrusive evaluations may have some use, but they “must be properly implemented, with a uniform test and an adequate sample and [their] application must be limited to the assessment of how best to manage library resources.”

Another article questioning the use of deceptive (err, unobtrusive) evaluation is the following article:

Jensen, Bruce. “The Case for Non-Intrusive Research: A Virtual Reference Librarian’s Perspective.” The Reference Librarian 85 (2004): 139-49.

Jensen argues against applying typical methods of unobtrusive reference evaluation to virtual reference services, because of both practical and ethical concerns. Practically, having pseudo-patrons ask fake questions online does not take advantage of the wealth of transcripts of virtual reference questions available to researchers. Ethically, such evaluation is “an irresponsible misuse of the time of librarians and research assistants” and can degrade the service, though, he notes, “there will always be researchers convinced that their own work somehow trumps the work and lives of the people under study.” This argument both further develops and contrasts with that of Hubbertz, developing the ethical critique of unobtrusive evaluation more and extending the criticism to the evaluation of virtual reference, but not considering the problems with typical unobtrusive evaluation of traditional reference services. He concludes with a call for more research on virtual reference that takes advantage of the wealth of transcripts available, shares the research findings with the objects of study, and does not attempt to deceive virtual reference librarians with pseudo-patrons and false questions.

Curiously, Jensen deems acceptable such methods to evaluate traditional reference services as “the price that must be paid for an intimate view of the reference desk from the user’s side.” Only here do I disagree with Jensen, since I don’t believe deception and time-wasting are worth the price to be paid.

Arnold and Kaske give us an example of such a study based on transcripts:

Arnold, Julie, and Neal Kaske. “Evaluating the Quality of a Chat Service.” portal: Libraries and the Academy 5, no. 2 (2005): 177-193.

Arnold and Kaske establish a clear criterion by which to evaluate their chat reference service: providing correct answers. Using the categories of reference questions supplied by William Katz in his Introduction to Reference Work, the authors analyze 419 questions in 351 transcripts of chat reference transactions at the University of Maryland and provide a model for assessing the value of that service. After coding and classifying the questions, they studied what types of users (students, faculty, other campus persons, outsiders, etc.) asked which types of questions (directional, ready reference, specific search, research, policy and procedural, and holdings/do you own?) and how often those users got a correct answer. Policy and procedural questions topped the list of almost all user groups and represented 41.25% of the total, followed by “specific search (19.66 percent), holdings/do you own (15.59 percent), ready reference (14.15 percent), directional (6.24 percent), and research (3.12 percent).” “Students (41.3 percent), outsiders (25.1 percent), [and] other UM individuals (22.0 percent)” asked the bulk of the questions, and the librarians staffing the service answered the questions correctly 91.72% of the time. Different user groups tended to ask different types of questions. Since other studies of reference transactions have claimed that reference questions are correctly answered about 55% of the time, the authors conclude that future research should study this apparent discrepancy. However, in light of Hubbertz’s study the discrepancy may be less puzzling.

Thus, it would seem that I’m certainly not the only one who believes that deception is ethically tolerable for assessing chat reference. However, there’s still the reference desk. Is deception ethically tolerable there? Certainly not. But is it even necessary?

For an alternative to the deceptive model of reference desk assessment, see the following article:

Moysa, Susan. “Evaluation of Customer Service Behaviour at the Reference Desk in an Academic Library.” Feliciter 50, no. 2 (2004): 60-63.

Moysa describes in a concise and readable article the process used by her library to evaluate their librarians’ customer service behaviors. Basing its criteria upon the ALA Reference and User Services Association’s “Guidelines for Behavioral Performance of Reference and Information Services Professionals” (1996) [ed. note: rev. in 2004, referenced above], the reference department used a combination of self-assessment and observation. Moysa considers both the ethical problems of unobtrusive evaluation and the practical problem that normal observation affects behavior. She concludes that the literature indicates that observation over a sustained time eliminates many of the negative practical effects and notes that having the reference staff participate in the process of creating this evaluation model from the beginning mitigates most of the ethical objections. Moysa has described a method of evaluation and assessment that deliberately avoids lies and deception, and for the reference desk at that, so it would seem that we both disagree with Jensen that deception is the price we pay for reference assessment.

Thus, there are other ways to assess reference. Then the question becomes, how are we to improve the quality of reference. Rather than (or at least in addition to) these sorts of ethically sound assessment tools, we should spend much more time thinking about the education, training, and culture of reference, and especially of the proper character required of a good reference librarian. If we have reference librarians with the proper ethos, the character appropriate to their profession–educated, intellectually curious, driven by a desire and equipped with a capacity to solve information problems, practiced in the appropriate ways to respond to various audiences, adaptable to changing circumstances–and a culture that supports them, then we won’t need such reference assessment, because good reference will take care of itself.

The Ethics of Fake Reference

My last post was written in a fit of pique, and understandably someone took issue with part of how I expressed my frustration. Meredith Farkas commented that “To call the student an ‘obvious liar whose intent is to waste my time’ is really awful.” That’s quite possibly true and I should have been more gentle. However, there are other things that are awful as well. Lying is awful. Deception is awful. Using people as unknowing participants in human experimentation is awful. Treating people merely as means to your own ends is awful. Betraying the implied agreement behind a reference query is awful. I opined that sending out students to ask fake reference questions is an ethically dubious practice. Today I would like to elaborate by briefly exploring the issue. I’ll approach the practice from several ethical perspectives and you can decide whether you agree with me.

First, let’s consider the Kantian perspective. You may be familiar with Kant’s categorical imperative, which says that one should act in such a way that one can will the principle behind one’s action to be universal law. Basically, don’t do it unless you think everyone should do it, or in a briefer version of the Golden Rule, do as you would be done by. This principle would, I think, preclude lying, deception, and using people as unknowing participants in some psychological reference experiment. It would also, I think, preclude wasting the time of professional librarians on fake reference questions, just to see how they responded. Would these same students and their professors want to be lied to and deceived in order to further someone else’s ends?

The other part of Kant’s ethics says you should treat people as ends in themselves, and not just as means to your ends. What is the fake reference interaction doing but using the reference librarian as an unknowing means to someone else’s ends? Most people resent being used as merely a means to someone else’s end. It might seem that reference librarians in general are always merely means to someone else’s end, but that’s not the case. If I willingly agree to participate in a certain relationship as a reference librarian, this is different than if I am deceived or manipulated. I’m here to help people answer actual reference questions. My action wills that reference librarians should answer such questions. My action does not, however, will that I be used as an unwilling guinea pig in someone else’s reference experiment. There is an implied contract in a reference transaction. You have a real information need and I help you with it. When that contract is broken by deception, then my part is ended.

Next, let’s consider the utilitarian approach. The utilitarian dictum is that an action (or a rule behind an action, depending on whether one is an act- or a rule-utilitarian) is good if it leads to the greatest happiness for the greatest number. This approach makes it less clear for this situation. Is the greatest happiness for the greatest number achieved by fake reference patrons lying to reference librarians? According to the felicific calculus, does the happiness of the fake reference patrons outweigh the resentment and unhappiness of the manipulated reference librarians?

Or we can expand this more broadly to a consequentialist approach. Instead of just considering happiness as the end, let’s consider the consequences of the action. What good consequences come from the lying and deception? Who benefits by a fake reference transaction? Does the student faking the transaction learn anything? And does this make up for the deception somehow? In my case the participant-observer approach was flawed because I spotted it. Asking an obviously fake question and then upbraiding me for not going along with the charade rankled. Thus, the fake patron didn’t learn how reference questions were answered, but instead learned how I deal with fake patrons trying to manipulate and deceive me. As the fake patron noted, things went quite well at first, but by the end the person was disappointed. We both were.

We should also consider the virtue ethics approach, derived from Aristotelian ethics. In this approach, ethics isn’t so much about rule following or calculating happiness, but about forming certain types of character in people. I think this approach makes more sense in terms of the ethical upbringing of children and the formation of character. We don’t people to have to stop and argue with themselves about whether they should lie or betray others. We want people who tell the truth and don’t betray others as a matter of habit. We want them to have a certain good character. We want people who as a matter of habit don’t deceive others just to obtain their own ends. We want reference librarians who willingly do all they can to answer legitimate reference questions or help with research problems, and we don’t want reference students who go out and lie to librarians. Is this the sort of character we want to develop in future reference librarians? You can probably see where I’m going with this, so I’ll move on.

I’d like to mention one other perspective. In his later work, Martin Heidegger discussed what he called technological thinking. Technological thinking involves considering everything that exists as a tool standing in reserve (bestand) to be used by human beings. This would include not only obvious tools such as hammers, but also the natural world as a whole. It doesn’t matter what happens to chickens or forests, for example, because they are just standing in reserve to be used by us. The ultimate problem with technological thinking is that eventually humans themselves come to be considered as tools or objects standing in reserve to be used by others at their will (consider the phrase “human resources”). This is related to Kant’s objections to considering other people as means to your ends rather than as ends in themselves. The technological thinker views everything and everyone as a tool standing in reserve to be used when appropriate. Thus, the hapless reference librarian becomes merely a tool or a means for the fake reference patron to achieve some other end.

Thus, we have a handful of ethical approaches, and it seems to me they all lead to the conclusion that lies and deception are bad things. Even if under some consequentialist approach one could justify the lies and deception and manipulation by some higher end, what is gained here? Do the fake patrons (or their professors who hand out these assignments) think that some higher end justifies lying to librarians and wasting their time? Do they think that reference librarians are so clueless that we can’t spot the faker? Do they think spotting the faker has no effect on the reference transaction? Do they think that other people are to be used as unwitting means to their own ends? That reference librarians aren’t there to help people with reference queries and research needs, but are just there as a little experimental tools for library school students to play with? Apparently, they do.

If students want to learn how I do reference, I’ll be happy to help (under the rule that it’s a good thing for librarians to willingly help library school students as I was willingly helped by librarians when I was a student). Shadow me. Interview me. Analyze my chat transcripts. See how I act when people approach me at the desk. Watch me from afar or a-close all you want. Read the reference behavioral guidelines that I helped write. But don’t lie to me and try to deceive me.

If however, after all this, you or your professor decide that your ends somehow justify using reference librarians as your tools and you still choose to lie and try to deceive them because they’re not worthy of respect as busy professionals or even as human beings and ends in themselves, then you should be a much better liar than this person was. Practice that lying and deception until you have it down to a fine art. It won’t make you much of a reference librarian or even a decent human being, but at least the librarian won’t be left feeling manipulated and betrayed and you might achieve whatever end you think justifies your means.

Attention, Time-Wasters

It’s not often that a professional interaction makes me angry. However, I had an odd experience tonight. During my regular Sunday evening chat reference shift I ended up being lectured by a library school student from an unnamed library school at a large public university in New Jersey about how my reference service needs improvement. As part of an assignment for a reference class, the library school student was supposed to go out and pester actual librarians with fake reference questions to see how they responded. I’ve read a bit about this ethically dubious practice, and in addition am able to spot a fake reference transaction pretty quickly. (Hint to library school students posing as fake reference patrons–I do this for a living. I can tell when you’re lying to me.) Something seemed fishy to me, so ultimately I responded with our standard line. Our reference service is for Princeton faculty and students or questions specific to the Princeton collection. Oh, and I don’t answer general reference questions from library school students at some other university. Then came the admission that the whole reference transaction was fake. What a surprise! And then the lecture. Spare me!

Is this what reference education is about these days? Are the students supposed to go out and waste the time of professional librarians? Because that was the impression I was left with. This student was a great big time-waster who had the erroneous opinion that a reference service at a private university should act as a general reference service for anyone in the entire world. I praised the librarians at the university the student attends and recommended the student approach them. Apparently, this wasn’t good enough. Instead, I should have played along with the game (which I suspected was a game) and spent a half hour of my time searching for items which no one actually wanted. That, apparently, is the sign of good reference service. So because I refused to help a fairly obvious liar, which is what this student was, somehow my reference service needs improvement?

What needs improvement is reference education, if this is what it has sunk to. Do teachers of reference think that librarians have nothing better to do than spend time answering fake reference questions from fake library patrons? Especially if those fake patrons more or less give the game away by saying they’re library school students at some other university who need help with a simplistic question a college sophomore would be able to answer? What else are we actual librarians to suspect given such information other than that we’re being conned? Reference librarians tend to be pretty savvy about these things. I don’t know about other reference librarians, but I resent being approached by obvious liars whose intent is to waste my time, especially when these liars are put up to their lies by their professors and then have the nerve to get self-righteous when I catch them and refuse to help them. I would be happy to help library school students or answer questions about what I do or allow students to see how I work. But that’s not what this practice is about. If students come in with obviously fake questions, it’s not my practice to play along with them. That’s not reference education. That’s a big waste of my time.

Attention, time wasters: please leave real librarians alone. Some of us have work to do.

[Since this post has been getting some new hits lately, readers should see the two follow-up posts: The Ethics of Fake Reference, and Alternatives to Deception.]

Copyright and the Code

You might not know this, and based on the attendance at open hearings and such you probably don’t care, but the ALA Council approved a minor revision of the ALA Code of Ethics last week. I had a very small part in this process as the RUSA representative to the ALA Committee on Professional Ethics.

Almost everyone with any interest in the Code was happy with it, except for Article IV, which since 1995 has read, “We recognize and respect intellectual property rights.” Apparently, this article was slipped into the Code during the last revision without much discussion. On the surface, it seems innocuous enough. To me it doesn’t say much more than, “we obey the law.” The problem that I and others had was with what wasn’t said, namely, that librarians also want to make information available to people. After many good suggestions and a lot of wrangling, the revised wording, and that which I think was approved, is: “We respect intellectual property rights and advocate balance between the interests of information users and rights holders.”

Near the end of the committee discussion, I found myself in the unusual position of being on the radical side of the debate. In general, I’m temperamentally moderate and believe in different spheres of justice, to borrow and inappropriately apply a term of Walzer’s. For example, I’m one of those who thinks the ALA shouldn’t take a position on the Iraq War, despite the fact that I have opposed it from the beginning on ethical, political, and military grounds. This would mark me as a “conservative” among some groups of librarians. However, the pendulum shifts when it comes to library issues. I proposed dropping the property rights clause entirely, and substituting something like, “We want to make as much information as possible as freely available to as many users as possible.” Perhaps not that very wording, since it isn’t bureaucratic enough, but certainly that sentiment.

At one point, I was accused of wanted to do away with copyright, but such was not the case. My argument for this change assumed that the ALA as an organization and the vast majority of librarians want as much access to information as possible, and that while we agree with the idea of copyright, we do not in fact agree with much of current copyright law, especially the Sonny Bono copyright extension and the Digital Millennium Copyright Act. Had those acts been in force in 1995, Article IV might not have been written.

Article I, Section 8 of the Constitution states the purpose of copyright: “To promote the Progress of Science and useful Arts, by securing for limited Times to Authors and Inventors the exclusive Right to their respective Writings and Discoveries.” My personal test case for copyright is T.S. Eliot, who has been dead for over 40 years, but whose estate maintains strict control over much of his work because of copyright extensions and charges high prices for inclusion of his work in anthologies. But any copyright that extends this far past the author’s death isn’t promoting the progress of anything. It’s of no benefit to the author or the common good. The idea of copyright as an unlimited right to ownership by a corporation to some piece of intellectual property in perpetuity has no justification, legal or moral.

I wanted to write this yesterday, because I was thinking of Martin Luther King Jr., and his use in the “Letter from Birmingham Jail” of Aquinas’s distinction between just and unjust laws. Under natural law theory, a law on the books (or a positive law) is only just if it’s in accordance with the natural law. We can have unjust laws, and it seems to me that people are much more likely to ignore an unjust law and not feel bad about it. How many people, I wonder, make copies of their own DVDs to put on their video iPods or other portable devices? Plenty. But why would so many people violate the law against this? Because it’s a stupid law, and also an unjust one, if we can imagine a clause of the natural law governing digital copies of DVDs. People violate this law all the time, and feel no remorse at all. Nor should they.

The new code wants to ensure that librarians are on record supporting the rights of people to access information that may be copyrighted, which libraries have been doing since there was copyright. With current copyright law, we might not be able to create libraries today if they didn’t already exist, and if we move to an all digital world with the extremely restrictive digital rights of today, then libraries will have a harder time serving users. A more radical approach would have been the acknowledgment that a lot of us only grudgingly accept current copyright law, and push it to the limit in various ways to get information to users. Upon reflection, I don’t think the more radical version would be a good idea, because we don’t want to give the copyright fascists any more ammunition to attack libraries with, but I still believe that in the dark shadows of our professional souls where the lawyers aren’t allowed, the needs of the users still trump excessive copyright laws. We just can’t officially admit it.

The Ethos of Librarians

The ALA has a Code of Ethics currently undergoing revision. (I have a tiny part in that process as the RUSA representative to the ALA Committee on Professional Ethics.) It’s important to have a code of ethics in case of disputes, just as it’s important to have collection development policies or other documents to refer to when a serious question arises. However, I don’t think librarians rely much on codes like this in their usual practice, but instead in times of stress or struggle. This isn’t a bad thing, because when it comes to the ethical culture of libraries, we shouldn’t be promoting a culture of rules, but a culture of character.

The Code of Ethics, indeed all codes of ethics, can be considered deontological documents. Here’s my very quick summary of ethical theories from an article I wrote on the virtue of reference last year: “To develop my argument I must give some background on “virtue ethics” and in particular on Aristotle’s Nicomachaen Ethics. Ethical philosophers have paid an increasing amount of attention to Aristotle’s ethics over the past few decades as “virtue ethics” has become prominent along with deontological and consequentialist ethics. While deontological ethics judges ethical actions by a particular standard of rightness or wrongness it is our duty to obey (e.g., the Ten Commandments or Kant’s categorical imperative) and consequentialist ethics judges ethical actions by their consequences (e.g., utilitarianism’s “greatest happiness for the greatest number”), virtue ethics follows Aristotle in focusing not on rules of conduct but on the character of the moral actor. What sort of person acts ethically? How do we raise and educate such people? What virtues (or excellences) does a person require to be an ethical human being? Those are some of the sorts of questions virtue ethicists might ask.”

Such codes are useful, as I said, in times of dispute, but in general we want to create a culture of character where librarians don’t think about the code or any code and don’t run to the rulebook to make any decisions. In culture of character, we perform virtuous actions because we have developed virtues or excellences through long habit, rather than through explicit rule following. We don’t keep lists of rules about proper professional or ethical behavior; we just behave appropriately, or we do not. For those who don’t behave appropriately, referring to codes is unlikely to reform their behavior because codes won’t overcome their lack of virtues and their habitual misbehavior. In library schools and libraries, we should strive to create this culture of character.

For the most part we already do this, and I’m not trying to present a radically new understanding of the ethical culture of libraries so much as trying to understand for myself what already goes on. Consider some practical examples. In a reference transaction, there are proper and improper behaviors. Failing to make eye contact while addressing a library patron would be inappropriate, as would lying to them. Some inappropriate behaviors are worse than others, obviously. In developing collections, we get a feel for what’s appropriate for the collection and what’s not. Perhaps there are people who frequently consult their collection development plans to see what they should be buying, but I’m not one of them.

As an example of habit and action being poorly formed, I’ll offer my own experience in chess, which I play at a mediocre level. I have a list of rules I keep in my head and that I run through before making moves, especially in the middle game. Fight for the center. Develop the pieces. Move the pawns only once. A knight on the rim is dim. Make a threatening move if you can. Leave nothing en prise. Etc. All the rules you might find in an introductory chess book. The problem is that I still rely on explicit rules because I’ve never developed adequate chess virtues, where moves just feel right. I read once of Boris Spassky analyzing a game and making a move. When asked, he couldn’t explain why it was a good move. It just had the feel of a good move, so he made it. Most grandmasters are probably like that. We all know, or at least should know, that feeling in our jobs. We do things because they just seem the right thing to do, even if we can’t articulate why.

Instead of a rule following culture, it would be better for libraries to develop an ethos where responsible action based on good habits was the norm, where people were allowed and encouraged and educated to act in these ways. The question is how to do this. Is the character of librarians something that can be taught, or do people who already have certain ethical habits just better at the job? Can one train a reference librarian to be considerate and thoughtful of the research needs of a library patron? Or is it that those people who aren’t already like that just won’t make good reference librarians? I don’t know. I can only speculate that certain cultures don’t encourage good habits. Rule following cultures. Rigid cultures. Micro-managing, permission-driven cultures, where people are afraid to act or are constantly being told what to do at every level. These are certainly not work cultures conducive to properly virtuous librarians.

Regardless, while I’m glad the ALA Code of Ethics exists as a normative set of rules to be depend upon if necessary, or that certain types of behavioral guidelines for librarians exist for various purposes, I can only think that if we have to go back to the rulebook, then we’ve already failed somehow.