a. Group num­ber and name

Group num­ber: 18
Group name: %eiip

b. Mem­ber names

Erica (eportnoy@), Mario (mmcgil@), Bon­nie (bmeisenm@), Valya (vbarboy@)

c. Project summary

Our project is a smart book­shelf sys­tem that keeps track of which books are in it. You can see the web appli­ca­tion por­tion of our project here, and our source code here!

d. Introduction

In this user test, we evaluated both the hardware and software components of our system. We had users actually use Biblio-File to add books to their shelf, search for books (both those that were and were not on the actual shelf) and play with the system. In particular, we were testing to see whether or not our system actually made searching for books easier. To test this, we had users search for book both with and without Biblio-File. Moreover, we wanted to see if adding many books was annoying or tedious for the users, and if the delay due to RFID reading was particularly annoying or frustrating.

e. Implementation and Improvement

Our P5 post can be found here. We also made a few changes since P5, which are listed below:

• Completed implementation of the server-client-daemon architecture – we can now user-test using only our system, without needing “Wizard-Of-Oz” simulation
• Modified the RFID library to enable low-latency use of multiple scanners (original code didn’t support multiple scanners)
• Alphabetized entries by title in the web app to let users scroll through books in a sensible order
• Added magic administrative functions to help with user testing: auto-populating bookshelf, clearing database, etc.
• Computer now beeps loudly after a sensor detects an RFID to give users feedback

f. Method

i. Participants
For our user test we chose three undergraduate students with varying levels of technical expertise. Our first tester is an engineer, who often uses technology, and was very interested in how our system worked. Our other testers were less experienced with technology as a whole, and were completely unfamiliar with the Arduino and the other tools that we used in our bookshelf. We chose them to see how intuitive our system was, and how annoying or frustrating the delays might be. People who are less familiar with these tools are also less used to delays, and will therefore have a more natural reaction to them. Similarly, people with less technical expertise are less likely to assume anything about the app, and we can therefore see how they use it, what they try to do but cannot etc.

ii. Apparatus
In order to conduct the test we used a stack of shelves in the TV room of Charter. The location might not have been optimal, because there were other people there playing video games, so there was a lot of noise and distractions. That being said, it was cool that our system was so mobile, and could be applied to any bookshelf anywhere. We attached our system to the shelves, and brought our own books to use for the tasks themselves. We let the users use one of our own phones to test it, to avoid the need to download pic2shop for barcode scanning. That being said, they could have easily used their own mobile device if they chose to.

This is the version of the system that we used for our user tests

iii. Tasks
The easy task is find­ing a book on the shelf, or search­ing for a book that is not present. Users can choose to use our mobile app to search for a book, or they may attempt to man­u­ally search the shelf. If our sys­tem pro­vides added value, we hope that they will opt to con­sult our mobile app. To do this task we provided the users with a very full bookshelf. Each user saw exactly the same books on exactly the same shelves. We timed how long it would take the users to find a book that was on the shelf, and one that wasn’t, both with Biblio-File and without.

Our medium task is adding a sin­gle new book to the sys­tem; it con­sists of adding an RFID tag to the book, adding it to the sys­tem using our mobile inter­face, and then plac­ing the book on the shelf. The purpose of this task was to test how easy our system is to use, and what a user would intuitively want to do given a system like ours.

Our hard task is adding an exist­ing book col­lec­tion to the sys­tem; this consisted of four books, for test­ing pur­poses. This is the last task a user would have to complete with our sys­tem, and it this is very sim­i­lar to the pre­vi­ous task. It consists of using the mobile inter­face and an RFID tag to add books to the bookshelf. The main purpose of this task was to test the tediousness of adding many books to a collection.

A video of one of our user tests can be found here!

iv. Procedure
To conduct the study, we first introduced our team and had the user read and sign the consent form, and fill out the demographic questionnaire. We then explained the concept of a Think-Aloud Study, and practiced the methodology on an unrelated problem (see script in appendices). After this, we demonstrated how our system works (in general, without showing them any of our tasks). We then ran the easy task, and timed it, followed by the medium task and then the hard task. Afterwards, we told the users how our system was meant to work (if they didn’t understand it to begin with), and asked some follow-up questions to check how annoying our system was (if it was at all) and how the user felt using it. The answers to these questions are included in the appendices. The users were encouraged to think aloud and ask questions throughout the study.

g. Test Measures

We measured two within-subjects variables related to book access and retrieval, whether or not a book is on the shelf and whether or not the user was using our system. We chose to measure these to see if our system gave the user any quantitative speedup in common book interaction processes.

• Time taken to retrieve a book on the shelf.
• Time taken to realize a book is not on the shelf.
• Time taken to interact with a book using only the physical bookshelf.
• Time taken to interact with a book using our system.

h. Results and Discussion

Our tests showed that in general, our design is sound, although a repeated measures ANOVA with a sample size of 3 showed no significant difference in using or not using our system (p > .05, see ANOVA output in appendices). Many users were enthusiastic about what we were able to do; in particular, many were delighted that we could gain a lot of information from a single photo of the ISBN barcode. We deliberately did not give users a complete demo of our system because we wanted to judge its intuitiveness. Even without complete instructions, our testers largely understood the system, which we’re very proud of. However, for some tasks, such as removing books from a shelf, it’s clear that more specific instructions would be helpful.

Users relied heavily on receiving some sort of feedback that an RFID tag had been sensed, which we implemented in the form of a loud beeping sound. This worked well. Also, users did not seem impatient when we asked them to add many books to the system at once.

There are some small changes that we’d like to make. For example, when there are no books in the shelf, we shouldn’t display the search bar, since users often attempt to add a book by typing the title into the search bar first. We may also want buttons to display a “not on shelf” message instead of a greyed-out “Light up!” button when they’re not on the shelf, since 1 in 3 users did not recognize the design motif of a disabled button. Also, there are some bugs we still need to fix, such as the barcode scanner redirecting to iTunes occasionally, and the lag with the LEDs on the shelf.

We also need to clarify the tap-in tap-out process to the user. While it is not the most intuitive, it is based on the technical limitations of our hardware, so we will have to compensate in instructions. Since some users attempted to tap a book in before adding it via the software, we should enable our software to allow either ordering. We should also change the instructions to say “place the bookmark inside the front cover,” since some users placed the bookmark too deep inside the book to be read.

i. Appendices

i. All things read or handed to participant

• Consent form and demographic questionnaire
• Demo, task, and interview scripts

ii. Raw data

• Merged critical incident logs, and questionnaire/interview responses
• ANOVA table of time taken

a. Group num­ber and name

Group num­ber: 18
Group name: %eiip

b. Mem­ber names

Erica (eportnoy@), Mario (mmcgil@), Bon­nie (bmeisenm@), Valya (vbarboy@)

c. Project summary

Our project is a smart book­shelf sys­tem that keeps track of which books are in it. You can see the web application portion of our project here!

d. Tasks we have chosen to support in the working prototype

Our hard task is adding an existing book collection to the system; this should consist of at least three books for testing purposes. This is the first task a user would have to complete with our system, and it consists of adding RFID tags to the books, adding them to the system using our mobile interface, and then placing the books on the shelf.

Our medium task is adding a single new book to the system; this is very similar to the previous task. It consists of using the mobile interface and an RFID tag to add a book to the bookshelf.

The easy task is finding a book on the shelf, or searching for a book that is not present. Users can choose to use our mobile app to search for a book, or they may attempt to manually search the shelf. If our system provides added value, we hope that they will opt to consult our mobile app.

e. Differences from P3 and P4

Our tasks have not changed from P3 to P4, though the interfaces the user will interact with and the workflows have changed somewhat. This is because our tasks are goal-oriented, and we believe that the goals of the system remain unchanged. Additionally, the hard task is the first task that an actual user would have to perform, so we believe that testing the proposed tasks in the order given is valuable.

f. Revised interface design

i. Changes as a result of P4 and the rationale behind these changes

The main feedback we received from our user testing was that it takes too long to add a book. In order to fix this we removed the screens that asked our users to verify the information, and we no longer require our users to photograph the cover. Instead, we now have them sim­ply scan the bar­code and we use the Google­Books API on our end to get the rel­e­vant infor­ma­tion. We also noticed that many users struggle with adding a book to the system. We fixed this by adding a clear, obvious, and noticeable “Add Book” button to the main page. In user tests, users also did not use our search feature because they could see all the books on one screen. In our high-fidelity pro­to­type, we utilized mobile design heuristics in cre­at­ing a page with large text and images. A consequence of this is that fewer books are shown at a time, and the user is able to search via predictive queries, implemented using a typeahead.

From the hardware side of things, we initially wanted to use Seeed Studio RFID readers placed behind the bookshelves to statically sense the presence of multiple books, and thereby be able to inventory the contents of the shelf. Users prefered this method to a “tap-in, tap-out” approach to managing books. However, it looks like we may have to take the “tap-in, tap-out” approach since a fundamental limitation of the readers’ hardware seems to be that they can only read one tag at a time. This means that the user will have to tap each book to the scanner when replacing or removing a book.

ii. Updated storyboards 

This storyboard illustrates our most difficult task – adding an entire collection to the system.

Here we illustrate our moderate task – adding one new book to the system.

Finally, here is our easy task – retrieving a book from the bookshelf.

iii. Sketches

Our new bookshelf will have a “tap-in, tap-out” system for adding books, because the RFID scanners are not good enough to simply sense what books are on/off the shelf.

g. Overview and discussion of the new prototype

i. Implemented functionality

In this new prototype we greatly improved the book-adding paradigm. Now, instead of having to go through multiple screens and steps, a user simply takes a picture of the ISBN, puts a sticker or bookmark in the book, and places the book on the shelf. We also implemented a search function, where the users can search by author or title.  We have added “Light Up” buttons to light the shelf the book is on (although the physical light is still done via wizard-of-oz techniques, as explained in more detail below). We provide no button for deleting a book, because the difference between removing a book from the shelf and deleting it from the database is unclear and we do not want our users getting confused.

We are using a computer to communicate between the Arduino and the web server, to maintain a seamless interaction from the user’s perspective. In particular, we need this setup in order to query the bookshelf state remotely from a web application. We are also using an Arduino to power, run, and com­mu­ni­cate with the RFID read­ers. We are using an Arduino because we need a micro­con­troller of some kind to interact with the RFID scanners, and we already have an Arduino, which is rel­a­tively easy to use.

ii – iv. Left out functionality, wizard-of-oz techniques, and the rationale behind these decisions. 

We decided to leave out the physical bookshelf from our high-fidelity prototype. Instead, we will continue to use a normal bookshelf and wizard-of-oz techniques to implement the physical finding of the book and the light to indicate its location. We are doing this because our user testing showed us that the aspect that we really need to work on is our application and user interface. Thus, when building our high-fidelity prototype we focused most of our efforts on that. By emphasizing these functions we hope to get better feedback from users. Our earlier feedback indicated that the workflow was most important, so we hope to test the new and improved workflow on users next week. Moreover, we had to wait on building the physical system because we had some issues with our RFID reader, and getting the wires needed to make it work. Furthermore, upon experimenting with the RFID reader we discovered that its range is smaller than we expected, and that it can only sense one RFID at a time. Because of this, we decided to change our system to have a “tap-in, tap-out” function – instead of having the bookshelf keep track of all the books that are on it, users will a tap a book to the reader when placing it on the bookshelf, and then tap it again to tell the bookshelf that they have removed the book. Getting an RFID reader that could do everything we wanted would have been expensive and not feasible, so we came up with this solution to do the best with the materials we have access to. We have a wizard-of-oz web-form which allows the user test administrator to indicate which shelf-number a user placed the book on, simulating what will later be happening via Arduino and RFID.

v. Code not written by us

For our project we used some code that was not written by our team, as well as many publicly available libraries and applications. In particular, to implement the barcode-scanning functionality of our application we used an app called pic2shop, which scans the barcode, reads the ISBN, and then sends that data to our application. We also used the Google Books API in order to get data about a given book (the title, author, and cover image) from the ISBN number. The web app itself is hosted on Heroku and built using the Python framework flask, and is based on the example application called flaskr. Twitter Bootstrap and jQuery are used for the frontend. We also used the RFID SeeedStudio Library for Arduino in order to operate the RFID scanner and read RFID tags.

h. Here are some images documenting our prototype!

Mobile interface/screen. From here users can search for the book by title or author, add a new book, or browse their collection.

Users can search for books and the typeahead will provide suggestions to auto-complete their query.

When the user presses “Go” the app takes them to the book. From here they can press “Light Up” to light up the shelf that the book is on.

This is the same application via web interface. This is what it looks like in a browser.

When a user adds a new book they are taken to another application (pic2shop), which takes a photo of the barcode.

Once the barcode has been scanned it returns to our application and the user is shown these instructions.

Though there is no delete button, administrators do have the ability to delete books if necessary.

This is the wizard-of-oz interface to simulate the RFID sensing on our shelves, which has not yet been implemented.

This is the browser resized to simulate what our application looks like on an iPad.

This is our RFID scanner.