The Elite Four (#19)
Jae (jyltwo)
Clay (cwhetung)
Jeff (jasnyder)
Michael (menewman)

Project Summary
We will develop a minimally inconvenient system to ensure that users remember to bring important items with them when they leave their residences; the system will also help users locate lost tagged items, either in their room or in the world at large.

Test Method

Informed Consent
We wrote out a consent form and gave it to each participant to read and sign before proceeding with any testing. The consent form briefly talks about the purpose of our project, and it outlines any potential risks and benefits. It also informs the participant that all information will be kept confidential. We made sure we were present while they were reading it in case they had any questions or concerns, but overall it was very straightforward.

Link to consent form:
https://docs.google.com/document/d/1RmM8eRv5mjBGGQm7pDlTCuOmP3u5cS82MRnoDfVA7iM/edit?usp=sharing

Participants
We selected our participants randomly from a sample of upperclassmen studying in a public area during the afternoon. We had one participant who lives in a single (and consequently has a higher chance of being locked out), one participant who lives in a quad, and one in a triple. All three participants, as upperclassmen who have used both the new electronic self-locking doors and the previous mechanical locks, are part of our target demographic. None of them are COS/ELE majors or students in COS 436.

Testing Environment
We performed the testing in Terrace’s upper dining room, using the back door as a simulated dorm door. We held the prototype device up to the door frame when appropriate and let the user carry it around for tasks two and three (as described below in Testing Procedure). The users were able to literally go outside for task three, and the dining room served as the user’s “dorm room.” Our equipment consisted of cardboard mockups of the device and a credit card form factor RFID tag that fits inside of a wallet.

Testing Procedure
Clay wrote the demo and task scripts and helped read them to the users. Jae wrote the consent form and helped read the scripts and explain the tasks to the users. Jeff simulated the functionality of the device, providing beeping and switching the prototype versions when appropriate. Michael took the most elaborate notes on the testing process, including implicit/explicit user feedback. We all helped write up the discussion and blog post.

We had our users perform the following tasks respectively: attempt to leave the room without a tagged item, attempt to locate a lost tagged item within the room, and attempt to locate a lost tagged item outside of one’s room. These tasks are ordered by difficulty, from easy to medium to hard.

User 1 presses the “find” button.

Demo & test scripts:
https://docs.google.com/document/d/1b2B5NOTYPpswJz7-M55SX8jZFXNKGVsP6iXeTWK93Tw/edit?usp=sharing

Summary of Results
User 1 is a senior who lives in a quad. She was able to perform the first task without a problem, although she was curious about alternate tag form factors (our prototype only features the credit card form factor for now). She was impressed by the usefulness of the second task and knew without being told that she should hit the FIND button, but she didn’t immediately realize that she was supposed to dismount the device from the wall. She also wanted to know if there was a way to disable the beeping after finding an item but before re-mounting the device. For the third task, she had no difficulty, which is not surprising given its similarity to the second task.

User 2 is a senior who lives in a single. For the first task, she wasn’t sure about the range of the sensor after syncing — does the user need to hold the tag close to the device? She also wanted to know if there was a way to tag only the prox, since she might not want to carry her entire wallet around. During the second task, she didn’t initially realize that she needed to press FIND, but was otherwise able to intuitively use the device. She suggested a FIND/FOUND toggle to stop the device from beeping after the lost item was found. The third task went more smoothly, although she did wonder if beeping speed would increase before the tag was in range (it won’t) and suggested that constant beeping might be annoying. She also suggested that the device might be easy to lose or forget to re-mount, and she wanted a way to disable the device — either an on/off switch or a sleep function.

User 3 is a junior who lives in a triple. He thought the device seemed useful and suggested that he would prefer a sleep function to an on/off toggle. He was able to complete all three tasks with basically no prompting or difficulty; he intuitively knew which buttons to press and what the beeping meant, and he even remembered to re-mount the device after finishing tasks two and three.

Discussion
Watching users attempt to use our lo-fi prototype with minimal intervention from us, we observed several flaws in our design. Ideally, the aim of this system is to be as intuitive as possible, but our users weren’t always able to intuit how to use our device. To fix this, we have decided to edit some aspects of our design. The find button will become a toggle switch, such that users know which state the device is in (“remind” or “find”). We also need to make it clearer to the user that they can remove the device from the wall and carry it around. This will likely take form as some reminder text on the device itself.

The users also provided information about possible new features. Some users expressed concern that they would lose their device when it is not mounted on the wall. In order to fix this, we will design the scanner to alert the user when it is neither in “find” mode nor mounted on the wall. This ensures the user remembers to re-mount the device. We also plan to add either an on/off switch or a snooze mode. Users pointed out that if they have guests over (and people are frequently coming and going) they would like to be able to turn off the system such that it isn’t going off all the time. A snooze mode is preferable to completely turning the system off, since the system returns to normal working order after the event is over. One user also suggested that the device beep in “find” mode only when the device is first switched to “find” and when the lost tag comes into range; otherwise, for task 3 especially, the beeping could become quite annoying.

Subsequent Testing
Based on the feedback we have received, we believe that we are ready to build a higher-fidelity prototype without further testing. The low fidelity testing revealed no catastrophic issues with our design. All of the other usability issues have been discussed within the team and with our test subjects and addressed adequately. As such, we feel confident that our design is ready to advance to a high fidelity prototype.