Group Number: 1

Names: Yaared, Kuni, Avneesh, Joe, John

Project Summary: Our project is a centralized serving system, called ServiceCenter, that allows waiters to efficiently manage orders and tasks.

Description of Test Method:

To obtain informed consent from the test subjects, we first asked if they would be interested in helping to test our system for a COS class.  Once they assented, we explained that they would be playing the role of a waiter in a restaurant and interacting with our system and that the testing itself would take about ten minutes plus set up time.  We then presented them with the consent form that can be accessed here, and they signed it. We also read the demo and task scripts to them in order to explain what they were going to be doing through the test. The demo and task script can be accessed here and here.

All three test subjects were affiliated with Terrace F. Club.  The first was a member of the kitchen staff, so he had experience serving food to others.  He was a relaxed and friendly waiter.  This subject expressed interest in our prototype system when he saw us setting up, and we were able to show him how it worked in detail by using him to test it.  The second subject was a student who has spent multiple summers working as a waitress in the restaurant at a country club.  Since she was used to demanding customers, her style was more professional and focused.  She was chosen based on her experience as a waitress who was used to working in a high pressure environment. The last subject was a male student.  He had a unique approach to customer service, but was an enthusiastic participant and waiter.  We chose him because we wanted to test the system on an inexperienced waiter/waitress.

We chose to run our low-fidelity prototype in the lower dining hall of Terrace F. Club. The lower dining hall is split into two sections: a solarium and a seated area. The solarium was where we set up our Motherboard – a screen that displays order information, order status, and cup status for all tables in the restaurant. For the actual board, we used a white board with the table floorplan for the restaurant drawn on. The customers sat in the seated area, where our test users took orders and served the food. When serving food, the waiter on duty served plates with labels on them, specifying what dish it was. We used cups, jugs, cutlery, and updated the Motherboard using dry-erase pens to simulate our system for this low-fidelity prototype.

For our low-fidelity prototype test, we divided roles amongst our group as follows. John, Yaared, and Joe acted as customers in charge of ordering for their respective tables, while at the same time making any test run observations from the viewpoint of the customer. Avneesh acted as our main test run observer and took notes. At the same time, he was available to help complete any tasks if the waiter/waitress requested help. Kuni managed the Motherboard and the “kitchen”. Given an order, Kuni would update the appropriate section of the Motherboard, put out completed orders to be served by the waiter on duty, and again update the Motherboard, thereby simulating the transfer of information between the kitchen and floor.

For our testing procedure, we decided to stagger 4 different groups of customers of varying size (i.e. 4 tables). The first group (i.e. John) enters and is served by the waiter. John orders for a large group of 7. Now, the second group (i.e. Joe) enters and is served by the waiter. Joe orders for a small group of 2. ‘5 minutes later’, the third group (i.e. Yaared) enters and is served by the waiter. Yaared orders for a medium-sized group of 4. The second group (i.e. Joe) then asks for the bill and leaves after paying. Now, the fourth group (i.e. Joe) enters and is served by the waiter. Joe orders for a medium-sized group of 3. The respective groups finish their meals, ask for checks, pay and then leave. We thought that staggering 4 groups of customers of varying size in quick succession would be a good way of simulating a situation in which servers may be hard pressed. Since our system is supposed to allows waiters to efficiently manage orders and tasks, we thought simulating a situation in which servers may be hard pressed, i.e. dealing with a lot of tables in peak hours, would best test the effectiveness of our system (after all this is the problem we are trying to solve!).

Summary of Results:

Our users understood and used the system well throughout our prototype test. All of our tasks were completed—calling for help was very simple and only happened when our users had to bring out multiple plates of food at once. Our other two tasks—checking customer status and determining task order—were performed regularly throughout the 10-minute period for every test run. An incident occurred where there was confusion over what a customer had ordered when two customers had placed similar orders. Another major incident was when our first subject was confused about which tables corresponded to the figures on the board. At times, Kuni, who was operating the Motherboard, had trouble keeping up with the speed of orders being placed and preparing the paltes. Otherwise, the user testing went very smoothly.

Users rarely (less than 5 times) looked at the motherboard for more than 10 seconds. Most of the time, they would glance in passing and quickly assess what their next task was. The only time they seemed to stop to look at the board was when there were multiple tables with ready plates. Throughout the test, there were times when the customer had to remind the waiter/waitress that their cup needed refilling. Overall, Kuni processing all the information and updating the board while also preparing the plates seemed to be the biggest bottleneck throughout the testing process.

Discussion of Results:

The most common “problem” faced by the test subjects seemed to be deciding how to order food deliveries to tables when multiple tables had food ready. One test subject volunteered the suggestion that orders should display an “estimated time left” that would show the rough time until each dish would be ready (the estimate could be determined in a variety of ways.) This could prove helpful in planning for future trips to tables when there is some down time. Also, it could be helpful to automatically sort the orders on the screen. All orders could be given a time value, positive for orders that have been sitting out (for how long they have been out), and negative for orders not yet ready (for how long until they will be ready). The orders could then be automatically sorted in ascending order by time. This would place them in priority order for the waiter/waitress.

We also realized that the dedicated board updater could be a bottleneck in a real restaurant setting even in the workings of the real system. If there are a lot of orders with many modifications and many tables being served, then the input of all these orders will most likely be the most time consuming part. But we realize that this is a worse case no matter what the system is. If we dedicate a person to input, then this could alleviate the problem. The dedicated input person would take the task from waiters/waitresses and possibly become quicker at inputting the information. Users were overall enthusiastic about the system, and they didn’t need too much explaining beforehand to understand how the system worked. We were happy that users did not need to scrutinize the board to figure out what the status of each table was—the layout and concept seemed to be clear.

Future Test Plans:

There are several areas of our low-fidelity prototype that could use extra testing before we move on to the high-fidelity prototype; many of these were brought to light during this testing session. As it is, we have already tested the usage of the information screen both in terms of entering data and reading data. However, we did notice during the testing procedure that simulating the information screen without a computer created a major bottleneck – Kuni had as much work to do in each task procedure as the actual outside participant. In order to better simulate the actual speed of the system (since speed and efficiency are at the core of our goals), we might have to find a faster method of displaying the data without computerizing it, if we want to continue to test with low-fidelity. Finally, we would also want to incorporate the input interface in the next round of testing. Since in our system, we would have a dedicated worker inputting the orders into the system, in our test Kuni essentially played this role by directly writing the orders onto the board.  In the creation of the paper prototype for the input interface, we went through some brief pseudo testing, going through 2-3 iterations of the input interface. But for future testing, we would have another subject input the orders as the simulation runs.

There are other areas which we found difficult to test but which could have yielded useful information. For one, we were not actually able to do testing in an open restaurant setting with waiting staff, instead using the dining room of an eating club with students with waiting experience as an alternative. Testing in the real setting would yielded the ideal results, but we are unlikely to find a restaurant willing to participate. Another smaller area which was difficult to emulate in low fidelity was the entrée timer (i.e. the timer which counts down the time until the food in the kitchen is done being prepared). It was simply infeasible for the board manager to be constantly updating the time, and somewhat less useful anyway since we used wait times of 1-2 minutes instead of 15-30 minutes as is common in actual restaurants. Testing this aspect might be done with an actual timer or stopwatch in the future.