Find out exactly what are the usability tests used? how did they measure the usability? did they use interview or others? who did the tetsing?did actual users part of the study

Usability field tests analyze the use of systems in their real context of use [1]. In field tests, effectiveness, efficiency, and satisfaction [2] of the task performance depend on where and in which context a task is performed respectively. Bevan points out that for usability testing especially the context of use should be as close as possible to the real context, in order to provide valid results [3]. On the one hand, field tests provide this highly realistic context, but on the other hand the reality setting includes a high number of variable actions and influences [4], which have to be observed. Especially in public transport, field tests pass different contexts along a journey, for instance stations, vehicles and footpaths. Consequently, public transport sets the following challenges for field tests:

Discontinuity of actions along variable locations

Variable external influences, for instance waiting times vs. time pressure for

changing or rainy footpaths vs. crowded stations

Higher workload due to high autonomy for reaching the destination

In this context, stationary equipment is not sufficient to observe the actions of the user and the possible influences. The equipment has to be extended with other observation methods and mobile observations techniques. In addition, the objectively observed data has to be interpreted. These interpretations require information about the subjective impressions and feelings of the users [5].

The aim of this paper is to design and evaluate a mobile test setup, as combination of test methods and equipment, for field tests with mobile devices in public transport. The challenge is to observe sufficient objective data about the interactions of the users in specific tasks and also data about the variable environmental influences [4]. The designed test setup has to ensure valid and reliable data, which is not influenced by the technical equipment or lacking acceptance of the observation methods by the users.

2 Analytical Pre-study

Usability tests include different qualities of data. First of all, objective data are gathered by quantified data or video and audio recordings. The next step is to interpret the objective data. The interpretation requires more detailed data about the subjective feelings and emotions of the users. The subjective data reveal motivations of the users for specific actions, as well as feelings and perceptions, according to the test incidents [5]. The following analysis is restricted to methods which are realizable in a field test in public transport.

2.1 Methods for Objective Data Collection

Position Tracking. Gathering detailed information about the varying local test position, which is combined with the synchronized test time, enables conclusions on additional usage contexts. Position tracking can be combined with most mobile smart phones or additional tracking devices. The detailed geo coordinates can be displayed on maps, for instance to compare routes and analyze differences in way-finding strategies.

Function Tracking. Server log files of user requests give an overview of the used functions during the test. More detailed technologies even provide log files, which record every single user action with screenshots [6] and the point of time, but require additional software installations on the test platform.

Screen Capturing. In every usability test it is essential to record the user interface of the test object as detailed as possible. But additional cameras, attached to the smartphone, influence the user comfort and the ease of use. The focus of cameras with other perspectives highly depend on the users handling position and the incidents of light on the screen. In contrast, software solutions, which enable a directly and synchronized capturing of the smartphone screen, is independent from external influences and nearly imperceptible for the test users. Important criteria for the choice of screen capturing software are the sufficient resolution and recording duration.

Evaluating Methods and Equipment for Usability Field Tests in Public Transport 547

Performance Measurement. According to the results of time measurement, tracking, or capturing, performance measurement compares the users times of task completion [5]. This method provides information about the efficiency of a task completion process. But due to the variety of influences, field tests generate widespread results of task completion times, which can be hardly interpreted.

Interaction Camera. Cameras, which can be flexibly attached and adjusted, provide the possibility to focus the test observation on a specific aspect [7]. For example, the interaction with smartphones can be observed by an interaction camera in field tests. Due to its small and unobtrusive construction, interaction cameras can be attached to the users clothes without causing limitations the mobility and free movements of the user