A Framework for Evaluating Wayfinding Systems

A Framework for Evaluating Wayfinding Systems
Sandra Gabriele, Anna-Lena Theus, Daniella Briotto Faustino, Bruce Tsuji
York University

Paper Summary

This paper reports on a study consisting of a physical walkthrough and simple tasks performed by users as a way to expose weaknesses in an existing wayfinding system at a university athletic complex. The relationship between the user and their interactions with the architectural and graphic components of a wayfinding system provided a framework for evaluation to reveal deficiencies. Valuable to educators, students, professional designers and clients this study provides a demonstration of how a design framework can be used to help identify and categorize areas of concern for evaluating wayfinding systems, post-occupancy.

Background

The design of wayfinding systems for buildings often begins well before construction is complete and with little more than floor plans, making it challenging for designers to anticipate precisely how users might behave in the context of use. Thus, post-occupancy observation of user behaviour while interacting with signs and other information in the space is essential for making adjustments (Kishnani 257). 

Born mainly out of professional practice, the design and development processes used for wayfinding systems in large-scale complex environments are well documented in several books (Berger; Calori and Vanden-Eynden; McLendon and Blackistone; Mollerup; Arthur and Passini,). While these publications contain well-developed procedures, spanning the initial planning to the final installation of signs, little emphasis is placed on evaluating wayfinding systems after completion. For example, Calori and Vanden-Eynden (77-78)recognize that while post-occupancy evaluation is necessary, often it is not part of the budget and therefore becomes an unpaid, informal activity undertaken by the designer. Furthermore, Berger (89) points out the importance of the evaluation process for signage and wayfinding projects and identifies two distinct types of evaluation: the first takes place during development phases and the second occurs post-occupancy. The former is concerned with details of sign design: legibility, sign size, position and fabrication materials while the latter helps to determine whether or not the system supports the user in navigating the space. 

Understandably, there are few studies reporting on post-occupancy evaluation because the design of large-scale wayfinding systems are projects taken on by professional designers, whose obligation is to the client first and foremost. Unless the professional designer is also an educator, there is little incentive to share the findings of the evaluation process with the wider community and more specifically, for educational purposes. Until now internships, on the job training, and mentoring have served as the training ground for acquiring knowledge about these processes. More and more, educational institutions are interested in ensuring that design students are “job-ready” upon graduation. Currently, some design-related disciplines (interaction design, user experience design and human-computer interaction) introduce students to methods involving user testing in their curriculum. This is largely due to the focus on user-centred design and usability, along with a strong culture of empirical research and the sharing of results. The lack of published information on post-occupancy evaluation of wayfinding systems provided the impetus to explore a way to test and formalize a framework for evaluation.

The few available studies of post-occupancy evaluation report on a range of methods including: sign inventory and photographing signs; service points and hallways (Bosman and Rusinek 73); task-based observations (Kanakri, et al. 253; Kishnani 268); user perception surveys (Bosman and Rusinek 73-74); and user route mapping (Kanakri 253). A study by Brown, et al. (35) describes how the authors used multiple methods to reveal problems with the wayfinding system in a pediatric hospital:

After initial walk-through evaluation tours and meetings with administrators, five more systematic methods were used to assess problems: staff and visitor interviews, staff-maintained logs to record visitor requests for wayfinding, photographed traces in problem areas, behavior observation and tracking of visitors, and cognitive maps drawn by patients and parents.

Related research in environmental psychology also provides novel methods that could be used in post-occupancy evaluation. For example, to understand environmental behavior when wayfinding in complex buildings, researchers used a case study method and self-reported participant responses through a questionnaire together with pointing tasks to identify locations in a shopping centre (Dogu & Erkip 731-755). Other researchers explored the use of eye-tracking data to compare simulated lab environments to real environments to understand how a complex environment and body movement affect the processing of information and spatial decision-making in wayfinding (Schwarzkopf, et al.31-36).

Researchers used navigation tasks to evaluate devices that assist users with cognitive and visual impairments and limited mobility during wayfinding. For example, Chang, et al. (27-37) evaluated a context-aware device by observing participants with cognitive impairments while they found their way along predefined routes prompted by the device, measuring success in terms of points along the route and completion. Veldcamp et al. (161-165) used a “Wizard of Oz” method to give audio instructions via a headset to elderly participants with early-stage dementia while they walked to various locations to test a GPS navigation aid. Researchers assessed error rates and participant attitudes. Coughlan and Manduchi (379-397) tested a camera phone that detects user locations by scanning colour markers. Researchers observed visually impaired participants as they made their way around indoor and outdoor locations. They were interested in finding how scanning strategies (i.e. panning left to right) affects the device’s capacity to read colours. Kulyukin et al. (303-311) designed an intelligent walker for individuals with mobility problems. They conducted three focus groups and user trials. To inform the initial design, they conducted the first focus group with healthcare workers to understand the conditions affecting clients who might benefit from the use of the walker. To obtain input on a number of designs for the interface design, healthcare workers participated in the second focus group, while clients participated in the third. Finally, clients participated in user trials to determine whether the verbal/written instructions sufficiently supported them in finding the target destination. Researchers documented time, errors, routes along with user comments and subjective evaluation of the device.

Users, cognitive maps and wayfinding

In their book, “People, Signs and Architecture”, Arthur and Passini combine what is known about spatial perception and cognition and the practical aspects of designing for wayfinding. They bring forward valuable guidance for designers, focusing on how humans perceive and negotiate space during wayfinding. They discuss successful wayfinding as “the ease with which the spatial layout of a setting is able to be understood and mapped”. Furthermore, they propose an integrated approach where the layout of a setting and the wayfinding information system provide support to solve users’ wayfinding problems. Planning for wayfinding considers users–their strengths, limitations and behaviours–with the architectural and graphical components of a system. 

Understanding the user, their goals and the subsequent actions and behaviours required to reach a destination is essential for the design of successful wayfinding systems (Arthur and Passini 27). Users are no longer viewed as one homogenous group of able-bodied individuals. Instead, many users have impairments that affect their perception, cognition and mobility, all of which have an impact on their wayfinding abilities (Arthur and Passini 62). 

A cognitive map (also referred to as mental model or mental map)is an image that humans create in their minds to help them understand the world around them. Donald Norman (31) describes how humans develop these images:

People create mental models of themselves, others, the environment and the things with which they interact. These are conceptual models formed through experience, training, and instruction.

The term wayfinding refers to a human’s cognitive and behavioral ability to find their way from one destination to another (Golledge 24). Humans encode information about their environment to develop cognitive maps for navigating a space (15). In terms of wayfinding, learning an environment occurs either by: 1) experiencing it; or 2) learning the layout from an overlooking vantage point or visual aid. (Golledge 9). 

Visual aids (maps, directories, signs and schedules/timetables, etc.) assist users in wayfinding by helping them create accurate mental models with visual and text-based information. Armed with this information, users understand the overall layout and relationships between the buildings, room locations in relationship to each other, the pathways, access points and the activities, along with where and when they occur. All of this information supports users in finding their desired destinations. In a complex system, especially when it’s an unfamiliar environment where visitors may be anxious or distracted (for example, in a hospital), an incomplete or inaccurate mental model and lack of adequate visual aids could result in difficulties in navigation. The characteristics associated with useful visual aids for wayfinding include: 

  • Message clarity
  • Coordination of information
  • Adequate information at decision points
  • Adequate lighting
  • Legibility
  • Accuracy of information
  • Reliability of information
  • No physical obstructions (Arthur and Passini 184-185).

A user must be able to understand or read the environment to create an accurate mental image of an environment to make a “plan” for wayfinding. Thus, the layout should be organized to have: a distinct identity for each spatial unit; units that are grouped into destination zones; and linkages between units and zones (Arthur and Passini 113). Furthermore, the environmental information should communicate the circulation system using the form and volume of a building and architectural features such as: legible entrances and exits and contrasting corresponding facades; visual cues through landscaping and arrangement of paths; gates and transition points that communicate a destination point or zone. Redundancy of information, both architectural and graphic, helps to ensure effective communication in wayfinding. (Arthur and Passini 116-139).

In this study, we use Arthur and Passini’s approach as the basis for a framework to reveal shortcomings in wayfinding systems by focusing on the interactions between the user, the architectural components and the graphical components of a system (see figure 1).

Figure 1. Framework for evaluating wayfinding systems

Methodology and Results

Context: The Physical Recreation Centre

In this study, our interests lay in finding an approach to test an existing wayfinding system and formalizing a framework for evaluation. We chose the Physical Recreation Centre on a university campus because the complex serves a variety of users: the university community (students, faculty and staff) as well as members of the public. The needs for each of these user groups vary depending on the activities they engage in, whether they live or work on campus or visit campus solely to use the athletic facilities. However, all patrons share a common goal; all are interested in arriving with ease at the right place, at the right time to enjoy the athletic facilities. 

An initial visit to the centre revealed various shortfalls with the current wayfinding system. The most obvious indication of problems with the system was the addition of supplementary signs, installed over time to support the initial standard university signing system. (The Centre’s management deemed it necessary because the system in place was not sufficiently supporting user navigation.) In all, we observed three types of signing “systems” in use: 1) the original standard university designed signs, mounted on the ceiling and walls; 2) newer wall graphics and wall mounted signs; and 3) ad hoc, laser printed signs and maps, affixed to the walls (see figures, 2 3, 4). In addition, we found inconsistencies in the support materials (colour coding and room names on class schedules and directories) and the “repurposing” of rooms, originally designed for a specific sport but then used for a different and unrelated activity. The athletic centre consists of five buildings, constructed over several years. The buildings are connected by: a bridge link, an enclosed walkway, a common area and a concourse with a reception area that functions as the access point to the “members only” areas.   

Our goal was to explore how to evaluate existing wayfinding systems. While we identified numerous problems, we were not concerned with revealing every instance that might need improvement. Rather, we were interested in extracting qualitative information about the user experience as revealed through actions and thoughts while users performed typical wayfinding tasks.

Figure 2. Standard university signage system

Figure 3. Newer, wall mounted signs

Figure 4. Ad hoc, laser printed signs

The Study

The study consisted of three consecutive parts: (1) a pre-test-questionnaire, eliciting demographic information and questions about participants level of familiarity with the Physical Recreation Centre and the kind of strategies they use when navigating through unfamiliar environments; (2) a walk through the athletic centre, during which participants searched for eight locations across the complex; (3) a post-test questionnaire, asking participants about their experience and suggestions for improvement. 

For the walkthrough, we gave participants a list of tasks, requiring them to walk through the Physical Recreation Centre and find eight locations. To support participants in navigating, we provided a set of visual aids (a campus map, a paper directory map for the main level and second level of the complex, a schedule of all fitness classes) (see figures 5, 6, 7) and encouraged them to usewall and overhead signs. If necessary, they could use their mobile phones and digital maps. We encouraged participants to resist asking for directions, however, when faced with a situation where they were unable to continue, they could ask other patrons or staff. During the walk through, participants used a “think aloud” method, so we could capture their experience. Additionally, they responded to some questions at each location. Each session lasted approximately one hour.

Figure 5. Campus map

Figure 6. Paper directory maps for the 1stand 2ndlevels

Figure 7. Schedule/timetable of fitness classes with locations, times    

Results and Analysis

The data collected included participants written responses to questionnaires, answers to questions at the end of each task and researcher observations of their comments and actions while searching for locations. To reveal the hidden insights, the analysis began with each researcher individually looking for possible common themes that emerged from the data. Next, the themes were compared, discussed and grouped. At that point, researchers separately re-coded the entire data set based on the themes. Results of the re-coding were compared, discussed and negotiated where discrepancies occurred, until we were satisfied. When we reached agreement, the total number of instances for each theme was tallied and the results were grouped into three broad categories as expressed by the Framework Evaluating for Wayfinding Design:1) Users – User Characteristics; 2) Graphic Components –Signs, Maps and Schedules; 3) Architectural Components – Physical Places, Buildings and Other Structures.

Users’ perceptions and actions during the walkthrough revealed various problems with the existing wayfinding system at the Physical Recreation Centre. Navigation errors and confusion resulted from the mismatch of the users’ mental models and the realities confronting them. This mismatch can be explained by inadequacies in the graphic information provided to participants and the structural complexities found in the physical environment at the Centre. While this section points out some of the findings from the study, it is not exhaustive and therefore, we do not provide a point-by-point recommendation for each finding. 

1) Users: User Characteristics

Responses to the questionnaires, participant comments and researcher observations during the walkthroughs revealed information about the participants (users) (see table 1). Using convenience sampling, we recruited eight participants (four female, four male) with an average age of 28 years. Six participants identified themselves as campus students, whereas two volunteers were members of the public and therefore not part of the university community. Interestingly, five participants indicated they had never visited the athletic centre before. In addition, all participants indicated they use digital maps such as Google Maps when navigating in unfamiliar environments and six indicated they use directional signs. Previous knowledge emerged as the most common strategy used in the successful completion of tasks. However, there were 14 instances where participants misinterpreted information or were visibly confused or disoriented.

Table 1
Study Results, Users: User Characteristics

Information about the user, their strengths, limitations and preferences proves essential in understanding the effects the graphic and architectural components have on the user experience and provide insight for improvements. As would be expected, those most familiar with the centre were more successful in finding the specified locations because they used previous knowledge. The stated preference for using digital devices for navigating suggest that digital technology might be an option that designers could consider to support the users. 

2) Graphic Components – Signs, Maps and Schedules

Participant comments and researcher observations during the walkthroughs revealed instances of errors made by participants due to the graphic information(see table 2). The errors could be attributed to: incomplete information on signs; incomplete information on maps and schedules; absence of signs, signs not visible, or signs not well positioned; lack of coordination between navigation aids; misleading information; poor use of colour coding; and the lack of clarity due to design. 


Table 2
Study Results, Graphic Components: Signs, Maps and Schedules

While each of the instances described above require attention to ensure successful navigation, the lack of coordination between navigation aids in relation to the Group Cycling Room is a particularly striking example of how signs, maps and schedules contributed to difficulties in finding a location. This example involves multiple aids and demonstrates a good case for revisiting a wayfinding system that has suffered from an attempt to solve problems as they occur in an ad hoc manner, rather than addressing the system as a whole. Variations of the name for the Group Cycling Room were as follows: 

  • the schedule provided to participants read, “Group Cycling Room”;
  • the directory plan used, “Spinning Room”; and finally, 
  • the identification sign at the entrance was labeled, “Squash Court 6” (see figure 8).

Figure 8.The schedule, directory plan and room identification sign use different names for the same room.

This lack of coordination occurred over time as new navigational aids were added, underscoring the importance of having an overall view of the entire wayfinding system functionality. The space that is now the Group Cycling room was originally designed as a squash court but was repurposed for spinning classes. While the use of multiple names caused navigation problems, the location and structure of the Group Cycling Room also contributed to the difficulties encountered in locating the room. These will be discussed in the following section.

3) Architectural Components – Physical spaces, Buildings and Other Structures

Participant comments and researcher observations during the walkthroughs revealed instances of errors made by participants due to the architectural components(see table 3). Barriers to the successful completion of tasks included: rooms where the spatial layout was not easily discernible; passageways that were added to connect buildings; the location of rooms where related/similar activities occur were not located in close proximity to one another; and doors and passageways that were either locked or forced participants to exit the building.

Table 3
Study Results, Architectural Components: Physical Spaces, Buildings and Other Structures

Locating a spinning class was particularly difficult for a variety of reasons. Firstly, the Group Cycling Room was located in a repurposed squash court. The room was equipped with spinning bikes and the opaque door was replaced with a dark tinted glass door. However, the enclosed squash area where the room was located was isolated and “invisible” from other locations in the building. The courts spanned two floors with access both at the court level and the gallery level, but the directory map did not adequately represent the open portion of the courts on the second floor gallery level (in addition “Squash Court 6” where the Group Cycling Room was located was not detectable on the gallery level because it was enclosed by walls). Most participants entered the gallery in error and were forced to cross reference two floor plans and the schedule and had to match these with the door identification sign in the squash court area on the first floor. As mentioned in the previous section, participants noted the lack of coordination between navigation aids and the multiple names for the “Group Cycling Room”, “Spinning Room” and “Squash Court 6”. In combination, the characteristics of the physical space and the inconsistencies in the graphic information caused confusion for participants trying to find the location of a spinning class.

Other architectural components also contributed to making navigation in the Centre difficult. Members use a card to access most of the facilities via the reception desk in the main concourse. Buildings are connected with a bridge, linking the Athletics Building to the Gym and Ice House, as well as a walkway from the Fieldhouse to the Athletics Building. While these links make the member’s area secure, they also force patrons to take much longer and more complicated paths. 

These examples demonstrate the complexity of the layout and the circulation system in this set of connected buildings and the problems that result from adding structures over time to accommodate new activities. The shortcomings in the current layout and the circulation system fail to help users create an accurate mental model of the individual units and how they are connected and caused difficulties in wayfinding (see figure 9). Because substantial funding would likely be required for structural renovations, recommendations for improving the wayfinding system are limited to redesigning graphic information. This would include the maps, signs, schedules/timetables and other supporting information, both in terms of the visuals and the terminology used to describe rooms, buildings and activities. 

Figure 9. While Physical Recreation Centre is made up of five distinct buildings (left), the mental model created in the user’s mind does not accurately reflect the user experience (right).

Recommendations for the Physical Recreation Centre

The results from our study represent a small fraction of the areas that require improvement at the Physical Recreation Centre. As stated previously it was not our intention to find all instances but instead, we wanted to find an approach to conduct effective post-occupancy evaluations. However, as members of the university community, we were interested in bringing forward our results in hopes that administrators and managers at the Physical Recreation Centre might recognize the need to examine the effectiveness of their wayfinding system for the benefit of its patrons. Recommendations for next steps included: a larger, more comprehensive walkthrough study with different user groups, a full audit of existing signs, a space usage audit and path analyses, all of which would provide administrators, managers and designers a full understanding of their wayfinding system status and a blueprint for improvement.

Contribution to the field, and implications for theory and practice

This study demonstrates the walkthrough as a method and a three-part framework for evaluation of wayfinding systems, post-occupancy and serves as a model to isolate areas for improvement. While the data we collected is specific to the site selected for the present study, it reveals an approach in which designers can identify areas of concern by observing the interaction between users, the graphic components and the architectural components in the context of wayfinding. Collecting, categorizing, analyzing and documenting data is the first step towards developing a plan for improvement. It is important to note that allowing themes and patterns to emerge from the data collected from typical users performing tasks at a facility is essential for understanding site-specific problems. Additionally, we presented varied methods that can be used to collect data on wayfinding behaviours. Further research can explore how these methods might broaden and strengthen post-occupancy evaluation, especially the methods that capture information directly from the user’s thoughts and actions. Collecting multiple forms of data could be especially valuable in more complex environments (e.g., hospitals), where anxious patients may have to navigate multiple locations during a single visit. 

This study contributes to professional practice, education and the growing field of experiential graphic design by: 1) providing feedback to improve the usability of new wayfinding systems, post-occupancy; 2) informing the redesign of existing systems; 3) adding to the knowledge base for curriculum development for the education of designers; and 4) contributing to usability research on wayfinding.

The evaluation of wayfinding systems requires a holistic approach to capture the complexities involved in navigating physical spaces. Ideally, evaluation should involve users at various points of the design and development process, from preplanning to post-occupancy. 


Works Cited

Arthur, Paul, and Romedi Passini. Wayfinding: People, signs, and architecture. 1992.

Reissued as a collector’s edition in 2002 by Focus Strategic Communications, Inc.

Bosman, Ellen, and Carol Rusinek. “Creating the user-friendly library by evaluating patron perception of signage.” Reference Services Review 25.1 (1997): 71-82.

Brown, Barbara, Holly Wright, and Craig Brown. “A post-occupancy evaluation of wayfinding in a pediatric hospital: Research findings and implications for instruction.” Journal of Architectural and Planning Research (1997): 35-51.

Berger, Craig, and James Grieshaber. Wayfinding: designing and implementing graphic navigational systems. RotoVision, 2005.

Calori, Chris, and David Vanden-Eynden. Signage and wayfinding design: a complete guide to creating environmental graphic design systems. John Wiley & Sons, 2015.

Chang, Yao-Jen, et al. “A context aware handheld wayfinding system for individuals with cognitive impairments.” Proceedings ofthe 10th international ACM SIGACCESS conference on Computers and accessibility. ACM, 2008.

Coughlan, James, and Roberto Manduchi. “Functional assessment of a camera phone-based wayfinding system operated by blind and visually impaired users.” International Journal on Artificial Intelligence Tools 18.03 (2009): 379-397.

Dogu, Ufuk, and Feyzan Erkip. “Spatial factors affecting wayfinding and orientation: A case study in a shopping mall.” Environment and behavior 32.6 (2000): 731-755.

Golledge, Reginald G., ed. Wayfinding behavior: Cognitive mapping and other spatial processes. JHU press, 1999.

Kanakri, Shireen, et al. “Wayfinding systems in educational environments.” Environment and Ecology Research 4.5 (2016): 251-256.

Kishnani, Nirmal. “Evaluation of the wayfinding system at Stansted Airport.” Visual Information For Everyday Use: Design And Research Perspectives (2003): 257.

Kulyukin, Vladimir, et al. “iwalker: Toward a rollator-mounted wayfinding system for the elderly.” RFID, 2008 IEEE International Conference on. IEEE, 2008.

Lawton, Carol A. “Strategies for indoor wayfinding: The role of orientation.” Journal of environmental psychology 16.2 (1996): 137-145.

McLendon, Charles B., and Mick Blackistone. Signage: graphic communications in the built world. McGraw-Hill Companies, 1982.

Mollerup, Per. Per Mollerup: Wayshowing. Lars Muller Publishers, 2005.

Norman, Donald. The design of everyday things: Revised and expanded edition. Basic Books (AZ), 2013.

Schwarzkopf, Sarah, et al. “What lab eye tracking tells us about wayfinding: A comparison of stationary and mobile eye tracking in a large building scenario.” Eye Tracking for Spatial Research, Proceedings of the 1st International Workshop (in conjunction with COSIT 2013). 2013.

Veldkamp, D., et al. “The use of visual landmarks in a wayfinding system for elderly with beginning dementia.” Med-e-Tel 2008 proceedings CDROM(2008): 161-165.