Usability Study for the Urban Ecology Center Website
Introduction
The Urban Ecology Center (UEC) is a local organization that strives to champion environmental research and education for natural resources within all age groups, bringing the community together. The website is unique, has a consistent theme throughout, and interesting interaction and graphics. This poses a question:
Target users in the community and in education use this website for registering for outdoor programming and educational events, becoming involved as a volunteer, donating, renting venues, and purchasing merchandise. A usability study that focused primarily on navigational elements and flow was conducted over a 2-week period to determine whether issues exist and where focus should be placed for continuous improvement. Key research questions that helped to inform this study include:
Methodology
Recruitment
Participants were recruited though an online screener containing 6 questions relevant to recruiting users that fit the user profile, ensuring individuals that were actively involved and uniquely invested in the behavior and values associated with this organization (Fig.2).
- Age – over the age of 18
- Geographiclocation – local or planned visit
- Leisurevocations – recent/regular participation in outdoor activities
- Technology – completed one registration or online purchase this year
- Opinion – felt natural resources are important.
- AvailableTech – had access to a laptop or desktop
Participants
Seven qualified participants were selected that fit the needed criteria. Additional information obtained during pre-test questioning validated participant selection as testers indicated they valued natural resources and regular outdoor activity participation
Study Design
The 7 individuals selected were enthusiastic about participation and engaged in 45-60 minute in-person or recorded Zoom sessions on their own laptops and computers to observe navigational behaviors, problem-solving actions, record emotional response and verbal input, as well as obtain quantitative data from metrics and surveys.
Establishing User Task Goals
Four realistic tasks simulated common user goals were and measured on time for completion and/or task completion. The tasks were carefully selected and ordered to simulate the effects of repeated use and learning throughout the test.
Tasks 1and 2 were similarly located program searches to identify effects of learning and any reproducible issues.
Tasks 3 and 4 benefitted from earlier website exploration and helped to validate navigational efforts (Fig. 4).
Defining the Questions
Quantitative data uncovered areas of interest or focus, while qualitative data and verbal input from participants on probing provided valuable reasoning behind the data and participants’ attitudes and behaviors. Questions included:
Pre-test
- Open-ended, rapport and interest building questions
Post-task
- Task success/completion (Tasks 1 -4)
- Task timed completion (Tasks 3 and 4)
- Likert Scale 7-point survey questioning navigational ease
- Open-ended probing questions to establish score confirmation
Post-test
- Standard 10-question System Usability Score (SUS) to assess perceived system usability (Fig.5).
- Open-ended comments elicited for system likes/dislikes and observations.
Looking at the Data
Task Success
All four participants completed the four tasks successfully; however, each encountered a variety of navigational obstacles along the way.
Timed Completion
Tasks 3 and 4 (The Hive and shopping) were viewed for time required for navigation, defined by completion of the question by the moderator to the time the correct location was selected. The mean time for these tasks was 60 and 83 seconds, respectively. One user that found the Hive within 10 seconds stated that she had used the site and recalled seeing it before. One user, that took 120 seconds to locate the merchandise area, nearly abandoned the task with frustration, before it was located.
Interesting Findings
End of Task Questions
When users were asked to rate ease of navigability, some key findings were exposed:
- Despite similarity between tasks 1 and 2, the second task was rated lower.
- The search for the Hive was rated somewhat easier, despite increased time and effort.
- The shopping task was rated lowest at a mean of 3.57.
Post-test Questions
On completion of the study, participants completed a standard 10-question, 5-point system usability questionnaire to rate the system as a whole resulting in a mean SUS score of 60.7 which is below industry standards. The results were perplexing as the previous as navigational ease scores were firmly at a mean of 4.25 on a 7-point scale (7=very easy). This score indicates a harsher rating of the system as a whole –but why?
Considerations
- Participants may have considered the cumulative effect of multiple issues, despite single task tolerance.
- Subjectivity within personal ratings may even out with a greater participant pool.
- Increased time for later tasks, may have increased frustration levels.
Complementary Analytic Data
A deeper dive into the site analytics would provide helpful information to further user studies with custom reports:
- Page/session times and abandonment – to help determine time spent and dropped tasks or forms.
- Internal search terms – may help reveal what terms users are directly searching for on the site.
- Setup custom click path tracking – to help define user journeys and uncover unusual patterns to explore further
Design Issues, Implications, and Recommendations
Calendar search engine functionality
When 3/7 participants used the search bar within the calendar, the search was unsuccessful as it required exact words and exact word forms.
Decreased visibility of calendar search elements
While the wave extension did not find a WCAG 2.1 violation for contrast and color, the container area for the search bar and visible filters was frequently missed by users in their initial search for programs in tasks 1 and 2.
Unexpected category groupings
The category filter titles within the calendar were inconsistent with users ’expectations. All 7 participants did not expect the Botany Walk to be categorized under “Classes and Workshops.” Users stated that they might have located it more easily if it had been under other or multiple appropriate categories
Confusing naming conventions
Unfamiliarity with terminology or thematic titles that do not provide immediate clarity may limit correct identification of desired events or programming, resulting in frustration or abandonment. Only one participant was able to associate “botany” with a plant/nature walk right away.
Finding the Hive sustaining community
While it is logically placed in the support menu as a link and a feature, it was initially missed in 4/7 attempts.
- The “Donate" link left uncertainty without Hive labeling, resulting in abandonment.
- Using the search bar did not directly bring up the Hive information page.
- No Hive features existed when scrolling the landing page.
Users did not think that the additional effort and time would prevent them from donating if that was their intention, but they did want to have confirmation that they were in the Hive as a member and wished it would have been easier to get there with certainty.
Locating the merchandise area
Finding the UEC merchandise area was a pain point for all participants. Users took an average of 6 wrong turns before finding the webstore. Common complaints were expectations of location with all participants expecting to see it under a different or its own menu or more prominently placed.
Conclusion
Participants in this study agreed that the website was visually engaging with meaningful images. Generally experienced technology-users, participants were able to achieve their task goals successfully; however, pain points in navigation, categorization, naming conventions, consistent messaging, and visual placement exist. Participant comments indicated that there was a discrepancy between the users’ mental models of conventional design and unique differences within this site. By considering recommended design standards and applying modifications where needed to clarify existing labels, optimize search functions, improve consistency of messaging, and provide visual focus where needed, this website has strong potential to maintain its beauty and uniqueness while improving the user experience and building the community the organization desires.
