Context
Context
Reach One Youth: Evaluating A #GiveTLH Winner
Reach One Youth: Evaluating A #GiveTLH Winner
The founders of Reach One Youth Sports noticed that their former YMCA youth basketball players were struggling to transition into adulthood, leading them to create an organization that has an enduring impact on youth development.
However, users are uncertain about where and how to access specific information about Reach One Youth's programs and services and they need a website that helps them find and discover information without endlessly scrolling and guessing.
The founders of Reach One Youth Sports noticed that their former YMCA youth basketball players were struggling to transition into adulthood, leading them to create an organization that has an enduring impact on youth development.
However, users are uncertain about where and how to access specific information about Reach One Youth's programs and services and they need a website that helps them find and discover information without endlessly scrolling and guessing.
The founders of Reach One Youth Sports noticed that their former YMCA youth basketball players were struggling to transition into adulthood, leading them to create an organization that has an enduring impact on youth development.
However, users are uncertain about where and how to access specific information about Reach One Youth's programs and services and they need a website that helps them find and discover information without endlessly scrolling and guessing.
My Role
Led the research team, facilitated communication between the organization and UX teams, and assisted in redesign.
Outcome
Evangelized UX research and garnered support for incremental site improvements.
Project
Mentor Me Collective UX Research & Design Fellowship
Methods
Surveys, User Interviews, Usability Testing, Card Sorting, and Heuristic Evaluation.
Team
1 UX researcher, 2 designers, and myself (research & design)
Timeline
September 2022 - November 2022
Results
Results
Key Findings
Key Findings
The homepage is the first point of contact that communicates value to first-time users. Without an organized navigation with consistent and clear content and corresponding visual elements, users may miss out on programs and services that meet their needs.
The homepage is the first point of contact that communicates value to first-time users. Without an organized navigation with consistent and clear content and corresponding visual elements, users may miss out on programs and services that meet their needs.
Unstable information architecture
Unstable information architecture
The website has some visual hierarchy with headings and subheadings. For frequent visitors and visitors familiar with common non-profit website patterns, it is difficult to find specific information because the content is unorganized and constantly shifting with videos and news.
The website has some visual hierarchy with headings and subheadings. For frequent visitors and visitors familiar with common non-profit website patterns, it is difficult to find specific information because the content is unorganized and constantly shifting with videos and news.
Disjointed Navigation
Disjointed Navigation
Primary and secondary navigation labels were disjointed and category options were not merged to focus on users main tasks.
Primary and secondary navigation labels were disjointed and category options were not merged to focus on users main tasks.
Lack of understanding of content value and context
Lack of understanding of content value and context
Users expect a description of programs, roles, and associated fees before signing up. However, the website often presents content (images, videos, graphics, and forms) without any context (accompanying written content), leaving the user to guess the significance.
Users expect a description of programs, roles, and associated fees before signing up. However, the website often presents content (images, videos, graphics, and forms) without any context (accompanying written content), leaving the user to guess the significance.
Prototype we created to help stakeholders visualize our solutions from report.
View Prototype
Prototype we created to help stakeholders visualize our solutions from report.
View Prototype
Prototype we created to help stakeholders visualize our solutions.
Prototype we created to help stakeholders visualize our solutions.
Kickoff
Kickoff
Building The Relationship
Building The Relationship
I scheduled and moderated our kickoff presentation. Most of the meeting involved intros, information gathering, and scoping exercises to create our research and design brief together.
I scheduled and moderated our kickoff presentation. Most of the meeting involved intros, information gathering, and scoping exercises to create our research and design brief together.
Research Strategy
Getting Solutions Implemented
From past experience working with organizations that are hesitant to embrace research, I aimed to get buy-in for our recommendations by being transparent and inviting during the research process. We aimed to emphasize small changes the organization could make and test to measure improvement before investing in a full redesign.
Research Strategy
Getting Solutions Implemented
From past experience working with organizations that are hesitant to embrace research, I aimed to get buy-in for our recommendations by being transparent and inviting during the research process. We aimed to emphasize small changes the organization could make and test to measure improvement before investing in a full redesign.
Stakeholder questions that stood out to me:
What information is not present on the website that users would expect?
How can we improve the experience of volunteers navigating the website?
What information is not present on the website that users would expect?
How can we improve the experience of volunteers navigating the website?
Considerations & Constraints:
1.
Small Budget
2.
Limited Platform Changes
3.
Rely on volunteers for tech help
4.
Users with low digital literacy
Considerations & Constraints:
1.
Small Budget
2.
Limited Platform Changes
3.
Rely on volunteers for tech help
4.
Users with low digital literacy
Considerations & Constraints:
1.
Small Budget
2.
Limited Platform Changes
3.
Rely on volunteers for tech help
4.
Users with low digital literacy
User Research
User Research
Understanding users frustrations, needs, & actions when interacting with Reach One’s Website
Understanding users frustrations, needs, & actions when interacting with Reach One’s Website
We probed for information and prior research on the organization's user segments during the kickoff meeting. Based on their responses and questions, we learned that the focus for this research study was volunteers and parents/guardians. The three primary user segments identified were:
We probed for information and prior research on the organization's user segments during the kickoff meeting. Based on their responses and questions, we learned that the focus for this research study was volunteers and parents/guardians. The three primary user segments identified were:
Volunteers, mostly local university students, that have experience playing youth sports or mentoring. Seeking volunteer opportunities for college credit.
Youth athletes, ages 5-18, that often learn about their programs from word-of-mouth or social media rather than their website.
Parents or guardians from low-income backgrounds with low to moderate tech literacy. Seeking a central location or platform to access information.
Volunteers, mostly local university students, that have experience playing youth sports or mentoring. Seeking volunteer opportunities for college credit.
Parents or guardians from low-income backgrounds with low to moderate tech literacy. Seeking a central location or platform to access information.
Volunteers, mostly local university students, that have experience playing youth sports or mentoring. Seeking volunteer opportunities for college credit.
Parents or guardians from low-income backgrounds with low to moderate tech literacy. Seeking a central location or platform to access information.
Jobs-To-Be-Done
Jobs-To-Be-Done
Realigning on user actions
Realigning on user actions
Our team created proto-personas in the discovery phase of our research to help build empathy with the users. When, research and design teams were later misaligned on the user’s jobs and the value the product could bring to address their needs, we had a joint Jobs-to-be-Done' (JTBD) workshop.
Our team created proto-personas in the discovery phase of our research to help build empathy with the users. When, research and design teams were later misaligned on the user’s jobs and the value the product could bring to address their needs, we had a joint Jobs-to-be-Done' (JTBD) workshop.
Parent or Guardian Desired Outcome
As a
the
it takes to
to
I want to
parent
Identifier
decrease
Direction
time
Metric
find information
Object of Control
register my child for the relevant program.
Contextual Clarifier
User Research
User Research
Test Design
Test Design
I recruited 6 participants. I collaborated with the organization to recruit participants in their network. After an unsuccessful first round of recruitment, I created a screener survey survey to recruit participants matching their user profiles online from their community.
I recruited 6 participants. I collaborated with the organization to recruit participants in their network. After an unsuccessful first round of recruitment, I created a screener survey survey to recruit participants matching their user profiles online from their community.
We designed our study with a hybrid-research approach.
The administered remote usability study (30- 40 min) included the following:
Think-aloud protocol
Homepage walkthrough
4 pre-defined tasks
Post-session questionnaire
10-minute pre-session interview
The administered remote usability study (30- 40 min) included the following:
Think-aloud protocol
Homepage walkthrough
4 pre-defined tasks
Post-session questionnaire
10-minute pre-session interview
Usability Tasks
Usability Tasks
Before conducting the usability sessions, the designers performed a heuristic evaluation of the website. Based on their prior knowledge of critical accessibility and aesthetic issues, we focused the majority of tasks on the findability, discoverability, and comprehensibility of the content.
Before conducting the usability sessions, the designers performed a heuristic evaluation of the website. Based on their prior knowledge of critical accessibility and aesthetic issues, we focused the majority of tasks on the findability, discoverability, and comprehensibility of the content.
Locate registration costs.
Find the responsibilities of a volunteer.
Determine the benefits of participation in Reach One’s programs.
Make a donation.
Locate registration costs.
Find the responsibilities of a volunteer.
Determine the benefits of participation in Reach One’s programs.
Make a donation.
Usability Testing
Usability Testing
Results
Results
We combined and analyzed responses from the interviews into a thematic affinity diagram. We identified a total of 6 themes that presented as issues for the current website.
From the post-test questionnaire (SUPR-Q), we calculated the raw usability score (illustrated below). From the tasks, we measured and illustrated the task completion rate and average time.
We combined and analyzed responses from the interviews into a thematic affinity diagram. We identified a total of 6 themes that presented as issues for the current website.
From the post-test questionnaire (SUPR-Q), we calculated the raw usability score (illustrated below). From the tasks, we measured and illustrated the task completion rate and average time.
Main qualitative findings:
Main qualitative findings:
First-time visitors seek clarity on the organization’s programs, services, and its value (5 users)
Expect program and service information before applying (4 users)
Forms are too long and not able to enter all required data (4 users)
Information is difficult to locate due to its inconsistent placement (5 users
Confusion over categorization of navigation and labels (4 users)
Confusion over relationship between content (images, graphics, and text) (4 users)
First-time visitors seek clarity on the organization’s programs, services, and its value (5 users)
Expect program and service information before applying (4 users)
Forms are too long and not able to enter all required data (4 users)
Information is difficult to locate due to its inconsistent placement (5 users
Confusion over categorization of navigation and labels (4 users)
Confusion over relationship between content (images, graphics, and text) (4 users)
Main quantitative findings:
Main quantitative findings:
Completion Rate
Completion Rate
Completion Rate
Completion Rate
Completion Rate
Completion Rate
Average Time
Average Time
Average Time
Average Time
Average Time
Average Time
User Research
User Research
Revisiting the Navigation
Revisiting the Navigation
3 ouf of 6 users verbally expressed that the categories and labels could be clearer for users. We conducted an unmoderated open card sort with Optimal Workshop to understand how users would group the content. From the card sort 70% of the participants agreed with the following labels and category groupings.
3 ouf of 6 users verbally expressed that the categories and labels could be clearer for users. We conducted an unmoderated open card sort with Optimal Workshop to understand how users would group the content. From the card sort 70% of the participants agreed with the following labels and category groupings.
Ideating on opportunities
How might we organize information on the Reach One Youth website to make programs and volunteer opportunities easily accessible for novice users?
Ideating on opportunities
How might we organize information on the Reach One Youth website to make programs and volunteer opportunities easily accessible for novice users?
Prioritizing Improvements
Prioritizing Improvements
Solution Space
Solution Space
The homepage is the first point of contact that communicates value to first-time users. Without an organized navigation with consistent and clear content and corresponding visual elements, users may miss out on programs and services that meet their needs.
The homepage is the first point of contact that communicates value to first-time users. Without an organized navigation with consistent and clear content and corresponding visual elements, users may miss out on programs and services that meet their needs.
Simplified labels that reflect user’s actions
Simplified labels that reflect user’s actions
The ways to get involved were spread across the top-level navigation. We removed some top-level labels and added them into the subcategories based on our findings from the open card sort. We also created a 'Get Involved' label to house mentoring and volunteering.
The ways to get involved were spread across the top-level navigation. We removed some top-level labels and added them into the subcategories based on our findings from the open card sort. We also created a 'Get Involved' label to house mentoring and volunteering.
Dedicated news section
News and events took up the top-half of the homepage, making it difficult to understand the organization’s mission. We incorporated a dedicated announcements section on the homepage that can be updated consistently in one location for parents and volunteers.
Simplified forms that show progress
Simplified forms that show progress
Forms took up entire webpages. We recommended dividing forms into smaller steps to make them less intimidating and easier to understand. Adding a progress bar allows busy parents or students to gauge how much time is left to complete the form.
Forms took up entire webpages. We recommended dividing forms into smaller steps to make them less intimidating and easier to understand. Adding a progress bar allows busy parents or students to gauge how much time is left to complete the form.
Getting parents answers and saving time communicating
Getting parents answers and saving time communicating
The organization discussed the idea of building a “parent portal” early in the project. We advocated for a FAQ section that they could implement quickly. This would allow the organization to provide information users want and need to know in a familar format.
The organization discussed the idea of building a “parent portal” early in the project. We advocated for a FAQ section that they could implement quickly. This would allow the organization to provide information users want and need to know in a familar format.
Reflection
Key Takeaways
Less is more when it comes to writing reports. I’d keep the findings short and visual.
I’d use SEQ after a task to get standardized data and be able to compare what part of the website has the most problems. There were moments I observed during the session and after the session when analyzing the SUPR-Q results that I noticed users contradicted themselves. I'd probe after the task for more context.
Reflection
Key Takeaways
Less is more when it comes to writing reports. I’d keep the findings short and visual.
I’d use SEQ after a task to get standardized data and be able to compare what part of the website has the most problems. There were moments I observed during the session and after the session when analyzing the SUPR-Q results that I noticed users contradicted themselves. I'd probe after the task for more context.
Reflection
Key Takeaways
Less is more when it comes to writing reports. I’d keep the findings short and visual.
I’d use SEQ after a task to get standardized data and be able to compare what part of the website has the most problems. There were moments I observed during the session and after the session when analyzing the SUPR-Q results that I noticed users contradicted themselves. I'd probe after the task for more context.