Contemporary user experience design requires the investigation of behaviors associated with emerging technology. Grounded in research, each project embraces the needs, attitudes, and motivations of people. The goal of each project to deliver value to people who use them and have a positive impact business metrics.
AUGMENTED REALITY
Investigating the relationships between screen-based mental models and extended reality
SUMMARY
A generative research study on ancillary augmented reality systems illuminates opportunities for integrated physical and digital design experiences
This self-directed project explores how building a deeper understanding of users inspires solutions. Inspiration for innovation spans from the built environment to mobile device screen. A survey and first-click usability studies provided data on interactions and mental models. The retail context created the scaffolding for the extended reality study. This includes brick-and-mortar environments and the application of augmented reality functions to eCommerce.
The objective of this generative project emphasizes the identification of a design opportunity. More specifically, it expands the utility of augmented reality functions. The design opportunity developed through the analysis of existing behaviors and motivations. Work on this project occurred over a two-week period in June 2022. Shifting patterns of content generation and consumption are ongoing topics of interest.
GOALS
Goal One: Understand participants' thought processes and prior experiences with extended reality
Goal Two: Determine the impact of traditional retail marketing channels on decision-making
Goal Three: Understand participants' existing shopping motivations
Retail merchant implementations are crucial factors in extended reality experiences. For example, how are touchpoints that link physical and digital environments valued? In addition, how might experiences integrate with existing software systems? How might extended reality experiences integrate with merchants’ point of sale systems? For merchants’ inventory systems? These questions create the foundation of another generative research study.
USER RESEARCH
An online survey collected data on thought processes and prior experiences with extended reality. More specifically, the survey differentiated between augmented reality and virtual reality. The survey contained a series of closed, multiple-choice questions and open-ended questions. As a preliminary survey to measure attitudes and preferences, participants are from the United States.
Demographically, the collected data from twenty-four responses contains all age groups except for being under the age of 18. This is due to legal conditions of participant recruitment. Three participants were between the ages of 18 and 24 (12.50%). Seven participants were between the ages of 25 and 34 (29.17%). Four participants were between the ages of 35 and 44 (16.67%). Two participants were between the ages of 45 and 54 (8.33%). One participant was between the ages of 55 and 64 (4.17%). Seven participants over the age of 64 (29.17%).
The data collected from a series of Likert scale questions illustrates the role of consumer marketing. Participants are slightly more knowledgeable about virtual reality than augmented reality. The responses illustrate the design challenge of differentiating between extended reality approaches. How might simple and plain language reduce the technological complexity of extended reality? Participants being more knowledgeable about virtual reality may be due to the reach of Meta Quest devices.
For entertainment, my kids play alot of these games.
—Participant response when asked about the role augmented (AR) or virtual reality (VR) technology plays in life.
Based on survey responses, the predominant role of augmented reality and virtual reality is gaming. Six of the twenty-four participants mentioned video games in their responses (25%). One participant responded with the role of augmented reality to try on makeup (4.17%). One participant responded with using the role of augmented reality as social media filters (4.17%).
It does not play much role in my life. The only role AR has played is Snapchat filter.
—Participant response when asked about the role augmented (AR) or virtual reality (VR) technology plays in life.
The open-ended question responses illustrate the current mental models of extended reality. Based on this group of participants, there is a relationship between virtual reality and gaming as a form of entertainment. Augmented reality has a relationship with social media filters and merchandise try-ons. These conditions raise the question of the role of social interactions in extended reality. For example, how might individuals share extended reality choices and preferences with others? What interactions and systems are necessary for sharable extended reality? Currently, one common social interaction is screen casting from a device to secondary screen for others to view.
A series of multiple-choice questions help build an understanding of current shopping behavior. Seven out of twenty-four participants find out about new projects from friends and family (29.16%). Eleven out of twenty-four participants find out about new, interesting products through online reviews, company websites, articles and news, and videos (45.83%). Two participants find out about new, interesting products through social media and online influencers (8.33%).
Of the twenty-four responses, two participants shop in-store more than five times per week (8.33%). Seven participants shop in-store 3 to 4 times per week (29.17%). Eleven participants shop in-store 1 to 2 times per week (45.83%). Four participants rarely shop in-store (16.67%).
Continuing, of the twenty-four responses, one participant shops online more than five times per week (4.17%). Ten participants shop online 3 to 4 times per week (41.56%). Nine participants shop online 1 to 2 times per week (37.50%) Four participants rarely shop online (16.67%).
Merchant adoption of extended reality raises questions on integration with existing channels. This group of participants indicated that they are shopping more frequently online than in-store. This supports the theme of augmented reality emerging as an eCommerce advertising channel. Augmented reality is a feature implemented within a merchant’s website or smartphone application.
INDUSTRY RESEARCH
A comparative analysis communicated the differences between platform approaches across the software industry. Through analysis, platform differentiation is a compelling element of extended reality. For example. 8th Wall provides the capability to create augmented reality experiences for the web. Apple attempts to reduce friction by creating a drag-and-drop editor for 3D objects within their ecosystem.
Snap AR offers a toolkit to create augmented reality experiences within Snapchat. Snap’s ecosystem approach is like Apple’s. Niantic, through their Lightship platform, offers a platform agnostic development environment. It is notable that Niantic acquired 8th Wall. Continuing, Google’s ARCore is usable across many development environments. Extended reality development platforms include Unity, Unreal, iOS, and Android. For example, the Meta Quest 2 runs on an Android-based operating system. System optimization focuses on virtual reality application performance. It is notable that at the time of this case study, Roblox does not have a native virtual reality experience. Roblox is an online gaming platform. Currently, users tether a virtual reality device to a laptop and select the VR option in Roblox.
DESIGN
Two high fidelity wireframes focus on user interactions within the retail context. The wireframes display a single retail fixture with items stacked on top of the fixture. After synthesizing the survey data, Material Icons communicate emerging paradigms of extended reality. The Group icon indicates the ability to share the experience with a designated list of contacts. The Sensors icon indicates the ability to connect with nearby experiences and devices. The Info icon indicates the ability to expand product information.
Other design elements follow common mental models found in merchant applications. The Explore icon communicates the ability to view more merchant content. The Qr Code Scanner icon indicates the ability to open the devices camera to scan a code. The Shopping Cart icon communicates the ability to view products added to the potential order. The Account Circle icon indicates the ability to view information relevant to personal use of the app. The two high fidelity wireframes vary slightly. The first version (left image) includes labels under certain icons. The second version (right image) does not include icons.
USABILITY STUDY
First-click testing provides data to evaluate the interface elements of the application. This method examines what a study participant would click on first on the interface to complete their intended task. This study consists of four problem-oriented tasks for participants to complete. Paths are set to measure the successful completion of each task. Clickmaps communicate how the participants perceive content and visual elements of the interface. Success rates and the time spent on tasks provide additional data for analysis. Two groups of approximately ten participants completed a series of four tasks for each wireframe.
Task One | Connecting to nearby elements
The first task of the study asks participants to connect with digital elements. The primary area for task completion is the Sensors icon. The secondary areas for task completion are the Explore icon and the Qr Code Scanner icon. The exact task presented to the participant was “You are in a smaller store. You want to connect to digital elements that enhance the shopping experience.”
For study one (left image, N=12), the success rate for task one was 33.33% with the average time taken at 1.86 seconds. Three participants clicked the Sensors icon primary area for task completion. One participant clicked the Qr Code Scanner icon secondary area for task competition.
For study two (right image, N=11), the success rate for task one was 54.54% with the average time taken at 8.57 seconds. Four participants clicked the Explore icon secondary area for task completion. One participant clicked the Qr Code Scanner icon secondary area for task competition. One participant clicked the Sensors icon primary area for task completion.
For this task, more participants clicked the Sensors icon primary area for task completion in study one than in study two. The average time taken is significantly less in study one, which illustrates the role of icon labels. Icon labels guide the user through process of elimination. The results of this task communicate the need to differentiate between products and digital elements.
Task Two | Viewing more options
The second task of the study asks participants to view available options. The primary areas for task completion are the Info icons. The secondary areas for task completion are the Explore icon and the Qr Code Scanner icon. The exact task presented to the participant was “You like an item. You want to see if there are any more options available.”
For study one (left image, N=12), the success rate for task two was 75% with the average time taken at 5.78 seconds. Five participants clicked the Info icons primary area for task completion. Three participants clicked the Explore icon secondary area for task completion. One participant clicked the Qr Code Scanner icon secondary area for task competition.
For study two (right image, N=11), the success rate for task two was 63.63% with the average time taken at 5.42 seconds. Seven participants clicked the Info icons primary area for task completion.
The results of task two highlight the importance of embracing mental models for viewing more item options in an extended reality retail context. This includes designing the physical touchpoint that initiates the experience and incorporating many paths for item information. In study one, the content displayed on the Explore screen becomes significant.
Task Three | Collecting Feedback
The third task of the study asks participants to extend the experience. The primary area for task completion is the Group icon. The secondary areas for task completion are the Info icons. The exact task presented to the participant was “With a few items in mind, you are looking for some feedback. You want to extend the experience to a group of friends.”
For study one (left image, N=12), the success rate for task three was 66.67% with the average time taken at 3.05 seconds. Five participants clicked the Info icons secondary area for task completion. Three participants clicked the Group icon primary area for task completion.
For study two (right image, N=10), the success rate for task three was 50% with the average time taken at 7.22 seconds. Four participants clicked the Info icons secondary area for task completion. One participant clicked the Group icon primary area for task completion.
Task three results communicate the next step of designing user flows. In this case, collecting feedback begins at the product information level and not at the groups level. In both studies, more participants clicked the Info icons secondary areas for task completion.
Task Four | Scanning for product information
The fourth and final task of the study asks participants to learn more about an item. The single successful area is the scan icon. The primary area for task completion is the Qr Code Scanner icon. The exact task presented to the participant was “You want to learn more about where an item originated. There is a code on the item’s tag.”
For study one (left image, N=11), the success rate for task four was 36.36% with the average time taken at 2.23 seconds. Four participants clicked the Qr Code Scanner icon primary area.
For study two (right image, N=11), the success rate for task four was 18.18% with the average time taken at 4.45 seconds. Two participants clicked the Qr Code Scanner icon primary area.
Similar to the results of task one, task four communicates the importance of icons and labels.
CLOSING THOUGHTS
This combination of design and usability testing provides insights into how screen-based mental models translate to extended reality experiences. The next phase of this project focuses on defining user flows by concentrating on feature development and content hierarchy. After this phase, it is necessary to conduct another usability study to collect insights. It is important to translate the high fidelity wireframes into a design prototype. An eye tracking study is an appropriate research method.
Simultaneously, research on the merchant experience with ancillary augmented reality systems becomes a priority. It is critical to understand existing content management systems and legacy software platforms. It is also necessary to understand points of friction along the physical to digital journey.
Questions for more research and design include: How might users begin augmented reality experiences? How might user device settings create points of friction? How might content adjust dynamically based on proximity to extended reality conditions? How might device sensors augment content sharing? How might content and object interactions create points of friction?
Social Audio
Designing for hyperlocal content discovery and communication
SUMMARY
A look into a location-based social audio application designed through digital media and information architecture research
This self-directed project explores the integration of location, music, and social broadcasting. An article in a college yearbook from the University of Tennessee provided inspiration for the project. A survey provides insights on user behaviors related to digital media consumption. The survey asks questions spanning from listening to music to streaming videos, television, and movies. Card sorting helped to design the initial information architecture of the application. Participants organized topics from the application into groups that made sense to them. Once sorted, participants named each group they created. The data collected from both the survey and open card sort resulted in two high fidelity wireframes.
This exploratory research project emphasizes the identification of a design opportunity. A smartphone application provides a new, exciting way to listen to music and connect with others. As noted above, the application concept developed through historical precedent and existing behaviors. Work on this project occurred over a two-week period in September 2022.
GOALS
Goal One: Understand participants' digital content consumption patterns
Goal Two: Uncover other websites and applications participants use to discover digital content
Goal Three: Collect relevant information to direct research scope for future usability studies
Business owners are important stakeholders in the development of the application. A survey designed to measure advertising sentiment is necessary. For example, would business owners value advertising to a small but focused audience? Would business owners allow application users to use business locations as stations? The initial assumption is that advertising generates revenue. Future research is necessary to identify the potential for paid features.
USER RESEARCH
A spread in the 1978 Volunteer Yearbook inspired the core experience of the application. The Volunteer Yearbook recapped yearly campus activity for the University of Tennessee. This spread discusses how a group of students launched a radio station for a particular dorm building on campus.
In 1978 residents of Hess Hall operated an AM radio station (79.5 on the dial) as a residence hall radio station. The station broadcast through the AC current in the hall and acquired its equipment from the defunct Reese Hall dorm radio station.
– Written by Betsey B. Creekmore
An online survey collected data on digital media consumption patterns. More specifically, the survey differentiated behavior between radio, online music, podcasts, live-stream content, and videos, television, and movies. The survey contained a series of closed, multiple-choice questions and open-ended questions. As a preliminary survey to measure attitudes and preferences, participants are from the United States.
Demographically, the collected data from the fifty responses contains all age groups. Two participants were under the age of 18 (3%). One participant was between the ages of 18 and 24 (2%). Seven participants were between the ages of 25 and 34 (14%). Eight participants were between the ages of 35 and 44 (16%). Six participants were between the ages of 45 and 54 (12%). Seven participants were between the ages of 55 and 64 (17%). Nineteen participants were over the age of 64 (38%).
Based on the survey results, thirteen participants (26%) responded that they spend more than five hours listening to the radio during a typical week. This group is active across all digital content channels.
Four participants from this subgroup were between the ages of 25 – 34 (30.77%). Five participants were between the ages of 35 and 44 (38.46%). One participant was between the ages of 45 and 54 (7.69%). Two participants were between the ages of 55 and 64 (15.38%). One participant was over the age of 64(7.69%).
Sixteen participants responded that they do not listen to the radio during a typical week. Analyzing this subgroup provides additional insights on behavior. One participant spends more than 10 hours streaming music online during a typical week (6.25%). Four participants spend 1 to 3 hours streaming music online during a typical week (25%). Two participants stream music online for less than an hour during a typical week (12.50%). Nine participants do not stream music online during a typical week (56.25%).
Two participants of this group were under the age of 18 (12.50%), Two participants were between the ages of 25 to 34 (12.50%). One participant was between the ages of 35 and 44 (6.25%). Two participants were between the ages of 45 and 54 (12.50%). Two participants were between the ages of 55 and 64 (12.50%). Seven participants were above the age of 64 (43.75%).
This cohort of sixteen participants spends the most time streaming videos, TV, and movies online during a typical week. Two participants spend more than 10 hours streaming videos, TV, and movies online during a typical week (12.50%). Four participants spend 5 to 10 hours streaming videos, TV, and movies online during a typical week (25%). Four participants spend 3 to 5 hours streaming videos, TV, and movies online during a typical week (25%). One participant spends 1 to 3 hours streaming videos, TV, and movies online (6.25%). Five participants do not stream videos, TV, or movies online (31.25%).
Continuing thirteen participants (26%) spend more than five hours streaming live content online during a typical week.
One participant was between the ages of 18 and 24 (7.69%). Four participants were between the ages of 25 – 34 (30.77%), Five participants were between the ages of 35 and 44 (38.46%). One participant was between the ages of 45 and 54 (7.69%). One participant was between the ages of 55 and 64 (7.69%) One participant was over the age of 64 (7.69%).
Of the fifty responses, nineteen participants responded that they do not stream live content online during a typical week (38%). Collectively, this cohort spends the most time streaming videos, TV, and movies online. Two participants spend more than 10 hours streaming videos, TV, and movies online during a typical week (10.53%). Three participants spend 5 to 10 hours streaming videos, TV, and movies online during a typical week (15.79%). Two participants spend 3 to 5 hours streaming videos, TV, and movies online during a typical week (10.53%). Two participants spend less than an hour streaming videos, TV, and movies online during a typical week (10.53%). Ten participants do not stream videos, TV, and movies online during a typical week (52.63%).
Of this group of nineteen participants, two participants were under the age of 18 (10.53%). One participant was between the ages of 25 and 34 (5.26%). One participant was between the ages of 35 and 44 (5.26%). Two participants were between the ages of 45 and 54 (10.53%). Two participants were between the ages of 55 – 64 (10.53%). Eleven participants were over the age of 64 (57.89%).
Continuing, this cohort also spends time listening to the radio. One participant spends more than 10 hours listening to the radio during a typical week (5.26%). One participant spends 5 to 10 hours listening to the radio during a typical week (5.26%). Four participants spend 1 to 3 hours listening to the radio during a typical week (21.05%). Four participants spend less than an hour listening to the radio during a typical week (21.05%). Nine participants do not listen to the radio during a typical week (47.37%).
As a part of the survey, participants filled in the phrase “The online platform I use most for online content is…” Participants responded with twenty different platforms across fifty total responses. Responses mentioned YouTube thirteen times (26%). Responses mentioned Facebook eight times (16%). Responses mentioned Netflix six times (12%). Responses mentioned Amazon five times (10%). Responses mentioned Windows five times (10%).
MARKET RESEARCH
Market research on social audio platforms provided more user experience insights. At the product level, standalone applications contain social audio experiences. Within larger social ecosystems, social audio is a feature. Each approach has strengths and weaknesses. For example, social audio experiences in larger social ecosystems enjoy the scale of the platform. Standalone applications enjoy the opportunity to establish new experiences. Sustainable growth is a challenge for both approaches.
For example, one social audio experience launched as a standalone application. Eventually, the platform’s core application absorbed the standalone application. Content approaches vary for social audio experiences. Social audio experiences contained in larger ecosystems focus on many topics. For standalone applications, content spans many topics but also specific niches. Standalone social audio applications also focus on specific audiences.
USER RESEARCH
Card sorting helped to design the initial information architecture of the application. Market research analysis generated a list of forty topics to include in the open card sort.
Participants organized social audio features from the application into groups that made sense to them. Once sorted, participants named each group they created. Twenty participants from the United States completed the task of sorting all cards into groups.
The groups created by participants provide insights into label categories and navigation. Unfortunately, the study provided inconclusive data. For example, participants created categories such as “action, archives, community, messages, and live.” Other participants created categories such as “good, good-1, group 1, and group 2. Applying the method of standardization to user-defined categories also provided inconclusive data. Standardization merges related categories together to create one category. The best merge method dendrogram provided insights on relationships between card topics. Patterns begin to appear at 67% agreement.
Group One
View what you have recently recommended, Vote on a song that will get added to a playlist, Connect device to live station | Play a song from a playlist, Share station playlist with friend, Vote on a discussion topic for a live station discussion
Group Two
View trending stations, View upcoming live station events, View message requests from another user | Select genres you enjoy listening to, Listen to a live station, Reposition map to your location
Group Three
View archived chats, view what you have recently listened to, search your playlist, view trending playlists, Follow stations to have access to their playlists, Recommend a new song for a playlist, Add station song to your playlist, Send a message to a station
Group Four
Share station with a friend, Search station playlists, Send a voice message to a station | Send a message to another user, Send an invite to a friend, View updates from the stations you follow, Find nearby stations
Group Five
Select topics you enjoy listening to, Search all stations, Read a message from another user, Compose a new message | Set notification for an upcoming live station event, View station details, View the recent activity of a station, View the stations you follow, Allow notifications to know when stations are live, View most active stations, View all stations on a map, Listen to a live discussion on a nearby station
As illustrated in the similarity matrix above, strong card pairings and potential groups are limited. Card pairings show the percentage of participants who grouped these cards together. Based on the responses to the card sort, another iteration of the study is necessary.
Card pairings above 70
View all stations on a map | View most active stations, Follow stations to have access to their playlists | Recommend a new song for a playlist, Recommend a new song for a playlist | Add a station song to your playlist, Join a live station discussion | Vote on a discussion topic for a live station event, Read a message from another user | Compose a new message
Card pairings above 65%
View all stations on a map | View station details, View station details | View the recent activity of a station, View the recent activity of a station | View the stations you follow, Follow stations to have access to their playlists | Add a station song to your playlist, Recommend a new song for a playlist | View what you have recently listened to
Card pairings above 60%
View most active stations | View station details, View most active stations | View the stations you follow, View station details | View trending stations, View the recent activity of a station | Allow notifications to know when stations are live, Reposition map to your location | Listen to a live station, Recommend a new song for a playlist | Vote on a song that will get added to a playlist, View what you have recently listened to | Vote on a song that will get added to a playlist, View what you have recently listened to | View trending playlists, View what you have recently listened to | Search your playlist, Search all stations | Search station playlists, Search station playlists | Find nearby stations, Connect device to live station | Join a live station discussion, Read a message from another user | Send a message to another user, Compose a new message | Send a message to another user, Send a message to another user | Send an invite to a friend
DESIGN
A series of wireframe iterations illustrate the process of synthesizing findings from both the survey and the open card sort. Each iteration of the wireframe focuses on the location feature of the social audio application. In this instance, a map view frames the location feature. The context of the map view includes stations.
Each iteration of the wireframe also explores the structure of the application. Iterations of the navigation and tab bars focus on information hierarchy.
The wireframe sketches result in two high fidelity wireframes of the home screen. The context of the wireframes positions the user on a college campus. After synthesizing the research, Material Icons communicate common digital media mental models. The Home icon indicates the ability to navigate to the applications home screen. The Library Music icon indicates the ability to navigate to saved media items. The Calendar Month icon indicates the ability to navigate to notifications of upcoming events. The Chat icon indicates the ability to navigate to messages with other community members. The Account Circle icon indicates the ability to navigate to account settings. The Near Me icon indicates the ability to reposition the map to the current location. The Location On icon indicates the ability to open details of the corresponding location.
The information hierarchy is different in these two iterations of the home screen. The second iteration of the home screen (right image) does not feature labels. While search is a text field on the first iteration left image), it is an icon on the tab bar in the second iteration (right image). Both iterations feature content category buttons. The second iteration (right image) also repositions the Account Circle icon to the tab bar. The second iteration (right Image) also moves the Calendar Month icon to the navigation bar. The Notifications icon, introduce in the second iteration, indicates the ability to view content alerts (right image). Continuing, content surfaces are also different in the wireframes. In iteration one (left image), the primary content surface is a map view, while a list view is the primary content surface in the second iteration (right image).
CLOSING THOUGHTS
At this stage, both home screen wireframes are ready for evaluation through a first click study. Since the application falls outside of the common approach to digital media, the next phase of usability studies focuses on the age group 25 to 54. The emerging design hypothesis is music serves as a type of time machine that reminds this age group of past events.
As an iterative approach to usability research, each high fidelity wireframe should reach between five and ten participants. This ensures quick discovery of usability issues while reducing the diminishing returns of a large study. The next usability study also validates or invalidates the insights gained from the open card sort.
Questions for more research and design include: How might representation of locations that do not exist anymore occur? Is there an opportunity to launch this application and then transition to a virtual environment where people listen to music in small communities? How might the application achieve scale? Are there enough users to provide meaningful interactions for each station?
Academic Projects
Invi | Invi explores the relationship between on demand, co-created retail and the hyperlocal production and distribution of the 3D printed objects as a method of reducing environment impact of supply chains.
Architectural | Architechural explores the integration of a low-energy sensor network and evaluative user research within the built environment to achieve interior and exterior comfort reciprocity.
Combine | Combine explores the integration of rural infrastructure with emerging technology systems as a method of generating economic growth through knowledge sharing.