User Experience Evaluation of Fashion Entities on Amazon.com

Original BeBe Store Design

The instructor for this course arranged for the entire class to work in coordination with Amazon.com on evaluating the user experience of stores that focus on a specific author, musician, or apparel brand. These stores were referred to as entities. For this project, I was in a group with two other students and we were to do our evaluation on fashion entities (apparel brand stores), and report our findings and recommendations to an Amazon.com panel who were working on these stores.

The primary function of this evaluation was to identify elements of the brand stores that lead to an enjoyable (or less than enjoyable) experience. In addition, we evaluated the ease with which users were able to find products and make purchasing decisions.

In our preliminary proposal we outlined our scope and the research questions the evaluation was designed to answer:

• Are Amazon.com customers experiencing a desirable balance of theme content and products on apparel entity pages?
• How well do the apparel brand stores on Amazon.com help users discover new products?
• Can users easily preview products and make purchases from apparel brand store pages?
• Is the information architecture of the apparel stores clear?

Also, we decided that the testing would be around 1- 1.5 hours, have pre- and post- surveys in addition to a series of defined tasks on the live Amazon.com website. These would be undertaken in the Laboratory for Usability Testing and Evaluation (LUTE) using Morae Recorder and Observer along with the Tobii X120 eye tracker.

Our plan initially was to recruit between three and eight users, through a variety of electronic and physical ads.

Following the preliminary proposal, we developed the test plan and kit for the study in addition to all the materials needed to undertake the study including:

• Recruitment ad
• Pre-screening questionnaire
• Phone interview questions
• Consent form
• Testing protocol and script
• Post-task questionnaires
• Post-evaluation interview

We had 81 responses to our advertisements and recruited six participants – one for the pilot test and five for the study based on how well their responses to our pre-screening questionnaire matched our qualifying criteria.

Eye Tracking Task

I was the LUTE research assistant at the time, so I set up the equipment for the testing which involved six tasks, the first of which was on the eye tracker and the remaining five were live moderated using think-aloud protocol using computers with Morae Recorder and Observer. Following each of the tasks, the user was presented with a post-task questionnaire. At the end of the session a final post-evaluation interview was conducted.

My role during the testing was as note-taker using Morae Observer for all of the sessions but one in which I was the moderator.

Following the testing, we analyzed the qualitative data (video recordings) for any common themes and outstanding statements concerning the participant's experiences, while descriptive statistics were used to analyze the quantitative data from the post-task questionnaires. Along with our observations, we triangulated our data to check for consistency within the participant's responses. Our findings and recommendations were based on these results and we presented these to the Amazon.com panel.

The most significant findings were:

• Participants would like to see a larger selection of products on apparel entities (i.e., the brand's entire selection)
• Additional brand information and media should be directly related to products
• Exploratory shoppers said Amazon.com recommendations are useful and important
• Participants were interested in composing outfits and seeing how different apparel items look together

Based on the usability issues identified, we recommended the following to be implemented:

• Provide a larger selection of products; allow shoppers to see the brand's entire collection
• Update apparel entities with seasonal trends and products to draw shoppers
• Provide media content that is directly related to products and styles
• Give brands greater flexibility to visually customize apparel entities

BeBe Brand Store with Recommendations Implemented in A/B Testing

Our findings and recommendations were presented to the Amazon.com panel and were positively received.