8 Months of User Testing - Is it Enough

It seems like a long time to spend in the discovery phase. Guess what? In this case it’s not (and it doesn’t take a superhero to realize this).

I was tasked with migrating an internal tool from an antiquated system into a modern, device agnostic application that’s streamlined to provide a betterUser Experience.

I’ve previously written about How I use Sketching to Start my UX process.I’d like to back up, and talk about where it all really starts. User Testing

Before I set out interviewing users I had to understand the system myself. This was no easy endeavor I spent weeks poking around, taking on different roles in the system and documenting obvious pain points.

The company is involved in changing driver’s behavior (specifically involving safety) across large and mid market trucking fleets. Over the past 10 years, they have compiled comprehensive amounts of data that has been captured using hardware installed in Vehicle Cabs.

Behavior alerts are triggered by sensors that set off a few seconds of video recording. These video clips are used by coaches within the companies to help drivers change behavior.

The benefit to the service is evident right away, in both private and public sectors. ROI is realized by lowering operating costs and reducing insurance claims, while achieving greater efficiency and compliance.

Before I could go in and propose changes based on best practices I needed to show them how their User’s were interacting with the product on a daily basis.

So the Testing Began

The overall goals were simple: to find common usability issues and patterns that can be improved upon to deliver a seamless customer experience (CX)

The first step was to Qualify User Testers.

This was easy, since the User’s were already familiar with the current system — and previous iterations before that. We qualified 18 users and offered an incentive of a gift card.


Overall goals

Examine the current workflow and determine what roadblocks exist. Understand how the user uses the current system on a daily basis to complete their tasks

The next step was to to Conduct Tests

Since a majority of our User’s were based all over the United Stated (and globally) we settled on testing them remotely. We used a moderator atUserTesting.com

Scripts and tasks were written based upon our objectives. The sessions usually lasted 30–35 minutes, and consisted of watching a tester run through the system live (while being encouraged to speak their thoughts out loud). Specific questions were asked as the User’s performed their tasks.

After that we Recorded the Results by cutting the videos into highlight reels, annotating the clips, and transcribing each session to deliver powerful results in multiple presentations to key stakeholders.

Without fail we were able to show top executives what their user’ s thought of their product Armed with this information, receiving “buy in” was no problem.

Time spent on each task, how long it took the user to complete it and how easy it was to use compared to other features

Time spent on each task, how long it took the user to complete it and how easy it was to use compared to other features

We captured time spent on each task, and specific frustrations.We put together a “Pain Points” matrix, and a list of user “Needs”and “Wants” (the later to be captured in a 2.0 version of the site).

 

 

Specific Pain Points The key takeaway was these things take time, but giving your user’s a voice is the most important part of the product discovery phase. Allow them to see that their feedback is valued and you will get a whole lot further wit…

Specific Pain Points

 

The key takeaway was these things take time, but giving your user’s a voice is the most important part of the product discovery phase. Allow them to see that their feedback is valued and you will get a whole lot further with your CX than making decisions in “recovery” mode and adding features with any thought to the over all User Experience.

What did I learn?

• User’s don’t always realize they’re on older software, but are frustrated with long loading times, constant crashes and buggy components

They don’t notice the pathways they take to complete tasks aren’t the quickest and most efficient

The back end software is cluttered and can be confusing to User’s going through a workflow for the first time (or any time)

People stick to what they’re familiar with and don’t want fancy interactions. They want to get their job done in a timely manor

Although a positive recognition program exists in the program today, not many people are using it

Most user’s in this demographic don’t like colorful graphic charts and graphs. They want their information presented to them in a organized way that makes sense end to end

Whats the plan?

We’ve gathered enough information to put together an “ideal” prototype to be tested. This includes all the “wants” and “needs” but doesn’t necessarily provide the most intuitive User Experience. We’ll test this with User’s and flesh out the most important parts.

We’ve created a dedicated panel of User’s who will act as a “sounding” board for future iterations.

I’m also working with a great group of Data Scientists to surface the relevant data and provide the information that makes sense (i.e. the information the user wants to see).

As we move forward with development, those needs will continue to be taken into account. We’ll flesh them out with the business and system requirements and continue to test early and test often.

I’d love to hear your thoughts. Do you think 8 months is a long time to spend in the testing/discovery phase on large enterprise software?

  •  

adrienne levin

LET'S WORK TOGETHER.

I AM A creative AND passionate individual,
Let’s talk about your ideas!

 
 
Cart (0)