Anyone who has worked on internal facing tools knows it is a different world than designing for typical users. At Asurion we handle the insurance products for all the major carriers. We launched a new internal platform in 2016/17. One of my responsibilities is working on the Agent experience focusing on optimization. I inherited the design of the platform as-is, so a big part of my work is identifying any pain the system is causing for agents. I have done cognitive walkthroughs, heuristic evaluations, and user testing, but wanted to take it a step further and utilize contextual inquiry and go "undercover."
I decided to partake in training classes with our frontline agents learning the system. I did it primarily to step into the shoes of my users. I hoped to come away being able to really empathize with agents, and understand their mentality. Luckily I came away with more than just that. I learned just how powerful this way of observing is. Some of my key takeaways are below.
1. Internal agents look at things differently than typical “users” and issues sometimes come up in group training easier than in one-on-one sessions.
Listening to agents talking during training some of my gut reactions when I first saw the platform were proven to be true, and other things that bothered me, they didn’t even notice. And of course, they noticed things that bothered them that I never would have noticed. Since they were in a “learning” environment surrounded by their peers they also felt more comfortable to bring up things that confused them.
i.e., in order to select an ‘action’ in the Agent Chanel one must select an actions menu, select the action that they would like to do and then they have to hit a button that says launch. The minute I saw this I was confused as every other item within the program requires just one click, and this is not a pattern they have ever seen before. It never came up in user testing specifically. Lo and behold in all of training classes this came up as something that they were confused about and thought was "weird."
Going forward I intend to continue doing one on one user testing but mixing in discussion groups and walkthroughs where one agent drives and others observe, as they seem to be more likely to discuss frustrating issues.
2. It’s easy to see if a feature is something that is easily trainable or not. This can aid in prioritization for product owners, developers, and any other role involved in a product.
For agent facing internal tools I would argue ease of use takes a back seat to speed and trainability; particularly in a role where employees are incentivized on a time basis. Although the two often go hand in hand, there are times when that is not necessarily the case. There’s no better place than a training class to see how long it takes to train a feature.
I.e., If a customer’s claim is placed on a hold, the agent must refresh a certain way to see if the hold was released. This whole situation was unclear to the agent, “how will I know if it’s released?” The trainer addressed it and it was immediately clear to the agent what to do.
The situation still brought up a point where there was an interaction that left the agent hanging. It’s important to look at the page again, but if the team comes across something that’s harder to train that’s an easier development lift we can make better informed decisions as a team.
3. It makes it easy to identify one-off issues versus issues for everyone.
Of course it depends on the training class, but in the the classes I attended one of the common methods of training is for one agent to carry out a scenario sharing their screen and to role play in front of the whole class. Other times they would all go through individual scenarios, if one person had an issue they could ask the trainer, and if anyone else had the issue they would chime in. You could therefore see if there was a pattern of behavior across multiple users or if it is just a one-off confusion.
Overall, I would definitely recommend this experience to anyone working on internal tools or really any tool where training is part of the experience. It gives you a much better holistic view of where your users mentality is. I absolutely intend on doing it again, and particularly look forward to when they start training new hire classes(these were experienced agents transitioning to the new platform), as I am sure it will bring up different feedback.