Pitfall #1: Using Only One Methodology to Develop a URRA

Pitfall #1: Using Only One Methodology to Develop a URRA

by Liz Mauer, MHCI

This is the second in a multipart series about how to effectively identify, understand, and communicate a product’s use-related risks.

While it is considered best practice to identify and characterize use-related risks by employing both empirical and analytical methods, we’ve noticed that when it comes to analytical methods in particular, most URRAs seem to rely on one methodology: Failure Modes & Effects Analysis (FMEA). This is a well-understood, bottom-up method to start identifying use-related risk, but it has some weaknesses and it should not be the only tool in your arsenal to identify and mitigate use-related risk.

First, FMEA really only considers failure modes that are single points of failure. If there is a potential harm that results from a combination of several failure modes, a “Use FMEA” probably isn’t going to find it, and if you do, you’ll have a tough time characterizing it. Consider the example of a large-volume infusion pump that is equipped with safety software to prevent medication errors made during programming. If you are creating a Use FMEA for the bedside use scenarios of this pump, you might identify use issues such as programming mistakes that result in an overdose to the patient. The mitigations for these use issues are the safety software that the pharmacy and medication safety officers at the facility have implemented on these pumps, so an overdose theoretically could not occur using this pump, right? But what if the pharmacist who programmed the safety software made an error in programming? What if the pharmacy committed a use error when setting the limits AND the bedside nurse also committed a use error when programming? The patient could indeed receive an overdose, and a Use FMEA would have a hard time identifying and characterizing this type of hazardous situation that requires multiple failure modes for it to occur.

Also, half the battle when creating and maintaining a URRA is being able to effectively review the analysis with the stakeholders throughout the development process, so that it can truly be a “living document” as designs mature. These types of reviews, where someone scrolls through an ever-growing spreadsheet of use issues projected onto a shared screen, can be fatiguing and ineffective in engaging stakeholders in thinking about how to manage use-related risk. How many rows can someone possibly review closely in such a format before their attention wanders? It would be more useful to lead these types of reviews with a question of “let’s think about the worst thing that can happen to a user or patient with our product – how could that happen?”

This is why, in addition to empirical methods such as formative usability testing, we recommend a complement of analytical approaches for informing URRAs: a bottom-up approach (such as a Use FMEA) along with a top-down approach (such as Fault Tree Analysis), where teams start by identifying a hazard (administering an incorrect dose, for example) and then hypothesize about what events or series of events could transpire to cause that hazard. By combining the two, you are more likely to identify the use-related risks that matter, that would result in actual patient or user harm, and you will be more likely to be able to keep the relevant stakeholders engaged in reviews when discussing these risks.