FDA Offers New HF Guidance

Introduction
This past June 2011 marked the release of a new guidance document from the FDA titled Applying Human Factors and Usability Engineering to Optimize Medical Device Design. It is the first new guidance on human factors from the Agency in more than 10 years and it marks a significant evolution in the Agency’s thinking about the role of human factors in improving medical device safety. Like its earlier counterpart , the guidance focuses on how human factors methods can be used to identify and mitigate use-related hazards. But the new guidance is much more prescriptive and raises the bar for device manufacturers. It includes new requirements for human factors validation testing. These requirements apply not just to “traditional” device manufacturers, but to companies producing “combination products” as well – drug products delivered using specialized syringes, injectors, and inhalers.

What is Human Factors Anyway?

Human factors is a marriage of psychology and engineering. It is the application of our knowledge of human capabilities, limitations, and predispositions to the design of work, workspaces, and technology. Of course, technology includes medical devices.

Poorly designed devices put patients and device operators at risk. Problems arise when there is a mismatch between how a device should be used and the physical, perceptual, or cognitive abilities of the person using it. For example, the thumbwheel on a glucose monitor may be too difficult for a diabetic with neuropathy to operate, information on a patient monitor may be too small for a nurse to read accurately from across the patient’s bed, or the sequence of steps to set the dose and prime an injector may be too lengthy for an elderly patient to remember. Environmental factors such as lighting, noise, and vibration can create use-related risks as well. Imagine setting up and adjusting a heart monitor in a speeding ambulance. And situational factors such as lack of sleep or the competing demands of a busy ER can make operating an already complex and non-intuitive device more difficult.

Use-related hazards arise when design problems trigger use errors that pose risks to patients or device operators. The good news is that human factors engineering can play a valuable role in identifying, evaluating, and addressing use-related hazards during device design. A thorough human factors analysis considers all aspects of the user, intended use, the use environment, and the device user interface:

• User Considerations include who will use a device (a physician, nurse, patient, non-medical care giver): their physical strength, dexterity, and stamina; their attitudes, and emotional states; and what training they will receive and their knowledge of related devices.
• Intended Use includes what the device will be used for (e.g. self-administration of a medication), or the work the device is designed to support (e.g., monitoring a patient’s oxygen saturation level).
• Use environments may be hospital ICUs, labs, physician offices, emergency transport vehicles, patient homes, and public spaces; the environment may be crowded or cluttered, dark or well lit, quiet or noisy; environments may include other equipment or require the user to engage in other activities while operating a device.
• The Device User Interface includes not only switches, dials, touch screens, icons, menus, indicator lights and alarms, but operating instructions, labeling, packaging, and training materials as well.

Well-designed devices accommodate user and environmental factors. Their user interface is consistent with a user’s abilities and operate in ways that are aligned with a user’s experience and expectations. They facilitate correct actions and prevent or discourage use errors that pose risks to patients or users. Human factors engineering provides the knowledge and methodology to reliably create well-designed, user- and patient-friendly devices.

The New Guidance: Applying Human Factors to Optimize Medical Device Design

The new guidance provides an excellent introduction to human factors and its application to medical device design. It is a well-written Human Factors 101. Like its predecessor guidance released in 2000, it describes how human factors methods can be applied during device design to effectively manage use-related risks. But the new guidance also introduces new requirements for Human Factors Validation Testing and outlines how human factors activities should be summarized and reported.

Human Factors Validation Testing

In principle, device manufacturers will need to conduct human factors validation testing for their device if their risk analysis has found a moderate to high risk of use error or the FDA feels testing is warranted to justify its concerns about human factors issues. In practice, the FDA is requesting human factors validation tests in support of submissions for a broad range of devices. This includes combination products utilizing drug delivery systems such as an injector or inhaler. Human factors validation studies have become part of the review for approval of many New Drug Applications (NDAs). These reviews are facilitated by the Office of Combination Products working in conjunction with the Human Factors Team within the Center for Devices and Radiological Health (CDRH) and the Center for Drug Evaluation and Research (CDER).

For most device manufacturers, human factors validation testing requires conducting a simulated use test, in which a minimum of 15 prospective users for each distinct user population are asked to operate the device in a meaningful way, in a realistic, but simulated environment. The test should be structured to mimic actual use, utilize a production version of the device, and be sufficiently sensitive to capture use-related problems, if they exist. This means test participants must be representative of actual users; the testing must be focused on the highest-priority tasks or use scenarios; and environmental and situational factors that can affect performance must be incorporated into the test environment (e.g., dim lighting, multiple alarms, distractions). The test should monitor participant performance for evidence of use errors and engage participants (after all test scenarios have been completed) in assessing any use issues that arose during testing.

Finally, test participants must be Americans . The FDA is responsible to US consumers. Rather than entertain arguments from foreign manufacturers about why Canadians or English-speaking Europeans are adequate surrogates, the Agency now requires that testing be done in the US.

The Human Factors Engineering Report
The new guidance also outlines how manufacturers should summarize and report on their human factors engineering activities. Table 1 outlines the FDA’s requirements for a Human Factors Engineering Report. The outline ensures that manufacturers discuss those topics of primary importance to human factors reviewers and follow a format that will facilitate the review process.

Manufacturers should draft the report as a defense lawyer might draft their closing statement in a legal case: first, review the facts of the case (the analysis you undertook to identify and address use-related hazards), then the evidence assembled (especially the results of your validation test), and finally, why the facts and evidence support the case (your definitive claim that the device can be used safely, and why any residual risks are acceptable without further efforts at mitigation).

Table 1. Outline for the Human Factors Engineering Report.

Canada, Europe and Beyond?
The new guidance is, of course, specific to the FDA and devices bound for the US market. What about Health Canada, Europe, or other jurisdictions? While Health Canada regulations do not deal specifically with human factors (a search of the Health Canada website for the term “human factors” turned up little in the way of guidance or requirements) Health Canada is following the FDA’s direction and has begun to ask device manufacturers to conduct human factors validation testing to support their submissions.

Manufacturers are already required to comply with ISO/EN 62366: Medical devices – Application of usability engineering to medical devices, in order to gain CE Marking for sale of their devices in the European Union. ISO/EN 62366 is similar in intent to the FDA’s new guidance document and includes a requirement for validation testing.

Moving Forward
If your company is a medical device manufacturer or is planning a new drug application that includes a drug delivery device such as a syringe, injector, or inhaler, plan on conducting a human factors validation, in the United States, to support your regulatory submission. Don’t wait for a request from a reviewer. Manufacturers need to start human factors work early in their product’s design. Companies will not meet the FDA’s requirements for a well-designed human factors validation study if they have not conducted an analysis of device users, use environments, and situational factors that affect how well people can use the device; prioritized the risk of user interactions; and identified and taken steps to mitigate potential use-related hazards. The very good news is that doing this human factors work will ensure that your company develops a safer, more usable product.

Formative Usability Testing

The simplest, most effective way to understand how to improve your product is to watch people use it.

Human Factors MD conducts Formative Usability Testing for clients to evaluate the usability of their medical software or device design. We put early product concepts or design prototypes into the hands of users and have them complete real activities under our observation. Six to eight users are sufficient to put a design through its paces. We measure how well people do, where they have problems, and how they experience the design. When testing is complete, our clients know which aspects of their design work well and where to focus efforts at improvement.

Solves These Problems

• We are investing heavily in the development of a new generation of product. But how can we be sure we have a winner?
• Clinicians won’t use our device unless it’s simple and easy to operate. How do we ensure that our engineers are on the right track?
• How do we meet our design verification objectives? How can we verify that our design outputs match our design inputs?

Our Approach

Conducting formative usability testing as part of an iterative design strategy is the most reliable way to develop a truly usable product. The very good news is that formative usability testing can be effective with as few as six to eight participants. Moreover, depending upon the design issues under evaluation, usability testing can be conducted using simple, low fidelity paper or foam mockups or higher fidelity software or presentation prototypes.

But for usability testing to be effective, it needs to conducted well. Usability testing isn’t simply about bringing users around a table to view and talk about your design. A usability test is not a focus group. For usability testing to be an effective tool for understanding user interface design strengths and weaknesses it needs to engage actual users in performing real work. Actual users. Performing real work. Using realistic clinical data. In a realistic setting.

We’ll work with clients to plan an effective usability test: to define the test objectives, prototype requirements, and activities test participants will perform to evaluate the design. We develop a participant screener and recruit representative users. We host and conduct the testing, keeping a video-log for subsequent evaluation. We analyze the results, prioritize usability issues, generate design recommendations, and summarize the key findings and recommendations in a presentation to our client’s development team.

Benefits

• Provides quick, cost-effective, and actionable feedback from real users.
• Identifies usability strengths and prioritizes usability weaknesses.
• Satisfies design verification requirements.

Deliverables

A Usability Test Plan, Participant Screener, Results Presentation, and video-footage.

Usability Bench Test™

Our Usability Bench Test™ provides a quick and cost-effective evaluation of potential usability issues with a software application or medical device user interface. The Bench Test can be performed at the earliest stages of your product’s development, while there is still an opportunity to introduce effective design solutions.

Solves These Problems

• How do we identify potential usability issues early enough in the design process to address them effectively?
• How to we identify usability issues that could negatively effect user acceptance and satisfaction?
• We conduct usability testing when we have a working prototype, but what can we do earlier in the process to ensure we are on the right track?
• How can we verify that the design we have meets our design input requirements?

Usability Bench Test™ Defined

The Usability Bench Test™ combines elements of several proven human factors techniques, including Task Analysis and Heuristic Review. The test is performed by one or more human factors and design experts, who take the following factors into consideration:

• The design of the user interface, instructions for use, and packaging.
• The specific steps (i.e., task steps) required to use the device safely and effectively.
• The device use environment (the environmental, social, and organizational conditions under which the device is likely to be used).
• The motor, perceptual, and cognitive capabilities and limitations of the target user population (i.e., what users can do relative to the requirements of the task).
• The predispositions of the target user population (what users are likely to do given their expectations and experience with the device).
• General human factors principles for good design (e.g., Provide unambiguous and timely feedback on use actions and device states, Provide a clear conceptual model, Make things visible, Anticipate errors, Reduce memory load).
• Relevant standards, including the AAMI’s HE75 Human factors engineering: design of medical devices.

Usability problems are identified and prioritized based on their impact to the user and likelihood of occurrence. Recommendations for design or labeling changes aimed at addressing the problems are provided.

Deliverables

Written report suitable for inclusion in a design history file and/or a PowerPoint™ presentation summarizing key findings and recommendations for design or labeling changes.

Human Factors and the QSR

As a manufacturer, you are required by FDA to demonstrate how human factors considerations were met during your product’s development. To assist manufacturers in understanding the regulations, FDA issued several guidance documents to support its human factors initiative, including Do It By Design: An Introduction to Human Factors in Medical Devices and Medical Device Use-Safety: Incorporating Human Factors Engineering into Risk Management. However, the “teeth” behind their human factors requirements lay in Section 820.30 of the Quality Systems Regulation or QSR, in paragraphs c, f, and g:

Our Medical Error Index

The past two decades have seen the problem of medical error come to light. Here is our version of the Harper’s Magazine Index applied to the problem of medical error:

The Psychology of Human Error

While the consequences of medical errors can be devastating, in reality, medical errors are not unique. Medical errors are simply errors in a medical context. As such, we can turn to what we know about the nature of human error in general to understand why medical errors occur, what factors produce them, and how to design to reduce them.

What is Human Factors Anyway?

Human Factors Defined

Human factors is the application of what we know about human capabilities and limitations to the design of equipment and devices in order to enable more productive, safe, and effective use.

Known also as usability engineering, cognitive ergonomics, or user-centered design, human factors is a marriage of psychology and engineering: the application of a scientific body of knowledge about human strengths and weaknesses to the design of technology.

Why Users Don’t Make Good Designers

Some approaches to product development advocate engaging users directly in the design process, as co-designers. Along this line, a new client recently suggested that we organize several joint design sessions with their clinical advisory panel, and bring along a developer to the meetings who could incorporate the panel’s design ideas into a software prototype, “on-the-fly.” We (gently) moved our client away from this approach, by convincing them that in general, users don’t make good designers.