Risk-Based Monitoring for Medical Devices—The New FDA Guidance
A Whitepaper by Dr. Nancy J Stark
Want to be on the mailing list? Opt-in for whitepapers.
To read or comment on the blog .

In August of 2011 FDA released a draft guidance document titled "Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring." At first I was very excited because the monitoring guidelines had not been updated since the 1988 "Monitoring of Clinical Investigations." But then, as I began to consider the ramifications of implementing the new guidance for medical device investigations I started to have mixed feelings. The first part of this whitepaper is a factual review of the draft guidance with ideas about how device manufacturers can take advantage of it. The second part is an op-ed expressing my views about what it will mean for the device industry. [1.Risk-Based Monitoring Guidance] [2. Monitoring of Clinical Trials]

Two points stand out as noteworthy before we begin: 1) the draft guidance was written by the Office of Compliance not the Office of Evaluation, and buy-in from an IDE reviewer is uncertain, and 2) the draft guidance addresses itself to drugs with a mere mention of devices as an afterthought, so most concepts have to be 'retrofitted' to device needs.

Part One: Review of Guidance
Risk-Based Monitoring
The draft guidance is based on the premise that most sponsors conduct on-site monitoring visits every 6-8 weeks with the goal of source verifying 100% of the data 100% for 100% of the subjects. It uses this premise to argue that sponsors will save money by moving to risk-based monitoring because such a move will result in fewer monitoring visits. Risk-based monitoring probably will result in fewer on-site visits for big pharmaceutical firms, who may be 'conducting hundreds to thousands of trials in locations around the world'. [3. Quality Management in Clinical Trials] But start-up medical device companies have never monitored as heavily as the guidance assumes and, in that sense, the principles of risk-based monitoring won't save on monitoring costs.

The guidance suggests that we use fancy statistical algorithms and analyze real-time data as it comes into electronic data capture systems to identify high-risk sites or high-risk data. The algorithm could be something like 'Risk Priority Number = severity x likelihood of occurrence x delectability'. A Risk Priority Number is calculated for each identified risk. When high-risk sites or data exceed a trigger, they merit immediate, onsite monitoring visits. You can out-source the monitoring visit if you don't have in-house resources. [3. Quality Management in Clinical Trials]

High risk sites and high risk data
A high-risk site is a site with a high probability of having a problem; the 'symptom' might be that they are enrolling at a slower or faster rate, experiencing a higher or lower rate of adverse events, or having a higher rate of non-compliances than other sites. For example, one site I worked with suddenly stopped enrolling subjects altogether. The sudden change in enrollment pattern should have triggered an on-site monitoring visit. We eventually learned the site had enrolled the same subjects in multiple trials and compromised everyone's data. The site stopped enrolling because they were being audited by other sponsors for fraud.

High-risk data might be a particular type of event or outcome that is a 'symptom' of a problem with the study. For example, a high rate of post-procedure embolism might indicate a poorly trained investigator or a problem with subject selection.

The guidance suggests that any site in the start-up phase of the study is a high-risk site and that on-site monitoring should be performed at every site in the study's early stages.

Device companies can adopt the same principles using low-cost methods. Have the sites fax or scan the case report forms to the monitor once a week for off-site monitoring (i.e. remote monitoring). With one trial and maybe 100 subjects, a monitor can easily examine the case report forms manually for high-risk sites or data without the need for statistical algorithms.

Critical data and processes
Critical data and critical processes are the types of data and processes most likely to be high risk. Critical data include: [1] evidence that the subject really exists, [2] evidence that the subject has the disease or condition being studied, [3] evidence that IRB approval was obtained, [4] evidence that informed consent was signed before the intervention or procedure, [5] endpoint data that address the hypothesis, and [6] serious adverse device effect data. For each study you'll need to make a unique list of endpoint data and adverse device effect data that is critical.

Critical processes include [1] the process of obtaining informed consent, [2] the intervention or procedure, [3] timely reporting of regulatory non-compliances or study deviations, [4] timely correspondence with a data safety monitoring board (if one exists), and the like. Because studies vary so much from each other, you'll need to make a unique list of critical processes for each one.

Critical data and processes are most likely to be high risk and should be closely checked during the off-site monitoring process.

Methods of monitoring
Next the guidance considers two different methods of monitoring; on-site monitoring and 'centralized monitoring', meaning the real-time analysis of data by an application running on a server in some data management center.

I propose that for device studies there are four methods of monitoring: [1] on-site monitoring, [2] off-site monitoring of faxed or scanned case report forms, [3] off-site monitoring of electronic ally captured data, and [4] off-site monitoring of source data that has been captured by the device (usually in vivo or in vitro diagnostic devices.) Often data card is changed out on-site and the data card analyzed off-site. Sometimes the data card may transmit data electronically via the internet—consider a digital camera used to take photographs of the procedure. In the future, of course, it may even be possible to remotely access electronic medical records.

The idea, then, is to use the most expensive method of monitoring (on-site) for sites or data or processes that are critical and high risk and to use less expensive methods of monitoring (off-site or remote) for sites and data that are less so. It is a welcome concept to device manufacturers who are cash-starved and looking for ways to economize.

Triggers
The idea of a trigger is that some server at a data management center somewhere is programmed to alert you if critical data or a critical process has become high-risk; i.e. has the value or frequency of a critical data element or critical process step changed in such a way as to demand attention. Indeed, if hundreds or thousands of sites are reporting data every day, you will need a program to screen the information for anomalies or outliers.

But since most device firms are doing only a few studies at a time, having the case report forms faxed or scanned to the monitor for remote viewing is an inexpensive alternative. The monitor checks the forms for, [1] completeness (no missing data), [2] contemporaneousness, [3] legibility, [4] logic, [5] adverse device effect reports. The forms can be checked for everything except source verification. The monitor issues and resolves queries for any issues observed. Most triggers are found by simple observation.

Then the case report forms are forwarded to the data center where the data are entered into a local (often non-networked) computer. Here the data may be statistically analyzed to see if a site, data element, procedure or process step is at risk and merits an unscheduled, on-site monitoring visit.

Monitoring Plan
A monitoring plan should be developed for every study. This requirement is not new to sponsors conducting IDE studies or European studies under ISO 14155, but the guidance makes clear recommendations for the format. I recommend you develop a template for a separate document that can be referenced in the protocol, and don't make the monitoring plan a part of the protocol itself; or you can purchase the CD which includes a monitoring plan template as a Word handout.

Section One—Study Description
As a separate document, the first section describes the protocol, investigational device, purpose of the study, and other obvious reference information. You need to describe, [1] the monitoring approaches you will use for the study, [2] criteria for determining the timing, frequency, and intensity of planned monitoring activities, [3] specific activities required for each monitoring method employed during the study, including
reference to required tools, logs, or templates, [4] definitions of events or results that trigger changes in planned monitoring activities for a particular clinical investigator, and [5] identification of possible deviations or failures that would be critical to study integrity and how these are to be recorded and reported.

Section Two—Reporting
Section two of the monitoring plan should discuss the communication of monitoring results to management, review boards, and regulatory bodies. It should describe the format, content, timing, and archiving requirements for reports and other documentation. For example, if the monitor receives case report forms by fax or email, it makes sense to require, say, a weekly or monthly report. Section two should discuss the process for appropriate communication of the findings to management, review boards, and regulatory bodies.

Section Three—Noncompliances
Section three of the plan should discuss the management of noncompliance. How, and who, will follow up with investigators found to be non-compliant with the regulations, protocol, or IRB requirements? If non-compliances persist will the site be retrained, terminated from the study, or will some other action be taken? If protocol deviations are detected you need a plan for root cause analyses. Problem-solving has always been a part of a monitor's responsibilities in device trials and it may require an on-site visit to really assess the problem. I have had the root cause to be such simple things as the data transcriber needing spectacles, a three-hole punch, or a dedicated phone line. Eyes-on may be the only way to detect such problems so they can be solved.

Section Four—Training
Section four of the monitoring plan should describe specific training for monitors and internal data auditors, especially with the detection of triggers and recommendations for on-site monitoring visits. For device studies, I recommend section four also discuss training plans for investigative site personnel. This section should also describe plans for random quality audits.

Section Five—Amendments to Monitoring Plan
Section five describes the process for amending monitoring plans. What events may require review and revision of the monitoring plan and establish processes to permit timely updates when necessary?

Much of sections Two to Five can be incorporated into the clinical research quality management system. By developing procedures for off-site monitoring you won't have to repeat them in every monitoring plan.

Documentation and reporting
Documentation is just as you would expect. Both on-site and off-site monitoring reports should be written, communicated to management, and archived.

Part Two: Opinion-Editorial
The draft guidance repeals the 1988 guidance and leaves a framework vacuum in its wake. Worldwide, sponsors have based their monitoring strategies on the framework of prestudy visits, initiation visits, routine study visits, and close-out visits. This framework forms the outline of most of the clinical research quality management systems for device sponsors, and was adopted by the ICH-GCPs [3. reference] and ISO 14155 "Clinical investigation of medical devices in human subjects—Good clinical practice" (2011). [4. reference] Now that the framework is withdrawn, where does that leave the clinical and regulatory professionals who struggle to find a common monitoring strategy between sites and countries?

The draft guidance references the ICH-GCPs six times. The ICH-GCPs were written in 1996 by pharmaceutical sponsors and regulators from the US, Europe, and Japan. It is 15 years old and is showing a few signs of age—there is no discussion of central data management or electronic security issues, for example. But because the phrase 'good clinical practice' is in the public domain, ICH was free to adopt it as part of the title of document E6, "Good Clinical Practice: Consolidated Guidance." The Center for Drugs published the guidance in the Federal Register giving it high standing in the regulatory community. And although used by device manufacturers, it does not work well. The Investigator's Brochure is chemically-centered not mechanically-centered, investigational devices are stored and dispensed but never installed, there is no mention of training an investigator to use the investigational product, there is no mention of software-controlled devices or software collected data, no mention of caregiver or healthcare provider safety; in other words, for medical devices, the ICH-GCPs are inadequate and out-of-date.

ISO 14155 'Clinical investigation of medical devices in human subjects—Good clinical practice' (2011)—the internationally definitive document for medical device clinical studies—is not mentioned even once. I could live with the lack of acknowledgement if FDA hadn't taken such a heavy hand in writing the standard. It is wrong to insist on practice-changing language (such as moment of consent versus moment of enrollment) and then turn your back on the work product as FDA seems to have done.

The draft guidance draws heavily on the ever-increasing presence of electronic data collection. A really cool EDC system can spot a lot of site problems before the human eye ever gets to the data. But these systems are expensive (think in terms of a $1M for a modest study) and unaffordable to small start-ups.

Finally, it was the Offices of Compliance from CDRH, CDER, and CBER who wrote this guidance. While they promise us the inspection manual will be promptly updated to match the guidance, they do not make assurances that the reviewers in the Office of Device Evaluation will follow along. Sponsors are strongly urged to get buy-in on the monitoring plan from reviewers before initiating a clinical trial.

References
[1] 'Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring' FDA, August 2011. http://www.fda.gov/downloads/Drugs/.../Guidances/UCM269919.pdf
[2] 'Guideline for the Monitoring of Clinical Investigations' FDA, January 1988.
[3] 'Quality Management in Clinical Trials', March 2009, Clinical case series from Pfizer.

Questions or Comments?
Please post your questions and comments on the blog.

Best Regards,
Nancy J Stark, PhD
President, Clinical Device Group Inc

Click here to receive CDG news via our XML/RSS feed

Follow us on Twitter