Assessing risk in Clinical Trials

Clinical Trials process is becoming a complex and capital-intensive process. We all know this. In doubt? Then read our previous Week Notes, or continue reading this one. However, Clinical Trials is also a risky process! First, risk is defined as the potential to lose something of value. Mathematically we can write: R = PxV , where P is the probability of the losing that thing (a number between zero and one), and V is how much it is worth (in dollars or other currency), and R is the risk. Therefore, encompassing money and likelihood, risk is a key driver for the current complexity of clinical trials.

Take some of the stakeholders in the clinical trials. These stakeholders perceive different levels and types of risk. For example, the patients mostly see health risks. These risks include 1) risk of adverse events (during the clinical trials or after the approved medicine hits the market), and 2) risk of disease progression due to unavailability of effective drugs. Pharmaceutical companies, on the one hand, would like to minimize the risk of their drug not being approved by the Food and Drug Administration (FDA). That is, they are not interested in losing their investment. FDA, on the other hand, mainly sees the risk of adverse events in patients (for instance, because of approval of unsafe drugs). These risks are related, and a decrease in one may come at the expense of an increase in the other. For example, the FDA could simply demand a highly comprehensive data to be absolutely certain (P = 0%) that the drug under question is safe and effective, but this means almost all drugs do not get approved. Therefore, the risk for pharma in this situation becomes astronomical (almost 100% likelihood of losing their substantial investment!) This also means that no new medicine will be available to the public (because no new drug will be approved).

In the interest of minimizing their own risk, stakeholders interact with levers (or variables) of the system. This system can be visualized in many ways, and at multiple fidelity levels. One simple and yet powerful way to achieve this “system mapping”, is through a causal loop diagram (CLD). A CLD shows the relationships between stakeholders’ actions in the clinical trials qualitatively, and consists of variables and arrows.

For instance, a preliminary CLD of the system is shown in Figure 1. The key variable, the one we would like to focus on, is the population size in Phase III. This variable is chosen because, Phase III is where many patients are enrolled, and is aimed to produce the data to show efficacy and safety of the new medicine to the FDA. We are not focused on a specific medicine here, and size could be extended to include complexity as well. There are a total of four variables in the CLD. These variables are connected by arrows, which indicate that the variable on the tail side has a causative effect on the variable on the head side. A blue arrow (with a plus sign) means an increase/decrease in the tail variable, leads to an increase/decrease in the variable on the head side of the arrow (these variables move in the same direction). A red arrow means the opposite.

Focusing on the key variable, size in phase III, we can identify two balancing feedback loops, shown by black circular arrows. A balancing loop means a change in the variable in one direction, after completing the loop, leads to a change in the opposite direction on the same variable. Take the balancing loop on the left (shown with B). We read this loop as: a decrease in size of Phase III, leads to an increase in probability of post market adverse events (because the drug has not been tested thoroughly), which leads to an increase in the size of Phase III. The FDA could mandate this increase in size. Observe that we started with a decrease, but ended up with an increase. This means we had a balancing loop! Feedback loops are imperative to system analysis. Some diagrams might include reinforcing loops, which may lead to a great change over time. Fortunately, we do not have a reinforcing loop in this CLD (what would happen if we had one?)

Figure 1 A preliminary causal loop diagram of the extended clinical trials system

At this point, I would like to note while we do not actually show the stakeholders (pharmaceutical companies, patients, and others), we see their effects on the system using the CLD. Also, note that the system boundary is extended beyond clinical settings, and to the general public. Level of detail and system boundaries will be chosen as adequate to the level of analysis.

CLDs are useful for developing a common “language” between research team members, and encourages collaborations between us to complete system maps together through consulting the literature and/or our expert advisors. We will also create several diagrams describing other key variables of the system before delving deeper into quantifying the risk with actual data. An outcome of this risky research would be to determine whether stakeholders have an imbalanced perception of risk when it comes to drug development (for instance the extreme case we talked above would lead to no new drugs). We will, potentially, propose acceptable risk levels or “risk sharing” strategies to make the system stakeholders work more synergistically together.

In closing, I hope this diagram develops a common language between you and us. Before we part, let me describe the other balancing loop: an increase in the size of phase III (who wants increase? why?), leads to a decrease in probability of new medicine… (Go ahead. You can finish this!)

Mehdi Jalalpour