The Culture of Clinical Trials

As mentioned in our first week note of 2016, during the first phase of this project we identified gaps and impediments in the clinical trials system that prevent patients from getting the best treatment in a timely and cost-effective manner. We visualize each gap and impediment as belonging primarily to one of five categories— clinical trials system structure and clinical trial design, information and data exchange, culture, market forces, and legal and regulatory issues. We are now working intensely in each of these five categories to generate new ideas and take the first steps towards developing a set of relevant, systems based recommendations. Each of those categories may seem self-explanatory, perhaps except for culture. What exactly do we mean by culture?

The word culture is incredibly difficult to define, never mind defining the culture of clinical trials. There are many definitions of culture which vary wildly by discipline. As a starting point, Merriam Webster defines culture as “the set of shared attitudes, values, goals, and practices that characterizes an institution or organization.” You might be thinking— but the clinical trials system certainly does not involve only one institution or organization. That is absolutely true, and a critical point given the state of clinical research today. A clinical trial almost never involves just one physician researcher and his/her patients, as we might imagine. It actually involves hundreds of research sites located within hospitals, medical practices, or dedicated research facilities; the sponsoring organization whether a pharmaceutical or biotech company or a government agency such as NIH; the government agencies providing regulation and oversight such as FDA and Office for Human Research Protections (OHRP) within HHS; clinical research organizations (CROs) often trusted with the actual running and supervision of the trial; and contractors who do everything from create pre-prepared kits for lab specimens to perform a second reading of every imaging study (e.g., x-ray, CT, etc.) performed during the trial. Oh wait, and then there are the actual people who become study subjects by signing an informed consent and agreeing to participate in a clinical trial. How could we ever define a set of shared attitudes, values, goals, and practices that characterizes all of these stakeholders?

We hypothesize that each stakeholder in the clinical trials system from the research sites to the CROs have their own organizational culture, but there is also a unique culture created where all of these cultures overlap. We visualize this shared culture of clinical trials with the diagram below. Yes, it’s an incredibly over-simplistic diagram, but it helps us think about the unique culture that is created when these stakeholders work together to create new treatments for patients.

So what are the characteristics of the shared clinical trials culture? Despite the volumes written on culture and clinical trials independently, it is actually incredibly difficult to find much of anything discussing the culture of clinical trials. As a result, we are working to piece it together ourselves. We are gathering what has been written about the culture of each stakeholder and look for places of overlap, speaking with advisors and contacts from each group, and observing ongoing clinical trials. We are looking for those attitudes, values, goals, and practices; but, we have come to believe that while these things reflect culture, they “fail to capture its essence.”

One of our strategic advisors, Clayton Clayton Christensen, wrote a case study on organizational culture called “What is an Organization’s Culture” in 1999 in which he synthesizes the works of a number of scholars including Edgar Schein who noted the problem with only capturing things like attitudes and values. Dr. Schein, from MIT’s Sloan School of Management, notes that culture is “a property of an independently defined social unit – a unit whose members share a significant number of common experiences in successfully addressing external and internal problems.” Further, “…because of these common experiences, over time this group of people will have formed a shared view of the way the world around them works, and of the methods for problem solving that will be effective in that world.” That shared view of the world has led to the formation of certain basic “assumptions and beliefs” that have worked well enough and long enough for solving the group’s problems, that they are taken for granted.

When we think of culture this way, the task then becomes separating out the basic assumptions and beliefs in order to more clearly describe the “shared view of the world” that is central to clinical trials culture. So what are some of those basic assumptions and beliefs that have become part of clinical trials culture over the last one hundred years of American history? What attitudes, beliefs, and values were formed as the various stakeholders faced external and internal problems in trying to create new treatments for patients? How did these attitudes, beliefs, and values come to be taken for granted long enough that their origins and purpose have become hard to explain?

One example for you to consider— when you ask someone who works in the clinical trials system why double blind, placebo controlled, randomized clinical trials (RCTs) are considered the gold standard or the pinnacle of the hierarchy of evidence, stakeholders tend say, “Because that’s how it’s always been.” In fact, the rise of RCTs to the pinnacle of evidence is the result of crisis after crisis where people were killed or injured as a result of taking some drug or other treatment (e.g., Ms. Winslow’s teething syrup – 1913, Elixir Sulfanilamide – 1938, thalidomide – 1962, etc.). In the search for certainty (of the safety and efficacy of drugs and other treatments), each of our stakeholders sought better methodologies, experimental designs, and standards to protect the American people and concluded that RCTs were best suited for facing the external problem of identifying whether a drug or other treatment strikes the appropriate balance of safety and efficacy.

However, many people across stakeholder groups have begun to note that perhaps our faith in RCTs has blinded us to other innovative methodologies and approaches. As Alejandro Jada and Murray Enkin note in their beautifully-written book, Randomized Controlled Trials: Questions, Answers, and Musings, RCTs are “valuable, useful tools” that have followed “a linear, cause-and-effect epistemology” and successfully and consistently produced results, i.e., responses for more straightforward, acute health problems like pneumonia. But perhaps the complex health problems of the 21st century demand a more innovative or radical redesign of the simple linear clinical trial. Beyond more innovative designs, other stakeholders suggest that we must remember that double-blind placebo controlled clinical trials are only one tool in the toolbox to answer a therapeutic question. Dr. Rob Califf, whose nomination as FDA Commissioner currently hangs in limbo has said, “…no amount of statistical analysis can substitute for randomization in ensuring internal validity when comparing alternative approaches to diagnosis or treatment… Nevertheless, the sheer volume of clinical decisions made in the absence of support from randomized controlled trials requires that we understand the best alternative methods when classical RCTs are unavailable, impractical, or inapplicable.”

Clinical trials culture continues to place RCTs at the top of the hierarchy of evidence, but many stakeholders in this shared culture are ready and willing to consider shifting their values and practices (even if it is only ever so slightly). This is because, as Dr. Christensen notes, culture is dynamic and can evolve in two ways— in the face of a clear and present crisis or through a more organized and deliberate “managed evolution.” The increasing use of adaptive trial designs and new clinical trials models such as the I-SPY 2 Trial seem to be some form of managed evolution. However, we believe that other basic assumptions and beliefs of clinical trials culture are facing threats due to a clear and present crisis— the widely-held perception that treatments are taking too long to come to market and cost too much money once they reach the market.

We are in a moment in history when we are technologically capable of doing research and examining evidence in completely new ways, and yet clinical trials have barely evolved. Why do some research sites still use paper records in the age of electronic health records and electronic data capture? Why do sponsors of clinical trials and CROs often fail to incorporate new technologies even basic ones such as remote data and site monitoring into their regular practices? Why do sponsors and CROs continue to use dozens of different electronic systems that don’t communicate with each other? Why is the collection and analysis of electronic health data so difficult, even taboo, when our every browse and click online as well as every purchase inside brick and mortar stores is being recorded and analyzed on a daily basis? Why do the sponsors of clinical trials fail continue to design and run studies that do not incorporate actual clinical practice flow, resulting in mountains of protocol deviations and troublesome FDA audit findings that impact the likelihood a treatment’s approval?

Why do all of these things continue to happen even as the ongoing conversation about the time and amount of money it takes new treatments to come to market in the US has seemingly reached a fevered pitch in the mainstream media and on social media? We would argue that to a great extent it is due to culture, and unless stakeholders manage the evolution of clinical trials culture, this deep and powerful crisis will force cultural change. Now is the time to identify which aspects of clinical trials culture have served the various stakeholders reliably and successfully for the last 50 years or so but are no longer successfully addressing those external and internal challenges. What aspects of clinical trials culture now constitute a “disability” rather than a “capability”? Now that the stakeholders are no longer facing the same tasks as 50-100 years ago, i.e., straightforward, acute health problems and now that there are more therapeutic questions than could ever be solved by RCTs, what will be the new processes, priorities, and shared culture that is desperately needed to respond to 21st century health problems? Stay tuned….
Jen Bernstein