Author: John Putzke
It is universally recognized, but often neglected, that asking the right question has profound implications not only on the end-result answer, but also the entire framework used to perceive and understand the problem. As Albert Einstein said,
“If I had an hour to solve a problem and my life depended on it, I would use the first 55 minutes determining the proper question to ask, for once I know the proper question, I could solve the problem in less than five minutes.”
The Genesis of Questions
It is mentally challenging and often disorienting to work within the space one step prior to the question. However, this “no-where” space is precisely where the separation between what could be and what is happens. It’s the place that holds the entirety of the problem and demands the hard, visionary work of an artisan to carve out the question. It’s hard precisely because it’s more natural to remain grounded in the realm of known answers, thankfully so as this facilitates organized navigation through the world by keeping the overwhelming chaos of “no-where” at bay. Adding to the difficulty, this natural state also molds questions that do arise into mere refinements of existing questions and answers. Despite the tremendous upside to this adaptive natural state, an awareness of its risks and limitations is critical for continued advancement. Most notably, the complete expanse of the problem (and possible solutions) is no longer fully comprehended.
The Clinical Research Trap
In clinical research, this natural state manifests as scientific questions being reflexively framed through a singular focus on the independent variable, which in turn are assessed through known outcomes. Clearly, this commonly accepted approach to scientific inquiry has resulted in an amazing set of discoveries and insights. However, as with all beliefs or practices that are widely adopted, what gets pushed out of awareness are the circumstances surrounding their adoption, the separations of “no-where” space into the definitions used, the expanse of possibilities, and the full range implications.
To get a sense of its grip over investigators, consider the following question. What if a significant proportion of the forthcoming new, disruptive and trans-formative discoveries in clinical research have nothing to do with devices, pharmaceuticals, interventions or any other to-be-dreamed-up independent variable? Notice this hypothetical immediately throws clinical research back into the pre-emergent chaos of “no-where”, and calls forth the great separations used to define research, its methods, aims, practices, etc. Questions that re-imagine the principles and methods of clinical research will be the mechanism that uncovers these new discoveries. To clarify, this is NOT advocating for questioning the scientific method (e.g., random assignment, staff blind to condition, etc.) in the manner of a naive post-modern nihilist. Instead, it’s a call for questions that re-draw the definitional lines of clinical research.
What type of questions will emerge? They are easy to spot as their scope applies across therapeutic domains and designs, as opposed to being centered around an independent variable. There are many such questions, but one of the cornerstones will be: Is Research Relational?
The Traditional Answer
By definition, clinical research involves people and where people are found, relationships exist. Thus, there is an obvious side to the answer. Yes, there is some type of relationship between subjects and investigative staff in all clinical research. Relationship as used here is not meant to include exchange between investigative staff and subjects that would jeopardize or influence the results (e.g., a discussion of the study rationale). Instead, relationship is meant to be mechanisms through which the investigator engages subjects in the research process.
The well-schooled empiricists would argue any relationship isn’t worth the risks and hail the virtues of “objectivity”, maintaining distance between subjects and investigators so-as-to minimize potential confounds and/or interaction effects with the object of study. Reasonable assertion and in line with the tradition of leaning heavily on the importance of internal validity (i.e., the extent to which changes in the independent variable [e.g., a drug], can be attributed to changes in the dependent variable [e.g., blood pressure]). Certainly, Hawthorne effects and other investigator / subject confounds (e.g., trial effects, secondary observer effects, etc.) have been well documented, so the position of objective detachment is well supported. However, the grounding in objectivity doesn’t deny a relationship, but instead forms the arena of constraints and liberties within which the research relationship game is played. This in no way means there are not risks, limitations, compromises and trade-offs associated with this approach that may be ameliorated, in part, through the building of a different arena.
Theoretical vs. Practical Rationale
In attempting to build a new arena, a re-examination of the original building materials is in order. The theoretical rationale behind relationship objectivity has been acknowledged, but to what extent was this merely the result of practical circumstances? Historically, relational exchange between investigative staff and subjects largely involved social pleasantries, scheduling discussions, words of encouragement (phone or face-to-face), and conversations about study events (i.e., discussion of consent documents, descriptions of procedures, etc.). The options available for staff / subject interactions have dramatically increased and changed with advances in information technologies. Is it reasonable to take the same detached, minimal engagement stance with these new technologies or may these new building materials offer considerable benefit? The answer requires weighing the limitations/strengths of the current vs. new ways to engage subjects, as well as an examination of whether the proposed levels of engagement introduce confounds (e.g., If so, under what conditions, etc.).
Poor Engagement Risks
The traditional objective, detached, minimalist relational approach between staff and subjects often worked. Indeed, it may be preferable given staff’s often pressing time demands. That is, subjects are minimally engaged by investigative staff and experience little beyond the burden of completing forms, visits, procedures, etc. The main risks and limitations are:
- Increased dropouts
- The weaker the relationship, the more likely to not comply or dropout
- Poor data quality:
An increased number of:
- Incomplete forms
- Missed study visits
- Procedure and/or data item refusal
The traditional empiricist may argue, “Yes, these consequences may be more likely in some cases, but these effects are the same across groups, and thus do not confound results.” Against this rationale, consider designs or interventions with characteristics that may be differentially affected by limited engagement (e.g., long-term studies with wait-list control, or those involving: a multi-step subject requirement, a heavy subject burden, painful procedures, etc.). Seems reasonable to assert the possibility of interaction effects abound using the current methods as well.
Moreover, what if higher levels of engagement significantly increases retention rates across groups? If so, adverse interventions are stopped sooner and positive results are confirmed earlier. Also, consider studies whereby positive or negative results tend to be associated with increased dropouts vs. control (e.g., studies of obesity, smoking, diabetes control, etc.). In such cases, increased engagement would minimize this dropout bias.
The empiricist may continue, “Yes, but these new methods may interact with the object of study (e.g., there may be greater benefit from increased relational interactions in the intervention vs. control group). This is an empirical question and worthy of further exploration. As seen in the figure below, ideally the level of engagement has no affect on outcome (blue line), and thus the benefits of higher levels of engagement could be leveraged. But research is needed to determine if results diverge across groups merely due to differing levels of engagement (red lines). It will be important to look at design, population, and outcome related factors in this regard, and explore engagement level as a potential mediator and moderator effect.
Although a topic well beyond what is to be pursued here, some examples in each area of characteristics that may be at risk for interacting with engagement level:
- Long-term studies
- The use of a wait-list control
- A high subject burden or multi-step requirement
- Involving adverse or painful procedures
Looking at the outcomes ladder in the figure below, those that are visible, palpable, or experienced have the greatest opportunity to trigger engagement and thus at most risk of potential interaction effects. For instance, c-reactive protein or blood pressure are generally not noticeable to subjects and thus not good targets for engagement methods. However, emerging devices promise continuous monitoring and feedback in areas not previously possible (e.g., retina scanning glucose detectors). Also, those that are not generally directly observable (e.g., Overall Health Related Quality of Life) have construct measures that produce a value, which in turn, could be used for differing levels of engaging, and thus open to potential confounding effects.
Building upon related design and outcome characteristics, populations that may be at increased risk of interaction effects with levels of engagement:
- Chronic disease
- Those with perceptible symptoms
In noting these areas of needed investigation, it is not being put forth as a rationale to keep from moving forward with leveraging increase subject engagement in clinical research as there are tremendous potential benefits across studies and therapeutic domains. Also, I’m not aware of research exploring the equally important downsides to the currently accepted levels of subject engagement which, in turn, hasn’t kept investigators from adoption.
This is an amazing time to be in clinical research! Investigators and information technology are coming together to explore not only new and exciting interventions (e.g., devices, pharmaceutical, behavioral treatments), but also transforming the way clinical research is conducted, particularly from the subject’s perspective and their level of engagement in the research process. These will be the topics of coming articles, but note some examples found in Studytrax (see video for how these work here: half way down page):
- Participants can earn points for exchange by completing forms and study events
- Built-in, secure conversation platform between participants and research staff
- Use a participant’s own data to create and deliver personally relevant information (e.g., videos, charts, web links, educational materials, etc.)
- Set and track goals that are reinforced through incentives upon attainment (e.g., weight reduction of 10%)