You’re sitting on a plane. The person next to you says he’s conducting a study about parachute safety and asks if you would be willing to be randomly assigned to one of two groups: jumping out of the plane with a parachute or with an empty backpack at its current altitude and speed.

Yes, this is a real study. The researchers included both commercial and private aircraft. The researchers were able to enroll 23 subjects in the study after screening 92 people; 69 people were unwilling to participate or were otherwise excluded from the study. After randomization, the experiment was conducted in two locations in the U.S.; all subjects completed the study. The results indicated that no subject from either group was injured or killed. There were no differences between the groups using the backpack or the parachute.

What?

How is that possible? The subjects who were asked on commercial aircraft at 450 mph and 30,000 feet would not volunteer (and a couple who did were excluded due to mental health concerns.) Those who were asked on a stationary private aircraft at zero altitude and zero speed all agreed to participate.

The Purpose

The researchers wanted to highlight that with or without realizing it, clinical randomized trials can be biased by the subjects who are recruited and the way they’re recruited. You can determine this only by drilling down into the research to see how subjects were selected.

Remember the study on bitter orange and caffeine? All the subjects were young and healthy. That happens often in exercise studies when testing dietary supplements, but people of all ages are active and use products designed to improve performance. Aging brings many differences in muscle mass, hormone levels, and other system changes that may reduce or exaggerate the affects of the supplement. The generalizability to other populations is often limited.

This parachute study was done tongue in cheek; no one would let people jump out of a commercial airplane with just a backpack and no human subject ethics committee would ever approve it. But subject selection can impact results and that calls into question the whole concept of doing a randomized clinical trial in the first place.

What are you prepared to do today?

        Dr. Chet

 Reference: BMJ 2018;363:k5094 http://dx.doi.org/10.1136/bmj.k509.