A series of articles and studies published recently have called into question the use of statistics in examining large health databases. It’s especially pertinent to nutrition research because placebo-controlled trials are virtually impossible.

See if this sounds familiar: eating meat is associated with an increased risk of colorectal cancer. The “association” is calculated by considering variables that may impact the development of cancer and estimating the hazard of eating various quantities of meat. Recent articles have called into question whether enough variables are being considered, because only a few people decide what else could impact the development of cancer other than meat. Then the analysis is run and the results reported.

What if they didn’t pick the correct variables or enough variables that could impact cancer? What if they didn’t use the correct statistical methods to analyze the data? That could be why one study shows fish oil lowers the risk of cardiovascular mortality and another says it does not. Let’s look at an example.

“Ultra-Processed Food is Bad”

Eric Schlosser is an author and filmmaker who wrote Fast Food Nation and made the movie Food, Inc.; Food, Inc. 2 has come out recently. As a guest on Real Time with Bill Maher, he commented that ultra-processed food (UPF) is the leading cause of obesity and other diseases, but he offered no evidence to support that position. He suggested that it was the artificial ingredients found in UPF that make us crave them and eat them without restraint. Again, no evidence.

Several epidemiological studies have suggested that UPF is related to obesity. The question that remains is how? If it’s the artificial ingredients, how do they do that? By stimulating appetite? Or turning off satiety signals? Or some other mechanism?

That data isn’t collected in most large studies. When the researchers select variables that could impact the results, are they considering the correct ones? It makes a difference because at the end of these large observational studies, that’s the question that remains: How?

The Bottom Line

There is no conclusion at this point. That’s not usually my style, but this is complicated. There are too many observational studies and meta-analyses that are being used as the foundation of medical care and health and nutrition education, because as I see it the data are incomplete. For now, there are some changes we know will work and can prove: Eat less. Eat better. Move more.

What are you prepared to do today?

        Dr. Chet