Tag Archive for: nutritional epidemiology

Nutritional Epidemiology: Still Confusing

/in /by

Remember where we began: frustrated with the conflicting studies on nutrition and their impact on our health. The researchers used specification curve analysis to illustrate several issues. The most important point is that there are many ways to analyze large datasets in nutritional epidemiology. Reviewing 15 studies in 24 papers, they found that the number of ways to analyze the data could reach 10 quadrillion (that’s 10,000,000,000,000,000). Obviously, that’s not realistic.

Instead, the approach that could be used by researchers doing these types of studies in any field is to select a randomized sample of different analytical approaches and present the results in the way I did in Tuesday’s Memo. Using that approach showed that fewer than 4% of the studies reached statistical significance. But how much of that could be just dumb luck? Setting the probability of significance at less than .05 (which is most common) means that out of 100 statistical approaches, five could show significance just by chance alone.

This paper addresses a long-standing problem in nutritional research and other areas as well. Researchers who do these types of longitudinal studies already use different analytic techniques in a haphazard way. They just keep chunking data until they find an analytic approach that’s statistically significant, and that’s the one they publish, sort of like a thief checking car doors until he finds one unlocked. Journals won’t publish results that don’t demonstrate significance, even though that would be beneficial for others to find out what not to do. “Publish or perish” just doesn’t work that way.

The Bottom Line

In this series of Memos, I’ve tried to lay out one of the reasons that long-term nutritional studies that look at morbidity and mortality can be flawed, if not contradictory. To be sure, the statistical analyses I’ve talked about are complicated, but that wasn’t my purpose. It’s to let you know that because of the lack of hard and fast rules for outlining the statistical approaches before looking at the data (as is done in randomized controlled trials), the results and the interpretation of those results will always be suspect.

In plain words, never get too excited about longitudinal studies, whether positive or negative. In the coming weeks, I’ll examine some studies on fish oil and multivitamins to illustrate the points I’ve tried to make.

By the way, for those of you really wanting to know whether you should eat red meat based on the analytic study the researchers tested as an example, they stated that there were some holes in the NHANES data that could impact the outcomes they reported. For now, it seems that women may benefit more from eating red meat than men will, but there’s no definitive answer yet.

What are you prepared to do today?

        Dr. Chet

References:
1. https://www.sensible-med.com/p/the-definitive-analysis-of-observational
2. Journal of Clinical Epidemiology 168 (2024) 111278

Nutritional Epidemiology: Specification Curve Analysis

/in /by

Did you look up quadrillion? It’s a 1 with a whole lot of 0s—15 to be exact.

When I finished Saturday’s Memo, the researchers had chosen an area of nutritional epidemiology to focus on: the analytics used to analyze the data. They began with the premise that there are many ways to analyze any data set. They then identified published research studies that examined the consumption of red meat and mortality. They identified 15 publications reporting on 24 studies that examined the effect of red meat on all-cause mortality.

They weren’t doing a meta-analysis to see all the results of all the studies combined; they used a newer technique called specification curve analysis. They identified the type of data used in the analysis, the number of variables, the number of covariates, as well as demographic data. From that information, they then calculated the total number of ways each data set could be analyzed—the specification curve analysis. Turns out that number is 10 quadrillion! That exceeds the capacity of the computing power of a small country, and I can’t even imagine how much electricity that would consume.

They decided to take a randomized sample of the possible ways to analyze the data with specific variables and covariates in each and came up with 1,440 different approaches to analyzing the data. They ran additional tests on the data and eliminated 232 approaches because the data exceeded norms.

Then they ran the remaining analytic approaches on data from several years of the NHANES study. What did they find?

  • The median hazard ratio (HR) was 0.94 for the effect of red meat on all-cause mortality. That means the mortality risk was decreased 6% if the subject ate red meat.
  • HRs ranged from 0.51 to 1.75; 435 approaches yielded HRs more than 1.0 (increased risk) and 773 less than 1.0 (decreased risk). Most analyses showed that eating red meat reduced the HR, and thus reduced the risk of dying.
  • Of all the results, 48 (almost 4%) were statistically significant; of those, 40 indicated that red meat reduced all-cause mortality and 8 that red meat increased all-cause mortality.

Does this mean that eating red meat decreases your risk of dying early? We’re not done yet. We’ll put it all in perspective on Saturday.

What are you prepared to do today?

        Dr. Chet

References:
1. https://www.sensible-med.com/p/the-definitive-analysis-of-observational
2. Journal of Clinical Epidemiology 168 (2024) 111278

Nutritional Epidemiology: The Problem

/in /by

The health headline shouts: “Fish oil increases your risk of heart disease.” The next week, “Fish oil beneficial for reducing risk of Alzheimer’s disease.” It makes one wonder what is going on in research. I’ve felt that way for a long time, and I know you have as well. I think I’ve found part of the problem, and I’m going to talk about in the next three Memos. As I said, this involves statistics, but I’ll keep it as simple as I can; we’ll be talking about some heavy duty statistical analytics, but there’s no math in my approach.

The frustration with conflicting research outcomes is especially prevalent in nutrition studies. It also happens in other fields as well including treatments for cardiovascular disease. Dr. John Mandrola, a cardiologist, wrote about a paper that examined why nutritional epidemiology is subject to conflicting results. After reading his post, I listened to the interview he had with the principal investigator of that study; then I read the paper itself. Statistics are not my forte but putting aside the high-level math, what they did was pretty amazing. By reviewing the paper here, I think you’ll get a better understanding of why research papers yield differing results even when using the same database of subjects.

Let’s begin with what has become accepted in the nutritional literature: eating red meat increases your risk of all-cause mortality. I know that many readers have given up or severely restricted red meat because so many physician, dietetic, and other health organizations have said it’s bad for your heart, among other organs. But is it really?

I used the phrase “nutritional epidemiology.” Those are studies that use some form of diet record, usually a Food Frequency Questionnaire, to track the food intake of a large group of people. Then they are tracked for 5, 10, even 20 or more years to see the differences in mortality between groups who ate red meat and those who did not. This contrasts with randomized controlled trials; they generally have as few as six subjects up to several hundred. They’re difficult to do because of the labor intensity of collecting data and ensuring subjects adhere to the protocol, whether a diet, specific foods, or dietary supplement.

With that in mind, the researchers examined all the nutritional epidemiological studies on the relationship of eating red meat and all-cause mortality. They then calculated all the ways that the data could be analyzed given the number of outcome variables and covariates used. We’ll pick it up on Tuesday, but as a tease, look up how much a quadrillion is.

What are you prepared to do today?

        Dr. Chet

References:
1. https://www.sensible-med.com/p/the-definitive-analysis-of-observational
2. Journal of Clinical Epidemiology 168 (2024) 111278