Tag Archive for: statistics

Where’s the Proof?

/in /by

A series of articles and studies published recently have called into question the use of statistics in examining large health databases. It’s especially pertinent to nutrition research because placebo-controlled trials are virtually impossible.

See if this sounds familiar: eating meat is associated with an increased risk of colorectal cancer. The “association” is calculated by considering variables that may impact the development of cancer and estimating the hazard of eating various quantities of meat. Recent articles have called into question whether enough variables are being considered, because only a few people decide what else could impact the development of cancer other than meat. Then the analysis is run and the results reported.

What if they didn’t pick the correct variables or enough variables that could impact cancer? What if they didn’t use the correct statistical methods to analyze the data? That could be why one study shows fish oil lowers the risk of cardiovascular mortality and another says it does not. Let’s look at an example.

“Ultra-Processed Food is Bad”

Eric Schlosser is an author and filmmaker who wrote Fast Food Nation and made the movie Food, Inc.; Food, Inc. 2 has come out recently. As a guest on Real Time with Bill Maher, he commented that ultra-processed food (UPF) is the leading cause of obesity and other diseases, but he offered no evidence to support that position. He suggested that it was the artificial ingredients found in UPF that make us crave them and eat them without restraint. Again, no evidence.

Several epidemiological studies have suggested that UPF is related to obesity. The question that remains is how? If it’s the artificial ingredients, how do they do that? By stimulating appetite? Or turning off satiety signals? Or some other mechanism?

That data isn’t collected in most large studies. When the researchers select variables that could impact the results, are they considering the correct ones? It makes a difference because at the end of these large observational studies, that’s the question that remains: How?

The Bottom Line

There is no conclusion at this point. That’s not usually my style, but this is complicated. There are too many observational studies and meta-analyses that are being used as the foundation of medical care and health and nutrition education, because as I see it the data are incomplete. For now, there are some changes we know will work and can prove: Eat less. Eat better. Move more.

What are you prepared to do today?

        Dr. Chet

Classic: Type III Error

/in /by

You know that eating fruits and vegetables is good for you, right? I tell you that all the time because that’s what the research indicates. But according to an editorial piece in the Journal of the American Medical Association (JAMA), maybe we’ve been wrong all this time (1)—at least as it relates to reducing our risk of cancer. Here’s a recap of the editorial and my take on it.


The Editorial

Mike Mitka is a well respected senior writer for JAMA; he’s published numerous articles that are well researched and contain interviews with the authors and other experts. In a 2010 JAMA article, he writes that an article from the European Prospective Investigation into Cancer and Nutrition (EPIC) clearly demonstrates that high intakes of fruits and vegetables do not reduce the risk of cancer, at least not very much.

Based on the study published in the Journal of the National Cancer Institute, increasing fruit and vegetable intake by six ounces per day reduced the risk of cancer only 3% (2). A couple of well-known researchers supported the notion that research does not support plant intake reducing the risk of cancer, including the director of nutrition for the American Cancer Society.

The problem is that it’s just not true, at least not in that study. The way it’s written, the study suggests that when subjects increased their intake of vegetables and fruits by six ounces, the reduction of cancer risk was a paltry 3%. This was an observational study, not an interventional study. No one kept track of who increased or decreased their plant material intake, and there’s no way to know that from the data collected.

The way I read the article, the EPIC study is central to this new belief about plant consumption not being related to decreased cancer risk. The first articles from the study were published in 2003, and that’s when scientists suggest the evidence grew weaker supporting reducing cancer by plant consumption.

I’ve written about EPIC before; it’s one of the largest observational studies ever conducted with over 520,000 subjects from 23 different centers in 10 European countries. While I suggested that there was strength in numbers when you want to try to tease out subtle effects, I mentioned that the researchers gathered nutritional information at the beginning of the study with a Food Frequency Questionnaire; subjects were asked as many as 256 questions about what they had eaten in the past year. There are two problems with that when you’re looking at the rate of cancer.

First, you have to assume that this one-time questionnaire on diet applies to the patterns of the way the subjects ate and will continue to eat. By the validation studies that were done for the EPIC study itself, that certainly was not true (3). Correlations between what people ate one year apart were as low as 36% in a sub-sample of the subjects. Researchers chose to adjust levels statistically, but that just isn’t the same as actually collecting the data. But when you’re dealing with a half million subjects, you just can’t collect dietary histories on every subject.

Second, the assumption is that the levels that the subjects ate were actually adequate to reduce the rate of cancer. In the study, the average fruits and vegetable intake for men was 17 ounces per day and about 19 ounces per day for women (4). Using an example from the editorial, a medium apple is 10 ounces and a serving of broccoli is about 3.5 ounces. That’s just two servings and that’s close to what the subjects actually ate.

The fact is that most subjects in the study did not eat very many vegetables and fruits and it got worse the further north the country was located. That should have been the overwhelming conclusion of the study to begin with before any other analysis was conducted.


My Take

Trying to explain statistics is not my strong suit, but I adapted this from a blog by Satoshi Kanazawa, an evolutionary psychologist in London and a contributor to Psychology Today (5).

There are two types of errors in statistics. There’s the error of a false positive: you think that the data reflect your hypothesis when it doesn’t. Then there’s the error of false negative: you think the results do not support your hypothesis when it does. Statisticians call the former type of errors “type I errors” and the latter type of errors “type II errors.”

My feeling is that Mitka and researchers who are using the EPIC study to make suggestions about the relationship between plant intake and cancer risk are committing a “type III error.” What’s that? An unwavering belief in statistics: you don’t look at anything but the numbers, and that rules what you believe. Maybe a better way of describing a type III error is hubris, an overbearing pride in statistics. These researchers are very smart people. Did no one consider that the way the data were collected didn’t make sense when tracking diet’s effect on cancer risk? That’s hard to believe unless pride gets in your way.

But when you get right down to it, if you eat an apple and some broccoli on a typical day and someone told you that adding a banana or a tomato would reduce your risk of cancer by 3%, doesn’t it seem like a great idea to add that banana or tomato? Imagine what you could do if you actually ate the recommended eight to ten servings!


The Bottom Line

This will not be the last time I write about EPIC because there are over 500 articles published to date. I still think that eating more vegetables and fruit reduces the risk of all disease including cancer based on the thousands of other studies that say that it does. It seems like the larger the study, the less beneficial effects anything seems to have whether it’s diet, exercise, or supplementation. Maybe someone should start considering whether the effects are being washed out by regressing to the mean. But that’s a message for another day. Time for some berries!

What are you prepared to do today?

Dr. Chet

 

References:
1. JAMA. 2010; 303 (21): 2127-9.
2. J Natl Cancer Inst. 2010 Apr 21;102(8):529-37.
3. International Journal of Epidemiology 1997; 26 (Suppl. 1): S26–S36.
4. Public Health Nutrition: 5(6B), 1179–1196.
5. www.psychologytoday.com/blog/bloggers/satoshi-kanazawa

 

The Dark-Chocolate Study: The Problems

/in /by

After Thursday’s memo, you’ve got to be wondering what could be worse with the dark-chocolate study than its misleading conclusions. I’m going to examine the process the author explained in his faux study to show how that applies to many research studies from legitimate health research.

Not enough of the right expertise
There was no input from a nutrition expert that I could find. The author used a physician to do the study and collect the data. Even though the purpose of the study was to prove that eventually something will be statistically significant . . .

We're sorry, but this content is available to Members and Insiders only.

If you're already a DrChet.com Member or Insider, click on the Membership Login link on the top menu. Members may upgrade to Insider by going to the Store and clicking Membership; your membership fee will be prorated automatically.

Problems with the Dark-Chocolate Study

/in /by

The journalist and his colleagues who perpetrated the dark chocolate and weight loss study I talked about Tuesday were out to prove a point: typical journalists, even the ones who specialize in health, don’t really understand the fundamentals of nutrition, weight loss, and fitness. They especially don’t understand statistics and how they’re misapplied even in legitimate studies. Nothing new there.

What I think they did show was that there are many news and information sources on the web that will publish anything provocative. Dark chocolate helping weight loss certainly fit that profile.

The author explained everything he . . .

We're sorry, but this content is available to Members and Insiders only.

If you're already a DrChet.com Member or Insider, click on the Membership Login link on the top menu. Members may upgrade to Insider by going to the Store and clicking Membership; your membership fee will be prorated automatically.