There was an Atlantic article published recently that punched a lot of my annoyance buttons. The article itself is not significantly worse than a lot of pop-sci stories, I’m just tired of reading the same kind of articles hyping “new” studies as if they held real answers for nutrition. Here are a few of the problems I have with it.
1) The study is an epidemiological study. These studies can sometimes find some interesting correlations, but they can never give conclusive results about causes and effects because of the enormous number of independent variables, even assuming the information used is impeccable.
The information is not impeccable. The way information is gathered in studies like these, through self-reported periodic questionnaires, is horribly prone to error, subjectivity, forgetfulness, and bias on both the part of the participants and the researchers. While using questionnaires may be one of the few practical ways to gather information on a large cohort, whatever you obtain is probably so muddled as to be worthless. Even if everyone was scrupulously honest and as exact as possible in answering the questionnaires, the fact of the matter is that there just aren’t that many data points to work with and the questionnaire categories are unavoidably no more than very crude estimates of intake.
To make things worse, there are confounding variables, which is why researchers sometimes have to just ignore data from huge chunks of the cohort. If the researchers are being honest, they do this to omit flawed data. If they’re not being honest — being biased either consciously or unconsciously — they do this because the information isn’t matching the conclusions they’d like to reach.
“Correlation is not causation.” If you read any kind of debates about science, you’ll hear this phrase. A lot. What it means is that sometimes one or more of the data sets you gathered correlates with another data set, but cannot be a cause of it. For example, if you find that brown-eyed people are statistically more likely to commit suicide, it doesn’t follow that having brown eyes makes you want to blow your brains out. Considering that brown eyes are more common than blue, it just means that there are more brown eyed people who have committed suicide than blue in this particular sample. If your statistical models are good and you’re being conscientious, and you’re free of conscious or unconscious bias, you’ve adjusted for this possibility. Probably. Unless you found a false correlation and didn’t realize it.
In longitudinal studies like this, there are also the effects of other factors to wonder about. How much of the change is due to age? How much to diet? How much to lifestyle? When you factor those things in, can you actually draw any useful conclusions?
Looking at the discussion section of the study, I think they actually did try to control for as many variables as they could. The problem is that there are so many things to take into account, and so many things that are impossible to objectively measure that might affect the outcomes.
Considering the number of problems involved in data gathering, the inherent subjectivity of the process, and the unreliability of the statistics involved in epidemiological studies, I would go so far as to say that most are junk science. They are a waste of money that could go toward something more worthwhile, like variable-controlled ward studies, where you have control over the subject’s environment, and measure exactly what goes in and what comes out. You don’t have to guess, estimate, or question, you measure it objectively.
2) The headline is a sensationalized version of what is suggested in the “Conclusions” section of the study. The actual title of the study is: “Associations among 25-year trends in diet, cholesterol and BMI from 140,000 observations in men and women in Northern Sweden”. In other words, “we looked at a bunch of stuff, including some basic biomarkers, and tried to find patterns in it.”
3) From this point, I’m going to just take a look at what was actually in the study, without quibbling about whether the information is accurate or not.
Their conclusion states that an increase in fat intake coincided with popular support for low-carb diets, but the data they actually reported don’t support that anyone in the cohort was following a low-carb diet. The foods they say are associated with high fat intake include many carbohydrate sources: bread, grain-based snacks, potato chips. Their reported greatest increases of fats were as spreads for bread, and oils (presumably vegetable oils) for cooking.
The carbohydrate intake was 45.9% for men in 1986, and 49.2% for women. They obscure the carb intake by reporting only fat and protein intake in the “Changing intake patterns for fat and carbohydrate 1986 to 2010” section. Interestingly, the increase in fat intake that they’re reporting between 1986 and 2010 is a 0.7% increase in men and 2.2% in women. We are looking at aggregate data here, but that small of a change is hardly something to point at as a smoking gun.
If you run the numbers for 2010 you get: Fat, Protein Carbohydrate intake for: Men (F39.95%, P14.3%, C54.2%) Women (F37.7%, P14.3%, C52%). The carbohydrate value is calculated using their figures for fat (separated by men and women) and protein (no separate figure for men and women reported). So according to their numbers, the carbohydrate intake actually increased from 1986 to 2010.
However, their reported figures for 1986 don’t total 100% (the total reported is 98.7%) so this may be an arithmetic anomaly. The problem is that they don’t show the actual values for carbohydrate intake in their paper and the graphs are too rough to tease that information out. If you simply subtract the 1.3 remainder from their 1986 numbers to approximate the values, you still have men at 52.9% and women at 50.7% carbohydrate intake. Either way you look at it, what the people in this study were eating was not a “low-carb” diet. Depending on your nutritional outlook, you could characterize it as moderate carb, perhaps. I would consider that to be a moderately-high carbohydrate diet.
For reference, an actual low-carb diet would look more like 30–40% protein, 10–20% carbohydrate, with the remaining 40–60% of the calories coming from fat. A ketogenic or cyclic ketogenic diet would limit carbohydrates even further, to about 10% of the energy intake or less.
The type of fat they report is interesting too. The highest intake values are for a butter and raps seed (sic.) blend. I assume they are referring to rapeseed, or canola oil. They seem to assume a priori that rapeseed oil is healthy, but it actually seems to be somewhat problematic. The main culprit may be erucic acid, which is limited to 5% of the oil’s content by weight for food-grade rapeseed oil in the EU, a higher amount than the 2% considered safe for human consumption in the US. Interestingly, the inclusion of saturated fat in the diet provides some protection from the undesirable effects of rapeseed oil, notably fibrotic lesions of the heart and vitamin E deficiency. So the finger wagging at saturated fat in this report is completely mis-aimed.
Two things that stood out to me were that BMI for the cohort steadily increased, and there was a 3 year lag between their reported “sharp” increase in fat intake in 2004 and the increased cholesterol values after 2007. They do acknowledge in the “Discussion” section that their study doesn’t allow them to make any conclusions about a causal relationship between the higher fat intake and blood cholesterol levels, but that doesn’t stop them from strongly implying that there is one.
In their figure 7, it’s notable that the lowest cholesterol levels were in 2002, and, even while tending upward, the current levels are still lower than they were in 1990, which showed a dramatic drop when dietary interventions were first attempted. If increased fat intake was such a bad thing for cholesterol levels — and I don’t necessarily think that’s the case — they still have better outcomes than they did when they were giving out the advice to cut fat.
I think a more reasonable explanation for an increase in total cholesterol has more to do with the steady increase in BMI and the increasing age of the cohort. It wouldn’t matter much what you fed them, high fat, low fat, or anything in between, if they were fatter on average their blood lipids would trend negatively. Interestingly, cholesterol can temporarily increase when you start to lose weight, especially when losing visceral fat. If anyone was actually on a low-carb diet — something which is impossible to determine from the data presented in the paper — it’s possible that weight loss is skewing blood lipid data and we could see a downward trend in body fat with a trailing reduction in total cholesterol as the lipids are cleared.
If I were looking to use the information from this study to make recommendations, I would look more at the sources of carbohydrate intake. What you eat, not just how much of it, does matter. The people in the study changed their carbohydrate sources from boiled potatoes and crisp bread (presumably knäckebröd made from rye) to soft bread (made from wheat), rice, and pasta. The total carbohydrate intake is fairly high too, especially considering the source of the carbohydrates is mostly bread and junk food, not vegetables or fruit.
I don’t think the total fat intake is much of an issue, but I do think the fat sources are a problem. Rapeseed oil isn’t a particularly healthy oil, despite lobbyist insistence that it’s perfectly okay. The people in this study are probably consuming the butter/rapeseed blend mentioned in the study out of a misplaced concern to avoid saturated fat from plain butter. They may instead be causing more health issues from the inclusion of an adulterant oil.
The pre-existing biases of the researchers is possibly what leads them to make a recommendation to reduce fat consumption when according to their own data it does not actually seem to be causing problems. Their report shows that the fat intake has only very slightly increased over the beginning 1986 levels, but the cholesterol levels decreased steadily until 2004 and are still lower than the 1990s post-intervention levels, when the medical community was presumably recommending a low-fat diet.