1999 Science in Society Journalism Awards

Magazine

Gary Taubes

“The (Political) Science of Salt”

Science

Part Four: Trials and tribulations

In the grand scheme of the salt controversy, a study such as Intersalt, revisited or not, should have been irrelevant. After all, as researchers on both sides agree, Intersalt was an observational study showing at best weak associations in a field of research where randomized, controlled clinical trials — the "gold standard" of epidemiology — should be able to establish a cause and effect, if any exists. "You kind of can’t believe it’s an issue," says Robins, for instance. "They can actually run randomized experiments [on salt reduction], and they’ve run lots of them." All a researcher needs is to randomize subjects into two groups, one reducing salt intake, one eating normally, and then see what happens.

But the results were as ambiguous as anything else in the salt dispute. Doing the trials correctly turned out to be surprisingly difficult. Choosing low-salt foods, for instance, inevitably leads to changing other nutrients, as well, such as potassium, fiber, and calories. Placebo effects and subtle medical intervention effects have to be avoided carefully. "If you just study people for 10 weeks, you will detect some changes over time which have nothing to do with the experiment you’re carrying out," says Graham Watt, who in the mid-1980s ran three of the first double-blind, placebo-controlled trials on salt reduction.

A technique known as meta-analysis has lately become the route to clarity in such situations. The idea is that if a host of clinical trials gives ambiguous results, the true size of the effect might be assessed by pooling the data from all the studies in such a way as to gain statistical power. But meta-analysis is controversial in its own right. It might have been the ideal solution to the salt controversy had not the salt controversy turned out to be the ideal situation to demonstrate the questionable nature of meta-analysis. As Harvard School of Public Health epidemiologist Charles Hennekens puts it: "It’s all so arbitrary, and you’d like to believe it’s arbitrary in a random way, but it turns out to be arbitrary in the way the investigators want it to be."

In 1991, Cutler, Elliot, and collaborators generated the first meta-analysis of randomized clinical trials on the salt question. They found 21 trials in hypertensive subjects, although only six were placebo-controlled, and six in normotensives, of which only those done by Watt were double-blind and placebo-controlled, and those showed zero benefit from salt reduction. By amassing these trials together, however, the controlled with the uncontrolled, Cutler and Elliot deduced that a 3- to 6-gram reduction in daily salt consumption would drop blood pressure by 5/3 mmHg in hypertensives and 2/1 mmHg in normotensives. This relationship was "likely to be causal," they then concluded, because "the results are consistent with a large body of epidemiological, physiological, and animal experimental evidence." This, of course, was exactly the point of contention.

Cutler’s meta-analysis was promptly overshadowed by the three-part extravaganza published in the BMJ in April 1991 by Malcolm Law and his colleagues. Their conclusions were unprecedented: They deduced that salt reduction has an effect on blood pressure nearly double that found by Cutler and Elliot. Law and his colleagues predicted that "moderate" universal salt reduction — cutting daily intake by only 3 grams — would benefit the population more than treating all hypertensives with drugs, while cutting intake by 6 grams a day would prevent 75,000 deaths a year in Britain alone.

They derived these conclusions in three steps. First, they analyzed the ecologic studies to estimate the average apparent effect of salt on blood pressure. They then "quantitatively reconciled" this estimate with the numbers derived from the intrapopulation studies after suitably correcting those upward for regression dilution bias. Having demonstrated that the ecologic and intrapopulation studies were not in fact contradictory, as had been believed for 20 years, they then proceeded to determine whether this reconciled estimate was consistent with all the relevant clinical trials. These, says Law, turned out to be dead on, thus demonstrating that all studies were in agreement about the considerable benefits of salt reduction.

Although this "quantitative review," as Law calls it, has its supporters, they are in a minority. Its critics — including epidemiologists and statisticians who read the paper at the request of Science — insist the work is so flawed as to be effectively meaningless. Take the selection of which studies to include and which to discard: In the analysis of the ecologic studies, Law and his colleagues chose 23 studies done between 1960 and 1984, and one from Szechuan, China, published in 1937. They then excluded Intersalt, the mother of all ecologic studies, from the analysis because its well-calibrated, standardized blood pressure measurements often yielded numbers 15 mmHg lower than those made in comparable communities by the older, uncalibrated, nonstandardized studies. Critics likened this decision to tossing the baby and keeping the bath water. Law told Science that they excluded Intersalt because the original results were "inadequate" and "too low," but that this was not the case with "Intersalt Revisited," a study he would have included had it been available.

As for the analysis of clinical trials, noted Swales, Law and his colleagues synthesized the results of 78 trials, of which only 10 were actually randomized. One study even predated the era of modern clinical research. The fall in blood pressure that Law and his colleagues attributed to sodium, says Swales, was likely due to the "impact of poor controls." Even Richard Smith, the BMJ editor who published the research, described it to Science as "not the best we’ve ever done."

Intersalt "did not show blood pressure increases if you eat a lot of salt."

— Lennart Hansson

"The position has been clarified; all the Intersalt analyses confirm salt as an important determinant of blood pressure."

— Malcolm Law

Law, however, says the study has stood up well, noting that its findings agree with those of Intersalt Revisited. And despite the critiques, Law’s meta-analysis is still one of the most highly cited papers in the salt literature and was one of the bedrocks — along with Intersalt, the study Law considered inadequate — of the 1993 NHBPEP primary prevention report.

• Part One: The salt controversy
• Part Two: Crystallizing a debate
• Part Three: Intersalt tries again
• Part Four: Trials and tribulations
• Part Five: Poles apart
• Part Six: Picking your battles
• Part Seven: Touchstones of the salt debate

Science in Society Journalism Awards:
2005 | 2004 | 2003 | 2002 | 2001 | 2000 | 1999 | 1998 | 1997 | 1996
about | books | broadcast | magazines | newspapers | radio | television | web | enter