1999 Science in Society Journalism Awards

Magazine

Gary Taubes

“The (Political) Science of Salt”

Science

Part Three: Intersalt tries again

In 1993, after the NHBPEP cited Intersalt as supporting a recommendation of universal sodium reduction, the Salt Institute, a Washington-based trade organization of salt producers, began a concerted effort to obtain Intersalt’s raw data. The institute’s director, Richard Hanneman, says he wanted to examine the reported association between salt intake and rise in blood pressure with age. He and some of the researchers who consult for the institute for $3000 a year — McCarron; University of Alabama, Birmingham, cardiologist Suzanne Oparil; University of Toronto epidemiologist Alexander Logan; and UC Davis nutritionist Judy Stern — were puzzled by what they saw as a contradiction in the data. If higher salt intake resulted in a greater increase in blood pressure as the population aged, they reasoned, the centers with high salt intakes should have had higher median blood pressures, which wasn’t the case. Only if the Intersalt centers with high salt intake had lower blood pressure to start with could their median blood pressures have come out roughly equal, as Intersalt reported. While this seemed counterintuitive, Intersalt had not published the data — the blood pressure of the 20- to 29-year-olds — that would allow the hypothesis to be checked independently.

Hanneman failed to get Intersalt’s raw data, but he did obtain enough secondary data to publish a paper in May 1996, in an issue of the BMJ dedicated to Intersalt. Hanneman claimed to confirm that Intersalt centers with higher salt intake did indeed have lower systolic blood pressures in their youngest cohorts. Accompanying editorials, all written by outspoken advocates of salt reduction, harshly rejected the analysis. Malcolm Law, for instance, dismissed Hanneman’s ideas as a "bizarre hypothesis" and an example of "the lengths to which a commercial group will go to protect its market when presented with clear evidence detrimental to its interests." But none of these commentators addressed the apparent contradiction in Intersalt’s claims. Other researchers who read the paper — Intersalt collaborator Friedrich Luft, for instance, a nephrologist at Berlin’s Humboldt University, and Freedman, who read it at Science’s request — noted flaws in Hanneman’s reanalysis but also agreed that the Intersalt findings seemed inexplicable.

This particular dispute turned out to be moot, however, given the controversy ignited by another paper in the same issue: Intersalt’s own reanalysis of its data. Under the title Intersalt Revisited, Stamler and his colleagues addressed what they considered a problem in their original publication: that they may have underestimated the true association between salt and blood pressure.

Their reanalysis stepped into one of the most controversial areas in epidemiology, known as regression dilution bias. The gist is that if an association between two variables — such as salt and blood pressure — is real, any errors in measuring exposure to either variable will only serve to "dilute" the apparent cause and effect. In this case, because both 24-hour urine samples and single blood pressure readings are likely to stray from the long-term averages, Intersalt’s analysis would have underestimated the true strength of the effect of salt on blood pressure. "If [the association] is real," says Elliot, "it is biased toward the null, and so you have to accept the reality that it must be larger than measured." Statistical techniques could then be used to correct it upward to its proper size. The catch, of course, is that such corrections would inflate a spurious association as well.

Stamler and colleagues, certain of the reality of the salt-blood pressure link, now corrected their 1988 estimates for regression dilution bias. With a few other corrections, the net effect was to enhance the apparent benefits of salt reduction from something ambiguous in 1988 to consistent, "strong, positive" associations in 1996. Cutting daily salt intake by 6 grams, they now concluded, would drop blood pressure by 4.3/1.8 mmHg, a benefit three times larger than originally estimated. "Now the position has been clarified," wrote Law. "All the Intersalt analyses confirm salt as an important determinant of blood pressure."

But the position had not been clarified. The BMJ editors had initially commissioned a commentary to run with Intersalt’s reanalysis from epidemiologists George Davey Smith of the University of Bristol in the United Kingdom and Andrew Phillips of the Royal Free Hospital School of Medicine in London. The critique they submitted was so damning of Intersalt Revisited, however, that the BMJ editors felt compelled to reveal it to the Intersalt authors before publication. According to BMJ editor Richard Smith, Stamler and his colleagues objected so strongly to the commentary that the BMJ agreed to run it 6 weeks later, disassociated, at least in time, from the work it called into question. Positive finding? Intersalt data show a correlation between salt consumption and the rise in blood pressure with age.

As Davey Smith explained to Science, their commentary identified a litany of problems with Intersalt Revisited, from "O-level mathematical mistakes" to basing their statistical corrections on assumptions unsupported by data. For instance, in order to correct for regression dilution bias, Stamler and his colleagues assumed that changes in sodium intake and blood pressure in any individual were independent of each other over periods of a few weeks. But if blood pressure and salt intake did fluctuate together, Davey Smith and Phillips noted, then the Intersalt corrections would result in "an inappropriately inflated estimate." The two epidemiologists cited studies concluding that blood pressure and salt intake are related in the short term and pointed out that "the very hypothesis under test — that sodium intake ... is related to blood pressure — would predict [these] associations."

In their response, published in the same issue, Stamler and his colleagues insisted that their corrections were legitimate because the "totality of the evidence — the only sound basis for judgment on this matter — supports the conclusion that this association is causal." They cited the "independent expert groups, national and international," that had concluded habitual high salt intake was a causal factor of high blood pressure, although they neglected to mention that those groups had all relied on Intersalt circa 1988 to reach their conclusions. Intersalt also listed seven reasons why their original estimate was "probably underestimated" but seemed to make no attempt to find reasons why it might have been overestimated. "It was embarrassing to read," Harvard School of Public Health epidemiologist Jamie Robins told Science, while describing Intersalt’s arguments as "arcane, bizarre, and special pleading."

The commentary and response led to yet more letters in the BMJ the following August. Now Davey Smith and Phillips were joined by a half-dozen other researchers criticizing Intersalt Revisited, such as Nick Day, head of the biostatistics unit of the British Medical Research Council (MRC) in Oxford. "As soon as you start making big corrections [to your original findings]," says Day, "people begin to get suspicious."

Day describes the problem with Intersalt Revisited as one of "garbage in, garbage out" and believed it had implications well beyond the salt controversy: Stamler and his colleagues, like many epidemiologists, assumed they could correct for underlying uncertainties in their data with statistical methods. "It doesn’t work," he says. "There will always be uncertainty surrounding what you’ve done, and if what you’ve done makes quite a serious difference to the crude observed relationships, then it puts a great haze of doubt over the whole thing. If you have an underlying uncertainty — that is, ’garbage in’ — it is never going to be refined into gold."

This assessment is rejected by Stamler and most of his Intersalt Revisited co-authors, although not all of them. Michael Marmot, for instance, an epidemiologist at the University College London Medical School and a signatory of Intersalt Revisited, told Science that, in retrospect, the reanalysis was not compelling. "Somebody looking at this from the outside," he says, "could well take the view that [the corrections] were done for one reason alone, which was to increase the size of the associations. They would not be crazy for taking such a view just based on reading the paper."

• Part One: The salt controversy
• Part Two: Crystallizing a debate
• Part Three: Intersalt tries again
• Part Four: Trials and tribulations
• Part Five: Poles apart
• Part Six: Picking your battles
• Part Seven: Touchstones of the salt debate

Science in Society Journalism Awards:
2005 | 2004 | 2003 | 2002 | 2001 | 2000 | 1999 | 1998 | 1997 | 1996
about | books | broadcast | magazines | newspapers | radio | television | web | enter