CAN SOCIAL SCIENCE RESEARCH BE TRUSTED? If you're one of those who have misgivings about social science research because you suspect the methodology is squishy, here's some alarming ammunition for you — and, implicitly, a warning for science writers. It comes from Wray Herbert at his blog We're Only Human. Thanx to Nancy Shute for the tip.
Herbert is describing a paper from researchers at Penn and Berkeley that seems to prove an absurdity: listening to music about old age — specifically the Beatles' “When I’m Sixty-Four” — renders the student subjects 18 months younger than students who listened to "Kalimba",
The scientists
did really run the experiment as described, and they used accepted practices for reporting and analyzing their data. They ran the experiment to demonstrate a serious flaw in the usual way that behavioral science data are collected and reported — a flaw that (they claim) allows scientists to prove that “anything” is a significant finding.
This they attribute to researcher freedom to make decisions that skew results in the behavioral sciences. One such decision is to beef up the sample size during the study. For example, if the results are not statistically significant when the n is 20 subjects, a scientist can go right ahead and add more — as many as he/she likes, apparently — in hopes that a bigger study will yield significance. This, the paper avers, can boost the false-positive rate by 50 per cent.
Bless my naive and trusting soul. I thought a routine part of study design was to decide ahead of time how many subjects were needed to demonstrate statistical significance (if any) in a specific research protocol. Apparently not.
The paper outlines other such techniques, concluding that employing all of them would lead to a false-positive rate of more than 60 per cent. (Today's rule of thumb is 5 per cent false-positives in the behavioral sciences, but a survey has shown that 7 out of 10 social scientists employ the sort of "adjustments" that would bias the data in that direction.)
In other words, a researcher with all the best intentions is more likely than not to falsely detect a positive result just by using the accepted practices in the field.
At the Evolutionary Psychology Blog, Robert Kurzban also views the situation with horror. He notes that the impact of social-science false-positives might, in fact, be even worse than the numbers suggest. That's due to the fact that high-profile journals publish them because they are surprising findings. So false-positive findings could plausibly have a disproportionate impact on the field.
They also get more media attention — that's us, folks. But those same journals will not be interested in publishing failures to replicate. So the false positives will go on counting as the truth even after they have been refuted.
There are a handful of comments on the Kurzban post, but I couldn't find any other blogging about this paper. This despite its stunning charge that the social science literature is not to be trusted. You'd think that allegation would cause more of a stir. Wouldn't you?
HAPPY FIRST BIRTHDAY, TON! Hard to imagine that there could be a science writer out there who doesn't know about that most valuable resource, The Open Notebook (Motto: The story behind the best science stories). But just in case...
TON is the brainchild of freelances Siri Carpenter and Jeanne Erdmann, who began by persuading other writers to take us, step-by-step, through the process of making and selling exceptional articles they did. But TON is now a year old, and getting much bigger.
First, it recently added a database of successful pitches to outlets you want to write for by writers you've heard of. Study up and learn how to explain that terrific idea so that an editor will think it's terrific too.
This week there's a new feature: Ask TON, wherein questions asked by science writers are answered by science writers (and editors). Questions like how to write a successful pitch, how to prepare in advance for a scientific conference so you can come away with lots of ideas for salable pieces, how to decide whether your embryonic idea can become a story.
Many, many happy returns.
HERE COMES THE HPV VACCINE AGAIN The human papilloma virus is on the march once more, killing trees in their thousands and gobbling up the world's electrons too.
Let's start with the new recommendation from a federal advisory group. It says that prepubertal boys, not just girls, should get the HPV vaccine, too. The chief point of giving girls the Gardasil vaccine has been to prevent deadly cervical cancer in later life. (As all within the sound of my voice know, cancer of the cervix is a sexually transmitted disease caused by HPV, but I have to explain that even here because that spectral editor who's always hanging over my shoulder says some readers might not and so I must. Boo! Happy Halloween.)
Vaccinating boys would not only keep them from transmitting the virus to females when they start having sex, it also would prevent their own very icky cancers (anus, penis, oral) and genital and respiratory warts. For straightforward reporting on the recommendations, see Deborah Kotz at Daily Dose, the Boston Globe and Shari Roan, at the Los Angeles Times's Booster Shots,
At the NPR health blog Shots, Scott Hensley reports on the committee's recommendations too and says Merck, which makes Gardasil, told him that vaccinating boys would be cost-effective.
Shari Roan, in a different Booster Shots post, notes that the committee also reviewed the HPV vaccine's safety data and concluded that there is no evidence of mental retardation (as Presidential candidate Michele Bachmann has charged) — or much of any other ill effect either.
The Alliance for Natural Health begs to differ. It calls the recommendation to extend the vaccine to boys "shocking" and "outrageous" and lists several reasons why — including, it says, many adverse events (among them 68 deaths) that have been ignored by the government and Institute of Medicine.
And then there's Age of Autism, an anti-vax blog. It reports the recommendation while saying "You might recall that the former head of the CDC now runs the vaccine division for Merck," and linking the story with a California TV station's report of an outraged San Diego woman charging that her 11-year-old grandson was given the HPV vaccine without parental approval.
OTOH, ERV expatiates on the vaccine's virtues, including its ability to prevent HPV-caused oral cancers — now becoming more prominent since so many people have given up smoking, which used to be the primary cause.
At Journal Watch Infectious Diseases, Larry Baddour describes a new study showing that Gardasil prevents anal cancers in men who have sex with men. That's another reason to vaccinate boys before they start having sex of any variety. And at sister blog Journal Watch Women's Health, Anna Wald reports on a study showing that only one or two doses of HPV vaccine might be just as effective as the current standard, three doses. This study didn't involve Gardasil, but rather a different HPV vaccine, Cervarix. But fewer doses would lower the cost and the nuisance factor, making it likely that HPV vaccines would be more widely used.
And at its Well blog, the New York Times attempts to take the public pulse by asking readers if they would give their sons the vaccine. 79 comments at this writing.
But here's an intriguing thing: there's new evidence that HPV contributes to cardiovascular disease in women, according to Larry Husten at CardioBrief. Which, if true, could vindicate the doc minority that has long argued atherosclerosis is an infectious disease. See also Katherine Hobson at the Wall Street Journal Health Blog, who reports some enthusiasm for reexamining HPV vaccine trial data to see whether an effect on the risk for heart disease shows up there.
At the Washington Post's Wonk Blog, Sarah Kliff describes two worrisome trends that could reduce vaccine use, both of them involving physician attitudes. First, she says, younger docs are reportedly more likely than the greybeards to believe vaccines do more harm than good. Second — and perhaps more important — docs are losing money on nearly half the common vaccines they administer because insurance reimbursement rates are so lousy.
COMING SOON: SCIENCEONLINE2012. COMING NEXT WEEK: THE SCIO12 REGISTRATION SCRAMBLE. The annual science blogging Mecca ScienceOnline2012 will happen January 19-21, but the all-important registration process will start next week. As veterans know, you don't dawdle when SciO registration opens; the demand to get in is so enormous that last year registration was closed again after just 44 minutes.
So here's the SciO12 registration schedule, staggered this year:
Tues, Nov 1st at 12 noon EDT
Thurs, Nov 3rd at 6 a.m. EDT
Tues, Nov 8th at 00:01 a.m. EST
Wed, Nov 9th at 6 p.m. EST
And here's the registration URL, along with info on fees.
This time #scio12 is moving to Raleigh NC from its previous base, the Research Triangle, in order to accommodate about 150 more of the hundreds who want in. The ETA for the draft program is here on Halloween, Oct 31. But the program specifics almost don't matter; if you write about science online, #scio12 will do you good and help you too. Guaranteed.
.png)
