Blah, blah, blah: a jaded calculation of the rhetoric-to-results ratio in public health

We had our mailboxes stuffed this week with really boring journal articles (what’s that, you say? That’s every week?). So we decided to step back and ask a simple question: how much paper does the public health field produce, and to what effect? More specifically, nearly half of the articles we were forwarded this week seemed to be commentaries that filled up a vast proportion of journal and email in-box space. These range from self-negating headlines like “Global health policy remains important in election year” (if true, the editorial would be unnecessary) to “September is infant mortality awareness month” (though we couldn’t figure out what such awareness would help prevent—“Oh, it’s September, I guess I won’t drop my baby for the next few weeks…”). More commonly, we’ve been seeing a lot of “calls to action” to address various diseases that appear to correlate to the authors’ specialty of study and/or funding stream rather than actual disease prevalence. Do these types of commentaries do anything? What’s the “rhetoric to results” ratio of these kinds of editorials? We decided to perform a little experiment and a few calculations to find out…

 We started by taking a look at longitudinal data on the number of “calls to action” that appeared in the public health literature for the last several decades. A quick search of the PubMed database revealed an exponential rise in such “calls to action”; in typical public health fashion, we’ll unnecessarily abbreviate these articles “CTAs”, so that they can be confused with the Chicago Transit Authority and probably hundreds of other more useful acronyms.

As shown in the above figure, the sheer number of “calls to action” in the literature indexed on PubMed has increased from under ten per year in the 1970s, 1980s, and (curiously) most of the 1990s, to over 70 per year over the last five years. While only five such calls were published in the year 1993, 106 (that’s a lot of calling!) were published in 2011. A lot of these CTAs were concentrated in just a few journals; The Lancet, for example, published 18 of them, while Circulation published another 11.

What on earth were all these people calling action to? Almost all the CTAs were calls to fund more research. Among top causes were researching the delivery of healthcare (154 CTAs), heart disease (73 CTA’s), women’s health (53 CTAs), and obesity (45 CTAs).

So naturally we asked, do these calls seem to correspond to actual changes in public health research funding? I think you can guess the answer, but we’ll look at the data anyway:

It looks like the absolute number of calls are possibly vaguely correlated to  research funding. Here, research funding was defined as NIH funding areas.

So does this mean CTAs are effective in garnering more research funds? Actually, we think it’s quite the contrary. When we introduced “lags” into our analysis to find out if the CTAs were leading to subsequent changes in funding, or vice versa, we actually found a curious fact: the funding typically preceded the CTA, not the other way around. That is, people were often calling public health practitioners to fund topics that were already being funded. We would hypothesize that CTA authors actually seem to follow the bandwagon of “what’s hot” in public health research, rather than actually leading to genuine public health revelations that produce new research initiatives. The CTAs appear to be published after the initiatives are already well underway (and far too long to be blamed on slow publication cycles). Furthermore, we did not find any significant correspondence between CTAs and changes to funding; CTAs did not result in an increase in funding from their pre-CTA level, for example. So maybe the rhetoric-to-results ratio is quite a bit higher than a mere correlation would suggest.

But we also unexpectedly found a couple of other curious facts about CTAs. Just a few authors seem to dominate the CTA writing scene. When we constructed collaborator clouds to see who was writing these pieces, a few clusters of people seemed to be responsible for over 6 CTAs each.

Less surprising was the geographic distribution of CTA writers. Most of them were from the US and Europe. Among those in the US, a large number of publications originated in Washington D.C. (21), followed by Boston (19) and New York City (18).

But most importantly, do CTAs correspond to disease burden? (Here, comparing CTAs to disease burden in North America in terms of prevalence in thousands of people). Are CTAs attempting to correct for gross injustices in what we pay attention to and dollars for, versus what’s really killing people?

Nope.

4 responses to “Blah, blah, blah: a jaded calculation of the rhetoric-to-results ratio in public health

  1. This is awesome. I wonder how many people will say that their action was because someone “called” them in a journal article.

  2. This is fantastic. I wrote a letter to the Lancet a year or two ago titled: A call to action to end calls to action. We cited all of the Lancet’s many published calls over the past few years and stated time for the (same) authors to stop calling and start acting.

  3. These authors should be required to write follow up articles addressing whether or not their “call to action” has been met. Or on the other hand, journals should conduct a crude assessment of funding- disease ratio to determine whether these articles are truly necessary. It would definitely help me out in cutting down on articles that I am assigned to read for class…

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s