Free Our Books

because books want freedom, too

Peer review contributes to creation of unfounded authority?

with one comment

falseAuthorityIn this fascinating and quite overwhelming study published in BMJ, Steven A Greenberg, associate professor of neurology, argues that  in the citation network that was the object of his research, backdoor invention affected  negatively the quality  of knowledge produce, since it: “repeated misrepresentation of abstracts as peer reviewed papers to fool readers into believing that claims are based on peer reviewed published methods and data” . Although author clearly states that it is through abstracts, which in his view is a way to avoid peer reviewing,  that backdoor is introduced,  my conclusion is that on the basis of this research, we can say that peer reviewing in medical science is capable of significantly contributing to the creation of  a large unfounded authority. Here’s authors conlusion paragraph in full:

Citation is both an impartial scholarly method and a powerful form of social communication. Through distortions in its social use that include bias, amplification, and invention, citation can be used to generate information cascades resulting in unfounded authority of claims. Construction and analysis of a claim specific citation network may clarify the nature of a published belief system and expose distorted methods of social citation.”

He develops a vocabulary of citation distortions, and backdoor is the method how peer review is bypassed:

“In another form of invention, claims are introduced as fact through a “back door” that bypasses peer review and publication of methods and data. This is accomplished by repeated misrepresentation of abstracts as papers (seven different papers, 17 citations to 12 different misrepresented abstracts; […] Back door invention—repeated misrepresentation of abstracts as peer reviewed papers to fool readers into believing that claims are based on peer reviewed published methods and data”

He lets peer review lightly off the hook here. Isn’t the task of peer reviews to spot any problems with the quality of work reviewed, regardless of what form the distortion of knowledge claims takes? falseAuthority2

In my reading of his text, the author confirms the responsibility of the peer reviewing implicitly when he states that  a medical claim (see his paper for specific claim used here) “is supported in this manner and accepted by peers as fact”. In other words, peers accepted as fact a claim that slipped through peer reviews, multiple times, in multiple papers. The graph on the right shows that “Only one of 32 citations flows to papers70 71 72 73 77 78 that present data that conflict with the validity of these models” – models that were the basis of knowledge claims supported as the fact by all the papers on the right of the graph in the blue. It’s a complex analysis, and it’s possible that i misunderstood. I emailed the author and asked for his view on my understanding that he let peer review too lightly off the hook. Here’s the full paper How citation distortions create unfounded authority: analysis of a citation network.
.

Advertisements

Written by KontraMraku

12 August 2009 at 21:07

Posted in Analysis

Tagged with

One Response

Subscribe to comments with RSS.

  1. Interesting paper!
    In my own experience with publishing, the peer review process also can tend to favour authors with a greater social currency than others. I published once with a professor who was, as he said, “good at publishing.” He had been able to build up a bibliography based almost entirely on systematic reviews, and the sheer amount of his publications gave him an unnatural deference from colleagues older and more experienced than he, who had spent their lives doing quality, front-door, peer-reviewed research.
    This professor’s experience also led him to develop a skill set – not in a research area – but in subverting the publishing system in order to get published whatever he wanted to.

    Jocelyn

    4 October 2009 at 21:14


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: