Can It Really Be True That Half of Academic Papers Are Never Read?
By Arthur G. Jago JUNE 01, 2018
https://www.chronicle.com/article/Can-It-Really-Be-True-That/243564
Arecent Chronicle opinion essay arguing that the tenure process can be quite unfair included this line: "At least one study found that the average academic article is read by about 10 people, and half of these articles are never read at all." In a commentary that I was otherwise in complete agreement with, I found that particular statement quite unbelievable. First, the magnitude of the assertions was simply astonishing. Second, I was perplexed by how someone could design a study to empirically determine that some published articles were never read. Such a study was beyond my imagination; the pseudo-logical fallacy of "proving the negative" came to mind.
I contacted the author and was provided her source, an article in Smithsonian, the magazine. This article actually qualified (somewhat) the implausible claim by asserting that 50 percent of papers are never read by anyone "other than their authors, referees and journal editors." I guess it is some consolation to know that humans do indeed write, review, and select most manuscripts for publication, although we do know that computer-written gibberish occasionally makes it into print, into citation indices, and into researchers’ h-values.
A link in the Smithsonian article points to Indiana University as its source for the statistics, but this proved inaccurate. The Smithsonian author redirected me to the actual source, a 2007 article by Lokman Meho in Physics World, the magazine of the London-based Institute of Physics. When I asked Meho for his source of the cited statistics, he told me that "this statement was added to my paper by the editor of the journal at the time and I unfortunately did not ask from where he got this information before the paper was published." The Meho article has been formally cited over 300 times.
In turn, I contacted the editor of Physics World from 2007. He told me that "it was indeed" something that he had inserted during editing, from material provided to him in a communications course taken at Imperial College London in 2001. I contacted the instructor of that course, now retired, who told me he could not provide me with a specific reference to what is now "ancient history" but that "everything in those notes had a source, but whether I cross-checked them all before banging the notes out, I doubt."
The Physics World editor suggested that the Imperial College course material may have been based on a 1991 article in Science. However, I discovered that that article was not about unread research but was rather about uncited research. Not being read is a sufficient condition for not being cited; however, not being cited is neither a necessary nor a sufficient condition for not being read — i.e., not being cited says nothing about an article having been read or unread. As a striking illustration of the difference, a 2010 article was recently identified in Nature as an online paper that has never been cited but has been viewed 1,500 times and downloaded 500 times. (The paradox, of course, is that this uncited paper is not now uncited, by virtue of it being cited for its uncitedness.)
Frustrated, I ended my search for the bibliographic equivalent of "patient zero." The original source of the fantastical claim that the average academic article has "about 10 readers" may never be known for sure.
In the bigger picture, it is certainly true that much of published research has limited readership. As a young scholar — and well before electronic journal access — I was quite amazed to learn that one of the five most prestigious academic journals in my field (business management) had a worldwide circulation, including all libraries, of a mere 800 copies. Indeed, our audiences are often quite small, and some large percentage of articles undoubtedly have very little impact.
However, because an assertion is intuitively appealing or reinforcing of existing beliefs does not justify misstatements of fact or the distortion or embellishment of what can be documented. In their communications with me, all of the participants in this tale — good people, to be sure — recognized an absence of sound justification in their actions.
Even when a primary source is accurate, a reference to it may still be quite problematic when an author relies upon a flawed secondary source but cites, instead, the primary source. Using statistical modeling of recurring identical misprints in bibliographic entries, two UCLA engineers estimate that "only about 20 percent of citers read the original" article that they claim as a source in their own reference lists. Stated otherwise, 80 percent of citers are not readers, and the secondary flaws they encounter they themselves propagate in their own articles.
This object lesson in the perils of relying on secondary sources reminds us all that our readers place a trust in us each time we put words to paper. We have a duty, on behalf of all authors, to do our best to fulfill that trust when we produce those words. A single mistake — a bibliographic patient zero — may be quite small and entirely unintentional. However, it can infect the literature like a self-duplicating virus and become amplified with time.
In a 2009 essay, the Pulitzer Prize winner John McPhee noted that "any error is everlasting" and quoted Sara Lippincott, a New Yorker fact-checker, that once an error gets into print it "will live on and on in libraries carefully catalogued, scrupulously indexed … silicon-chipped, deceiving researcher after researcher down through the ages, all of whom will make new errors on the strength of the original errors, and so on and on into an exponential explosion of errata." Lesson learned.
Arthur G. Jago is a professor emeritus of management at the University of Missouri at Columbia. He has published articles in, among other journals, the very prestigious but not widely read Organizational Behavior and Human Decision Processes.
-
cwinton • 4 days ago
We might as well admit that fake news has plenty of company in academic publication, in this case an unverifiable statement that for reasons unknown resonates as likely being true. It's the age old story of rumors spreading far faster than the truth. We all can cite similar examples, although I have to admit this is an unusually juicy one.
-
Joe S • 4 days ago
Nice try. Seems to me like the original "source" was exaggerated or fabricated.
-
schultzjc • 4 days ago
This is not the first article to point out the fallacy (or at least apocryphal nature) of the original contention about readership of academic journal articles. But Dr. Jago has done some excellent forensics and contributes an interesting observation about the degree to which citation and readership may or may not be related. That situation has change dramatically very recently. An "...online paper that has never been cited but has been viewed 1,500 times and downloaded 500 times" couldn't exist until online publication began, quite recently. We could never know how many paper copies were read previously. And of course neither "viewed" nor "downloaded" mean the paper was read. As the data about cited papers being unread suggest, there isn't much reading going on, even for cited papers.
I think academics continue to publish mainly for their tiny disciplinary clubs (and families), often ignoring a much larger audience that includes folks who paid for the published research. If the goal is large readership, writing for a general audience and publishing in nonacademic venues is the way to go.
Then again, I may be feeling morose because a report in the journal Nature this week pointed out that only 1% of papers published in the 80s (a peak for me) continue to be cited now. Sigh. At least I know my mom read my papers. (Or said she did.)
-
mkant69 • 4 days ago
This reminds me of the scene in the movie Real Genius, where Dr. Meredith tells Mitch "A bit of advice... Always... no, no... never... forget to check your references."
Too often people include a reference without ever having read it. When reviewing journal articles, I often encounter references that don't support the asserted claim. Sometimes the reference argues against the claim. Even when a reference supports the claim, it is usually just the first in a chain of references that merely mention the claim, as opposed to the primary source of the claim.
-
EconProf • 4 days ago
An example of Colbert's "truthiness" where a statement feels like it's true and, even if it's not (or there's no empirical evidences to support it), it ought be true. Such an extreme statement is just close enough to one's beliefs or limited, nonrandom experience to be credible and confirmatory.
Also, we should be disturbed about the complementary set of contributions to research that is virtually unpublishable due to the blind review process and editor's judgment. Of course, thousands of submissions should be rejected for sloppy methodology, irrelevance, etc., but many others cannot be accepted as long as the authors of the primary cited papers are alive to reject any paper that may diminish their academic stature. Research areas are so narrow that it's often easy to predict who top journal editors will send your paper off to review.
Worse yet, in a publish or perish world, profs self-censor for fear that powerhouse reviewers defend their self-serving status quo against critical re-assessment. Instead, tenure-track profs must be risk averse by fawning over the leading researchers by writing, "The pathbreaking work on this vital subject (Elderfuss, 1992, 1994a and b, 1997a, b, c) changed the universe as we know it today." The result is that research can't progress, delayed by an establishment as reactionary a censorship, in its own way, as Galileo confronted. That's why even Nobel winners like Paul Krugman have been forced to abandon journals submissions in favor of working papers networks and conferences that allow work to see the light of day without the corrupt mediating filters. Otherwise, a tree falls in the forest and doesn't ever make a sound as long as academic heresy is defined as challenging the powers that be.
-
Tom Aaron • 2 days ago
A fabrication.
However, in my field not far from reality. Perhaps 5 researchers in the world understand the material. All would be consulted (in the loop) and a couple of those would be doing the peer review for our Geological Survey journal. Sadly...more or less at the end of the 125 year legacy...no new blood to pick up the torch.
Until someone 'reinvented the wheel' to understand our field of study...never to be read.
-
andrewp111 • a day ago
Most journals these days are all-electronic, so if a paper is never downloaded, you can presume that it is never read.
-
Don Hubin • a day ago
"Not being read is a sufficient condition for not being cited." Would that it were so! Alas, in some disciplines it is not. In my own discipline, philosophy, I think it is very rare for a paper to be cited without being not only read by the author but being, in some way, influential on the author's thinking. However, this is not true in all disciplines.
When my son, then a law student at the University of Chicago, was assigned a paper written by a very distinguished legal scholar that had a citation to one of my own papers, he was understandably excited. I pointed out to him that, given the practice of writing for law reviews, the fact that the author cited my work did not provide any strong evidence that the author had read a single word of my paper. It's possible that a research assistant did, of course. But it's also possible that the research assistant merely noted from the title and abstract that my paper was relevant to the specific topic discussed by the author in that paragraph and threw in the citation without having read my article.
Charitably, one could propose that the practice of citing lots of unread articles is engaged in to assist readers who want to pursue a topic in greater depth. But, cynically, I suspect it is primarily to provide the patina of deep scholarship where there is no substance to that appearance.
-
Jane Robbins • a day ago
With e-journals and, increasingly, e-access to print (and the trend toward eliminating print versions), it should be increasingly easier to affirmatively determine which articles are read online or at least downloaded (if not read), and for which only an abstract is read, through software and web-scraping that tracks clicks of article links, scrolling, time on page, downloads, etc. I'm afraid I suspect that greater, not less, than 50% are read, whether cited or not.
-
Andy Brower • a day ago
It seems to me that there is no way to measure how many times any academic publication or other document has been read by anyone. Proxies might be downloads, or citations, or book sales, or some other altmetric. But as noted by Don Hubin, even a citation does not necessarily mean that someone has read the paper. I think that is especially true for many highly-cited papers, where people have read about them in a secondary source and do not feel obliged to read the actual document. This all begs the question of what one means by "read", as well: how many people read the supplementary material (even reviewers and editors!) of articles in Science and Nature that might be orders of magnitude more voluminous than the article itself?
By the way, for what it is worth, 11 of 77 of my publications that appear in Web of Science are reported as having zero citations- about 15%, but four of those are from 2018.
The Journal Cladistics has published 978 articles (omitting meeting abstracts and corrections), and 938 of those have been cited at least once (4% with zero citations). Again, several in the latter category are 2018 publications, so perhaps will eventually get cited as well.
-
-
sand6432 • 20 hours ago
Lindsay Waters, editor at Harvard University Press, famously made a similar claim about articles in the field of literary criticism, which led him to call certain journals "publish only" journals compared with journals that actually contain articles that are read. As for the perpetuation of errors, there has been a study that shows that many scholars do not go to the original source to check whether a quotation is accurate but instead rely on the author of the source they are using to have gotten it right. ---Sandy Thatcher