Comparing open vs.
The Journal of Medical Internet
uses a mix of open and
closed peer reviews to adjudicate the
articles it publishes. Editor Gunther
Eysenbach and journal authors rate
the quality of reviews on a scale of
one to five, with one being of poorest
quality and five the best.
So how do the two processes
stack up? It depends who you ask.
The editor gives a slight advantage
to closed reviews. His mean rating
for open reviews was 3. 9, compared
to 4. 4 for closed reviews. But the
authors seemed to prefer open
reviews, giving them a mean rating
of 4. 3 compared to 3. 9.
lacked substance and were of little use in making publishing decisions.
The physics community adopted the practice – albeit unofficially –
earlier still. Since the early 1990s, physicists have posted pre-published
drafts of their research papers on arXiv, an online repository of scholarly
articles now hosted by Cornell University. Rob Myers, senior faculty
member at the Perimeter Institute for Theoretical Physics in Waterloo,
Ontario, says the idea behind arXiv was to make research findings available
to a broad community as quickly as possible, rather than have researchers
wait for months for the findings to appear in a journal. “It really wasn’t a
question about peer review,” he explains. ArXiv doesn’t include an online
forum for commentary, but it is common practice for readers to e-mail
their feedback to authors.
It may be that certain fields or subfields are better suited to open
review than others. Gunther Eysenbach, editor of the
Journal of Medical
, a Canada-based, online and open-access journal, says its
readers have readily accepted open review. “Our audience is very tech-
savvy and intrinsically interested in this kind of experiment.”
Dr. Eysenbach says the journal adopted the system in 2009 to expand
its pool of referees and make its articles accessible to lay readers. In some
ways the experiment has fallen short of its goals: most of the self-selected
reviewers are still academics, and sometimes the ones who aren’t don’t
fully grasp the idea of what’s required.
But overall the experiment has been a success, and the journal in-
tends to continue the practice, mainly because authors seem to prefer it
(see “Comparing open vs. closed reviews” at the top of the page). Another
benefit has been that the number of self-selected referees helps Dr. Ey-
senbach gauge reader interest in topics the magazine covers. “We will
never fully replace the role of the editor in assigning reviewers,” says
Dr. Eysenbach, an associate professor in the department of health policy,
management and evaluation at U of T. “But we can complement it.”
In the humanities, the success of the
has spurred other journals to follow suit.
, a quarterly pub-
lished by Palgrave Macmillan, conducted an open peer review last year
with similarly positive results. Jen Boyle, assistant professor of English at
Coastal Carolina University and guest editor of the edition, was pleased
that so many commentators took part and with the quality of their feed-
back. But she admits the comments lacked the critical heft of private
reviews. “It’s not that there weren’t any negative comments,” she says.
“They just weren’t quite as feisty as you’d see in a private peer review.”
Canadian scholars are also testing the waters. Researchers at the
University of Victoria’s Electronic Textual Cultures Laboratory recently
posted on Wikibooks (a Wikipedia sister site) an electronic version of the
, a collection of 16th-century English poetry and
verse. The manuscript contains an introduction, biographies and genea-
logical tables, all of which can be edited and commented on by readers.
in this issue
Click to subscribe to this magazine
article text for page
< previous story
next story >
Share this page with a friend
Save to “My Stuff”
Subscribe to this magazine