The open access dragnet 
In Who's Afraid of Peer Review, recently published in Science, John Bohannon reports on an experiment he did with open access science journals. He sent them a spoof paper that had the form of a serious article but was chock full of horrible errors. Only about 38% of the journals rejected it. Bohannon also talked about the paper in a recent NPR interview.

The blog Games with Words points out that this fails as a test of open access journals, because it is not clear what the acceptance rate would have been at traditional, closed-access journals. Note that it doesn't help to say (as Bohannon does) that acceptance of the spoof paper probably occurred when an open access journal simply didn't bother with peer review, even if we also accept that closed journals all genuinely conduct peer review. The study has no way of distinguishing the absence of peer review from crap peer review, and some closed journals may conduct crap peer review.

This criticism is sharpened because failure to have a control group was one of the howlers in Bohannon's spoof paper. He says in the NPR interview:
It looked like real paper, not a joke. But if you peer-reviewed it, you would within five minutes see that it was so flawed that it could never be published. ... if you're claiming to have evidence that some chemical is a promising new drug, well, you better have tested at least on healthy cells. Because even if you show that it hurts cancer cells, how do you know what you have there isn't just a poison? So that's one thing that's just awful about the paper, is that it doesn't compare cancer cells to healthy cells at all.

So GamesWithWords retorts: "Science -- which is not open access -- published an obviously flawed article about open access journals publishing obviously flawed articles."

Nevertheless, it seems that Bohannon wasn't even trying to test open-access as such. He says, "Open-access is great and everyone believes that."

Instead, he says that his results could be used to separate the sham open-access journals (which print anything so as to scam author fees) from genuine ones (which conduct authentic peer review and try to publish good science). That doesn't require a control group. If the goal is to sort rotten open-access journals from good open-access journals, then we don't need to consider anything but the open-access journals.

Yet if that's the goal, then the percentage of acceptances really doesn't matter. What matters is using a reliable sorting process and publicizing the results. Journals by the Cairo-based publisher Hindawi rejected the paper. Journals by Elsevier, Wolters Kluwer, and Sage accepted it.

And the fact that the results are presented as a percentage and series of anecdotes means that numerous readers will read it instead as a poor score for open-access as such. Brian Leiter, for example, links to the NPR interview just by writing "Open access journals in science: This story is a bit worrisome!"

Also note that Bohannon's experiment only tested journals that charge author fees. He explicitly excluded journals that don't. As I've noted before, it's the author-pays model that gives publishers like Elsevier, Kluwer, and Sage the incentive to run bogus, review-free journals.

[ add comment ] ( 3343 views )   |  [ 0 trackbacks ]   |  permalink
There's open and then there's "open" 
I was an invited speaker last week at DIY Publishing and the University, an event held by the NorthEast Regional Computing Program. I was there because of forall x; the organizers had found me through the Creative Commons database.

Most of the speakers talked about electronic resources, like institutional archives, student videos, or MOOCs. I was the odd man out, because forall x is a traditional textbook. Although it is distributed electronically, it is designed to be used as a physical workbook. Today I ran across an article in Salon which supports my old-school approach; it argues that people just don't comprehend material read on screen or on a tablet as well as material read on paper. Yet the difference is not so clear or strong that we should imagine it is inevitable. Future students may be better at learning from electronic documents, and future technology might present them in better ways. I will still be teaching a decade or two from now, and my preference for paper may be something I'll need to get over.*

One nice result of the NERCOMP event is that I now have a better understanding of MOOCs. A MOOC (the acronym stands for Massive Open Online Course) allows people from all over the world to register for and take a course. They might watch videos of lectures, participate in a discussion forum with other students, take tests on their computer, and so on. Because the courses are free, they are often mentioned in the same context as open-access textbooks.

The thing I learned is that "open" in MOOC just means open enrollment. Anyone anywhere is free to take the course. The course materials might be released under a Creative Commons license, but they might just be under traditional copyright. MIT labels its MOOC material "some rights reserved" and, although that is a standard label on CC-licensed material, MIT does not specify an specific permissions. Because they don't say, "some" is legally equivalent to "all". The difference is just a flourish, because "all rights reserved" would not sound as welcoming.

So I was also the odd man out because I was one of the few presenters specifically concerned with open access issues.**


* Even if there are cyborg students in the future, though, there are still concerns of equity. Even if it gets to the point where digital natives with advanced e-readers think better with electronic documents than with paper documents, economic disparities will mean that other students don't. Of course, the future of the university as an enterprise is also up for grabs in the next decade or two.
** Not the only one. Thomas Dodson, a librarian from Harvard, talked about their open access repository of faculty research.

[ add comment ] ( 273 views )   |  [ 0 trackbacks ]   |  permalink
The Leiter side of open access 
This morning, Brian Leiter made this post about Open Access publishing:
The Not-so-High Standards at (at least some) "Open Access" Journals

Not a great advertisement for the genre.

I hammered out a reply, which he added as an update. Here's what I said:
Your recent blog post rightly decries "The Not-so-High Standards at (at least some) "Open Access" Journals" and describes the case as "Not a great advertisement for the genre".

Importantly, the genre in question is not Open Access journals tout court. The real problem here is OA journals that use an author-pays model. Lots of them are straight forwardly scams to chisel money out of institutions that cover that kind of publishing and out of authors who need a line on their CV.

There are other models of OA. Quality OA journals don't charge author fees. I'm thinking here especially of Philosophers' Imprint, but also of less well-known and less prestigious ones like Logos&Episteme. We can argue about their stature in the field, but their being OA is not a demerit.

There is also the model which is sometimes called "green OA", in which authors' papers are systematically hosted in institutional or disciplinary archives. Although this does not result in OA journals as such, traditional journals can facilitate or thwart the practice depending on how they handle rights.

Qualifying your post with the caveat "at least some" is importantly not enough, because we can state precisely what's wrong here. For-profit publishers have an interest in suspicion being raised about OA in general, when really it's a specific business model that leads to egregious abuses like the one that you point to.

Long time readers might remember that I've fumbled this distinction in the past, so my point isn't to excoriate Leiter. Given the long-term importance of Open Access publishing for academia, it's important not to let bad practices tar the whole enterprise. The distinction between different kinds of OA (author-pays, author-doesn't-pay, and self-archived) is important.

[ add comment ] ( 4022 views )   |  [ 0 trackbacks ]   |  permalink
It takes a village to write a book 
My open access logic book, forall x, is going to be used this Fall for the first year logic course at Cambridge. I was contacted by a librarian there, who said that the course leader had edited a version especially for their course. So, she wanted to know, how should the book be listed and how should I be credited?

My reply, below the fold.
Read More...

[ add comment ] ( 3306 views )   |  [ 0 trackbacks ]   |  permalink
What journals do 
There's an interesting roundtable discussion in Theoria about journal publishing. The editors of several journals discuss the role that journals play in professional philosophy, including the evaluation of philosophers and funding decisions. They also discuss the possibilities of open access publishing.

Social responsibility of editors


Late in the discussion, someone from the audience asks "Why should the editor of a philosophical journal think so much about how the academic system appoints people to their jobs?" The alternative, one of the panelists suggests, is to just "produce good journals."

Sven Ove Hansson resists this suggestion, saying that "journals are part of, and very much integrated in, the academic system." And he's right. Whether journal editors face up to it or not, what and who gets published has real consequences for what and who gets attention and funding. There are institutional consequences of running a journal one way rather than another; that is to say, running a journal has political consequences. So editting a journal is (among other things) a political act.

More than that, there no such thing as a good journal simpliciter. What it is to be a good journal depends on how the journal will be used. The standards for how things should be written, how much citation is required, and so on all depend on the audience. For a journal with no intended audience at all, there is no difference between a good journal and a bad journal. To consider the most extreme possibility, imagine philosophers who toil to produce a journal which is immediately dropped into a black hole.*

So part of an editor's responsibility is to consider who the audience of the journal is and how they are going to use it. Given the academic system, evaluators and administrators are among the audience and the appointment and promotions system is part of how it is used. So producing good journals requires thinking about the academic systems in which the journals exist.

In the first place, journals are aimed at scholars interested in advancing the discussion about such-and-so. So, to more charitably read the original question, one can worry that editors are concerned too much about the academic system. The political reality means that editors should think a lot about it, though.

Open access


The fact that journal articles are part of an ongoing discussion underscores the value of open access publishing. Simply put, a more accesible article is more likely to be available and be read in places where it can make a difference. To their credit, most of the editors in the roundtable voice support for open access. Theoria itself is an OA journal.

One strange dissent is from Vincent Hendricks. Discussing the case in his own country, he says:
[The Danes] are introducing an interesting angle on open access. The idea is to have all Danish research articles included in a database, which is basically an open access database belonging to the state. Since the money for the research is paid by the state, everybody should have access to the database. Of course, there is a potential clash here between the state on one hand and, say, the journals on the other. There's no way Springer is going to publish papers that have already been published on a state-controlled website. If researchers have to put up their publications on a state-owned database, then nobody is going to publish their papers anyway. I doubt that those who decided this have any idea what they are getting their hands into.

I call this 'strange' because it describes open access in some bizarro universe. There are already funding agencies that require open access publishing (e.g. the Wellcome Trust circa 2004) and it's not a novel Danish innovation. Contra Hendricks' insistence, a requirement to put papers in a central database is compatible with later publishing in a journal. In physics, almost every new paper is placed in a public archive (the arXiv) even when it later appears in a traditional journal. Somehow physics journals survive at publishers including Springer.

The most charitable interpretation is that Hendricks just doesn't know any of this. He coedits Synthese, a philosophy journal, so perhaps developments in physics and biomedicine just haven't shown up on his radar. Yet, for reasons I suggested above, part of his responsibility as editor is to think about the broader context.**

The less charitable interpretation is that he does know about the developments in physics and biomedicine but conveniently overlooks them. Commercial publishers do this when they argue that open access would destroy academic publishing. Paranoia about open access is probably the narrative encouraged by Springer itself.*** Even though Springer could survive in an open access world, there is more money to be made if they can go on exploiting free labour and a captive market.


* I am here setting aside any value that writing the journal might have for the writers and editors themselves. Although they may derive some benefit from the process, it's incidental to academic publishing.
** Hendricks would disagree with the argument above, I think. He says in response to the question from the floor: "We have to produce good journals. That's all I really have to say about that."
*** There is some reason to suspect that the editors of Synthese are corporate toadies. I don't want this to be another post about the Synthese debacle, though... oops.

[ 2 comments ] ( 5901 views )   |  [ 0 trackbacks ]   |  permalink

<<First <Back | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | Next> Last>>