This was not an admission that featured articles were especially reliable, but just that they were different. They needed to be thought of as a separate population.
Now somebody has taken a look at them. In this week's First Monday, David Lindsey directly evaluates the quality of featured articles; Evaluating quality control of Wikipedia's feature articles. The upshot is that many featured articles are good but that some are terrible. They are, despite the 'feature' glitter, much like the rest of Wikipedia. He concludes with the suggestion that, "[t]o put it simply, being a featured article may not mean much at all."
As a methodological aside, Lindsey evaluated the current version of specific articles rather than the development of those articles across time. I still suspect that Wikipedians do pay more attention on average to featured articles than they do other articles. If that's true, then random vandalism is probably caught more quickly and reliably on featured pages. That is compatible with Lindsey's conclusion that the articles can still be poorly written, misleading, or just downright bad.
[ add comment ] ( 356 views ) | [ 0 trackbacks ] | permalink
We often assess claims based on plausibility of style and content. In writing about Wikipedia, I argue that these assessments can be frustrated by community editing. The implausible details can be taken out of false accounts, making the falsity harder to detect. Some people respond to my argument by denying that this happens.
Reading Eugenie Samuel Reich's Plastic Fantastic, I bumped into a similar phenomenon. Reich is a science journalist, and the book is about fraudulent science. Her claim is that peer review does not do an especially good job of catching deliberate fabrication. Moreover, scientists who perpetrate fraud often exploit reviewers' comments and questions in order to make their fabrications more plausible. Reich writes:
Not only in there no guarantee that a thorough review process will detect a false claim, but even more disturbingly, a thorough review may do little more than reveal to authors what changes they need to make in order to turn a false claim into a more plausible scam.[p. 122]The parallel with Wikipedia is not precise, but in both cases conscientious but imperfect editorial oversight results in public versions which are more plausible false accounts than the original submissions.
Scammer scientists exploiting this can publish more plausible scam papers than they could have otherwise. Yet one might hope that, although this helps fraudulent papers on the timescale of months, fraudulent research programs will still be uncovered in the course of just a couple of years. The parallel hope for the Wikipedia is that false claims will be corrected eventually.
So the hope is that fraud burns brighter by exploiting peer review but will still burn out in relatively short order. Consistent with this is the fact that none of the cases of fraud which Reich describes have gone undetected for more than a few years, and each was discovered in time to ruin the scientists responsible. Yet that may just be because she can only report scientific fraud which was ultimately detected.
[ add comment ] ( 241 views ) | [ 0 trackbacks ] | permalink
In these two related items, Wikipedian prose appears in print:
1. Dublin student Shane Fitzgerald invented a quotation and attributed it to the recently-deceased composer Maurice Jarre in the latter's Wikipedia entry.* The quote was subsequently printed by several major newspapers in obituaries for Jarre. [coverage in the Irish Times, here]
Regarding Wikipedia, this just corroborates things I already knew. Even though the quote was written to sound like something that the composer might plausibly have said, it was quickly removed from Wikipedia. Fitzgerald had to add it repeatedly until it slipped by Wikipedia's first responders. Even then, it only persisted for about a day. Wikipedia did its usual decent but imperfect job of filtering out fibs.
Regarding the state of journalism, it's more depressing. When people lament the demise of newspapers, they often say that real journalists do important work that citizen bloggers do not. Surely that is true in some cases, but not here. Newspapers played exactly the same echo chamber game that bloggers play.
2. Chris Anderson, editor-in-chief of Wired magazine, has a forthcoming book titled Free: The Future of a Radical Price. Waldo Jaquith at the Virginia Quarterly Review discovered that some sections of the book had been plagiarized. Anderson replied that the original draft had included footnotes, that the editor had decided to eliminate the apparatus at the last minute, and that errors had been made when incorporating attributions into the body text. Both he and his publisher have said that the footnotes will be available as an on-line supplement.
This much seems fine. He tried to acknowledge sources, made honest mistakes, and has made a good faith effort to correct for those mistakes. In any case, the standards are somewhat fuzzier for popular books than they are academic monographs.
The more worrisome thing is that some of the passages relating facts (about usury, for example) are copied verbatim from Wikipedia. One might worry again about plagiarism. It is verboten to repeat text verbatim without indenting it or putting quotation marks around it. Yet perhaps in the original draft, along with a footnote crediting Wikipedia, there were quotation marks.
The more substantive concern is that the text uncritically turns to Wikipedia as a relevant and reliable source. Anderson wrote a popular, nonfiction book and so is effectively operating as a journalist. Just as I expect reporters to take the few minutes required to follow up on what Maurice Jarre said, I expect a book author to follow up on whether charging interest was made a heresy in 1311.
In the case of Jarre, it is possible that many of the papers just took stories from the wire. It might even be that only one reporter knowingly took the quote from Wikipedia, and subsequent newspaper editors just unwittingly traded it around. In Anderson's case, we know that he was the one who did the cut-and-paste job.
* I let this news item pass without comment a couple of months ago, but blogging about it now lets me stick a pin in it. I live in the 21st-century, and the internet is my scrapbook.
[ add comment ] ( 305 views ) | [ 0 trackbacks ] | permalink
Poking around on Google Scholar, I can check how often my publications have been cited.* Subtracting instances of me citing myself, my most cited papers are Epistemology and the Wikipedia (with 7 citations) and Distributed cognition and the task of science (with 6 citations).
In science studies, the number of citations made to an article is often used as a measure of the article's scholarly impact. It is sometimes even used as a proxy for the article's quality. Citation counts give social scientists a quantifiable handle on ineffable factors. Sometimes, the same measures are used by administrators to assess the productivity of scholars and departments - again because it gives an objective procedure for assessing such things.
As far as I know, nobody uses such measures to gauge the quality of philosophical work.** It is a good thing, too. Citation patterns vary widely across the field, with some specialties cluttering articles with clouds of citations and others providing a few exemplary citations. There is little difference in substance between a footnote that cites 20 articles without comment and one that cites a recent survey article or anthology.
Considering my two most cited papers, neither of them are straight-up philosophy: The first was only a conference presentation. Anyone who has read it found it on the internet - either on my website or the SUNY digital archive. It has been cited mostly by people thinking about IT issues. The second was published in Social Studies of Science, an interdisciplinary journal.
Suppose these two examples are typical and imagine what would follow if citation counts were used as a measure of scholarly productivity for philosophers. Insofar as scholars in other disciplines cite more, one would want to write papers that pique the interest of those guys. One would want to publish outside the mainstay philosophy journals. In short, one would do more interdisciplinary work.
I do not know whether this would be a good thing or a bad thing. In the long run, though, philosophers would probably just start citing each other more in uninformative, cloud-of-reference footnotes.
* Admittedly, Google Scholar's database is somewhat quirky. It has the virtue of being readily available.
** Some time ago, I discussed an attempt to measure the impact of philosophy journals.
[ 1 comment ] ( 2012 views ) | [ 0 trackbacks ] | permalink
Some people have suggested to me that I should try my hand at writing some newspaper op-ed pieces. One natural topic for me, given where my research intersects with the interests of the guy down at the Dairy Queen, is nattering about the Wikipedia. So last month, in response to then current events, I wrote a piece that essentially recapitulates the thesis of my Episteme paper.
I submitted it a couple of places, but no luck. Rather than leave it in a directory on my hard drive where no one will ever read it, I've opted to put it here on the blog where no one will read it...
Read More...
[ add comment ] ( 635 views ) | [ 0 trackbacks ] | permalink
Random Entry
