Footnotes on Epicycles http://laser.fontmonkey.com/foe/index.php Copyright 2013, P.D. Magnus P.D. Magnus en-US SPHPBLOG 0.7.0 What I'm reading now http://laser.fontmonkey.com/foe/index.php?entry=entry120629-120935 What to believe now [Amazon/GoodReads], David Coady sets out to do applied epistemology. Most of the book is about expertise and democracy,* which is fine. With the caveat that I haven't read it all, I'll lament that fact that most of the book seems to be Coady summarizing and critiquing Alvin Goldman's work on these topics. Missing, for example, is any discussion of Harry Collins' work on expertise.

Wikipedia is discussed for three pages in the conclusion, and I'll focus on that because he quotes me. Coady writes:
Most contributors to Wikipedia, unlike most rumor-mongers, see themselves as engaged in a single collective enterprise. This enterprise is governed by rules, and Wikipedia has a hierarchy that seeks to enforce those rules. So, when P.D. Magnus characterizes the claims made in Wikipedia as "more like 'claims made in New York' than 'claims made in the New York Times" he is mistaken. ... Wikipedia is a reasonably reliable source for a reasonably wide range of subjects because of the contingent fact that it has a reasonably good culture at the moment.

Perhaps the rhetorical flourish in the passage he cites overstates my point, because claims in Wikipedia fall under one institutional umbrella in a way that claims made in New York do not. But my point is that there is sufficient variation in the quality and reliability of Wikipedia articles that it is wrong to treat them all together. Even though it is 'reasonably reliable' across a 'reasonably wide range', it is better to pay attention to the kind of article that you are consulting. Wikipedia is large enough that it is better to think of it as multiple overlapping communities, rather than as a single monolithic culture.

* EDIT: As Coady points out in the comments, there are also chapters on rumours, conspiracy theories, and blogging.]]> wikiosity http://laser.fontmonkey.com/foe/index.php?entry=entry120629-120935 P.D. Magnus Fri, 29 Jun 2012 19:09:35 GMT http://laser.fontmonkey.com/foe/comments.php?y=12&m=06&entry=entry120629-120935 Digesting the whole Wikipedia http://laser.fontmonkey.com/foe/index.php?entry=entry111210-110739 First Monday, Royce Kimmons has an interesting analysis of community contributions in Wikipedia. His results suggest that most particular entries are the work of separate contributions by a small number of people, rather than the efforts of an ongoing community. The cool thing is that it is a systematic study of all Wikipedia entries and histories.
Abstract: Wikipedia stands as an undeniable success in online participation and collaboration. However, previous attempts at studying collaboration within Wikipedia have focused on simple metrics like rigor (i.e., the number of revisions in an article’s revision history) and diversity (i.e., the number of authors that have contributed to a given article) or have made generalizations about collaboration within Wikipedia based upon the content validity of a few select articles. By looking more closely at metrics associated with each extant Wikipedia article (N=3,427,236) along with all revisions (N=225,226,370), this study attempts to understand what collaboration within Wikipedia actually looks like under the surface. Findings suggest that typical Wikipedia articles are not rigorous, in a collaborative sense, and do not reflect much diversity in the construction of content and macro–structural writing, leading to the conclusion that most articles in Wikipedia are not reflective of the collaborative efforts of the community but, rather, represent the work of relatively few contributors.
]]> wikiosity http://laser.fontmonkey.com/foe/index.php?entry=entry111210-110739 P.D. Magnus Sat, 10 Dec 2011 19:07:39 GMT http://laser.fontmonkey.com/foe/comments.php?y=11&m=12&entry=entry111210-110739 Short subject on featured articles http://laser.fontmonkey.com/foe/index.php?entry=entry100415-103801 my little study of Wikipedia, I initially stumbled on the difference between featured and regular articles. If I had thought about it in advance, I would not have tested any featured articles at all. I had included them, however, so I reported the results and suggested that the data about featured articles be set aside.

This was not an admission that featured articles were especially reliable, but just that they were different. They needed to be thought of as a separate population.

Now somebody has taken a look at them. In this week's First Monday, David Lindsey directly evaluates the quality of featured articles; Evaluating quality control of Wikipedia's feature articles. The upshot is that many featured articles are good but that some are terrible. They are, despite the 'feature' glitter, much like the rest of Wikipedia. He concludes with the suggestion that, "[t]o put it simply, being a featured article may not mean much at all."

As a methodological aside, Lindsey evaluated the current version of specific articles rather than the development of those articles across time. I still suspect that Wikipedians do pay more attention on average to featured articles than they do other articles. If that's true, then random vandalism is probably caught more quickly and reliably on featured pages. That is compatible with Lindsey's conclusion that the articles can still be poorly written, misleading, or just downright bad.]]> wikiosity http://laser.fontmonkey.com/foe/index.php?entry=entry100415-103801 P.D. Magnus Thu, 15 Apr 2010 17:38:01 GMT http://laser.fontmonkey.com/foe/comments.php?y=10&m=04&entry=entry100415-103801 How to be better at fraud http://laser.fontmonkey.com/foe/index.php?entry=entry100206-110621 writing about Wikipedia, I argue that these assessments can be frustrated by community editing. The implausible details can be taken out of false accounts, making the falsity harder to detect. Some people respond to my argument by denying that this happens.

Reading Eugenie Samuel Reich's Plastic Fantastic, I bumped into a similar phenomenon. Reich is a science journalist, and the book is about fraudulent science. Her claim is that peer review does not do an especially good job of catching deliberate fabrication. Moreover, scientists who perpetrate fraud often exploit reviewers' comments and questions in order to make their fabrications more plausible. Reich writes:
Not only in there no guarantee that a thorough review process will detect a false claim, but even more disturbingly, a thorough review may do little more than reveal to authors what changes they need to make in order to turn a false claim into a more plausible scam.[p. 122]
The parallel with Wikipedia is not precise, but in both cases conscientious but imperfect editorial oversight results in public versions which are more plausible false accounts than the original submissions.

Scammer scientists exploiting this can publish more plausible scam papers than they could have otherwise. Yet one might hope that, although this helps fraudulent papers on the timescale of months, fraudulent research programs will still be uncovered in the course of just a couple of years. The parallel hope for the Wikipedia is that false claims will be corrected eventually.

So the hope is that fraud burns brighter by exploiting peer review but will still burn out in relatively short order. Consistent with this is the fact that none of the cases of fraud which Reich describes have gone undetected for more than a few years, and each was discovered in time to ruin the scientists responsible. Yet that may just be because she can only report scientific fraud which was ultimately detected.
]]> ideas, wikiosity http://laser.fontmonkey.com/foe/index.php?entry=entry100206-110621 P.D. Magnus Sat, 06 Feb 2010 19:06:21 GMT http://laser.fontmonkey.com/foe/comments.php?y=10&m=02&entry=entry100206-110621 Publishing in the echo chamber http://laser.fontmonkey.com/foe/index.php?entry=entry090702-111356
1. Dublin student Shane Fitzgerald invented a quotation and attributed it to the recently-deceased composer Maurice Jarre in the latter's Wikipedia entry.* The quote was subsequently printed by several major newspapers in obituaries for Jarre. [coverage in the Irish Times, here]

Regarding Wikipedia, this just corroborates things I already knew. Even though the quote was written to sound like something that the composer might plausibly have said, it was quickly removed from Wikipedia. Fitzgerald had to add it repeatedly until it slipped by Wikipedia's first responders. Even then, it only persisted for about a day. Wikipedia did its usual decent but imperfect job of filtering out fibs.

Regarding the state of journalism, it's more depressing. When people lament the demise of newspapers, they often say that real journalists do important work that citizen bloggers do not. Surely that is true in some cases, but not here. Newspapers played exactly the same echo chamber game that bloggers play.

2. Chris Anderson, editor-in-chief of Wired magazine, has a forthcoming book titled Free: The Future of a Radical Price. Waldo Jaquith at the Virginia Quarterly Review discovered that some sections of the book had been plagiarized. Anderson replied that the original draft had included footnotes, that the editor had decided to eliminate the apparatus at the last minute, and that errors had been made when incorporating attributions into the body text. Both he and his publisher have said that the footnotes will be available as an on-line supplement.

This much seems fine. He tried to acknowledge sources, made honest mistakes, and has made a good faith effort to correct for those mistakes. In any case, the standards are somewhat fuzzier for popular books than they are academic monographs.

The more worrisome thing is that some of the passages relating facts (about usury, for example) are copied verbatim from Wikipedia. One might worry again about plagiarism. It is verboten to repeat text verbatim without indenting it or putting quotation marks around it. Yet perhaps in the original draft, along with a footnote crediting Wikipedia, there were quotation marks.

The more substantive concern is that the text uncritically turns to Wikipedia as a relevant and reliable source. Anderson wrote a popular, nonfiction book and so is effectively operating as a journalist. Just as I expect reporters to take the few minutes required to follow up on what Maurice Jarre said, I expect a book author to follow up on whether charging interest was made a heresy in 1311.

In the case of Jarre, it is possible that many of the papers just took stories from the wire. It might even be that only one reporter knowingly took the quote from Wikipedia, and subsequent newspaper editors just unwittingly traded it around. In Anderson's case, we know that he was the one who did the cut-and-paste job.


* I let this news item pass without comment a couple of months ago, but blogging about it now lets me stick a pin in it. I live in the 21st-century, and the internet is my scrapbook.]]> wikiosity http://laser.fontmonkey.com/foe/index.php?entry=entry090702-111356 P.D. Magnus Thu, 02 Jul 2009 18:13:56 GMT http://laser.fontmonkey.com/foe/comments.php?y=09&m=07&entry=entry090702-111356 Vanity searches and scholary productivity http://laser.fontmonkey.com/foe/index.php?entry=entry090611-132642 Epistemology and the Wikipedia (with 7 citations) and Distributed cognition and the task of science (with 6 citations).

In science studies, the number of citations made to an article is often used as a measure of the article's scholarly impact. It is sometimes even used as a proxy for the article's quality. Citation counts give social scientists a quantifiable handle on ineffable factors. Sometimes, the same measures are used by administrators to assess the productivity of scholars and departments - again because it gives an objective procedure for assessing such things.

As far as I know, nobody uses such measures to gauge the quality of philosophical work.** It is a good thing, too. Citation patterns vary widely across the field, with some specialties cluttering articles with clouds of citations and others providing a few exemplary citations. There is little difference in substance between a footnote that cites 20 articles without comment and one that cites a recent survey article or anthology.

Considering my two most cited papers, neither of them are straight-up philosophy: The first was only a conference presentation. Anyone who has read it found it on the internet - either on my website or the SUNY digital archive. It has been cited mostly by people thinking about IT issues. The second was published in Social Studies of Science, an interdisciplinary journal.

Suppose these two examples are typical and imagine what would follow if citation counts were used as a measure of scholarly productivity for philosophers. Insofar as scholars in other disciplines cite more, one would want to write papers that pique the interest of those guys. One would want to publish outside the mainstay philosophy journals. In short, one would do more interdisciplinary work.

I do not know whether this would be a good thing or a bad thing. In the long run, though, philosophers would probably just start citing each other more in uninformative, cloud-of-reference footnotes.


* Admittedly, Google Scholar's database is somewhat quirky. It has the virtue of being readily available.

** Some time ago, I discussed an attempt to measure the impact of philosophy journals.]]> wikiosity, trivia http://laser.fontmonkey.com/foe/index.php?entry=entry090611-132642 P.D. Magnus Thu, 11 Jun 2009 20:26:42 GMT http://laser.fontmonkey.com/foe/comments.php?y=09&m=06&entry=entry090611-132642 Two dead senators and an extra Wilhelm http://laser.fontmonkey.com/foe/index.php?entry=entry090308-152435 my Episteme paper.

I submitted it a couple of places, but no luck. Rather than leave it in a directory on my hard drive where no one will ever read it, I've opted to put it here on the blog where no one will read it...


The Wikipedia is the encyclopedia that anyone can edit. This makes it a simmering information soup, with a mix of hearty stock and confused froth. Sometimes it boils over, as it did in January when someone edited the entries on Ted Kennedy and Robert Byrd to say that both men were dead.

To be fair, eruptions like these do not show that the Wikipedia is bankrupt as a source of information. In both cases, the premature claims of death were removed within a few minutes by Wikipedia users. False claims inserted into Wikipedia entries are often removed quickly.

In recent research, reported in the journal First Monday, I tried to suss out just how quickly and reliably the Wikipedia community fixes errors. I added short, false claims to Wikipedia entries and watched them to see if they were fixed. Because I was interested in short-term reaction time, I corrected the entries myself after 48 hours if no one else had done so. I made changes to the entries on deceased philosophers, so there was no danger that any living person was defamed.

Over one third of the fibs I inserted were corrected or marked for correction. When someone corrected one of my fibs, there was a one in five chance that they would find the other fibs I inserted at the same time and correct those too.

This is a moderating result. Wikipedia correction is not perfect or immediate, but neither is the Wikipedia a place where you can say anything and have it stand.

Back to the case of Kennedy and Byrd: Wikipedia co-founder Jimmy Wales used it as an excuse to propose changing the way that Wikipedia works. Presently, anyone with a net connection can access and edit pretty much any article on the site. With the proposed change, anonymous users would not be able to edit the biographies of living people and have their changes appear immediately. Instead, under the Flagged Revisions proposal, a trusted Wikipedia user would have to sign off on the change before it would appear.

In February, Wikipedia error is in the news again. The full name and noble title of Germany's new economy minister, Karl-Theodor zu Guttenberg, is fifteen words long. A prankster added an extra 'Wilhelm' to the middle of Guttenberg's name in the Wikipedia article about him. When several German media outlets made jokes about the minister's name, they printed it including the spurious 'Wilhelm.' They later printed retractions.

Flagged Revisions are already required in the German Wikipedia. So Wales' proposal would not stop falsehoods like the extra 'Wilhelm' in Guttenberg's staggeringly long moniker.

Note, however, that none of these notables were harmed by the errors in Wikipedia. Kennedy and Byrd were said to be dead for only a few minutes. Even if some credulous Wikipedia user loaded the pages during that brief window of time and falsely believed that the men had died, no one would rely on a brief consultation of such a precarious source for any purpose where the life or death of the men had actually mattered. And the German papers would have made fun of a name that was over a dozen words long, with or without the extra Wilhelm.

The great thing about Wikipedia is not just that anyone can contribute, as if it were a collaborative sandbox or experimental theater. That openness is valuable because it means that Wikipedia has entries about an enormous range of topics, many of them too recent to be covered at all in traditional print encyclopedias.

This also means that, regardless of how diligent Wikipedia users are, errors will get into it. Some will be corrected in minutes, others in hours, and others will persist for days or even months. When we use Wikipedia, we should be mindful that any entry we look at might contain errors living out their time between insertion and correction. We cannot just rely on the skills we have developed for gleaning information from traditional media. Learning to use Wikipedia will mean comprehending its novelty. ]]> ideas, wikiosity http://laser.fontmonkey.com/foe/index.php?entry=entry090308-152435 P.D. Magnus Sun, 08 Mar 2009 22:24:35 GMT http://laser.fontmonkey.com/foe/comments.php?y=09&m=03&entry=entry090308-152435 There is no 'you' in 'Wikipedia' http://laser.fontmonkey.com/foe/index.php?entry=entry090126-105817 NY Times reports, the free-wheeling days of Wikipedia editing may be over. The crackdown follows a recent incident in which Wikipedia entries reported Ted Kennedy and Robert Byrd were reported to be dead. As the Washington Post admits, the false claims only persisted for a few minutes. Nevertheless, the story is headlined "Kennedy, Byrd the Latest Victims of Wikipedia Errors", suggesting that the misreports somehow harmed or inconvenienced the two old and frail senators. Piffle, of course, and the Post story concludes by giving examples of traditional news media misreporting obituaries. [insert apt quote from Mark Twain]

This has led Wikipedia cofounder Jimbo Wales to call for changes in the way Wikipedia works. Wikipedia visitors who are not logged in as trusted users would no longer be able to change articles and have the revisions appear immediately. Instead, their changes would have to be approved by a trusted user before they would become part of the Wikipedia corpus.

It is unclear how much delay this would produce. Wales hopes it would not be more than a week. There is a tension here: If the restriction is only for some articles rather than others, then there will still be an evanescent flux of falsehoods in the rest of the Wikipedia. If the restriction is extended to all or most of the Wikipedia, then the delay will become intolerable.

Delay is also problematic because several users may change a page before any of the changes are accepted. If they are all adding the same information and making the same corrections, then some editor will need to decide which version to use. If they are making different changes in overlapping parts of an article, then some editor will have to fix the grammar and usage to make the changes fit together. In short: Jumbled nightmare.

In addition to adding delay, the process puts more power in the hands of approved Wikipedia users. Note that this is not simply the divide between registered users and anonymous users; the Ted Kennedy death reports were entered by registered user Gfdjklsdgiojksdkf. So the elite corps of trusted Wikipedia users will have responsibility for what appears in these shielded articles.]]> ideas, wikiosity, trivia http://laser.fontmonkey.com/foe/index.php?entry=entry090126-105817 P.D. Magnus Mon, 26 Jan 2009 18:58:17 GMT http://laser.fontmonkey.com/foe/comments.php?y=09&m=01&entry=entry090126-105817 Popping the stack http://laser.fontmonkey.com/foe/index.php?entry=entry090108-132037 daring fireball and makkintosshu, I learned that the URL http://www.apple.com/hypercard now redirects to the Wikipedia entry for Hypercard. This is a counterpart to the more common sin of bloggers linking uncommon terms in their prose to the Wikipedia entry for that term.* So I'll talk about that first.

Suppose I am reading a post and come across a word or topic that I am not familiar with. I always have the option of opening a new tab and searching the web for more information; if I were so inclined, I could just start my search with the Wikipedia.

If the author of the webpage has bothered to include a hyperlink, however, it suggests that they are specifically recommending that I look at whatever source they've hyperlinked. Suppose they actually have looked at the Wikipedia entry and deemed it to be quality. They have thus used whatever expertise they have to vouch for the Wikipedia entry. Since the Wikipedia entry might have changed since they vetted it, I might or not be able to trust the present entry. So bloggers who really have looked at the entry to confirm its quality should link to the dated version of the entry that they read, rather than the always-current entry. Alternately, they might link to both. (This argument is part of my forthcoming paper.)

If the author of the webpage inserts the link without really looking at the Wikipedia entry, as seems too often to be the case, then what do they think they are doing? If I am puzzled by the term, then the link doesn't give me anything more than what I would turn up if I did my own web search on the topic. If I am not puzzled, then the link is an annoying distraction. I might waste time clicking on it, mistaking it for seem actual content.** The link is clutter in any case, and it adds no real functionality to the page.

The take home lesson for bloggers: Stop it!

For Apple: If the Wikipedia entry were edited to say that Hypercard assisted in the assassination of Robert Kennedy, then Apple would be somewhat complicit in the fib. At the same time, it is unclear how the redirect is any more helpful than a spartan page which says that Apple no longer maintains Hypercard. Anyone coming across such a page while actually trying to learn about Hypercard could easily go find the Wikipedia page on their own.

It seems that links to Wikipedia are to webpages in 2009 like Comic Sans was to allegedly-funny print outs in 1999.


* What I say about bloggers applies the authors of webpages more generally. Samuel Arbesman discusses the London Tube map in the contect of introducing his really neat map of the Milky Way and includes some gratuitous links to Wikipedia.

** One might change the stylesheet to include a type of link that looks about like ordinary text. At least then the gratuitous links wouldn't be distracting clutter.]]> ideas, wikiosity http://laser.fontmonkey.com/foe/index.php?entry=entry090108-132037 P.D. Magnus Thu, 08 Jan 2009 21:20:37 GMT http://laser.fontmonkey.com/foe/comments.php?y=09&m=01&entry=entry090108-132037 Playing telephone with the echo chamber http://laser.fontmonkey.com/foe/index.php?entry=entry081001-100552 fibs in Wikipedia paper. That's unsurprising, since the paper is freely available on-line and addresses a topic close to some bloggers' hearts.

What surprises me a bit is that all of the reactions interpret my study as vindicating Wikipedia. My result is certainly a midpoint between despair and celebration for Wikipedia; about a third of the fibs were fixed in the studied time window, but (it follows) about two-thirds of them were not. Since there is good reason to think that the probability of being fixed falls of rapidly after the edit is made, this figure does not allow us to extrapolate a half life for fibs. Nonetheless, Jason Pannone mentions my study and says, "I'm not sure that this article will sway skeptics, but it does offer some additional empirical evidence that minor errors in Wikipedia are corrected quickly."

Kent Anderson points to my paper and briefly summarizes the result. He misinterprets it slightly, taking 1/3 of the fibs fixed to include only those cases in which the fib was removed entirely. This allows him to give the optimistic spin that "additional entries were flagged with 'need citation,' indicating that they had been caught and the time to correction was near." Blogging librarian Rhondda read about my paper in Anderson's blog and summarizes it this way: "The study showed that Wikipedia's methods for checking for small inaccuracies are validated. ... Within 48 hours, those that had not been corrected, had been flagged as needing adjustment." Some others were flagged (Anderson's error) becomes all others were flagged, so that every single fib was caught by someone. If the game of telephone continued, someone down the line might summarize my study as showing that Wikipedia fixes errors with divine inerrancy, before they occur.]]> wikiosity, trivia http://laser.fontmonkey.com/foe/index.php?entry=entry081001-100552 P.D. Magnus Wed, 01 Oct 2008 17:05:52 GMT http://laser.fontmonkey.com/foe/comments.php?y=08&m=10&entry=entry081001-100552