Wikipedia on Cartesian Free Masonry
Thu 27 Jul 2006 03:14 AM
I am presenting on the reliability of the Wikipedia in a few weeks, and I wish I had more data.
A study, reported in Nature earlier this year, tested science entries from Wikipedia and Britannica. I have done some similar work on philosophy entries, although on only a handful of subjects. Although these studies can tell us about the density of errors in Wikipedia as opposed to in other resources, they cannot tell us about the dynamics. Boosters of the Wikipedia talk of its self-correcting nature, but what is the life cycle of an error in a Wikipedia entry?
Two about mysticism
A few days ago, I decided to try a small test. I inserted this sentence in the entry on Bertrand Russell, just after a passage describing his reaction to Wittgenstein's Tractatus:
Despite his disagreement with mysticism on essential points, Russell did not deny that mystical experiences do occur; he acknowledged attending a seance while he was a student at Cambridge and seeing what appeared to be the ghost of John Dee.Ten minutes later, a user in the Czech Republic undid the change.
Such a test is rather shady, so I had only intended to try it once. (More on the ethics of it in a moment.) Nevertheless, this put me in a bind. I could not ignore it as a data point, but neither was it plausible that every falsity is scrubbed out so quickly. So the following day I tried another test. This time, in the entry on Descartes:
At La Flèche, Descartes first encountered hermetic mysticism. Although he was briefly a Free Mason, he later abandoned mysticism in favor of reasoned inquiry.The change survived past my bedtime.
About four and a half hours after I inserted the test sentences, an anonymous user replaced the entire section of the entry in which they appeared with the sentence "u have been terrorized!!!!!." Almost immediately, a user in Oxfordshire undid that change and restored the section that included my test sentences. The same anonymous user deleted that section again. A minute later, the user in Oxfordshire restored it.
Lest one think that the comment about being "terrorized" was a reference to the change I had made, I can say this much: An anonymous user from the same IP address made a similar attack on the Nero entry several days prior, erasing sections of the entry and replacing them with strings of obscenities. The style of the vandalism leaves little doubt that it was the same guy.
Thirteen hours after I inserted the test sentences, a self-described "pedant" and "university professor in Ontario, Canada" edited the entry so as to exchange an ampersand for the word "and."
Twenty-four hours after I inserted the sentences, I removed them.
The tests I performed are somewhat suspect, from an ethical point of view. I inserted fabrications into a public record. To be blunt, I lied. A large scale study involving a campaign of such lies would be irresponsible. I am not going to do it, and I do not want to hear that someone did it after getting the idea from me.
Moreover, the methodology has all of the advantages of theft over honest toil. Every change ever made to the Wikipedia is saved. The wiki interface facilitates comparing different versions. I used this feature in tracking the response to my two little fibs about mysticism, in tracking the vandalism of Mister Anonymous, and in recovering the history of the 'church key' entry (as discussed in a previous entry). Questions about the dynamics of the Wikipedia could be answered by a systematic examination of the history entries.
The history entries confirm the lesson of my modest tests: There are Wikipedia users who react quickly to vandalism. If they detect an error or lie, then it will be removed within minutes. But not every ill-change is detected by these minute men. When I removed it, my change to the Descartes article was no longer fresh in the queue of changes. Most users would encounter it as part of the article, seemlessly connected with the rest of Descartes' biography.
Thu 27 Jul 2006 07:39 AM
It would be ironic if most of the errors were due to people testing Wikipedia's error tolerance.