Over at Terra Nova there is news of the release of a study conducted with the help of Sony, which is essentially a large survey of MMO users’ role-playing style, their attitude towards the game, mental health and degree of social exclusion. It’s an interesting attempt to characterise the qualities of MMO players by their degree of interest in role-playing and their sociodemographic and personal profile, and the first study of its kind to use data from the underlying game database. I have some problems with the statistics (outlined after my rant, below) which maybe will be clarified when the final version of the paper is released, but I have bigger problems with the interpretation of the results, and the view that the researchers at Terra Nova are taking of role-players as compared to the “non” role players in the survey.

Specifically, in the summary of the paper, the first author Dmitri Williams states that

Role players come much more often from offline marginalized groups, suggesting that some may engage in the practice to find acceptance or a safe outlet for their identity.

Role players engage in the practice for a number of reasons, but the standout one tended to be for creativity. Escapism was present, but was rarely the main reason.

which suggests a reasonably balanced view of gamers’ reasons for playing in the second paragraph (escapism is rarely the main reason) but a very blunt and anachronistic explanation in the first paragraph. It seems to assume that there is a higher level of escapism in these marginalised groups, which is supported only by a tautological hypothesis. The authors argue that marginalized groups would be more likely to role-play than the non-marginalised, because role-playing is a form of escapism, or a safe outlet for their identity. Having found this statistical difference, they conclude that escapism must be the reason for this higher representation. But the original hypothesis is untested. I see no realistic or reasonable link between marginalization and greater role-playing. It’s not like you get to be gay in an MMO, or your blackness becomes more acceptable, or your non-christian religion. You get to be an elf, or a magician. That there should be a relationship between taking another role in a computer world and being dissatisfied with your role in the real world is a highly dubious claim. The truth of this claim needs to be established before the next postulate can be finalised.

However, the claims get a little more disturbing in a subsequent piece on RMT (Real Money Transactions) by a non-author of the paper, Castronova, who states that this paper

shows pretty clearly that players who desire strong refuge from reality, the sincere role-players, are a distinct minority. My arguments were delivered with a background assumption that very large numbers of people were scrambling over themselves to get out of the real world. Not so. That doesn’t make the arguments wrong, it just indicates that any plea for the right to live in a deep fantasy is less socially resonant than I thought… I’m an advocate for a minority, a somewhat disturbed one at that according to Williams, Kennedy, and Moore.

So Castronova’s assumption is that role-playing is about escapism, and plain and simple – people want to “get out of the real world”. Note in this paragraph Castronova doesn’t change his view that role-playing is about escapism, he just discovers that most people in MMOs don’t role-play much and therefore aren’t doing it for escapism. He goes on to use the loaded language of the claim that they are a “disturbed [minority] at that.” Judging the loonies is always a good look in academia, I find.

My problem with this is that, as far as I can tell, all media are a form of escapism. You can’t run around claiming that only 5% of people who watch movies do it for escapism – they all do! So what’s different about MMOs? Why should it only be some select group of extreme role-players who are doing it for the escapism? Couldn’t it be that everyone is doing the game as a type of escapism, and role-players just have a different style? A style more suited to minorities, apparently, but so what? The assumption underlying the paper and Castronova’s further comments are that those people at the “low” end of the role-playing spectrum, grinding out the levels and the monsters, are not doing it for escapism. I’m sorry, but no matter what style of play you have, when you pay by the month to engage for hours in a computer game where you play an elf, orc or rogue, you’re in it for the escapism. The rest of it is just about style.

So no, role-players are not a “disturbed” minority (at that!) who want to escape reality. They are a small subgroup of a large number of people who play a game as a form of escapism, and do it with a particular slightly pretentious style.

Problems with the statistics of the paper are:

  • they claim the survey is a “stratified random sample” taken on 4 strata (4 different servers) but there is no evidence in the analysis that the stratified random sample has been taken into account
  • They don’t report a response rate for the overall survey or the servers. Maybe “marginalized” heavy role-players were more likely to answer the survey than the non-marginalized heavy role-players?
  • The differences in the groups are in some instances very small and only significant due to the large numbers in the survey, and Cohen’s D statistics don’t really give any additional weight to the results (there are significant problems with the use of these kinds of stats in my experience). Consider the loneliness scale: high role-players differ from the low ones by 2 points on a scale of 4 to 80 (about 2.5%), which is not a big difference no matter how significant it might be. It appears that there was only 1 woman in the High RP group (out of 300 or so people!) but the gender difference between this group and the medium RP Group was statistically significant! These are large-sample anomalies
  • There is no multiple regression analysis, so no adjustment for confounders. Given the supposedly significant demographic differences between groups, it might be wise to have done this. Particularly, adjustment for the 7 categories of education, and for social marginalisation, might have removed the mental health differences between groups
  • Mental health appears to be estimated by a form of self-report. This is always a dubious measure.

So the stats could probably have been better explored…