• I stumbled on a problem with out-of-sample prediction in R, which I think is an example of idiosyncratic programming (and possibly insoluble) and, in the spirit of Andrew Gelman’s stats blog, I thought I’d put it on the internet to see if a) I’m just being stupid and b) it really is as much of a barrier to automation of statistical functions as I think it is. Perhaps someone with more clout than me will find this blog and notice this problem – or perhaps it’s not a problem and I’m just being stupid (or unfairly demanding of my prediction modeling software).

    In case you haven’t worked this out already, if you aren’t interested in stats, R, or nerdy shit that wastes everyone’s time, you should stop reading right now. Everything beneath the dotted line is going to kill your love life, and probably your pets.

    First, let’s consider the basic out-of-sample prediction process. We run a very simple linear model and then we provide R with a set of new data – containing exactly the same variable and type of variable – and get it to work out what the expected value of the line is on the new predictor variables. Taking the example straight from the manual, here is the code for a trivial predictive model:

    x <- rnorm(15)
    y <- x + rnorm(15)
    predict(lm(y ~ x))
    new <- data.frame(x = seq(-3, 3, 0.5))
    predict(lm(y ~ x), new, se.fit = TRUE)

    This is very simple code: we produce a set of x values and make the y values equal to these x values plus a random amount[1]. Then we run the model and get the predicted values (just for shits and giggles, btw), and then we make a new data set that contains values ranging from -3 to 3 in steps of 0.5, and run the predictions on these new x values. In this code, the data frame called new is the out-of-sample data, and we get the expected value of y given the observed values in new, for the given model. This is stats 101, right?

    In my case, the x values are

    1.5313262  1.5307600 -0.5067223 -0.1366010 -0.6557527  1.4245267 -0.3917136  1.7941995  2.0511560  0.3602334 -0.8798598 -0.5755816 -1.6419118 -0.7885237  1.1478910

    and the coefficients of the resulting linear model are

    [0.367,1.047]

    That is my model has an intercept of 0.367 and a slope of 1.047. This means that, e.g. for a new x value of -3 I would expect y to be 0.367+-3*1.047, or about -2.774. The correct predicted values for the new data set, based on these coefficients, produced by the 5th line of my code, are:

    -2.7751002 -2.2514963 -1.7278925 -1.2042886 -0.6806847 -0.1570809  0.3665230  0.8901269  1.4137307  1.9373346  2.4609385  2.9845424  3.5081462

    Now, let’s produce our first example of a problem. Consider the following very simple code:

    yy<-cbind(y,x)
    pred.m1<-predict(lm(yy[,1]~yy[,2]))
    pred.mnew<-predict(lm(yy[,1]~yy[,2]),new,se.fit=TRUE)

    Here, I’ve bound my previous x and y values together to make a matrix, with y in column 1 and x in column 2. Then I’ve rerun the model using the perfectly natural R method, of referring to each variable in the data set by its column reference ([,1] for the y, [,2] for the x). Then I’ve rerun the out-of-sample prediction on the same set of x values as I used before (the vector new). This is the simplest possible non-trivial linear model, and out-of-sample prediction in this case should be as simple as in the case of the previous model. So what happens? First, the following “warning”:

    Warning message:
    ‘newdata’ had 13 rows but variable(s) found have 15 rows

    Note this is a warning. So what values do we get? Here they are:

    1.97013967  1.96954674 -0.16412052  0.22347339 -0.32018633  1.85829845 -0.04368247  2.24542264  2.51450943  0.74376222 -0.55487302 -0.23623050 -1.35289974 -0.45922514  1.56860335

    These values are different. In fact, they’re the in-sample predictions. You can check it – they’re the result of applying the coefficients to the original observations of the predictor variable. This means that two important things happened:

    1. R failed to produce a solution to the simplest possible linear modeling problem, simply because I changed the input method for my regression model – even though the revised input methods (for the formula and the data) are entirely consistent with the R language
    2. The “warning” was not a warning: R didn’t just warn me about a problem, it failed to apply the command I told it to, and didn’t tell me about its decision. I could publish this shit, and not realize that I’ve been given my original values back.

    Note how thoroughly horrible this would be if I were running some kind of automated series of regression processes – I once did a capture/recapture model that was heavily dependent on automation in R, and I’m wondering if I fell for this crap then. R should not be declaring a “warning” when it is actually refusing to run the command I asked it to and producing completely different output. I know I’m only a statistician, but imagine if this kind of programming were allowed in Jumbo Jets. Unhappy punters we would all be, young Jedi.

    Also let’s just pause and dwell on the meaningless nature of that warning. I’ve supplied a vector of 13 observations, and the vector of 1s (for the constant) is implicit in everything I’ve done to date. So this means I’m (reasonably) assuming R will construct a 13×2 matrix with a column of 1s and a column of (out-of-sample) x values. Then, we have a 2×1 vector of coefficients. Thus, R should be able to calculate a vector of outputs from the information I’ve supplied it. The only possible explanation is that it is expecting me to supply it with a full-rank matrix, i.e. it has decided arbitrarily to stop implicitly assuming the constant. So, shall we try that?

    new.fr<-cbind(rep(1,13),new)
    pred.frnew<-predict(lm(yy[,1]~yy[,2]),new.fr,se.fit=TRUE)

    Same prediction process, only now the out-of-sample data that I provide is full-rank, containing a column of 1s and a column of (out-of-sample) x values[2].

    The result? The same problem: the same warning in red, and in-sample expected values. So the problem is not that I’m providing a less-than-full-rank matrix of predictors on the assumption that one column of the design matrix is implicit. The problem is deeper than that.

    I can, however, fix the problem in a weird way. If I assign column names to the original data, and also assign the new data a name that matches the name of the predictor in the original data, I get the correct output:

    yy2<-data.frame(yy)
    names(yy2)<-c(“Outcome”,”Predictor”)
    new2<-new
    names(new2)<-c(“Predictor”)
    pred.mnamed<-predict(lm(Outcome~Predictor,data=yy2),new2,se.fit=TRUE)

    In this case I’ve made a new data frame for the original data and the out-of-sample data, to avoid confusion, and I’ve given them all consistent names. Note now that in the prediction process the out-of-sample data, new2, is not full rank – the column of 1s is assumed in the calculation process. But it doesn’t cause any trouble: the previous warning about seeing only 13 observations doesn’t appear, and I get the correct out-of-sample predictions. I still only have 13 observations, but R doesn’t care. I get correct predictions and everything is fine.

    This might seem like a trivial problem until you’re doing automation. I recently wrote a program to do a search through a large number of univariate models, and then combine the variables from the top 10 in a predictive model. The predictions were the only thing I wanted. But I spent hours trying to work out why R wouldn’t give me out-of-sample predictions: my training sample had about 50k records, and my validation sample had about 20k. I was using numeric column references for the purposes of automation (the process should be fairly obvious: you make a matrix of variable numbers with AIC values from the corresponding linear models, sort by AIC values, choose the variable numbers corresponding to the best 10, and then put them into a formula in a linear model) but it wasn’t working. Eventually I worked out this problem, made a vector of column names and used the paste() function to produce a formula.

    But this work around is stupid. In both cases – using column numbers or referring to names – the design matrix is implicit, not specified by the user. The underlying R code has to construct this design matrix. So why does it screw me around like this? Why provide two ways of referencing columns and then have one of them bork your most basic modeling function? What would Tibshirani say? Dividing data into training and validation sets is an essential part of modern machine learning principles, but R can’t handle it because – omfg – one has a different number of records to the other. Even though the matrix mathematics is in no way affected by the difference in the number of records.

    That, dear reader, is called crap program design. Especially crap is the issuing of a warning that isn’t a warning at all, but an indication that the program has arbitrarily decided to produce a result different to the one you asked for! Rather than declaring a “warning,” R should say clearly “Catastrophic failure! Providing default prediction using original design matrix only!” Or better still it could say “What are you doing, Dave?” and refuse to let me out of the cargo bay doors.

    The manual says the following about giving out-of-sample data to the program:

    If the fit is rank-deficient, some of the columns of the design matrix will have been dropped. Prediction from such a fit only makes sense if newdata is contained in the same subspace as the original data. That cannot be checked accurately, so a warning is issued.

    and

    A warning will be given if the variables found are not of the same length as those in newdata if it was supplied.

    This is not exactly the most informative help in the world. I don’t think that what is described in the first sentences (to the extent that I understand the English) is what happens in practice, and the last sentence is certainly misleading. In R, warnings usually indicate something you should be aware of that does not affect the numerical processing of results (see e.g. Andrew Gelman’s statement in the above blog link, “I don’t care about warning messages.” Except you should, because sometimes R issues a message called a “warning” that is actually an indicator of insouciant and catastrophic failure to enact the expected mathematical process.

    Open source software: it sucks. Except that R is far and away the best software for automation in stats – until you run into crap like this. Fix it, CRAN!

    fn1: of course the R manual makes this part just slightly more obscure than it needs to be, because hey! R is a really pithy language, so why would you use comments to make the process easier to grasp, or separate the process of generating x from the process of generating epsilon? Anyone who knows what they’re doing can figure it out, and manuals are for people who already know what they’re doing – right?

    fn2: I guess I could do this more accurately by supplying a variable containing only 1s, and specifying “no constant.” That might work, I suppose. How horrid.

    Here is the full code to reproduce all these results:

    ## Predictions
    x <- rnorm(15)
    y <- x + rnorm(15)
    pred.v1<-predict(lm(y ~ x))
    new <- data.frame(x = seq(-3, 3, 0.5))
    pred.vnew<-predict(lm(y ~ x), new, se.fit = TRUE)

    ## now make a data set yy that contains a column for x and a column for y

    yy<-cbind(y,x)

    pred.m1<-predict(lm(yy[,1]~yy[,2]))

    pred.mnew<-predict(lm(yy[,1]~yy[,2]),new,se.fit=TRUE)

    ## now give the data set names and retry it
    yy2<-data.frame(yy)
    names(yy2)<-c(“Outcome”,”Predictor”)
    new2<-new
    names(new2)<-c(“Predictor”)

    pred.mnamed<-predict(lm(Outcome~Predictor,data=yy2),new2,se.fit=TRUE)

    # now try a full rank matrix
    new.fr<-cbind(rep(1,13),new)
    pred.frnew<-predict(lm(yy[,1]~yy[,2]),new.fr,se.fit=TRUE)

    x.mat<-as.numeric(lm(yy[,1]~yy[,2])$coefficients)

    # try again with matrix maths
    new.fr<-as.matrix(cbind(rep(1,13),new))
    pred.frnew2<-new.fr %*% x.mat

  • image

    This is one part of my Valentine Day haul, chocolate from the Delightful Miss E. I will of course receive more, and got some more before the actual  day. This is because Valentine Day in Japan is not a festival if mutual affection and dating, but a cunnibg marketing scheme concocted by the faceless men of Big Chocolate.

    And as a marketing  scheme it is of unparalleled wickedness, corrupting the western dating element and melding it to Japanese concepts of obligation and group membership – then targeting the whole  thing exclusively at women to make sure the meme manifests as fantastic profits. I love it for its sublime union of evil and chocolate.

    You see, in Japan Valentine Day has been reconstrued as a day for women to give chocolate to men, starting with their lovers and proceeding through all the men in key positions in their life: family, friends, club members, colleagues, even their boss and teachers. This last set of recipients reflect the women’s oblugations to men who have helped her during the year, and is referred to as giri choco. By this point most women are so deep in chocolate that they give to female friends too.

    As an example of the reach of this ritual, on saturday I went to see the band ElupiA, and their singer gave me chocolate just to show her appreciation for my support.

    What’s not to love about Valentine Day in Japan? Just one thing: i have to repay the lot on White Day, March 15th. As I said, it’s a wicked scheme…

  • Rebels don't endorse standard public health messages

    This is an excellent interpretation of Stieg Larsson’s page-turner of the same name. For my sins, I read the novel and enjoyed it despite its sometimes crappy writing, because the story is compelling and the characters are fun. Both the main male character, Michael Blomquist, and the eponymous female lead Lisbeth Salander are excellent depictions of their particular archetypes[1]: crusading journalist and lunatic hacker, respectively. The movie brings them to life well, perhaps even improving on them through good acting (’cause lord knows they were held back in the original through bad writing!) It also brings out the setting, both the historical part and the modern Swedish setting, so that they were just exactly how I’d imagined them when I read the book. It also makes the investigation interesting, and you can understand how the combined talents of Blomquist and Salander are capable of solving a mystery that no one else managed to. It also managed to cut out some parts that would have made the movie too slow, and to interweave the three stories (Salander, Blomquist, and the historical part) nicely without being confusing or chaotic. This is surely good movie-making …

    The acting was also great. The woman who played Lisbeth Salander, Rooney Mara, was superb in the role and did a brilliant job of holding together the tension, intelligence, viciousness and strangeness of that character without over-doing any of it, or pushing Salander into a stereotype of a hacker. Salander is a complex personality and a complex emotional story – simultaneously vulnerable and fragile and extremely tough, uncaring about convention but very aware of how other people think and feel – and Mara did a superb job of getting her right. In his own way, Blomquist, though superficially simpler, is also hard to get right, though perhaps more from a direction point of view: Blomquist is a man who respects women but doesn’t put them on a pedestal, who has deep passions but doesn’t lose control of them, and who probably isn’t a particularly expressive guy. Daniel Craig does a very good job of getting it right. The cast were also chosen so that everyone felt real, and many scenes that one might expect a movie remake to change, gloss over or misogynize were very well crafted.

    This is an important and unavoidable problem in bringing this book to cinema: handling the gender relations. This is a book about getting vengeance on rapists and murderers of women, but it’s also a story about a young woman who falls in love with an older man (cliche 101!) and a couple of Scandinavians who have an open relationship. The temptation here for your average movie director is to make the rape scenes sexy or shallow, to make the young woman a victim of the man’s charm or the age-gap completely normal and believable, and to make the women in the open relationship young sexy babes, or just crazy fucked-up people. None of this happens: the rape scene is horrible and the vengeance enormously satisfying, while also repulsive; the young woman is not a victim of the older man’s charms, and the nature of their relations with other people are such that you understand the situation is not normal for him or for her – it’s the first such old man/young woman affair I’ve seen in a movie that feels believable. And the older women in the open relationship – 40 something career women – look their age, like attractive 40-something career women in control of their own lives and sexualities. It’s through Blomquist that we mainly encounter these people, and his approach to the women in his life is straightforward, respectful and understanding. A perfect counter-point to the men that he and Salander are engaged in foiling, who are sleazy liars who only know how to use people, and especially women, for their own gratification.

    This movie also has one sex scene in it that really does describe the difference between a movie that depicts the real relations between modern men and women, and most of the rest of the American movie industry. The scene is nothing special, but its execution made me happy for its frankness and realism. We see the young woman and the older man having sex, but she is on top grinding away to her own orgasm, largely oblivious of him, and he is just kind of going along with it. In the end she comes and he doesn’t, and I think it’s the first time I’ve ever seen a sex scene where the woman gets something and the man gets nothing. Usually either the man comes or they both do, simultaneously, without any effort on his part except the glorious power of his amazing dick. The reality of course is that sex is not often like that, unless the girl is faking it, and everyone who has more than the barest experience of sex has experienced the woman who takes her own orgasm astride the man, often quite aggressively. It’s like the guy who made this movie actually wanted to show sex as it happens between real people who love each other, rather than as it is imagined in the minds of people who insist on reproducing the imaginary gender relations of the American culture industry. For that reason alone, the scene made me happy.

    The flipside of these scenes, though, is that there is a lot of nasty stuff to wade through in this story. The rape scenes, graphic evidence from the murder scenes they are investigating, the final tense showdown, animal cruelty … if seedy under-belly-of-society type movies don’t appeal to you or you just can’t watch films that involve rape or the cruel mistreatment of women, then I suggest avoiding this one. You’re not going to get much satisfaction. If you can bear this sort of thing in order to see a good story and fine acting, and you can get pleasure from fairly nasty revenge scenes (I certainly can), then I recommend taking this one in on the big screen. In addition to a tense story, fine settings and excellent acting, it also has some very cool cinematography and a great soundtrack, so its well worth the effort if you can endure that sort of cruelty on screen. But you need to go in ready for some nastiness, and if you don’t think you are, then you probably should give it a miss …

    fn1: Archetype is the word you use instead of “stereotype” when you enjoyed the book.

  • The devastated coast of Minami Soma City

    On Monday I took a two-day business trip to Minami Soma City, in the disaster-affected area of East Japan. Minami-soma city was hit by the tsunami, and although there does not seem to be much online footage of Minami Soma City’s experience, the effect on nearby Soma City (just north of Minami Soma) can be seen in this terrifying video (the main wave is at about 8 minutes). Minami soma city is just 23 km from the Fukushima Number One nuclear power plant, so soon after the tsunami hit the town was included in the government’s 20-30km voluntary evacuation / limited outdoor activity zone. Its population reduced enormously in the weeks that followed, and has now returned to just over half its pre-tsunami numbers. The town is also home to about 6,000 displaced persons, living in temporary housing. My purpose in visiting – along with some of the students of my department – was to help the local hospital with some research they are doing into the health of these displaced persons and of the residents of the town generally.

    As part of my stay I was taken to the area where the tsunami hit, which is a stretch of coastline extending a few kms inland from the sea. These photos show some of the damage that I saw there. The full set can be viewed in my flickr account.

    Entering the ruins

    When we drove into the area I thought perhaps it used to be farmland, because aside from the piles of rubble it is completely flat, but in fact this whole area used to be houses and businesses. The ground is flat like fields because it was scoured clean of all but the largest structures, and the resulting rubble has been gathered together into great piles of debris (visible in the photo above). This gives the area the impression of being a moonscape or wasteland, where once houses used to be, and the area from which one enters the destruction zone is lined with these piles of rubble. If one drives for a few more minutes, one can reach the sea wall and look back over the entire devastated area, as in the picture below.

    Sunset over the devastation

    The sea wall itself is about 4m high on the outside, made of huge slabs of concrete. On the seaward face there is a small stony beach and then some lines of tetrapods (concrete structures that act as further wave barriers). The sea wall survived the tsunami, but was heavily damaged and didn’t serve to impede it. Parts of it were broken off and swept away, and its landward side was heavily damaged. I think the wave just ran over the top of it. We walked along this wall and the two photos below show the wall in both directions. The photo at the top of this post, of Miss A returning to our car, was taken from the top of the wall.

    Looking south in the lee of the sea wall
    On the sea wall, looking north

    Facing North (the second picture of the sea wall), one can see the only surviving structure near the sea – that strange orange building that has been hollowed out but withstood the wave itself. I guess other smaller objects survived but have subsequently been removed in the clean up, because as we left the area we entered the rubble zone and passed huge piles of broken stone that must have been taken from this area. We also passed a graveyard of cars. In amongst all this neatly-arranged debris there were also a few boats.

    A fire engine

    We spent the night in the hospital, in one of the unused wards. Staff levels have declined at the hospital, though the patient load has as well, but business seems to be going on pretty much as normal there. In fact, this is one of the strangest things about Minami Soma City: once you pass over the low line of scrub and hills that separates the unscathed part of the city from the damaged area, life goes on pretty much as normal. It’s as if nothing ever happened here, except that it’s quieter than a normal town and a lot of the businesses are on reduced hours or shut. It’s not a ghost town, though, and everything is perfectly normal there – we went out drinking and having fun with the other doctors in the evening, and everything was just like any other rural Japanese town, if a little quiet.

    You are 23km from Fukushima Number One plant

    As the sign in the hospital says, Minami Soma City is quite close to the power plant, and the only road leading to it from Fukushima passes even closer. It’s quite hard to get to Minami Soma City – there are only 3 buses a day from Fukushima, so it’s either a long bus trip to Tokyo (probably 6-8 hours) or a bullet train to Fukushima city and then one of those three buses. The buses were also not able to run at first, because of the exclusion zone, but now they’re open and the journey to Minami Soma City takes us through several deserted towns in the exclusion zone.  I think a few people might still be staying on in these towns but they were largely deserted, and the only people I saw were a work crew in full radiation suits, cleaning an outside area in the driving snow. The only other signs of life in the area were animal prints in the gathered snow.

    This whole area is obviously struggling with the triple challenges of dealing with the aftermath of the tragedy, rebuilding, and the fallout of the nuclear plant. Many people have left and the area is certainly suffering physically and economically. With the one year anniversary of the events coming up soon I imagine a lot of sad memories of the tragedy will be rekindled. But the people there are kind, friendly, and full of warmth and energy not just to rebuild their town but to turn the tragedy into useful lessons for disaster management in the future. I’m hoping through my advice and support to provide some small contribution to that process, and perhaps to be able to learn some lessons about post-disaster management in an aging society. The atmosphere there was one of hope and energy to make a better future, and despite the sad story that these pictures tell, the people there offer a great deal of inspiration to make a better future. Let’s hope that working together the people of Japan can overcome this disaster and, through their experience, offer other countries lessons they can use to overcome their own future challenges.

  • In Wednesday’s Guardian, Charlie Brooker continues his series of articles on his trip to Japan, and in the same tone: where he started his first article with a long paragraph that combines toilet humour and assertions about the kookiness of Japan, this article starts with a description of a computer game about bouncing turds, and finishes the introduction with

    Unfathomable, futuristic madness: that’s what made me want to visit Japan.

    So, in case you weren’t sure from the first article, Japan is strange and fascinated with toilets and poo.

    Except, really, it’s Brooker who is fascinated with poo. He seems quite taken with the abject, if his first article was anything to go by. But once again, after he’s got the obligatory toilet humour and stereotyping of Japan’s “futuristic madness” out of the way, he carries on with a valid observation about this place:

    it’s a place where being a geek (or otaku) is comfortably mainstream. Former Prime Minister Taro Aso is an enthusiastic manga-collecting otaku, the TV ad breaks heave with glossy commercials for collectible card games, and multi-storey games arcades are commonplace.

    This is very true. Of course, he immediately follows this important observation with another example of drawing the wrong conclusion due to limited data:

    the subway is eerily silent: thanks to a strong underground signal, everyone’s staring at their smartphones, texting, playing games, or reading. Only after a fortnight did it strike me: not once did I hear a single person actually speaking into their phone on the Tokyo subway. Everyone – and I mean everyone – seemed to be perpetually tapping and swiping in silence. Unnerving to many: to a geek like me, it felt strangely comforting.

    This, Charlie, is not because everyone is madly playing some game or other. You might actually have noticed a lot of people reading these things called “books.” But the reason they’re not talking into their phones is because there are signs and announcements asking people not to. It is considered very poor manners in Japan to talk on your phone in restaurants, cafes, bars or trains. i.e. in public. And people in Japan follow these rules. If it’s “unnerving to many,” this is because that’s another one of those things about the west that don’t make sense once you haven’t lived there for a while. Those people you saw on the subway being quiet aren’t doing so because they are obssessed with a game; they’re doing so because they are refraining from offending others. That’s not “strangely comforting,” it’s perfectly ordinarily comforting.

    The rest of the article consists of a fairly nice description of one of Japan’s bigger game arcades, from the perspective of someone who is familiar with what should be going on but can’t understand it because he is in a foreign land. Again, though, he pushes the unfathomable nature of the thing too far, and again reminds us that Japan is exotic and incomprehensible:

    a roomful of sombre youths vying for individual supremacy using some form of networked arcade strategy game that uses collectible cards. Imagine witnessing a game of bridge being played in the Cabinet War Rooms in the year 2072 AD … whatever the theme, the nature of the action is absolutely impenetrable to the casual onlooker.

    Charlie, here’s a real-life hint for you: to people outside the nerd world, this kind of stuff is absolutely impenetrable in their own language. Now that you don’t speak the language, you can be reminded of how people feel when they watch you at your normal hobby. Eye opening, isn’t it?

    Other than this, the article struck me as a missed opportunity. There’s a photo of an “otaku girl” at the top of the article but she doesn’t look otaku to me, and (probably because he hasn’t had time to notice), Brooker hasn’t mentioned how different gender relations are amongst nerds in Japan compared to the West. To wit: in Japan, being a nerd is not only more acceptable, but it’s especially more acceptable for girls. There are adverts on the trains targeting Wii at old people, and the latest computer games (like Mario Land and Monster Hunter) at young women. There is an advert for a trading card game in which a member of a currently-popular boy band goes to a game shop and plays the card game with the lonely kid in the corner; there are adverts for a new trading card game where some of the cards are based on members of a famous boy band (Exile, I think). In Ikebukuro there is a whole series of shops devoted to targeting pornographic manga at women. This is a hobby world that is not just mainstream, but mainstream for both genders – and this is why Wii was invented in Japan, not the USA. It’s a shame that Brooker didn’t find a way to comment on this, and on how much easier that makes being a nerd in this country. He also didn’t find any opportunities to talk about the darker side of the nerd world in Japan: pachinko, or AKB48. Instead, he just took a last chance to remind us that Japan is crazy and incomprehensible. Just in case we didn’t know that.

    I wonder what his next article will tell us? Feel free to put your predictions in the comments …

  • I’m currently watching the latest season of Dexter, and as we were watching my partner suggested that it would be excellent to see an episode of either show in which Dexter visits New York (or vice versa). Perhaps he is chasing a serial killer operating in NY, or perhaps the Miami police have to go to New York on some task. Or vice versa. Some fun aspects of such an episode:

    • Deborah Morgan would be competing with Kate Beckett to solve the case; both of them would, of course, be competing with Dexter. How would Beckett handle Morgan? How would Morgan view Beckett?
    • Castle is very fond of making wild conspiracy theories: the whole episode/sequence could run with him consistently treating the truth about Dexter as a conspiracy theory to explain the weird events of the case; but of course everyone is laughing it off
    • Esposito could figure out how dangerous Dexter really is, but no one (except Castle!) believes him
    • Castle finds out the truth, and confronts Dexter
    • The story could be played in both TV series, so we see it from Castle/Beckett’s point of view and from Dexter/Deborah’s point of view.
    • There could be a spin-off where, having identified Dexter, Castle conceives of a new series of popular novels about a serial killer with a conscience, and moves to Florida for a year to become Dexter’s shadow instead of Beckett’s
    • Or better still, Beckett’s lieutenant palms Castle off onto Miami, and he spends a whole season tracking the Miami metro homicide squad, getting closer and closer to uncovering the truth about Dexter …
    • And, the novel Castle writes is called Darkly Dreaming Dexter

    Of course such an idea is just silly. But I think it would be pretty funny.

  • This slide is a little busy, but …

    Today I sat through about 15 presentations, and in the x 100th of them, the powerpoint failed. Just stopped working. It was a disaster for everyone concerned, not least of the concerned being the master’s student whose grade depended on getting the software working again[1]. As the crash came down, I realized that actually Powerpoint is a quite awesome piece of software, and we’d all be stuffed without it.

    Bear with me here.

    This young man’s presentation had a diagram on every slide, and the diagrams were essential, and consisted of mitochondrial thingies with strange boxes and structures in them, and usually also a flow chart of some kind to describe the process being used to break into or out of the mitochondriality of the thingamy. Every couple of slides was a table, or a graph. Then some chemical structures, maybe a picture of a bug, and then back to some of those fiendish flowcharts and another mitochonria with a box up its arse[2].

    Watching him desperately trying to get his slides working again I realized that if he had to present this material to us in any other way, his 15 minute presentation would have been drawn out to about an hour as he drew painstaking diagrams and structures on – shudder! – a blackboard. Or worse still, one of those hideous transparency thingies that ate my soul back in 3rd year. Before the internet (do you remember those times? No, neither do I. Why would you?) Perhaps he’d have had to bring a couple of old-fashioned 3-D models of chemical structures. So we’d have got through only a third as many presentations, at a great cost in blood and treasure, and would be watching Masters presentations until Friday afternoon. And I’ll bet you a groat that most of those presentations would still have been bad, although by Thursday afternoon my brain would be mush and quality control would be out the window.

    Actually, the presentations were largely of very high quality, and almost all of their failings were in content rather than presentation – the presentation was of universally high quality. And the bad presentations failed for largely one reason: the presenter was reading the same words that were written on the screen. That is, the worst of the presentations were bad because they were like a speech. Had those presenters followed basic principles of powerpoint – short bullet points, diagrams and graphs that you hang a speech on – they’d have been improved. The main reason that powerpoint fails us is that we use it as a speech-making prop rather than a presentation tool. Sure, there is probably still the odd weirdo out there who uses animated gifs, and occasionally you see someone try to present a sophisticated jobby, with transitions and the like – these usually fail and make the presenter look like a wanker. But if you use powerpoint for its main benefits, you get a fine addition to your presentations. Bullet points and charts give you something to hang your words on, and by hanging your words on the points you make the points more interesting, and give your audience a way to process information at a greater density and with more focus than you would get if you were just reading it.

    So, I think the standard view of powerpoint – that it dumbs down presentations and makes them shallow – is flat out wrong. There’s a heavy dose of condescension and, I would say, luddism, in this view. Embrace your slide-y overlords!

    fn1: A word of advice to all masters students out there: don’t use the timer on slides. You will go faster or slower than your timer allows you. In either case it looks bad, and then it gets annoying, and if you make your reviewers dizzy flicking backwards and forwards on the slides, it’s probably not going to help you

    fn2: My poor description of this presentation is not an indictment of Powerpoint, but a sign that I am not a biologist. I was there to support students from my department, who most certainly would not touch a mitochondria no matter how sweetly it talked to them. They’re all about the epidemiology!

  • Magnetism, by Ahmed Mater

    The campaign setting I am currently playing in, Punjar, has a vaguely middle Eastern subtext, with the city of our adventures presented as a chaotic, slightly exotic free state of souks and temples, such as western readers might associate with somewhere in pre-modern Oman or Turkey. While gaming there I try to hold in my head images such as the opening scenes of The Exorcist, though obviously (unlike the priest of that ill-omened scene) my character is a local who understands what is happening around him (and might even understand the meaning of the statue he dug up, if he could make the Arcana check!)

    Simultaneously with my entry into this world of bazaars, brothels and giant barking toads, the British Museum has opened what looks like a fascinating exhibition on the Hajj, the Islamic pilgrimage to Mecca that constitutes one of the five pillars of the faith. The Guardian has an interesting review, with links to some of the artists involved (one of the artists’ pictures is on the top of this post). The review certainly makes this exhibition sound like a masterpiece of the curator’s craft: it combines historical documents, objects and art with modern art, video of some of the scenes of the Hajj, old news footage, and modern diaries and spoken accounts of people’s pilgrimages. The review makes reference equally to high art and the diary of a North London schoolgirl. It also appears to show something of the complex relationship between Britain and its ex-colonies in South Asia.

    I’m not in London now so I can’t visit things like this anymore (though sometimes the British Museum’s exhibitions end up in Japan), but it looks like something that would be well worth visiting for those living in London. This exhibition also hints at the complex and fascinating campaign setting that the Islamic world offers to enterprising GMs. Obviously most of us, as outsiders to that world, can only really hope to present a cheap simulacrum of that world (like, say, Punjar) but even a very shallow investigation of the world of Islamic art, history and culture would no doubt throw up a wide range of interesting and exciting adventure settings. I’ve no doubt, too, that the political context of almost any period in Islamic history – from the time of the prophet onward – would be easily as challenging as those of the Victorian era. Also playing on the opposite side of the nations of the Great Game – e.g. as Afghan adventurers during the Russian and British interventions there in the 19th century, or as adventurers in any city of the Middle East during the Crusades – could be a lot of fun.  The breadth of the Islamic world, which ranges from modern-day England to 12th century Indonesia, and the diversity of its cultures, offers a plethora of settings, and the Hajj is the classic opening scene (“the adventure starts with the PCs on a routine mission, guarding a rich merchant on his pilgrimage to Mecca”). In fact, it could be like Monkey, with the entire campaign occurring on the journey to the Hajj. You set off from somewhere in India at level 1, and 8 months and 20 levels later you arrive in Mecca. Your ultimate mission, of course, is the pilgrimage itself. But in the face of a hazardous journey over a whole continent, can you even keep the faith that you set off in service of? Or, in the words from one piece in the exhibition: “Are you leaving as you had come?”

  • Continuing my series of posts on sex work, public health and feminism, I turn my attention now to the modern feminist response to sex work. First I’ll outline a common strand in modern feminist responses to sex work and pornography, which I think it should be pretty obvious contrast with the public health approach I described previously. In subsequent posts I will discuss the use and abuse of the contentious issue of “sex trafficking,” and then I will close this series by discussing what I think all this says about modern feminism’s relationship with ordinary women, with reality-based policy-making, and with the ways in which society has liberalized in the past 20 years.

    Prohibition and Pornography

    The first great feminist incursion into the sex work debate in modern times was the great pornography debate of the 1980s, when Andrea Dworkin and Catharine McKinnon became active in attempts to both ban pornography in several states, and contributed to an inquiry established by Ronald Reagan to inquire into the “harms” caused by porn. Dworkin and McKinnon are probably the two most famous radical feminists involved in the anti-pornography campaigns of the ’80s, and had a huge influence on the debate. They are often characterized as having teamed up with christian conservatives in their contribution to the 1986 Attorney General’s Commission on Pornography, and the methods used by the movement they represented, Women Against Pornography, were fundamentally illiberal.

    The Attorney General’s Commission on Pornography ultimately led to the publication of the Meese Report, a highly controversial document that found many negative effects of pornography, and infamously associates pornography use with rape and child sexual abuse. It also gives a hint of how the anti-sex work feminist movement was prepared to treat women in the industry. In Chapter 4, which describes the way in which women are treated in the pornography industry, we find the following introductory discussion of their methods:

    we have not had the power to issue subpoenas summoning reluctant witnesses to appear; thus all information at our disposal was presented to us voluntarily or obtained through our review of materials on the public record. In addition, the severe time constraints imposed on our work were particularly damaging in this area because, as discussed earlier, this aspect of the pornography “industry” has received only the scantiest attention in the past. We, therefore, did not have the benefit of knowing from the outset what were the most likely avenues to discovery of pertinent evidence about activities that are largely underground. Finally, both the difficulty of locating witnesses and the pressure of time meant that we were not able to spend substantial time in cross-examination of their testimony or in background investigations to corroborate their statements.

    In the end, this inquiry just did some convenience sampling of a sub-culture that was under attack in the US and whose female participants are generally seen in … well, in less than positive terms by most members of the community (especially in the 80s!) So is it any wonder that from amongst their extremely biased sample they find that the industry is seedy and dangerous and in need of reform? This is a constant problem in the modern feminist approach to sex work: in a society where anyone who enjoys or seeks out casual sex or selling sex is derided as a slut, a fool or an enemy of women, it’s no wonder that the accounts that surface from this industry tend to be one-sided and self-exculpatory. Who wants to be reported in a national commission of inquiry during a conservative era as a loose woman whose morals are so poor that she enjoys fucking strangers for cash? These women either don’t come forward, or lie.

    Which isn’t to say that the industry wasn’t dubious in the 70s and 80s, but the natural public health response to a dangerous working environment is to set up a regulatory and occupational framework that will ameliorate the risks. However, the radical feminist approach to porn was to attempt to get the industry banned, and this proceeded with efforts at municipal level. Because the first amendment protects free speech the movement attempted to redefine pornography as a form of sexual harassment and to pass civil laws that would enable women to sue makers and distributors of pornography on civil rights (rather than censorship) grounds. Hearings were held into the laws, and the process of these hearings is described in Mckinnon and Dworkin’s book In Harm’s Way, which is reviewed here and seems to present a fundamentally dishonest depiction of what actually happened.

    Not only is this a fundamentally illiberal approach to pornography and the sex industry, but it shows that the anti-pornography movement are willing to cut deals with any unsavoury characters – including Ronald Reagan’s christian conservative movement – to get their goals. We’ll see this again in later responses to sex work, when we see the way the anti-sex work movement has sided with the US State Department to use coercive methods to impress its preferred “solution” to sex work’s public health risks on developing nations. Perhaps more seriously from a feminist perspective, the 10 years of this movement’s activities in the US fundamentally divided feminists from the pornography industry, denying them a chance either to influence women-centred pornography or the depiction of women in porn aimed at men, and separating them from an industry which represents the natural consequence of second wave feminism’s greatest achievements: the liberalization of sex and the discourse about sexuality. So it was that from the 1980s onward pornography headed off down an increasingly misogynist and extreme path, at least in the West, and feminist influence over its development was lost. Now that the internet enables widespread porn delivery this is obviously a significant loss for feminism – instead of beaming pro-feminist images of sexual behavior into every teenage boys brain, Larry Flint’s degenerate cultural progeny are face-fucking them into misogynist oblivion. These activists also created a dominant discourse in feminism (and much of popular culture) about the destructive influence of porn that is almost completely groundless. This is not a great cultural legacy, and it certainly doesn’t create an atmosphere which is conducive to accepting and non-judgmental approaches towards women who work in what – in infectious diseases terms – is a very dangerous industry. While there is a sex-positive feminist movement, it is new and less influential on modern cultural attitudes towards porn due to the legacy it fights. We’ll return to the debate between these movements when we look at what this legacy of anti-sex work activism means for the relationship between modern feminism and young women.

    Feminism and Sex Work in Sweden and the UK

    While Dworkin and Mckinnon were active in the USA, a similar movement – influenced by similar people – was also growing in the UK. It’s most famous member, Sheila Jeffreys, staked her colours to the mast very clearly in the 1970s when she wrote a pamphlet declaring that all heterosexual feminists should eschew heterosexual sex and become “Political lesbians.” For feminists like Jeffreys, any woman who has sex with a man is a traitor. This makes sex workers quislings, the worst of traitors, and as a marginalized minority obviously easy front line targets in an ideological battle clearly aimed at changing the nature of the relations between the sexes. Her colleague and protege, Julie Bindel, is an anti-sex work campaigner in the UK with significant public influence through her journalism (she writes for the Guardian), who was deeply involved in a highly controversial and biased report for the POPPY Project, that presents an unscientific and potentially unethical review of sex work in the UK. Even though subsequent police action showed that many of the claims about trafficking and forced sex in the British sex industry were highly flawed, the campaigning of this group was instrumental in convincing the then Labour government to introduce a Swedish-style law on sex work. This law criminalizes the purchase of sex where the person selling it is working for someone else, on the flawed assumption that any sex worker who is working for someone else is (to use the radical feminist term) being “prostituted” (or “pimped,” as it’s more commonly known).

    This law is similar to the Swedish law, which criminalizes the purchase of sex but not its sale. These laws are based on the soothing fiction that by banning the purchase of sex but not its sale we can drive sex work out of business without punishing sex workers, only the men who visit them. These laws also have an explicitly moral, rather than public health agenda, as described by their architect[1]:

    In Sweden, prostitution is officially acknowledged as a form of male sexual violence against women and children. One of the cornerstones of Swedish policies against prostitution and trafficking in human beings is the focus on the root cause, the recognition that without men’s demand for and use of women and girls for sexual exploitation, the global prostitution industry would not be able flourish and expand.

    This article also mentions trafficking a lot, and includes some entertaining assertions about the Dutch sex industry (apparently Dutch job centres recommend brothels as work options for unemployed women!)

    So, the Swedish laws were introduced to prevent men purchasing sex, on the assumption that the view that women are commodities to be consumed is at the root of discrimination against women. This is a classic case of attacking the easiest symptom rather than the problem. If the problem is an attitude towards women which enables commodification, attacking the market place is no good – you need to attack the attitude. Unless the purchase of sex is common amongst all Swedish men, all that will happen is that it will target only the most extreme representatives of this attitude. And given most men don’t purchase women, how can we be confident that this commodification of women is the root cause of the non-purchasing men’s sexist attitudes?

    Both of these countries have acted to prohibit the purchase of sex but not its sale. Does this materially change the nature of sex work, help women leave the industry, protect women from trafficking and forced sex slavery, or make them safer? The opinion of most sex worker representative organizations is that it has the opposite effect: it drives sex workers back to a system of working individually, in rooms by themselves or on call-out jobs rather than in brothels, without security guards or drivers. It certainly doesn’t protect women from trafficking or sexual slavery, since these activities are illegal everywhere regardless of the status of the sex industry. The laws will only help women leave the industry if they are being forced into it in the first place (assuming the laws work in the way they are intended). But here the laws are driven by a fundamental misunderstanding of how the industry works and of what women want. Even with the best will in the world, you cannot drive women out of the sex industry, because it pays well. The only way the sex industry will disappear is if society can find a way to make men not want to purchase sex, and the surest way to do that is to attack all the other aspects of our screwed up system of gender relations that makes seeking casual sex such a complex and one-sided affair (I’ll have more to say about this when I review Big Bang Theory). Until then, men are going to want and need to pay for sex, especially if they are busy, traveling, disabled, or just plain ugly. Women, too, buy sex, and this fact alone presents a big problem for feminist approaches to the sex industry. It’s not going to go away until we restructure the nature of our non-commodified sexual relations, and this is happening very slowly (and, I hope to show later, the very feminists who oppose the sex industry also have very reactionary opinions about non-commodified sexual relations).

    From a public health and public order perspective, though, the main problem with these laws is that they drive women back into sole-trader arrangements, where they are vulnerable to rape and theft, and where their decisions about safe sex are driven by their own personal circumstances, work practices, and vulnerabilities rather than by the kinds of workplace policies, union rules and sense of shared responsibility that are most likely – in every area of employment – to change attitudes towards safety. It will also encourage people who are interested in running brothels – which are highly profitable businesses – to seek weaker, more vulnerable women who they can hide and who have little recourse to the law. That is, illegal immigrants. It also encourages police corruption (since sex workers and brothel owners need to get the police off their backs, and it’s the time honoured way). This is particularly tragic for women in the UK, because the UK police are extremely corrupt and there is no political will at any level to restructure the force to make it robust against corruption. When the Police Commissioner is willing to accept a gift of a five week massage holiday here from a media organization that had been hacking murder victims phones, paying police for private information on citizens, and even hacked the Prime Minister’s phone, what chance is there that ordinary police will turn down the odd back-alley shag from a girl who needs a break at work? None, I’d say. The Labour Party was willing to leave policing a law involving young women and sex to a police force that allowed its under-cover police to form sexual relationships – and have children – with activists they were supposed to be spying on. This is a recipe for corruption, and these laws will simply mean a return to the bad old days of vulnerable women being exploited or, at best, working in high-risk settings for lower pay and/or predatory criminal organizations.

    Sex Workers as Tools for a Political Goal

    The architects of these laws have made clear that they think the structure of modern sexual relations is wrong, and that they see sex work as the ultimate expression of the dysfunctional nature of modern sexuality. Often, they see commodified sexual relations as the problem – including but not limited to the idea of marriage as prostitution – but unlike the union-influenced and socialist feminist politics of Australia and of the earlier second wave feminists overseas, they don’t see the commodification of sexual relations as a result of distorted economic models. It is a hallmark of radical feminism that flaws in all other economic and social relations are believed to derive from the model of gender inequality, and so radical feminists don’t believe that problems like sex work can be solved through changing labour relations (whether radically, as in the case of feminists influenced by Marxism, or through the institutions of civil society, as in feminists influenced by the politics of the labour movement[2]). Instead, they see sex work as the most vulnerable link in a chain of social structures where women are dominated by men, and through public policy they see an opportunity to attack the underlying structures of the sexual relations of our society through attempts to abolish the sex industry. Unlike the prohibitionists of previous eras, they see prohibition as an opportunity to change the moral under-pinnings of gender relations, rather than to protect the moral fabric of existing society; but in both cases, they see public health, and laws affecting sex workers, only in terms of its relevance to the moral debate that concerns them. This means that they instrumentalize sex workers as a tool of public policy in the pursuit of their own moral goals, rather than treating them as fully independent people deserving of dignity in their own right. In my final piece in this series I will attempt to show why I think this similarity is not a coincidence, and derives in both cases from an inability to accept different perspectives, especially those of poor and non-white women. But first I will digress a little, to discuss the problem of sex trafficking. Things can only get more controversial from here …

    fn1: Ekberg, G. The Swedish law that prohibits the purchase of sexual services: Best practices for prevention of prostitution and trafficking in human beings. Violence Against Women. 2004; 10(10): 1187-1218.

    fn2: Sullivan, B. Feminist approaches to the Sex Industry. Proceedings, Conference on Sex Industry and Public Policy. Australian Institute of Crimonology, 6-8 May 1991. Available online (with many other interesting links) here.

  • Charlie Brooker, the British screenwriter, zombie reality TV expert and culture commentator for the Guardian, is doing a series of articles on Japan. I wouldn’t usually care but I quite like Charlie Brooker’s style of criticism, usually directed at television culture, which is ascerbic and filthy but also well educated and very fond of the medium (TV) that he mostly writes on. His cultural commentary can be a lot of fun and occasionally insightful, and certainly his first article on first impressions of Japan contains a few, such as his description of a lot of Japanese TV:

    Imagine watching an endless episode of The One Show with the colour and brightness turned up to 11, where all the guests have been given amphetamines, the screen is peppered with random subtitles, and every 10 seconds it cuts to a close-up shot of a bowl of noodles for no apparent reason. That’s 90% of Japanese TV right there.

    However, I’m concerned that he’s going to fall back on the same tired tropes that always get trotted out to describe Japan by westerners, especially those just visiting or who don’t have at least a passing familiarity with the language, and especially especially British and American commentators, whose level of introspection about their own cultures is, in general, profoundly lacking. The common tropes tend to be a combination of weirdness, exoticism, and a sense that you’ve stepped back in time to an earlier cultural period in the west, which almost certainly never actually existed. He certainly doesn’t start or end well, with both the opening and closing sentences describing Japan as “another planet.” He goes on in the first paragraph to say

    while the world around you is largely recognisable, it somehow makes little sense

    This is the classic expression of the cosseted western view. When did western cultural commentators decide that their own country is the arbiter of what “makes sense”? Once you’ve lived in Japan for a little while you start to see a lot of things about western life that definitely make no sense: when I watched TV in the UK and saw adverts for furniture, for example, inevitably some idiot actor would flop onto a couch and put their fully-shod feet up on it. Since I’ve lived in Japan I’ve come to realize that this is a truly disgusting habit, and it makes no sense that we in the west ever conceived of wearing our shoes into the house as a good idea. Perhaps, then, instead of phrasing things in terms of a culture that is full of “sense” (the one Brooker came from) and one that isn’t, Brooker could talk merely in terms of difference? And while he’s at it, learn to take his shoes off inside.

    So already Brooker has established Britain’s cultural mores as the background from which all else deviates, and has portrayed the Japanese as alien and strange (incomprehensible, even). His green kit-kat comment follows the same pattern: kit-kats as representative of British cultural norms, are rendered green in Japan for no apparent reason. It’s left to the people in comments to mention that the chocolate is green because it is tea flavoured, a common practice in Japan, but from the body of the text we’re left to assume that the Japanese just like to make western chocolate green for no reason. Here we see the essence of the depiction of the other as strange: present something they do as an idiosyncratic or incomprehensible phenomenon and avoid a description of the extremely simple reason for the action.His description of TV also contains an element of this: those subtitles aren’t random, Charlie, because by definition sub-titles are not random. They are the words that the person speaking is saying. As the Suicidals once famously said: “Just beause you don’t understand it don’t mean it don’t make no sense.” In this case, the thing you don’t understand is this thing called “language” and you should ask yourself how you would feel if an Asian were commenting on the “randomness” of elements of your own culture’s TV without knowing a single word of English.

    This perhaps also is what underlies his segue to a full two paragraphs of quite gross description of Japanese toilets. Why are the British focused on toilets? And whatever gave Brooker the impression that, as a member of a nation whose public toilets (not to mention its chocolate!) are universally poor-to-terrible, he is the best person to judge Japan’s extremely high standards of hygiene? Of course, toilet habits are a fundamental example of the way in which cultures differ, and a culturally introspective look at Japanese toilet habits could be an ideal opportunity for a Londoner like Brooker to discover that actually, his own culture has a lot to learn on this front. But instead it’s again a way of depicting the Japanese as weird and different, and these two paragraphs manage to incorporate a nod to the classical/modern binary of Japanese life, a good bit of British toilet humour, and bemusement at Japanese weirdness, all in one. To his credit Brooker finishes it with a sentence about machines overthrowing humans that serves to reunite Japanese and British as having cultural commonality; this is nice. But there is no chance to compare this with a British pub toilet – I bet Brooker doesn’t dare take a crap in your average British pub toilet, as just the stink alone would hurt his brain.

    The remainder of the article, however, is good stuff, giving impressions of TV from the perspective of someone who apparently doesn’t speak any Japanese. Once he’s on his favourite ground (TV commentary) Brooker ditches the cultural-analysis stereotypes and manages to give a fairly nice description of how Japanese TV looks if you don’t understand Japanese. He also is much more introspective here, making jokes about crazy Japanese game shows without missing the point that reality TV is just as degrading and terrible a phenomenon. The use of the word “yelping” is a bit unfortunate in the context of a man in a country where he doesn’t understand the language, but overall it’s good. I think he’s wrong about the content of Japanese TV ads though: they aren’t mostly about food, they’re mostly about hair products.

    Anyway, I’ll be watching this series of articles by Brooker with interest to see if he can rise above his colonialist heritage to give a genuinely interesting analysis of Japanese cultural life. I think he can do it, though I’m doubtful about whether he’ll be at all aware of how much he privileges his own cultural viewpoint. Japan is an almost completely blank slate to the British – the “far East” is something they know almost nothing about, in my experience. If he can give them a slightly deeper insight into Japan than “they’re weird and nothing makes sense” then he’ll have achieved something. Here’s hoping …