Friday 21 June 2013

Why the Wimbledon draw is a warning to policy makers


 Seeding matters in tennis.  Get the seedings – the assessment of who is most likely to win the tournament - wrong, and the public are denied the best final they might see – as happened at the French Open two weeks ago.  The tournament organisers at Wimbledon have just committed that error and produced an unbalanced draw – though less a bad one than there might have been.

Most of the world’s top tennis tournaments apply a simple rule.  They seed players according to their current place in the world rankings – which are calculated on a player’s performance over the past year.

Wimbledon doesn’t do that.  It has always said that, since the grass court season is so short, and thus plays so little part in determining the rankings, they reserve their right to adjust to recognise grass court performance.  For the men’s tournament they have an algorithm to do that.  For the women’s they exercise judgement – and that has led them in the past to bump the formidable Williams sisters up the seeding chart.

But even with these get-out clauses, the sports pages are full of condemnation of the decision to seed former champion, Rafael Nadal at No.5.

Data driven

The reason for Nadal’s seeding is quite simple.  Despite winning the French Open, Nadal is currently at No.5 in the world.  That is what the data says and data does not lie.

Context matters

But there is a reason for Nadal’s lowly ranking – he spent 7 months of the last season at home in Mallorca unable to play.  So the people ahead of him in the rankings have six months more points than him.  The fact that he is fifth in the rankings despite being absent for half the year is pretty amazing.

Formulaic adjustment

So why is Nadal not benefiting from the adjustment formula?  Because that too is very rules bound.  It takes account of performance on grass.  At first sight Nadal – twice a winner at Wimbledon (remember 2008 and his gloomy vanquishing of the seemingly undefeatable Federer), and four times a beaten finalist looks to have a pretty good claim to be bumped up.  If he were a woman he would be.  But the men’s formula looks at “recent” performance on grass – and last year Nadal’s knees were beginning to go – and he made shock early exits in the two grass tournaments he played and gave the Olympics a miss.  His earlier performances count for nothing.

Applying judgement

No tennis pundit thinks Nadal is the fifth most likely person to win the tournament.  Pre-draw betting markets had him ranked second alongside homegrown favourite Andy Murray, and ahead of both Federer and his compatriot, who has never won a Grand Slam (Rafa has 12) David Ferrer.   The smart move would be to seed him third or fourth which would have made sure there was no risk of him playing the top two in the quarter finals – and also avoiding what has happened – a quarter final versus Federer.

Off the courts

There has been much discussion on the benefits of more data driven decision making – in sport and in policy.   And in general, basing policy on data is much better than relying on prejudice alone.

But the case of the Nadal seeding adds a caution.  Data alone only tell you so much.  You also need to look behind the data and apply judgement to get the most sensible result.  This is the point US forecaster Nate Silver makes in his book “The Signal and the Noise” – you need to start with a hypothesis you refine as more data becomes available.   

This view also lines up with the points Jeremy Hardie and Nancy Cartwright made at Institute for Government on Monday about the way to use evidence in practice.  If you don’t understand the key elements that drive a result, successful replication will be impossible.  What looks like the “same” policy will turn out to have very different results if you misunderstood what bits of sameness mattered.  They called for the new What Works centres to develop guidance on how to make what works somewhere work somewhere else. And you need to find different ways of looking at the same problem to sense check your results.

If Wimbledon had looked at an alternative measure of performance – the rankings of performance in 2013 - they would have found that Nadal isn’t the fifth ranked player in the world.  By a considerable margin, he is the best.

No comments: