Freakonomics and User Errors

| |

This week’s featured reviewer is Nigel Magson. Nigel is the Founder and Managing Director of Adroit consultancy. He specializes in data application, data management, system design and data analysis. If you haven’t already, check out his in-depth article about KXEN. Contact us if you would like to be one of our guest bloggers.

I’ve reached that age…already.  Finding a university place for my son – yes, a rather disturbing thought, but one that Nigelrekindles memories of my own steps away from home into the world of work. So last week we trouped off to an open day at Bristol University.  In the Wills lecture theatre we listened to what studying Economics at Bristol would be like.  The lecture was punctuated with less than subliminal slides warning “If you don’t like maths, don’t study at Bristol”. That’s ok, Floyd, my son is doing A level maths, and says he enjoys it. The Lecturer picked, Freakonomics – Levitt’s pop culture meets economics treatise – to dissect, and in particular the chapter where Levitt argues that an increase in the abortion rate decreases the crime rate. Soon the various datasets were appearing on the board, and the statistical code which Levitt purportedly used, the associated stats tests, coefficients, p-values etc.. The lecturer exposes the programming and statistical errors and flaws in Levitt’s theory. There is no significant relationship in the data, Levitt got it wrong, concludes the lecturer with a satisfied academic smugness. His theory is bunkum.

As I research afterwards, the lecturer was by no means the first to show this (and he didn’t claim to be). Whilst I’m loving it, I sense my son has drifted off, somewhat phased by the amount of stats and some of the arcane elements. Afterwards, I try to explain one of the projects we’re involved in, income forecasting using a reporting engine we’ve written in VBA on top of Excel and cranking survival curves in a stats engine. He’s not listening (whose son listens to their Dad anyway?), we head off to Modern Languages department…

User errorI reflect. We help clients gain insight from their data, sometimes through doing the work, or setting up training them and helping them run analysis systems such as Apteco’s FastStats, Smartanalyser or SPSS. They also end up making mistakes in their application of the tools, or like Levitt, getting the code wrong. Well, they do. We’ve all seen it, and this can cost their organisations millions in lost opportunities or more likely simply go unnoticed. The human part. So replace user? Not that easy. Positively we’re part of an industry out there to help them maximise their investment, and avoid the errors.

I suspect know many of the clients I have worked with in selecting or even using tools or software have similar reactions to my son – and sometimes that’s understandable (if it’s not their day job and we go “geeky” on them). So now we have IT Central Station – a great place to access and share reviews. As reviewers, our responsibility is to avoid them glazing over, to keep them listening, and as recipients we must try not glaze to listen… that of course, is easier said than done.

Read reviews of Data Mining Solutions from real users at IT Central Station. See reviews of SPSS, KXEN, and other Data Mining products.


The Weekly Roundup: Oracle Business Intelligence Enterprise Edition (OBIEE) Reviews

Fooled By Data Visualisation