It seems to be that the blooming of Big Data and suchlike hasn’t in any way coincided with a commensurate uptick in the prescience of political polling. If anything, pre-election, professional prediction appears merely to be holding steady on its success rate vis-a-vis the examination of chicken entrails. Hot on the tail of crisis talks in the polling industry to understand the complete Horlicks they made of the 2015 UK general election (sorry about the technical jargon), their performance in the US presidential race isn’t doing anything yet to restore reputations. So, fun and games for us all to point and laugh. Unless of course what we do relies in any way on using data to understand or plan anything involving people. In which case one is forced to consider: am I doing any better than the pollsters?
So what’s going on? Why doesn’t more data necessarily lead to better insights or decision-making?
I like a bit of BBC Radio 4. Principally it’s there for The Today Programme and Test Match Special but when I tune in at any point of the day – while the kettle’s boiling or I’m in the car waiting to pick up my son or daughter – there’s always something to learn or laugh at: unless, of course, it’s the authored tinnitus of The Archers.
So I was listening to Woman’s Hour around the time of the Women of the World Festival and learned that one of the speakers at London’s South Bank would be Kimberlé Crenshaw and her topic would be Intersectionality, a term she coined in 1989. Intersectionality Theory grew out of Crenshaw’s personal experience as a black woman: that the discrimination she faced was more than the sum of her race and sex but the two interacted to reinforce each other. Intersectionality would contend that another woman could not say that she understood Crenshaw merely by dint of shared gender. As Crenshaw put it we are not simply the sum of our experiences but that:
‘we are the product of our connections.’
Another way of thinking about this is that it is the interaction of our defining characteristics that does the defining.
This takes some thinking about because the numbers and types of experience and connections that any one person might have seem to be expanding. When everyone in the middle of France lived in caves, the experience of everyone living in the middle of France was pretty homogeneous. In the UK population of the 1950s there was a much broader set of experiences and world views, but if you were able to pin down a couple of socio-economic markers, most people within those categories would have had a very similar life. Today, though, if you sit at the canteen table and talk to co-workers about their lives and family histories you’ll soon appreciate that whilst a pollster might see you as being part of telling, common datasets – people of working age, employed in your industry, living in your area – the breadth of life’s experiences is wonderfully diverse. And throwing more data at it doesn’t help because the closer you look, the pattern of an individual’s behaviour and belief is imprinted in ever finer details. Fractal-like depths that data can’t plumb.
The answer is at hand though. In fact, it’s in the preceding paragraph. If you want to understand people, their character and motivations, then sit down and share stories with them. This might be literally, in the canteen for example, or virtually. Although if you choose the latter route, don’t be fooled by that apparent ease of digitally ‘connecting’, ‘linking’ and ‘befriending’. You’ll have to make the running just the same: go to someone, show interest in their life, listen, respond. After all, I might pop up as a fan of Today but not The Archers in market research, but I wouldn’t even self identify as a Woman’s Hour listener let alone sit comfortably in their audience profiling data. Read the four line story of my day that is paragraph three above though, and you can infer an awful lot about me, my connections and experiences.