3 key trends in big data analytics

Like it or loathe it, we’re in the big data analytics space.  So we reflect continuously on it.  We ask “what are the 3 key things going on in big data analytics”?

As data volumes explode – and become big data
the challenge lies in their complexity not their  size.

However large a database is, asking the question ‘how many’ is really not challenging any more.

Asking complicated questions, or even more problematic, identifying the questions to ask, these are the challenges we face. Size has little to do with that.

Recognising this, some argue that the scientific paradigm is past its best.   These big data zealots argue that we should rely on the correlations offered by big data analytics and forget questions of causality. If the data suggests that storks bring babies, then they do.

I can’t over estimate my nervousness at this data determinism.

It’s foolish, wasteful of resource as it leads us down wrong turnings and downright dangerous.  That’s not to say that correlation isn’t important.  We know that smoking kills because Sir Richard Doll noted the correlation.  But we needed to understand the causality before condemning the practise, and it’s the understanding that makes it difficult for the tobacco industry to defend smoking.

As governments release more data and as commercial data becomes more widespread,
the challenge is in blending it with customer data to secure unique insight.

There’s no getting away from it, Governments have the best data.  Developed countries know most about their populations and increasingly they’re publishing what they know. So whether a business is micro-targeting a new product or thinking about opening a pizzeria, it’s the State that can identify the best spot.  Managing the blend of government data with yours requires knowing what’s out there in some detail, real business acumen to see the potential and technical skill actually to do the blending.

Decision makers, as well as data scientists, want access to data – anytime, anywhere.

Or, big data and the analysis of it, must be cloud based. That means good data governance and the development of competitors to the US domination of the big data scene – for those who don’t want their data sucked up by the National Security Agency

There’s another meaning to the word ‘access’.  The traditional divorce between data and decision maker is organisationally damaging.  Data scientists are not necessarily the best placed people to recognise the business significance in data; to ask the right questions; recognise appropriate answers.

Inevitably, data scientists make guesses about organisational priorities.

The challenge is in making data access intuitive, secure and informative.  To get ‘Big Insight’ from ‘Big Data’.