Big Data, Small Data – The More Data, the more we learn that our “Common Sense” may not be all that correct

Jane Aubrey: Do you lose very much?
Billy Chapel: I lose. I’ve lost 134 times.
Jane Aubrey: You count them?
Billy Chapel: We count everything.
From the movie, For Love of the Game


There it is , on the front page of this morning’s Dallas Morning News. (I had already seen the article on-line yesterday). The former contestants on The Biggest Loser, who had lost so much weight, have pretty much put it all back on. (Read the article here).

Call this one of the new findings from the era of much better, longer-term measurement. We are measuring everything. And we are learning that what we thought would work may not work after all.   (From this article, exercise is not enough; and cutting calories may simply slow down metabolism).

Small DataYes, we really are counting everything these days. We hear the phrases: “big data,” “small data.” Here’s a quote from Chip Heath, in his foreword to Martin Lindstrom’s Small Data: The Tiny Clues that Uncover Huge Trends:

Our businesses will not improve through Big Data alone. We need to follow Martin and explore Rich Data. Even if it comes in the form of Small Data. Our businesses will be better for it.

But, here’s the point. It requires data – lots of data! We are getting better and better at accumulating and collecting such data, and slowly we are getting better at analyzing and interpreting that data.

And now we are slowly realizing that one of the problems facing us is that what we thought would work – call it the “common sense” approach to things – may not work at all. (Like, exercise, eat fewer calories… alas, it appears that losing weight and keeping it off is not that simple).

There are many examples of this. Here in Dallas, we are in the process of shutting down the “tent city” that sprung up among the homeless. The real problem – not enough homes for the homeless. And over the years, we have tried a number of “common sense” solutions that have not worked. They make sense, we think. And then, they do not work. (Read this excellent article, also in the Dallas Morning News, by Larry James, CEO of CitySquare: Larry James: 6 frustrating things about closing Tent City).

The point of this post is simple. State your opinions. But, your opinion (and mine) really will be better informed by finding and analyzing the available data.

And, beware – tomorrow’s data may throw quite a curve into today’s data.

After all, every sample size keeps getting bigger and better.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s