Big data and great peril

Sir Galahad: Look, let me go back in there and face the peril.Sir Lancelot: No, it's too perilous. Sir Galahad: Look, it's my duty as a knight to sample as much peril as I can. Sir Lancelot: No, we've got to find the Holy Grail. Come on. Sir Galahad: Oh, let me have just a little bit of peril? Sir Lancelot: No. It's unhealthy.

- Monty Python and the Holy Grail

I couldn't help thinking of this scene in one of my favorite movies the whole time I was reading Vivek Wadhwa's excellent article in the Washington Post: "The rise of big data brings tremendous possibilities and frightening perils".

That, it does. I'll summarize (but read the article if you can):

  • "Big data" means the mindbendingly freaking ginormous amount of data coming to an electronic device near you, in the near future. That number is already somewhat more than we can get our heads around, and it's growing exponentially by the day (I won't quote it here because it will have changed by the time you read this, but check here).
  • The more data we have about something or someone, the more inferences we can make about that something or someone, and the more accurate those inferences will be. Right? So, the better quality decisions we can make based on that data. (That's assuming we have the tools to do that, but we will).
  • That's awesome. We'll be able to do all kinds of things that were impossible in the past, like offering truly personalized education, health care, consumer products, and other things as yet unimagined, for all and everyone. Everything Peppers and Rogers foretold, and then some (if you're not old enough to remember Peppers and Rogers, go ahead and move along to the next bullet).
  • That sucks. We'll be able to do all kinds of things that were impossible in the past, like spy on people in ways never imagined. You think what Snowden leaked about the NSA was something? It was nothing compared to what's coming.

Quoting from Wadhwa here:

"We will be revisiting crime cases from the past, re-auditing tax returns, tracking down corruption, and learning who were the real heroes and villains. An artificially intelligent cybercop scanning all the camera data that were gathered, as well as phone records, e-mails, bank-account and credit-card data, and medical data on everyone in a city or a country, will instantly solve a crime better than Sherlock Holmes could. Our grandchildren will know of the sins we committed; Junior may wonder why grandpa was unfaithful to grandma.

What is scary is that we will lose our privacy, opening the door to new types of crime and fraud. Governments and employers will gain more control over us, and have corporations reap greater profits from the information that we innocently handed over to them. More data and more computing will mean more money and power. Look at the advantage that bankers on Wall Street have already gained with high-frequency trading and how they are skimming billions of dollars from our financial system.

We surely need stronger laws and technology protections. And we need to be aware of the perils. We must also realize that with our misdeeds, there will be nowhere to hide—not even in our past.

Not all that good.

Here's what I think will happen. Security will be essentially ignored until a major incident takes place ... someone losing their life or livelihood because of big data ... a president or CEO brought down, something like that. Then, reputations will be damaged, the regulators will step in, and we will retrofit security and privacy into our data, tools and devices, at a princely sum. It always goes that way (went that way with e-commerce), which is good for security pros and cyber criminals, but pretty much nobody else.

Data is the new oil, we're told. There will be spills to clean up, some of them big, like that one in the Gulf of Mexico in 2010. All of us will have to go back in there and face the peril. No matter how unhealthy it is.