Many questions about big data have yet to be answered in a vendor-neutral way. With so many definitions, opinions run the gamut. Here I will attempt to cut to the heart of the matter by addressing some key questions I often get from readers, clients and industry analysts.

What is the role of intuition in the era of big data? Have machines and data supplanted the human mind?

Contrary to what some people believe, intuition is as important as ever. When looking at massive, unprecedented datasets, you need someplace to start. In Too Big to Ignore , I argue that intuition is more important than ever precisely because there's so much data now. We are entering an era in which more and more things can be tested.

Big data has not replaced intuition -- at least not yet; the latter merely complements the former. The relationship between the two is a continuum, not a binary.

A key piece of big data is its reliance on "unstructured" and "semi-structured" data. Can you explain what's going on here?

Roughly 80% of the information generated today is of an unstructured variety. Small data is still very important -- e.g., lists of customers, sales, employees and the like. Think Excel spreadsheets and database tables. However, tweets, blog posts, Facebook likes, YouTube videos, pictures and other forms of unstructured data have become too big to ignore.

Again, big data here serves as a complement to -- not a substitute for -- small data. When used right, big data can reduce uncertainty, not eliminate it. We can know more about previously unknowable things. We can solve previously vexing problems. And finally, there's the Holy Grail: Big data is helping organizations make better predictions and better business decisions.

NEXT: Data visualization is becoming more popular than ever.

... Read full story on InformationWeek


Post a comment to the original version of this story on InformationWeek