Wall Street & Technology is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management

01:10 PM
Greg MacSweeney
Greg MacSweeney
Commentary
Connect Directly
Facebook
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Petabytes of Data? Analyzing Endless Data To Create Value

As the size of the data warehouse grows, Wall Street firms must invest as heavily in data analytics as they do hard disks.

Gigabytes, terabytes, petabytes? (You lost me at gigabyte, by the way.) Financial firms are scrambling to handle terabytes and even petabytes of data. But why?

Many don't know exactly why they're ramping up their storage capabilities, other than because they've been told that using all the data that's available will provide insight that will help them make better business decisions, and ultimately lead to happier customers and a healthier bottom line.

Well, color me skeptical. Yes, diving into data can uncover some surprising things about a business. The numbers don't lie and mining data can help improve business processes in a variety of ways.

[Video: The Future Of Data Management]

For instance, before sabermetrics became popular in baseball, very few teams actually did much with their data. Baseball is one of the most data-intensive sports, and each team had volumes of stats on how players performed with, say, two strikes or with runners in scoring position or in day games versus night games. Every team had access to the same data, but very few did anything with it.

Today, almost every team in baseball uses statistical analysis to some extent to improve performance. Previously, teams didn't know what they could do with the data or how to analyze it. Thus, it provided little value to baseball franchises.

Data Heavy, Insight Light

Capital markets organizations have been data heavy and have always used data to analyze investment opportunities and, especially since the financial crisis, to monitor and uncover risks.

Today, however, firms are looking for new types of data in order to find new insights and to gain an edge on the competition. Many are considering unstructured data and information from a variety of sources -- Web logs, mobile tracking, weather and census data, for example -- to help create an advantage. In order to gather all of this data, terabytes and possibly even petabytes (or dare we say an exabyte -- 1 billion GB) of storage are going to be needed.

To have a petabyte of data is a remarkable feat, and many firms are looking to big data technologies and data analytics engines to mine all that data. But if a firm doesn't know what to do with the data it gathers, that isn't much different from the baseball teams sitting on mountains of data for decades, not knowing how to get any value out it. After all, it only took about 80 years for sabermetrics to take off and change the way the game of baseball is played.

For banks, the trick will not be to amass the largest data store. The goal will be to figure out ways to mine and analyze the sometimes-unrelated data sets to create some sort of value. As the size of the data warehouse grows, firms must invest as heavily in data analytics as they do hard disks.

Greg MacSweeney is editorial director of InformationWeek Financial Services, whose brands include Wall Street & Technology, Bank Systems & Technology, Advanced Trading, and Insurance & Technology. View Full Bio
More Commentary
A Wild Ride Comes to an End
Covering the financial services technology space for the past 15 years has been a thrilling ride with many ups as downs.
The End of an Era: Farewell to an Icon
After more than two decades of writing for Wall Street & Technology, I am leaving the media brand. It's time to reflect on our mutual history and the road ahead.
Beyond Bitcoin: Why Counterparty Has Won Support From Overstock's Chairman
The combined excitement over the currency and the Blockchain has kept the market capitalization above $4 billion for more than a year. This has attracted both imitators and innovators.
Asset Managers Set Sights on Defragmenting Back-Office Data
Defragmenting back-office data and technology will be a top focus for asset managers in 2015.
4 Mobile Security Predictions for 2015
As we look ahead, mobility is the perfect breeding ground for attacks in 2015.
Register for Wall Street & Technology Newsletters
Video
5 Things to Look For Before Accepting Terms & Conditions
5 Things to Look For Before Accepting Terms & Conditions
Is your corporate data at risk? Before uploading sensitive information to cloud services be sure to review these terms.