Data Management

01:10 PM
Greg MacSweeney
Greg MacSweeney
Commentary
Connect Directly
LinkedIn
Twitter
Facebook
Google+
RSS
E-Mail
50%
50%

Petabytes of Data? Analyzing Endless Data To Create Value

As the size of the data warehouse grows, Wall Street firms must invest as heavily in data analytics as they do hard disks.

Gigabytes, terabytes, petabytes? (You lost me at gigabyte, by the way.) Financial firms are scrambling to handle terabytes and even petabytes of data. But why?

Many don't know exactly why they're ramping up their storage capabilities, other than because they've been told that using all the data that's available will provide insight that will help them make better business decisions, and ultimately lead to happier customers and a healthier bottom line.

Well, color me skeptical. Yes, diving into data can uncover some surprising things about a business. The numbers don't lie and mining data can help improve business processes in a variety of ways.

[Video: The Future Of Data Management]

For instance, before sabermetrics became popular in baseball, very few teams actually did much with their data. Baseball is one of the most data-intensive sports, and each team had volumes of stats on how players performed with, say, two strikes or with runners in scoring position or in day games versus night games. Every team had access to the same data, but very few did anything with it.

Today, almost every team in baseball uses statistical analysis to some extent to improve performance. Previously, teams didn't know what they could do with the data or how to analyze it. Thus, it provided little value to baseball franchises.

Data Heavy, Insight Light

Capital markets organizations have been data heavy and have always used data to analyze investment opportunities and, especially since the financial crisis, to monitor and uncover risks.

Today, however, firms are looking for new types of data in order to find new insights and to gain an edge on the competition. Many are considering unstructured data and information from a variety of sources -- Web logs, mobile tracking, weather and census data, for example -- to help create an advantage. In order to gather all of this data, terabytes and possibly even petabytes (or dare we say an exabyte -- 1 billion GB) of storage are going to be needed.

To have a petabyte of data is a remarkable feat, and many firms are looking to big data technologies and data analytics engines to mine all that data. But if a firm doesn't know what to do with the data it gathers, that isn't much different from the baseball teams sitting on mountains of data for decades, not knowing how to get any value out it. After all, it only took about 80 years for sabermetrics to take off and change the way the game of baseball is played.

For banks, the trick will not be to amass the largest data store. The goal will be to figure out ways to mine and analyze the sometimes-unrelated data sets to create some sort of value. As the size of the data warehouse grows, firms must invest as heavily in data analytics as they do hard disks.

Greg MacSweeney is editorial director of InformationWeek Financial Services, whose brands include Wall Street & Technology, Bank Systems & Technology, Advanced Trading, and Insurance & Technology. View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Jackie Alvarez
50%
50%
Jackie Alvarez,
User Rank: Apprentice
4/8/2013 | 2:58:49 PM
re: Petabytes of Data? Analyzing Endless Data To Create Value
We are an information business and the business with the best
information is going to be the winner in the long term. In today's era
most firms have been focused on instituting best practices to streamline efficiency. But creating efficiency is not a competitive strategy.

As business optimism increases and firms experience a renewed confidence, I anticipate firms will begin moving from focusing on efficiency, and keeping the lights on, to defining how they can compete more effectively into the future. This is where information strategy can make a significant contribution to the aspirations of the firm and help them become "smart" companies. Being smart is about acquiring managing, mining and distributing information will can have an enormous impact your firm's ability to meet it's business objectives.

But, as with most programs, firms will evaluate both the short- and long-term costs associated with big information projects and get executive buy-in, track metrics to make sure they stay on track, govern their programs to reduce chaos, and deliver small projects frequently to stay on course. Above all, buy-side firms need to remain agile to take advantage of new technologies. A tough program to do well and one which remains elusive for most firms.
Greg MacSweeney
50%
50%
Greg MacSweeney,
User Rank: Apprentice
2/5/2013 | 4:34:59 PM
re: Petabytes of Data? Analyzing Endless Data To Create Value
The larger firms traditionally would have done this internally, but budget pressures may force them to look outside. It makes sense to bring in some expertise, since the many purchased data analysis apps are extremely powerful now. Smaller firms will likely look to providers to help them do this...the costs are too great and the ramp up take a long time.
ischmerken
50%
50%
ischmerken,
User Rank: Apprentice
2/5/2013 | 4:20:08 PM
re: Petabytes of Data? Analyzing Endless Data To Create Value
-áThis sounds like a big undertaking but something that financial services firms will explore, especially those with a quantitative/statistical orientation. As you said, many firms are sitting on top of mountains of data about their customers and transactions. Do you expect firms to build the expertise for analyzing pedabytes of data inhouse? Or do you see them partnering with analytics providers on the outside?
Ivy Schmerken
50%
50%
Ivy Schmerken,
User Rank: Apprentice
2/5/2013 | 4:20:08 PM
re: Petabytes of Data? Analyzing Endless Data To Create Value
-áThis sounds like a big undertaking but something that financial services firms will explore, especially those with a quantitative/statistical orientation. As you said, many firms are sitting on top of mountains of data about their customers and transactions. Do you expect firms to build the expertise for analyzing pedabytes of data inhouse? Or do you see them partnering with analytics providers on the outside?
More Commentary
The BYOD Challenge
Having a policy in place to manage mobile devices used by employees for work purposes is necessary in this current day.
Getting Onboarding Right in the Age of the Customer
Disparate “Frankenstein” systems slow down the onboarding process and impede customer service, says Pegasystems.
Performance Monitoring Key to Smooth Infrastructure Modernization
As banks consider how to shift infrastructure and storage solutions, they can’t afford to lose visibility into performance.
Accuracy in the Financial Data Supply Chain
Some of the best-practices from the retail world’s physical supply chain, including data standards and visibility, can be leveraged for efficiency in financial services.
Business Continuity 2.0: We’re Gonna Need a Bigger Boat
What would happen with a long-term outage to financial systems or the nation’s critical infrastructure? Businesses aren’t even close to being prepared.
Register for Wall Street & Technology Newsletters
White Papers
Current Issue
Wall Street & Technology - July 2014
In addition to regular audits, the SEC will start to scrutinize the cyber-security preparedness of market participants.
Video
5 Things to Look For Before Accepting Terms & Conditions
5 Things to Look For Before Accepting Terms & Conditions
Is your corporate data at risk? Before uploading sensitive information to cloud services be sure to review these terms.