March 28, 2012

As one industry expert recently observed, on Wall Street, “It’s all about the data.” But for financial companies to win the “faster, smarter, dirtier” game of Big Data, they must make data more actionable. That is, financial firms have to be able to put data to work in a way that informs smarter business decisions that help the business change and grow.

To some, that may sound obvious – it’s hard to imagine a firm that would want its data to inform really bad business decision making. But the reality is that most firms don’t have a clear enough understanding of how their data is managed therefore they can’t paint a clear picture of why that data is so important. More to the point, most companies simply don’t have the right systems in place to provide the kind of transparency into data to make the information truly meaningful.

So there’s a lot of talk about Big Data and how companies need to put it to work, but the reality is that so far, many firms have been at a loss to understand why and how to deal with it.

Managing Big Data: Why It Matters Effective Big Data management will be the defining issue of the financial marketplace in the year to come. This is due to a number of reasons, including:

• Data volume + regulations = big headaches – The challenge to the financial industry is not just how “tough” regulatory requirements are, but how tough it is to answer to them. This is made infinitely more difficult by the massive volumes of trade and transaction data that companies have to manage every day. For example, the Dodd Frank requirement to report derivative trades in near-real time will require superb Big Data efficiencies for financial services companies. The challenge is, how do you measure the risk associated with your trades when each trade is associated with many gigabytes of data?

• Business users + IT users = competing priorities – From the trading desk to the analyst function, business users want fast and easy access to information that they can put to work immediately in risk models and other applications. But IT users need to ensure they have recognizable and replicable logic flows and rules associated with data so that it moves from one part of the organization to another whilst maintaining integrity. Higher volumes of data make this competition more intense.

• Cost of data storage + concerns over data security = a wary market place – Operations and IT infrastructure experts know that the cost of managing data is going up, even when the cost of storage is relatively low, because of the amount of information that has to be managed and cached. Although the cloud offers the potential for a firm to reduce its Big Data footprint, many are still unconvinced about the security of data in the cloud. In addition, some worry that cloud-based data might not be as easily accessed for analysis and manipulation in databases as it needs to be for fast decision making.