Wall Street & Technology is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Data Management

10:43 AM
Alex Raitt
Alex Raitt
Connect Directly

Big Data in Capital Markets Crosses the Chasm

The difficulty in transplanting big data applications and their benefits from other industries has created a vacuum.

Alex Raitt, SunGard Consulting Services
Alex Raitt, SunGard Consulting Services

Despite the hype of big data and its associated technologies, large scale adoption within capital markets has proven elusive. One of the main reasons most securities firms have been slow to adopt big data strategies is that the capital markets industry does not lend itself to the consumer-driven applications seen in other industries. Typically, with some notable exceptions in equities business, capital markets firms do not have internet scale challenges with data volume or the same level of unstructured data. Technology focus is often elsewhere – on improving real-time systems, transaction processing, operational efficiency and establishing data warehouses of structured data. The difficulty in transplanting big data applications and their benefits from other industries has created a vacuum.

That’s about to change. As Geoffrey Moore suggests in his book Crossing the Chasm, with the adoption of discontinuous or disruptive technologies, there is a gap or chasm between innovators, early adopters and the early majority, and that’s where big data is today in capital markets. Recent pilot projects are starting to bear fruit, the education gap – where an intimate knowledge of the business needs to be combined with an understanding of statistical computing methods to generate that creative spark – is closing, and the industry is increasingly realizing that it’s in the advanced data analytics that the much promised disruptive effect will be found. We are on the verge of big data crossing Moore’s chasm, and when it does, we’re going to see big data technology used in novel, and sometimes unexpected, areas.

For example, big data is going to be increasingly embedded within decision support systems to increase automation of the decision-making process, as well as to enhance (predictive) analytics’ ability to provide additional insight. A form of this already exists in the equity front office with the use of algorithmic trading, and this pattern is now starting to be found in other businesses as well. In the credit line approval process, for example, internal structured and unstructured data can be combined with external data (like client prospectuses) in a big data repository. This helps streamline low risk approvals and thereby frees up staff for more complex decision-making tasks.

Big data will also play a large role in improving operational efficiency. Trade surveillance, for example, is experiencing increased regulatory and public scrutiny and the multitude of available communications channels in today’s digital world is making it all the more difficult for firms to manage. Big data technology will play an increasing role in firm’s response to monitor its trading activity. The ability to consume different channels and types of data – including instant messages, phone recordings, emails and blog entries – and consolidate this structured and unstructured data into a usable database will allow advanced pattern matching analytics to spot anomalous behavior.

The capital markets industry will also increasingly leverage big data for enhanced business intelligence by employing techniques including sentiment analysis and advanced analytics. Customer service communications with institutional investors, across a multitude of channels, can be recorded and sentiment measured, leading to more proactive handling of potential issues. And by consuming external data sources, a firm’s own analytics can be improved. Mortgage analytics, for example, can now consume household economic data at a neighborhood level of granularity to provide more accurate pricing of mortgage backed securities in some markets.

There are still challenges to overcome for more widespread adoption of big data in capital markets. The education gap needs more time to close and new technology skillsets time to mature, but as more and more successful deployments occur (and lessons are learned from those that fail), big data technology is set to cross the chasm. The applications discussed here will be just the tip of the iceberg.

For more on the disruptive effect of big data technology, read SunGard’s report, "Capital Markets Embrace Big Data.”

Register for Wall Street & Technology Newsletters
5 Things to Look For Before Accepting Terms & Conditions
5 Things to Look For Before Accepting Terms & Conditions
Is your corporate data at risk? Before uploading sensitive information to cloud services be sure to review these terms.