That’s about to change. As Geoffrey Moore suggests in his book Crossing the Chasm, with the adoption of discontinuous or disruptive technologies, there is a gap or chasm between innovators, early adopters and the early majority, and that’s where big data is today in capital markets. Recent pilot projects are starting to bear fruit, the education gap – where an intimate knowledge of the business needs to be combined with an understanding of statistical computing methods to generate that creative spark – is closing, and the industry is increasingly realizing that it’s in the advanced data analytics that the much promised disruptive effect will be found. We are on the verge of big data crossing Moore’s chasm, and when it does, we’re going to see big data technology used in novel, and sometimes unexpected, areas.
For example, big data is going to be increasingly embedded within decision support systems to increase automation of the decision-making process, as well as to enhance (predictive) analytics’ ability to provide additional insight. A form of this already exists in the equity front office with the use of algorithmic trading, and this pattern is now starting to be found in other businesses as well. In the credit line approval process, for example, internal structured and unstructured data can be combined with external data (like client prospectuses) in a big data repository. This helps streamline low risk approvals and thereby frees up staff for more complex decision-making tasks.
Big data will also play a large role in improving operational efficiency. Trade surveillance, for example, is experiencing increased regulatory and public scrutiny and the multitude of available communications channels in today’s digital world is making it all the more difficult for firms to manage. Big data technology will play an increasing role in firm’s response to monitor its trading activity. The ability to consume different channels and types of data – including instant messages, phone recordings, emails and blog entries – and consolidate this structured and unstructured data into a usable database will allow advanced pattern matching analytics to spot anomalous behavior.
The capital markets industry will also increasingly leverage big data for enhanced business intelligence by employing techniques including sentiment analysis and advanced analytics. Customer service communications with institutional investors, across a multitude of channels, can be recorded and sentiment measured, leading to more proactive handling of potential issues. And by consuming external data sources, a firm’s own analytics can be improved. Mortgage analytics, for example, can now consume household economic data at a neighborhood level of granularity to provide more accurate pricing of mortgage backed securities in some markets.
There are still challenges to overcome for more widespread adoption of big data in capital markets. The education gap needs more time to close and new technology skillsets time to mature, but as more and more successful deployments occur (and lessons are learned from those that fail), big data technology is set to cross the chasm. The applications discussed here will be just the tip of the iceberg.
For more on the disruptive effect of big data technology, read SunGard’s report, "Capital Markets Embrace Big Data.”