The term "big data" seems to be everywhere these days. You even see it in discussions about science, government, and mathematics, just to name a few. But one area where big data is making a difference is in the financial services sector where the profit and loss pendulum can swing in a matter of seconds.
So why all the fuss about big data? First let's start with defining what the term actually means. In the context of this discussion big data refers to the ability to process and analyze vast amounts of information in milliseconds. It's important to understand how technology impacts the way financial markets operate and how the adoption of new forms of technology, such as big data, change the IT landscape for the financial sector. In addition, the knowledge gained from this type of analysis can arm financial institutions with quite a bit of valuable information, but this knowledge comes with both benefits and risks.
How Fast Is Too Fast?
Over the years financial exchanges have continually made improvements to their technology infrastructure, all in the name of speed and accuracy. For example, direct feeds from the New York Stock Exchange (NYSE) allow companies to receive and process information in real-time and then react to the results. Moving forward, however, financial institutions must be cautious. Analyzing and reacting to big data has become so aggressive that the SEC has created a Quantitative Analytics Unit to police high frequency traders, market movers and trades that that have sub-second irregularities. In fact, the parent company of the NYSE was even fined $5 million for delivering data several seconds later to some customers.
[For more on how capital markets firms are tackling big data challenges, read: Demand for Deep Analytics Challenges Data Managers].
The increase in speed has also changed the way data is gathered and decisions are made. New techniques, such as "news aggregation," can trigger trading without having a person intervene. Massive amounts of data can be parsed, from headlines to weather patterns, to determine the effect on a company, stock or commodity. One example of how these new techniques can have a negative impact on a company involved United Airlines (UAL). On September 8, 2008 Google News issued a story about UAL's bankruptcy, however, the real story had come out in 2002. News feeds, such as Bloomberg's subscription service, picked up the story automatically and within seconds millions of shares of UAL were dumped. While this situation was eventually corrected, it led to a significant capitalization loss to UAL.
Does Big Data Give A Competitive Advantage?
So why is it important to analyze large amounts of data? There's always been data available about customers and trends in the market. The difference with big data is that vast amounts of information can be analyzed in milliseconds, which can help improve decision making and allow financial institutions to customize programs for specific customers.
As Gordon Gekko, the fictional character in the movie Wall Street said, "The most valuable commodity I know of is information." What is so promising about big data is that vast amounts of information can help identify customer behavior patterns and industry shifts, which can help financial institutions make better decisions.
Let's take a look at a specific customer base such as large depositors. Whether information comes from customer feedback, social media channels, or simply by analyzing trading patterns, a financial institution can identify trends about these customers that might not have been apparent in the past. For example, a financial institution may analyze data and find that 90 percent of customers who make deposits of over $100,000 each month are also dog owners, C-level executive, wine experts, and drive cars that cost over $75,000. While this four-variable example may highlight some personal attributes of the investor, it may also point out that these individuals have an unusually low risk-avoidance in their investments. Understanding this information can help the financial institution tailor investment programs to fit this particular profile.
Another example of how big data can help improve decision making is in identifying trends. For example, a financial institution can analyze customer cash flow based on time of year and the resulting information may help set expectations and improve forecasting. One example of this is when a regional bank sees that a customer's cash deposit is lower than what the Federal Reserve allows for an outstanding loan. This could happen based on employment turnover that occurs every five years, or something like government program funding that indirectly impacts their region.
Lastly, big data can help identify patterns that occur during a common event and make decisions based on historical data. For instance, when a company undergoes a corporate relocation it can have a tremendous impact on the employees and their cash flow. The financial institution can set up special payment programs, for both paychecks and bills, which can help alleviate the burden and stress on the employee. While this type of historical data has been around for many years, the ability to act on the information in real-time is what makes it so unique.
The Other Side Of The Coin
While big data can help with decision making and customer segmentation, there are issues surrounding security breaches, regulatory issues and additional costs that must be understood.
First of all, having so much customer information available and segmented in so many ways may actually create a new risk. What if the information is leaked or lost? We've all read stories about how a mid-level employee has left a laptop in a car and the information was stolen. Valuable and very personal information about tens or even hundreds of thousands of customers can be lost or stolen in the blink of an eye. In addition, even if the information is secure, many customers don't want to even share the information with an outside source.
Second, there are regulatory issues to consider. On the one hand there are very strict guidelines established by the SEC and the federal government about how data is to be stored and for what length of time it must be kept. On the other side are the company's lawyers who may want information kept for a short period of time to reduce risk. For many institutions, federal law and SEC regulations require that data be kept for a specific period of time. But what if the company is subpoenaed after the time frame has expired? Who is responsible? The data's life expectancy must be managed against the risk of keeping it for analysis.
Lastly, there is the issue of cost. Who picks up the cost for storing and managing all of this information? The costs associated with housing the information, Internet connectivity and connecting remote offices can become a huge drain on corporate profits. These are not small systems that a simple day trader can use. The disk space is well into the terabyte range and the computing systems can be in the hundreds or thousands of nodes depending on what is being done and the type of data.
For trading, external connectivity to an exchange may be high-bandwidth (OC-3 or greater), but is also needs to be low-latency. This may require that many of these systems sit close to the exchange, if not co-located in the exchange's datacenter. NYSE Euronex claims 76 microsecond messaging within its datacenter. The bottom line is that the costs involved can be substantial.
Benefits Outweigh The Risks
It's clear that technology can help improve the performance of financial institutions through the analysis of big data. Improved customer segmentation and identification of industry trends can greatly enhance competitive advantage. While there are always issues with utilizing new forms of technology, it's clear that the benefits outweigh the risks when leveraging the information learned from big data.
About the Author:
Devon Buffington is a Regional Architect at Advanced Systems Group.