In 2011, NYSE Technologies launched a hybrid cloud that offered investment banks, hedge funds and other firms seeking to benefit from the latency advantages of being located within close proximity to both the publication of market data and exchange order management systems an opportunity to maximize technology resources without massive capital expenditures. NYSE promoted this platform as a community cloud where market participants could collaborate and provide a wide range of services to each other within a secure, isolated, yet flexible environment.
As a product manager in NYSE's Global Market Data group at the time, I worked closely with my colleagues and several clients to provide access to real-time, reference and historical data sets within this community cloud platform. Working with a diverse set of clients, we quickly saw the benefits these firms realized by leveraging cloud technology to accelerate and simplify key tasks related to virtually all aspects of capital market activity.
Since the broad adoption of the ticker tape in the 1880s, capital markets have driven, and been driven by, technology. Today, the technology to manage all aspects of market operations is both vast and complex. For better or worse, markets are dependent on technology, and when that technology malfunctions, the results can be costly.
Consider, for example, the reception and consumption of real-time market data. Investment banks, hedge funds and other proprietary trading firms all employ trading strategies that span global markets. These firms require real-time trade, quote and order book data from multiple exchanges on several continents. In order to receive and process this data firms require:
• Connectivity to a dedicated financial extranet with sufficient bandwidth to support incoming data feeds.
• Data feed handlers to receive trade, quote and order book data and then normalize it for use within the firm.
• Data caching systems to mange the internal distribution and consumption of market data by traders, analysts, researchers and others.
• Client applications to display incoming data and manage resulting orders submitted to the various exchanges.
• Entitlement management systems to ensure compliance with exchange data distribution policies.
Management of these systems requires system administrators and software developers with highly specialized skill sets as well as access to a wide range of hardware and software platforms. Before cloud solutions were available, managing this kind of complexity was a challenge.
[Will The Next Era Data Center Be Hosted on the Cloud? ]
Case Study in Complexity
For example, in 2008 I was working with a client that was trading heavily on both the Nasdaq and NYSE Arca markets. This client was experiencing unexpected latency internal to its feed handler, caching systems and client applications. This latency caused the firm's traders to be "out of the market," as they were making trade decisions based on stale data, 1 or 2 milliseconds behind true real-time quote and order data. It was unclear where within the systems this latency was being introduced. To isolate the problem, it was necessary to replicate the client environment and monitor the flow of data from the data link layer all the way up to the client application. This required significant lab resources to build out new Linux and Solaris environments as well as Windows XP workstations with the necessary APIs to support the client applications.
After an extensive collaborative effort among the client and the development, support and testing teams at NYSE Technologies, we were able to isolate the problem, test an adjusted configuration, and roll the new configuration into production, eliminating the added and unnecessary latency.
While this issue was certainly one of the more complex with which I was involved, it was not unique. NYSE Technologies offered and supported 150-plus data feed handlers, caching systems and APIs for all major development languages. In environments with so many unique variables and seemingly limitless combinations of technology, troubleshooting and resolving technical issues was often a daunting task.
Unlike most enterprise software applications, feed handlers and caching systems are constantly changing and require frequent updates and modification. Changes are driven by regulatory as well as technical issues. After the Flash Crash of May 2010, for instance, the SEC implemented a circuit breaker to temporarily halt trading in securities in the S&P 500 whose price changed dramatically over a given time period. The introduction of this rule required that a new data field be added to all real-time U.S. equity data feeds. From a vendor perspective, it was necessary to write this new field into the code, perform complete regression testing and release the code to clients. Clients then were required to take receipt of new software binaries, fully test and then deploy new software into production -- all within an incredibly short time frame. Given the complexity of market data systems, it was always a challenge to help clients prepare for even minor changes to real-time market data feeds.
NYSE Technologies' community cloud platform significantly reduced the time and investment required for both vendors and clients to manage their market data systems. The ability to quicky deploy and configure server farms, applications and complex network configurations allowed firms to proactively focus on strategic initiatives while still maintaining robust and reliable market data and trading systems.
Firms in similarly complex industries can also leverage the cloud to manage development, testing and production environments that support multiple operating systems, network configurations and development languages. The maturation of cloud computing now provides the flexibility and stability necessary to allow skilled workers to focus on solving problems, rather than building test and development environments.
As cloud computing continues to grow, and the line between physical and virtual computing environments continues to blur, many more industries will embrace cloud virtualization platforms to reduce costs and development times while streamlining the deployment of production-ready enterprise solutions.
Seth Payne is a senior product manager at Skytap, an enterprise cloud computing provider, where he's responsible for defining cloud computing capabilities for enterprises. Prior to Skytap, Payne spent several years working on Wall Street in technology at both Gravitas and the New York Stock Exchange.