Wall Street & Technology is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


02:45 PM
David Meitz, Managing Director, Chief Technology Officer, ITG
David Meitz, Managing Director, Chief Technology Officer, ITG

The Art and Science of Leveraging Cloud Infrastructure

Now that cloud providers have addressed many of the more practical concerns of their users, data segregation has become the major challenge in cloud deployments.

Having demonstrated an initial, and understandable, wariness of cloud services, trading firms and other players such as hosting providers in the institutional trading environment are slowly coming to embrace the technology. With storage demands for mid-size firms growing at approximately 90 terabytes a year, perhaps this is not surprising. Building proprietary infrastructure that can keep up with this level of growth is comparable to painting the Golden Gate Bridge: a never-ending task.

The growth of data volumes brings the advantages of the cloud into clear focus. The immediate benefit of cloud services is that, being built on a shared infrastructure that occupies a shared space, they offer significant economies of scale. Leveraging the enterprise architecture and facilities that have been put in place by cloud service providers can therefore be a far more economical option than building out in-house resources.

Commodity providers, such as Google, Amazon and Microsoft can also be a faster option for accessing infrastructure, hosting software services or managing storage – particularly when the alternative is building large data centers, with all the routing, networking and security that entails.

Specialist providers have also established themselves in greater numbers as more industry sectors and individual businesses engage with the possibility of using cloud-based services. These specialists tend to offer more predictable and always-available service with extremely high levels of computational power to serve more data-intensive businesses.

As a result of these developments, we have a diverse cloud ecosystem that can meet a wide variety of needs. From simple hosting of servers and applications for firms that do not require or cannot house a data center of their own, to hosted spaces for distributing software, hosting web services or maintaining a virtualized environment, the cloud is adding value.

But remaining concerns are legitimate, particularly for trading firms. In scenarios where latency is a concern, or time-sensitive data needs to processed, then a cloud-based option is clearly not appropriate. Worries about security and protection of sensitive data are equally pressing, and concern about the location of data centers remains as firms have to ensure that the necessary connectivity and market data is available.

More attractive than a wholesale adoption of cloud computing therefore, is a hybrid model whereby a firm maintains its own infrastructure and data center alongside the service from a cloud provider. In this set-up, data is segregated and treated in the most appropriate environment.

In a hybrid scenario, a client’s personally identifiable information remains in a firm’s local infrastructure, while tick data used for compliance purposes is directed to cloud storage. By the same token, pricing data whose value and utility is determined in large part by the speed at which it can be accessed, understood and responded to remains in the local environment. Conversely, data acquired in the name of market research is directed to the cloud since its value is considerably less dependent on the speed at which it is available.

Now that cloud providers have addressed many of the more practical concerns of their users – security being the obvious example – data segregation has become the major challenge in cloud deployments.

This is more than separating the signal from vast amounts of data. Not all signals are needed in real-time after all. There is also little to be gained from a static, binary decision-making process, particularly as sensitivity to latency can change over time. As the gap between ‘historical’ data that is used for dynamic or intra-day trading analytics and real-time pricing data gets narrower and narrower, the need for more dynamic data analysis grows more pressing.

Interestingly, the answer to the dilemma lies in human engagement as much as automated analytics. Greater collaboration between clients, traders and IT teams will be a key feature of the more successful hybrid models, since it allows for feedback regarding data types and usage to be incorporated into development plans.

This may appear counter-intuitive at first. After all, in cloud computing we have a phenomenon that is both driver and consequence of greater automation and the unfortunately named ‘rise of the machines’. But the truth is that this level of data analysis is an art as well as a science. As the key that opens up the multiple benefits of cloud computing, it is an art that is worth mastering.

David Meitz is Managing Director, CTO at ITG, the execution and research broker. Responsible for software development, technology and trading support services, and information security, Mr. Meitz joined ITG in 2002 from Reuters America, where he was an EVP responsible for ... View Full Bio
More Commentary
A Wild Ride Comes to an End
Covering the financial services technology space for the past 15 years has been a thrilling ride with many ups as downs.
The End of an Era: Farewell to an Icon
After more than two decades of writing for Wall Street & Technology, I am leaving the media brand. It's time to reflect on our mutual history and the road ahead.
Beyond Bitcoin: Why Counterparty Has Won Support From Overstock's Chairman
The combined excitement over the currency and the Blockchain has kept the market capitalization above $4 billion for more than a year. This has attracted both imitators and innovators.
Asset Managers Set Sights on Defragmenting Back-Office Data
Defragmenting back-office data and technology will be a top focus for asset managers in 2015.
4 Mobile Security Predictions for 2015
As we look ahead, mobility is the perfect breeding ground for attacks in 2015.
Register for Wall Street & Technology Newsletters
7 Unusual Behaviors That Indicate Security Breaches
7 Unusual Behaviors That Indicate Security Breaches
Breaches create outliers. Identifying anomalous activity can help keep firms in compliance and out of the headlines.