Lehman, NYSE, CME, Forex Capital Pursue New Latency Killers
Sidebar: Forex Lowers Latency By Upgrading Its Messaging Platform
The first thing Wall Street executives want to know when they tour the New York Stock Exchange's data centers to check on the servers they've colocated there is, who is hosted in the other cabinets and why aren't their own cabinets closer to the NYSE's servers, reports Steve Rubinow, the exchange's CTO. "It's part of this obsession that we all seem to have about how can I be faster, even if the amount of acceleration is negligible," he says. "It's a constant topic of conversation: How can I shave off a few microseconds? Over the course of time, a few microseconds here and there add up."
Because many firms have similar trading strategies, they're often racing against each other -- whoever can see a new opportunity and execute first wins. And latency affects more than execution; it also impacts prices and the distribution of data inside an enterprise.
The reality is, moving a server rack closer to an exchange server would save mere nanoseconds -- a time increment too small to affect even the most time-sensitive algorithmic trading program, according to experts. But colocating servers at an exchange, Rubinow says, does reduce the delivery time of market data by milliseconds, which do have an impact.
For a large Manhattan institution sending orders across the river to the NYSE's New Jersey data center, although the distance is not long, data often takes a meandering path through network switches and routers, each hop adding a bit of latency to the process. That is why many Wall Street firms park their servers in the NYSE's New Jersey and Chicago data centers. "The shorter the distance and the fewer the obstacles in the data path, the faster the data transmission will be," Rubinow explains. (To accommodate demand, the NYSE is building a new data center in New Jersey that should be completed in 2009, according to Rubinow, who adds that the exchange currently handles just under 1 billion transactions a day and that number is rising.)
Such colocation and installing a fast internal network (such as Infiniband or 10 gigabit Ethernet) appear to be the two most effective ways to reduce latency in market data delivery and order execution. Receiving market data directly from exchanges -- rather than going through Reuters, Bloomberg or Thompson -- is another effective time saver.
But where are the next places to shave milliseconds off of trading processes? To find out, we asked some top Wall Street IT managers, who are experimenting with network redesigns, efficient application programming, data compression, distributed memory and more to reduce latency further.
Simplifying Networks
While a short, straight data path is ideal, such is not the norm for Wall Street. "Many network topologies haven't been created with performance in mind, but with fault tolerance and reliability as goals," says Thanos Mitsolides, SVP, fixed-income technology, at Lehman Brothers. According to Mitsolides, in his area at Lehman, data can make as many as 10 hops to get from one machine to another. Changing the network topology should help the firm reduce latency by a factor of four, he says, adding that in the second half of 2008, Lehman's network engineering group will redesign the firm's 1 gigabit Ethernet network to reduce hops and perhaps move some servers closer to the exchanges with which the fixed-income group works.
Mitsolides notes, however, that upgrading to an InfiniBand or 10 gigabit Ethernet network wouldn't automatically reduce latency. "Ten gigabit Ethernet is about having a bigger pipe -- not necessarily a faster pipe," he insists. "In plumbing terms, if you double the diameter of your pipes, you're going to get more water on the other side. But that doesn't mean that when you open your faucet in the morning you're going to get hot water any faster." Therefore better network design trumps higher bandwidth, Mitsolides asserts.
The more commonly held view, however, is that higher-capacity networks do help reduce latency. "Everybody's either on or trying to be on 10 gig Ethernet because everybody knows they're going to need it sooner or later, so they might as well invest in it now," says Rubinow at the NYSE, which has been testing InfiniBand for several months. "We like some of the results, and some of our customers have been asking us when we'll be InfiniBand-ready to connect their colocated servers with ours." Other customers will wait and see how that market evolves, he notes, pointing out that the NYSE will accommodate both InfiniBand and 10 gigabit Ethernet.
Tweet This