Infrastructure

03:05 PM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Data Opportunities: The Concepts Don't Change Over Time

Data volumes are growing and the technology is better, but the fundamental principles of data analysis remain the same.

Discussion abounds in the industry as to how firms are handling data to improve performance. Although the investment and trading environments have changed over the past few years, the quantitative principles driving the analysis of data are still the same. What has changed are the means by which data is handled and processed. This has been tied closely to technology use and development.


Mobile: The Road AheadAdvanced Trading's March issue examines how the buy side is crunching transaction data, automating algorithmic trading decisions and building analytics in an effort to more effectively track down alpha. To read more, download our March 2013 digital issue now.

Pick up any statistics book. Speak with any quant. Talk to any finance professor. The principles of finance, risk and mathematics have not changed since they were advanced in the mid-20th century. Fundamentally driven investment firms still conduct financial statement and industry analysis the way they have for decades. Quantitatively oriented firms still apply the same mathematical principles to time series data to benefit from divergences from established relationships. Trading firms still look for opportunities among undervalued and overvalued securities and related derivative products.

Technology is what has changed. Computer power, memory and storage are significantly cheaper than they were five, 10 and 30 years ago. These drivers have given modern-day analysts and supporting technologists the ability to process, store and normalize considerably more data than in the past. Analysts, leveraged by technology, arguably spend more time analyzing than performing the underlying tasks of data collection, scrubbing, etc.

[Optimizing Your Data Management Strategy in 2013 ]

Cheap technology and the off-the-shelf availability of sophisticated analytics packages have been the game changers. The commonly discussed "race to zero" -- the tremendous technology spend undertaken by a handful of firms whose profitability, if not survival, depends on the success of latency-sensitive strategies predicated on the fastest execution speed -- is a result of cheap, widely available technology backed by tremendous financial resources. Those less inclined to play the "speed game" can play the "smarts game." While at one point those with Excel spreadsheets had an advantage over others, the availability of packages like Matlab -- if not its open source equivalents -- puts analytical power in the hands of newly minted quants (and many not-so-quants).

Clearly, the availability of cheap technology and analytics has set the stage for extremely short-term holding periods and time horizons, fostered the rise of high-frequency traders, and brought about considerable industry, regulatory and political discussion as to the merits and detriments of such practices. Regardless of one's view as to whether or not such changes are beneficial to overall market operations, no one denies that they have in fact occurred.

Changes in investment horizon and analytics have shifted the overall trading and investment environment. Still, this hasn't altered the fundamental objective of data analysis or the underlying means by which it's performed. On the short end, firms that garner returns through extremely short holding periods still examine time series data for opportunities, just more of them across increasingly smaller windows.

The former "short-term" traders who can no longer play in the smallest window must make do examining data in longer windows. Then there are new, ancillary data types working their way into the information equation, such as news made actionable for trading triggers. While all of the above are legitimate topics for discussion and analysis in their own right, none invalidates that the premise and underlying use of data is the same as it has always been: to look for return-generating opportunities for a given level of risk.

The trading and investment landscape has changed over the years. A substantial element of these changes has been from technology-driven initiatives. On the surface, it may be plausible to suggest that the use of data has changed as well. However, this fundamentally is not the case. Certainly, particular elements of data handling and analysis have changed and evolved. The overarching principles of alpha generation still apply.

Matt Samelson is a Principal at Woodbine Associates, Inc. focusing on strategic, business, regulatory, market structure and technology issues that impact firms active in and supporting the global equity markets. He brings to the firm a wealth of experience in U.S. and ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
MSAMELSON000
50%
50%
MSAMELSON000,
User Rank: Apprentice
3/8/2013 | 5:11:14 PM
re: Data Opportunities: The Concepts Don't Change Over Time
All valid points guys. Interesting discussion. I agree that the sheer volume of data over time has changed the game in terms of technology, telecom, and infrastructure requirements as you say John. And a firm's ability to benefit by having better capability to manipulate, scrub, and analyze may well set the stage for superior returns versus other firms with "inferior" capabilities. Its the basic "heavy lifting" argument which has merit. Yet the fundamentals - pattern detection, time series analysis, statistics, etc. remain largely unchanged. In a sense, "everyone" is looking for the opportunity - but what is there leverage and what are the resource backing their efforts? And to your point Greg - there is certainly a lot more "noise" than there used to be - and that just emphasizes the need to have the infrastructure and technology resources to more efficiently comb through noisy data to discern underlying value.
MSAMELSON069
50%
50%
MSAMELSON069,
User Rank: Apprentice
3/8/2013 | 5:11:14 PM
re: Data Opportunities: The Concepts Don't Change Over Time
All valid points guys. Interesting discussion. I agree that the sheer volume of data over time has changed the game in terms of technology, telecom, and infrastructure requirements as you say John. And a firm's ability to benefit by having better capability to manipulate, scrub, and analyze may well set the stage for superior returns versus other firms with "inferior" capabilities. Its the basic "heavy lifting" argument which has merit. Yet the fundamentals - pattern detection, time series analysis, statistics, etc. remain largely unchanged. In a sense, "everyone" is looking for the opportunity - but what is there leverage and what are the resource backing their efforts? And to your point Greg - there is certainly a lot more "noise" than there used to be - and that just emphasizes the need to have the infrastructure and technology resources to more efficiently comb through noisy data to discern underlying value.
Matt Samelson
50%
50%
Matt Samelson,
User Rank: Author
3/8/2013 | 5:11:14 PM
re: Data Opportunities: The Concepts Don't Change Over Time
All valid points guys. Interesting discussion. I agree that the sheer volume of data over time has changed the game in terms of technology, telecom, and infrastructure requirements as you say John. And a firm's ability to benefit by having better capability to manipulate, scrub, and analyze may well set the stage for superior returns versus other firms with "inferior" capabilities. Its the basic "heavy lifting" argument which has merit. Yet the fundamentals - pattern detection, time series analysis, statistics, etc. remain largely unchanged. In a sense, "everyone" is looking for the opportunity - but what is there leverage and what are the resource backing their efforts? And to your point Greg - there is certainly a lot more "noise" than there used to be - and that just emphasizes the need to have the infrastructure and technology resources to more efficiently comb through noisy data to discern underlying value.
Matt_Samelson
50%
50%
Matt_Samelson,
User Rank: Apprentice
3/8/2013 | 5:11:14 PM
re: Data Opportunities: The Concepts Don't Change Over Time
All valid points guys. Interesting discussion. I agree that the sheer volume of data over time has changed the game in terms of technology, telecom, and infrastructure requirements as you say John. And a firm's ability to benefit by having better capability to manipulate, scrub, and analyze may well set the stage for superior returns versus other firms with "inferior" capabilities. Its the basic "heavy lifting" argument which has merit. Yet the fundamentals - pattern detection, time series analysis, statistics, etc. remain largely unchanged. In a sense, "everyone" is looking for the opportunity - but what is there leverage and what are the resource backing their efforts? And to your point Greg - there is certainly a lot more "noise" than there used to be - and that just emphasizes the need to have the infrastructure and technology resources to more efficiently comb through noisy data to discern underlying value.
John Panzica
50%
50%
John Panzica,
User Rank: Apprentice
3/8/2013 | 1:58:39 PM
re: Data Opportunities: The Concepts Don't Change Over Time
I agree the process of data analytics has not changed, volumes of data, which in turn drives the "capacity" to compute has changed tremendously. One of the ways capacity to compute manifests itself is technology, infrastructure, telecom, data center, space power interconnects. Then there is compute and storage, hardware compute capacity, complex event processing and storage. Latency performance is the other driver, non-HFT type capacity to compute if much more friendly to the IT budget then the race to zero game. Great article, thanks for sharing. JP
JP Panzica
50%
50%
JP Panzica,
User Rank: Apprentice
3/8/2013 | 1:58:39 PM
re: Data Opportunities: The Concepts Don't Change Over Time
I agree the process of data analytics has not changed, volumes of data, which in turn drives the "capacity" to compute has changed tremendously. One of the ways capacity to compute manifests itself is technology, infrastructure, telecom, data center, space power interconnects. Then there is compute and storage, hardware compute capacity, complex event processing and storage. Latency performance is the other driver, non-HFT type capacity to compute if much more friendly to the IT budget then the race to zero game. Great article, thanks for sharing. JP
Greg MacSweeney
50%
50%
Greg MacSweeney,
User Rank: Apprentice
3/7/2013 | 3:48:02 PM
re: Data Opportunities: The Concepts Don't Change Over Time
Great points, Brooke. Data manipulation (quote stuffing, mass cancellations, whatever you call it) is a major gripe of traders. It makes reading the "real" data very difficult. In fact, with so much extra data coming from, as you say, "manufactured" sources, it is a challenge for everyone. The exchanges have to build infrastructure to handle all of the extra data. Traders have to be able to handle the data...and so on. It's a tough challenge.
BrookeAllen
50%
50%
BrookeAllen,
User Rank: Author
3/6/2013 | 7:02:55 PM
re: Data Opportunities: The Concepts Don't Change Over Time
I would like to propose a new data type and analysis tool.

It is hard to argue against the idea that fundamental principles are time invariant because we can always say that if something changes then it must not have been fundamental or a principle in the first place.

One fundamental principle is that we must examine how our data is generated - not just the data itself. And we must be prepared to adopt or invent new tools to evaluate new situations and change our understanding of what is going on. I offer two examples.

When exchanges started sharing tape revenue people began shredding their orders. When exchanges began sharing quote revenue then people tried to make money by sending orders without a sincere desire to ever hold a position. When the only reason a number appears in a data set is because you are willing to pay for numbers, then the data itself has changed in a fundamental way.

Occasionally some participants blast non-marketable orders immediately followed by cancellations that act like denial of service attacks. Whatever the motive, establishing or liquidating positions does not seem to be among them.

Samelson says that although many things have changed, and there are new data types, the underlying use is the same: to produce return generating opportunities at a given level of risk.

I propose a new data type called "manufactured." Perhaps data manufacturing strategies have been going on since the beginning of time, however I believe that when Gǣalpha generationGǥ involves market manipulation or meaningless number generation, and the Gǣlevel of riskGǥ calculations center around Gǣnot getting caught,Gǥ then appropriate data analysis and noise reduction tools should include search warrants.
Register for Wall Street & Technology Newsletters
White Papers
Current Issue
Wall Street & Technology - Elite 8
The in-depth profiles of this year's Elite 8 honorees focus on leadership, talent recruitment, big data, analytics, mobile, and more.
Video
7 Unusual Behaviors That Indicate Security Breaches
7 Unusual Behaviors That Indicate Security Breaches
Breaches create outliers. Identifying anomalous activity can help keep firms in compliance and out of the headlines.