Data Management

04:24 PM
Connect Directly
Google+
Twitter
Facebook
RSS
E-Mail
50%
50%

Big Data Influences More Long-Term Storage

More firms are keeping data they could legally toss. Is this "keep everything" approach sustainable?

The pursuit of big data has led to an increase of companies keeping older files they would have previously discarded, confirms Scott Gillespie, expert on regulations and compliance and senior VP of business analysis at Quadron Data Solutions.

There is a six-year retention requirement of client account data, he explains. Account records, trades, holdings, profile information, goals and objectives and so on fall into this category. There are three reasons these records, which have historically been discarded with regularity, are now starting to stick around.

1. The cost of data storage has come down dramatically.

When technology directors face decisions around deleting versus saving, the lower cost of storage makes to easier to choose the latter. This "just let it be" attitude is further supported by the complexity of what data to actually cut.

Those what-to-delete problems are two fold. First, the connectivity of data means even an old events can be tied to current data, meaning the deletion of an old file could have an adverse impact on the quality of newer information. The other is identifying the proper account data, tagged to the right account.

Gillespie provides an example that demonstrates the challenges of deleting aged records of two customer accounts, one belonging to Smith and the other to Jones:

Smith's account experiences an address change on 1/1/06, and the bank is legally required to maintain the old address for six years. Smith's suitability changes on 1/1/09, and the bank is required to maintain the old suitability information for six years. Smith's account closes on 1/1/14, and the bank must maintain the account records for six years after closing. In all that time Jones's account hasn't changed.

Aging and deleting Smith's aged records without impacting Jones's account records may realistically require cherry picking from the database. Between the cost of data storage coming down, and the risk of explaining to regulators why data was prematurely deleted, firms may be better off just holding on to all that data for a very extended period, Gillespie explains.

2. Deeper analytics require larger data sets.

The big data mega-trend is driven by the idea that having more data will help uncover more information. Analysis can uncover trends and correlations that enable market differentiation and a competitive edge. Theoretically, the more data sets a firm collects, the more information they can derive.

If a firm has a policy that says to delete things seven years and older it raises the question of big data needs. A company may find interesting information relating to client transactions and holdings over the past five years, but what if they want to then look further to the last six to ten, or even fifteen years?

In Gillespie's example, a firm is doing some big data analysis that shows a particular region is doing the most investment advisory business. To learn more they will need to know the particulars of account profiles including the demographic of clients, objectives, and how long they've been customers. This allows for deeper questions like, "Which regions and reps are doing the most Investment Advisory business and what is the demographic profile of their client base?" and "What is the changing mix of securities trades over the past five years?"

Firms are quickly realizing that if they want to track trends and perform analytics beyond the last six years they cannot delete older data. Gillespie says his clients who were deleting data only a few years ago are now holding onto it going forward, hoping to glean more from a longer time frame. "Many are recognizing the need to perform analytics and posing the question of what kind of data we need to keep," he says, "It's a big conversation in the executive offices of broker dealers."

3. Regulatory expectations are rising.

Regulators are performing risk-based exams, which are highly dependent upon ready access to client, trade and position holding data, he explains. From a data storage and usage standpoint regulators used to just have a checklist (e.g. books of records, images of account records), "now they are posting big data analytics questions and expecting firms to hand over the results of their exam."

For example, the SEC can make the following request with the expectation that it will be fulfilled by a query of warehouse data: Deliver all purchases of leveraged ETF equities over the past 24 months by clients over 55 who changed their objective to growth or speculation during the same period.

No matter that the above question will leave some firms without robust warehousing to scramble for answers. The reality is that regulators understand data is cheap to store and they should be able to get any report from a investment firm. Would it then be so unreasonable to assume future questions put forward by regulators will be aided by "outdated" information? If old account records will benefit data queries, would it not be foolish to delete it today?

These possibilities are weighing on the minds of executives, according to Gillespie. "Clients are contenting to add storage at this point," he says, adding that more and more clients are electing to hold onto older data older than even a couple years ago. "They will address it ultimately but not exactly at the minimum requirements." Becca Lipman is Senior Editor for Wall Street & Technology. She writes in-depth news articles with a focus on big data and compliance in the capital markets. She regularly meets with information technology leaders and innovators and writes about cloud computing, datacenters, ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Becca L
50%
50%
Becca L,
User Rank: Author
1/30/2014 | 3:35:29 PM
re: Big Data Influences More Long-Term Storage
Yes, what interests me is if the legal requirement to hold the data has passed, and the firm has dumped it in accordance with the law, then they must be excused, right? But if they had kept the data for "big data" purposes, could the firm still rightfully withhold it from regulators? probably not.
IvySchmerken
50%
50%
IvySchmerken,
User Rank: Author
1/30/2014 | 3:24:22 PM
re: Big Data Influences More Long-Term Storage
Deleting data that is considered evidence for a legal case could end up getting a financial firm in hot water with regulators. If a customer complains about a broker, the broker can be asked to produce the client's records. The Doctor has some good advice - firms should have a data and records retention/deletion policy to protect data and to defend the firm from potential requests for e-discovery.
Greg MacSweeney
50%
50%
Greg MacSweeney,
User Rank: Apprentice
1/30/2014 | 2:45:13 PM
re: Big Data Influences More Long-Term Storage
There are regulations that require firms to keep records. For instance, communications between a broker and a customer must be recorded and available to regulators. Intentionally 'dumping' data is probably even more dangerous, legally, than storing data. Regulators do not accept the "woops, i deleted it" excuse.
Becca L
50%
50%
Becca L,
User Rank: Author
1/29/2014 | 10:12:20 PM
re: Big Data Influences More Long-Term Storage
That is a very interesting point, one I haven't heard mentioned yet in the big data discussion, and probably the strongest case for dumping data. Is there no statute of limitations in this area?

(The Doctor? Dr. Who? What an honor. )
The Doctor
50%
50%
The Doctor,
User Rank: Apprentice
1/29/2014 | 8:25:33 PM
re: Big Data Influences More Long-Term Storage
Readers should remember that stored data is discoverable in legal actions. The benefits of maintaining such data for business purposes must be weighed against its continued availability for discovery in legal proceedings. A risk-benefit evaluation should be performed, including legal concerns, when creating a firm's record retention policy.
IvySchmerken
50%
50%
IvySchmerken,
User Rank: Author
1/28/2014 | 10:26:41 PM
re: Big Data Influences More Long-Term Storage
From a compliance perspective, financial firms are required to store data for six, if not seven years, so it's not hoarding. But now they are saving beyond that time frame because they want apply big data analysis to perhaps 15 years of data to gain a competitive edge. I got the impression that they have strategy but it's possible that they are first figuring it out.
Jonathan_Camhi
50%
50%
Jonathan_Camhi,
User Rank: Author
1/28/2014 | 9:13:15 PM
re: Big Data Influences More Long-Term Storage
I would imagine that for a lot of FI's making use of some of this saved data is a matter of executing on an overall strategy that is data-driven, and executing on such strategies can be a difficult and lengthy process. So it doesn't surprise that they would be hoarding data while trying to figure things out.
Becca L
50%
50%
Becca L,
User Rank: Author
1/28/2014 | 7:06:05 PM
re: Big Data Influences More Long-Term Storage
Great point, Kathy. It does resemble a hoarders mentality of "I'll keep it because I may need it later," and that is definitely not cost efficient, even if prices are coming down. It may also bog down the speed of their queries, impacting efficiency.
KBurger
50%
50%
KBurger,
User Rank: Author
1/28/2014 | 5:17:43 PM
re: Big Data Influences More Long-Term Storage
I guess a "glass half full" way of thinking about this is, this is one way of leveraging investments in compliance, which is something that has been urged for years. The broader question (beyond costs and sustainability) I think is: are firms just holding onto this data "in case" they need it, or are they actively developing big data strategies and processes (whether around compliance, risk management, customer insight, or whatever) that draw on that data that's being stored? If it's just being stored because some day it might be needed and they are not actively figuring out ways to leverage it, then they are no better than hoarders and need an intervention!
IvySchmerken
50%
50%
IvySchmerken,
User Rank: Author
1/27/2014 | 5:22:58 PM
re: Big Data Influences More Long-Term Storage
Saving data for a customer that changes address and suitability requirements for six years is a response to regulatory scrutiny. Firms are afraid as you say that they will be hit by Big Data query from the SEC or FINRA that requires them to produce answers within in 24 hours. Keeping data for longer and longer periods - even 15 years - is a sign that firms want to do deep analytical dives on data from customers, adviser performance, etc. Cheaper storage is making this possible, but at some point, each firm will be running a data empire. Is this going too far?
Register for Wall Street & Technology Newsletters
White Papers
Current Issue
Wall Street & Technology - July 2014
In addition to regular audits, the SEC will start to scrutinize the cyber-security preparedness of market participants.
Video
5 Things to Look For Before Accepting Terms & Conditions
5 Things to Look For Before Accepting Terms & Conditions
Is your corporate data at risk? Before uploading sensitive information to cloud services be sure to review these terms.