Data Management

07:00 AM
Jonas Olsson
Jonas Olsson
Commentary
50%
50%

Single Source, Many Truths

If the data quality is not at fault, why then might departments reject the data? The answer is that there are two aspects to data quality: factual correctness and contextual correctness.

To stay competitive, financial services organizations must develop three core competences: identifying new business opportunities, understanding exposure, and streamlining compliance processes to minimize cost and risk.

As such, financial services organizations must equip their sales, risk, and compliance teams with the deep, timely insight they need to operate effectively. Because each department has its own context-specific definition of core concepts -- such as market value, customer, security, product, and price -- departments often meet their operational requirements using complex spreadsheets.

However, because spreadsheets create the significant risk of human error, many organizations decide instead to build centralized data repositories.

Data quality vs. data context
Building data repositories can be very difficult, and to reduce the complexity of the design process, a business often enforces a single set of static definitions for its data, despite the fact that each department depends on particular data definitions for operational reporting. However, without a way to tolerate contextual variations in financial definitions, businesses risk investing time and capital in an inflexible data warehouse that will most likely fail to meet its users’ operational needs.

This failure often manifests itself in complaints from users in different departments about the perceived quality of the data. In fact, the quality of data -- that is, the factual correctness of information on transactions, securities, accounts, and client names -- is often perfect.

If the data quality is not at fault, why then might departments reject the data? The answer is that there are two aspects to data quality: factual correctness and contextual correctness. For example, a rigid data warehouse that was only designed to support the bid price for a given source, security, and date will store factually correct bid prices day after day, despite the fact that many departments may require the close price for their operational requirements.

Although the bid price and the close price may occasionally be identical by pure chance, on most days the prices will differ. Faced with prices that are contextually correct on some days and incorrect on others, departments logically assume that there is a problem with data quality. In fact, the real issue is more likely that the data repository is unable to deliver reports in the specific context that each department requires.

To meet their business goals, organizations must therefore stop forcing static data definitions and instead deploy a solution that enables the underlying raw data to support multiple contexts.

Narrow scope, limited insight
One way to reduce the risk of end-users rejecting a data warehouse altogether is to narrow the scope of the project. Rather than building a single, strategic data warehouse solution, organizations can extract subsets of their raw data from source systems into a number of departmental datamarts, each of which is designed to address a very limited number of specific departmental use cases.

Datamarts enable businesses to bypass the challenge of supporting each department’s competing requirements for data definitions in a single system, but this benefit comes at a price. Although the time, effort, and resources required to create a single datamart are significantly less than building a data warehouse, datamarts scale poorly.

For example, whenever any department has a new reporting requirement, the IT team will face the time-consuming task of locating the relevant raw data from multiple source systems, then extracting and integrating it into the datamart. When changes are required -- and they certainly will be -- this time- and cost-intensive process must be repeated. Multiplied over many departments and numerous reports, the datamart approach rapidly becomes costly, complex, and time-consuming.

Moreover, datamarts only meet department-specific reporting requirements, at the expense of the bigger picture. They are effectively silos of analytics data, which makes reconciling datamart reports from all departments virtually impossible. With the regulatory landscape shifting towards deeper reporting in more frequent cycles, financial services organizations that rely on datamarts may find they lack the enterprise-wide overview required to meet their compliance requirements -- a serious threat to their ability to compete effectively.

Choosing a strategic solution
To offer their sales, risk, and compliance departments the reports they need without generating unacceptable costs and complexity, businesses need to move away from the spreadsheets and datamarts by establishing a central repository of all relevant financial data, one that addresses the need for context.

The answer is to create a flexible data warehouse that can deliver a comprehensive range of context-specific operational reports without incurring the cost of continually re-extracting and re-integrating the raw source data whenever new reporting requirements arise.

This kind of data warehouse would reveal the full lineage of each definition, tracking it back to the underlying raw data in each report in order to help dispel perceptions of poor data quality and to encourage end-user acceptance. Furthermore, it would make it possible to instantly roll up reports from each department into a company-wide overview, without the need to enforce single definitions for context-specific concepts.

By supporting multiple data sources and definitions, a properly designed -- and more flexible -- data warehouse would erase the common misconception of poor data quality, as each department will see its own specific definition of the same underlying source data.

Beyond the context/quality benefit, a single data warehouse for all data scales far more cost-effectively than datamart solutions. And although the complexity of uniting multiple source systems, stakeholders, and use cases into a single data warehouse may be a daunting prospect for most organizations, the ready availability of packaged data warehouse solutions makes it unnecessary to develop a whole solution from the ground up. 

Jonas Olsson is the CEO and founder of Graz , a provider of data warehouse and business intelligence software built specifically for the needs of investment managers, insurers and banks worldwide.  Olsson founded Graz in 2000 as an IT services firm, and transitioned it ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Threaded  |  Newest First  |  Oldest First
Byurcan
50%
50%
Byurcan,
User Rank: Author
8/20/2014 | 10:38:04 AM
Data
Interesting read. Providing context for data, no matter how factually correct it is, is important.
Becca L
50%
50%
Becca L,
User Rank: Author
8/31/2014 | 1:47:01 PM
Re: Data
Thanks for this insight, Jonas. Agreed, it's interesting how each department's data needs can be so separated from the corprate level. And easily forgotten, I imagine, when infrastructure is being upgraded at a high level.
Greg MacSweeney
50%
50%
Greg MacSweeney,
User Rank: Author
8/21/2014 | 8:40:28 AM
Context, Context, Context
Absolutely. Factually correct and contextually correct are both important, but the business units really need the contextually correct data (as long as it doesn't conflict with the factually correct version).

This is not a new problem and it is something that financial firms have been dealing with for years. Will firms ever figure out a way to solve these data accuracy problems?
KBurger
50%
50%
KBurger,
User Rank: Author
8/22/2014 | 10:53:38 AM
Re: Context, Context, Context
This also illustrates the complexities of data governance, which is a hot topic as a growing number of FIs embark on big data and other data-related initiatives. It's all well and good to declare, "Data governance is critical" but Jonas' article shows how complex an effective data governance effort and policy can be. It's not just about context, but also flexibility -- these approaches cannot be cast in stone, they need to be dynamic in some way in order to be responsive to changes in the business, regulatory requirements, etc.
Greg MacSweeney
50%
50%
Greg MacSweeney,
User Rank: Author
8/26/2014 | 5:59:01 AM
Re: Context, Context, Context
You are really describing "data management 2.0" or maybe even "data management 3.0," since most traditional data management projects are not flexible at all (one version of the data, no adaptability).

After 30 years of struggling with data, I wonder if the technology, the skills and the management strategies have finally evolved to the point where data management finally fulfills its promise
KBurger
50%
50%
KBurger,
User Rank: Author
8/26/2014 | 9:48:00 AM
Re: Context, Context, Context
We can dream, can't we?
Greg MacSweeney
50%
50%
Greg MacSweeney,
User Rank: Author
8/26/2014 | 11:23:26 AM
Re: Context, Context, Context
Yes, dreams do come true. But somehow, i can't picture a Chief Data Officer inspiring his team with a data management version of the "dream" speech. LOL
More Commentary
Banks to Increase IT Spend on Big Data Challenges, Finds Aite Report
Big data has presented the greatest challenges and dissatisfaction for banks, yet it is the most likely to see upward spending in the next two years.
Scotland Independence Vote: Haggis & Fragmentation
Scottish independence has far-reaching consequences for the global financial markets.
5 Tips to Save the Wall Street Datacenter
Though cloud computing and SaaS are all the rage, there is still a need for proprietary Wall Street datacenters, as long as they are run efficiently.
Preventive Measures for Post-Interview Anxiety
Most professionals leave interviews thinking that it went well, and then they wait... and wait. The Caring Recruiter has a cure for the typical post-interview trauma.
Leaving Out the Welcome Mat for Financial Services Hackers
Everyone knows the financial services industry is a prime target for hackers. Despite the dangers, many applications have software vulnerabilities that expose real risks.
Register for Wall Street & Technology Newsletters
White Papers
Current Issue
Wall Street & Technology - July 2014
In addition to regular audits, the SEC will start to scrutinize the cyber-security preparedness of market participants.
Video
5 Things to Look For Before Accepting Terms & Conditions
5 Things to Look For Before Accepting Terms & Conditions
Is your corporate data at risk? Before uploading sensitive information to cloud services be sure to review these terms.