Data Management

06:34 AM
Sean O'Dowd
Sean O'Dowd
Commentary
Connect Directly
LinkedIn
Twitter
RSS
E-Mail
50%
50%

Building A Big Diverse Data Platform For Wall Street

Although there is no one panacea for increasing the analytical quotient in capital markets and there is no big data analytics in a box, there are, however, components or architectural ingredients that are essential in getting value throughout the information supply chain.

The entire capital markets industry is buying into the big data hype, or what I'm calling business intelligence optimization (BIO). I touched on this change in wording in an earlier piece that wasn't focused on ‘big data', but the process to harvest and exploit new knowledge -- regardless of where the data resides.

In reviewing this definition, there is no panacea for raising the analytical quotient in capital markets. There is no big data analytics in a box. There are, however, components or architectural ingredients that are essential in getting value throughout the information supply chain. And, how are those insights operationalized to best impact the bottom line? There are straightforward and relatively simple approaches that offer different ways to build a platform, without the usual complexity associated with data analytics projects.

To build a successful BIO platform, start by understanding:

Data Skills: Talent is scarce in the data science/quant market to leverage and utilize these data and tools.

Standards: Standards, data quality and governance are changing rapidly and not fully formed, so deployment and management must be taken in measured steps.

Culture: Some companies may not be internally built for BIO. A data driven culture that is willing to experiment and understand there are some activities that can generate immediate ROI and others that will take some time is required. It's called continuous improvement and failing fast, and some firms will not be able to access fast insights.

Complexity: It's not easy. Firms have difficulty mapping and stepping through the fundamental factors for any project -- people, process and a clear vision for a "future state" or "next generation" platform -- and BIO is more complex than most projects.

Dedication: Let's not mince words -- it's hard. This will take dedication. The "edge" firms can gain takes hard work, time and patience. Gartner suggests that only 20% of the global 1,000 organization having a strategic focus on "information infrastructures" by 2015.

[For more on how Wall Street organizations are approaching big data challenges, read: Big Data Could Transform Global Financial Markets.]

Is there a path of least resistance? At this time and for the near-term future, a unified data architecture (UDA) is one of the best ways to build a platform that not only enables and operationalizes analytics, big data, BIO or information -- whatever term is used -- but also leverage existing IT assets. The goal is to take current systems, introduce new technology only when needed, and maintain flexibility so the company can rapidly develop applications and analytics that result in better performance. This is a hybrid ecosystem.

The components of a UDA include open source Hadoop, a discovery platform and an integrated data warehouse. Hadoop is an excellent solution for data staging, preprocessing and simple analytics among other applications. The data discovery platform should enable rapid exploration of data using a variety of analytic embedded techniques accessible by business users. And to operationalize findings and insights, a warehouse puts factors, rules or other outputs from Hadoop and the discovery platform into an integrated environment for production to be used across the organization or for a specific business project or unit.

The current combination of components are not just adding value, but also enabling speed to market. A UDA is a sound approach that injects new complimentary components, but does not compete with the fundamental architecture. Rather it enables firms to fill voids in their systems and create the capabilities to rapidly develop analytics and applications. Most importantly, it delivers business intelligence that is not just interesting, but actionable.

All the speed, analytics, insights and connectivity are nice, yet without an actionable outcome to improve the business, the former is just a piecemeal approach that is creating an expensive big data science experiment.

Sean O'Dowd leads the Global Capital Markets program at Teradata for Industry and Marketing Solutions. In this role Sean focuses on industry strategy, marketing and field enablement. Areas of focus span financial market structure, regulations and technologies that impact the ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
so'dowd015
50%
50%
so'dowd015,
User Rank: Author
7/29/2013 | 6:16:24 PM
re: Building A Big Diverse Data Platform For Wall Street
Exactly - ease the path to insight as much as possible. It will take some time to get all the pieces right - it will also take a few years for the market to fully embrace the new approaches too.
so'dowd015
50%
50%
so'dowd015,
User Rank: Author
7/29/2013 | 6:15:08 PM
re: Building A Big Diverse Data Platform For Wall Street
Thanks for your thoughts and good points. The ease of analytics, its use and setup is without doubt the priority for business. Aside from the IT side of things, the ease of analytical use is what I agree with as I wrote about the business intelligence optimization approach. And to your idea - the "expertise in a box" is something that should evolve and is frankly where some of the market will head. Certainly, for smaller downstream firms, I would see these top to bottom, plug and play packages or cloud based offering to show up eventually. Larger firms are always hesitant given the change requirements, security, etc.. There are already hardware/software appliances that have bundled hadoop, discovery database, algorithms and in database visualization capabilities with analytical roadmap services, so the development side of the "expertise of the box" seems to be taking shape to some degree. Open source will develop more and play a larger role without doubt - how quickly that becomes accessible and workable for wall street firms is another question.
Greg MacSweeney
50%
50%
Greg MacSweeney,
User Rank: Apprentice
7/29/2013 | 1:30:45 PM
re: Building A Big Diverse Data Platform For Wall Street
I agree. Successful big data initiatives will allow the users to manipulate the data through an easy to understand (and use) interface that has rich functionality. If executives need to rely on a developer or analyst to build reports, big data won't reach its full potential. After all, the business users know what they need, but they don't have a method to get to the data (easily) right now.
algon3rd
50%
50%
algon3rd,
User Rank: Apprentice
7/27/2013 | 10:31:14 PM
re: Building A Big Diverse Data Platform For Wall Street
I see one of the hardest challenges is to connect the understanding of the business from top to bottom and the understanding of the way to extract analysis from the data. How is an executive supposed to go about data mining their collected data? Or is it left up to the developers etc to determine the filters and what analysis to generate? In my opinion, the executive needs a way to manipulate the data on their own. Maybe a next generation "Excel" for big data / NoSQL databases?

Big data "in a box" is also interesting model. IBM has "cloud in a box" with Pure Systems. I think there is great value in providing a single brand experience from the top down in big data. "Expertise in a box".

As far as hadoop, it is great for sorting, grouping, and filtering data. The problem is, it is non-intuitive for users not familiar with how to solve problems in a map-reduce architecture. Therefore, I believe it is widening the gap above, the gap between the decision makers and the people who can use, manipulate, and extract information from big data.

Also, I see NoSQL databases, such as MongoDB, to begin playing a much bigger role in big data. But still, I think the analytics tools geared towards decisions makers is priority one, and providing a full stack, single brand, simple setup, scalable, big data solution as number two.
anon2709660884
50%
50%
anon2709660884,
User Rank: Apprentice
7/27/2013 | 10:31:14 PM
re: Building A Big Diverse Data Platform For Wall Street
I see one of the hardest challenges is to connect the understanding of the business from top to bottom and the understanding of the way to extract analysis from the data. How is an executive supposed to go about data mining their collected data? Or is it left up to the developers etc to determine the filters and what analysis to generate? In my opinion, the executive needs a way to manipulate the data on their own. Maybe a next generation "Excel" for big data / NoSQL databases?

Big data "in a box" is also interesting model. IBM has "cloud in a box" with Pure Systems. I think there is great value in providing a single brand experience from the top down in big data. "Expertise in a box".

As far as hadoop, it is great for sorting, grouping, and filtering data. The problem is, it is non-intuitive for users not familiar with how to solve problems in a map-reduce architecture. Therefore, I believe it is widening the gap above, the gap between the decision makers and the people who can use, manipulate, and extract information from big data.

Also, I see NoSQL databases, such as MongoDB, to begin playing a much bigger role in big data. But still, I think the analytics tools geared towards decisions makers is priority one, and providing a full stack, single brand, simple setup, scalable, big data solution as number two.
More Commentary
Voice Biometrics Improve Transaction Monitoring Fraud Detection
Why voice biometrics should be a part of your fraud prevention strategy in the call center.
Fintech Fast Forward 2015
What will shape the future of Fintech in 2015 and beyond?
Look Deeper at Business Connections
When a business person or practice crosses the line, what should a professional do?
Big-Data Analytics & Cloud: The Perfect Storm
Most signs are pointing to a big increase in investment in big-data analytics and cloud in the coming year.
Verifying Behavior, Not Input, to Detect Sophisticated Attacks
Understanding how users interact with the touch points with biometric information is an increasingly important part of digital security.
Register for Wall Street & Technology Newsletters
White Papers
Current Issue
Wall Street & Technology - Elite 8, October 2014
The in-depth profiles of this year's Elite 8 honorees focus on leadership, talent recruitment, big data, analytics, mobile, and more.
Video
5 Things to Look For Before Accepting Terms & Conditions
5 Things to Look For Before Accepting Terms & Conditions
Is your corporate data at risk? Before uploading sensitive information to cloud services be sure to review these terms.