Proficient at handling and interpreting big piles of market data to pinpoint trading opportunities, financial services firms have pioneered innovative data management for decades. However, things are different today. The piles of data are much bigger, faster, and far more unstructured and complex than they ever have been, necessitating new industry responses with implications that go far beyond trading. For example, in the wake of the Great Recession of 2007-2012, regulators require financial services institutions to provide increased transparency and risk mitigation to allay concerns about the safety of capital. This has led to a spike in new market data types and sources, creating exploding volumes of multi-structured big data that cannot be stored in conventional systems. So, beyond the pursuit of trading efficiencies, one of the biggest IT pain points that banks and capital markets firms face today is the need to tame this “Big Data” conundrum (i.e. by better understanding, integrating and aggregating their data) to comply with a bevy of more rigorous regulations like Dodd-Frank. As a result, according to the CEB TowerGroup “Capital Markets: Top 10 Technology Initiatives for 2012” report, analysts expect 2012 to be the year when “[capital markets] firms begin to employ big-data techniques to address problems in risk management, regulatory compliance, and portfolio analytics.” So, what does this mean for banks and capital markets firms?
Before answering this question, perhaps a definition of Big Data is in order. It is my view that Big Data occurs when increasingly voluminous, varied and complex data stores and movements encumber efficient data analysis with tools currently available to an enterprise. Since the threshold at which Big Data occurs will depend on a variety of enterprise characteristics (i.e. company size, the weight of legacy data workloads and the precise mix and capability of legacy data management tools), I define Big Data holistically and inclusive of all data workloads and classes. In other words, I view Big Data solutions as a superset of the entire data management tools spectrum, including BI and Analytics, RDBMS’, NoSQL Databases, etc. Gartner and Bank of America also define Big Data this way. For example, in a recent report “CEO Advisory: ‘Big Data’ Equals Big Opportunity published in March 2011”, Gartner defines Big Data as “the point where your current data collection and analytical paradigms start to fail to meet your needs. Big data varies, depending on technology available to you.” As such, in Dell interviews conducted in May, 2012, Gartner analysts estimated the 2012 WW Big Data market size to be $45-$100B (Beyer, 2012), while Bank of America estimates the 2012 Big Data opportunity at $100B in a recent study.
Alternatively, some experts view Big Data as being anchored on specific volumes and classes of data that defy conventional data processing and analysis (i.e. Big Data as a subset of overall data). Predictably, these experts estimate the Big Data opportunity to be much smaller. While the market size and definition of Big Data is still being debated, one thing appears certain: There is a burgeoning need for Big Data solutions within the financial services industry. Beyond trading, Big Data applications within banking and capital markets include Risk Analytics; Price Discovery; Fraud Analytics; Customer Behavior Analytics, etc. Lastly, a Gartner 2010 data management survey of banks and investment services firms underscores the vast potential of Big Data IT solutions within the banking and capital markets sector. It found that only 43% of responding firms give the quality of their data for supporting operations a high rating. Similarly, only 33% gave the quality of their data for supporting business intelligence and management decision making a high rating. (Knox, 2011)
Today, Dell is introducing a new solution that is helping our customers, banking and securities firms included, significantly reduce the pain and costs of retaining big data while improving management for easier retrieval and analysis. Being able to cost-effectively capture and store all of the relevant data makes it possible for financial services organizations to meet compliance but also gain insights to help grow the business. We anticipate that our new solution will help them do just that.