Are you a spreadsheet engineer or a mashup builder?
These monikers are colorful terms for sure. But, they are also descriptive of the choices you will soon have for constructing meaningful views into your IT infrastructure data.
Now that the EMC World hangover has subsided, I thought it might be timely to go deeper into one of the more interesting technology announcements made at this annual event. For this purpose, I enlisted the help of our crack product team and without much effort convinced the chief architect behind the EMC DataBridge enterprise software solution to explain some of the concepts herself.
Every IT organization needs to have visibility into their environment. And, there is no shortage of products and solutions that provide deep insights into your compute, network, and storage environments. However, when it comes to higher-level views and understandings of how services get consumed, DataBridge makes it easier than ever to assemble your existing infrastructure management data into meaningful information.
Spreadsheet Engineers
All IT organizations have various people who are responsible for specific aspects of delivering IT services, whether it’s the storage, backup, or VMware administrator. While they all tend to focus on their area of expertise, they are often asked to provide visibility into how their services impact other parts of the business, whether it be about consumption or usage, total assets employed, or some other cross-functional bit of information. Senior management is usually behind the request for this information; however, the administrators benefit from a broader understanding of how their services are consumed too.
Almost all of the people we spoke with at EMC World who routinely get asked to provide this level of visibility do so by consolidating data from multiple systems into multiple spreadsheets. It’s extremely time consuming and error prone, not to mention a waste of valuable resources. They all laughed when called “spreadsheet engineers” but it is an all-too-true analogy to what they do.
Mashup Builders
Why be a mashup builder?
Virtual data centers (VDCs) and cloud architectures offer the promise of agility and efficiency, but many IT organizations fail to harness the full power of the new technologies because they use legacy products that manage resources independently. Converged infrastructure requires software that spans across domains to control the entire environment to speed deployment, services, and ongoing management.
Enterprise mashups enable business users to combine silos of data in new, meaningful ways to gain greater visibility into their IT infrastructure, resulting in actionable data enabling them to make better, faster decisions. Using an enterprise mashup toolkit such as the DataBridge enterprise software solution and some basic principles for building these mashups empowers organizations to more easily and systematically combine data silos in new, more meaningful custom dashboards. It’s an alternative to the manual effort required of the spreadsheet approach to consolidating information across domains—as well as much faster and less prone to human errors.
The DataBridge Toolkit
The basic constructs of DataBridge were covered here in a recent previous post. With some grounding, let’s now go a bit deeper and look into the architecture behind this mashup solution.
As senior technologist, Cornelia Davis from the EMC office of the CTO explains in the accompanying video, DataBridge mashups perform data integration, which then have visualizations applied to form widgets. The architecture for the data integration portion which happens in DataBridge studio (where the magic happens) is generally broken into three (3) layers:
1. Mashup level 1 (M1): Extract data
Building enterprise mashups starts with extracting or accessing data from a number of sources at the M1 level (that’s 
mashup level 1). This data can come from EMC infrastructure management data sources (think, EMC ProSphere, EMC Data Protection Advisor) through the EMC Data Access API (EDAA) RESTful interface. Or, data can come from customer files in a variety of common formats such as CSV, Excel, and XML, or via mechanisms such as SQL, SOAP/WSDL and REST web services, and more. This could mean accessing primary storage data from ProSphere, backup storage data from Data Protection Advisor, and tenant storage rates, applications and locations from customer’s own Excel files, and bringing all this data together.
2. Mashup level 2 (M2): Combine data
While the M1 level addresses data access, the next level up is where you combine, filter, and process data. This layer employs a visual programming environment for constructing relationships among data for analysis purposes. You link data sources together, apply filters and sorts, and prepare the data. For example, combine primary and backup storage data for a single tenant in terms of revenue and capacity. The result becomes a building block for the next level.
3. Mashup level 3 (M3): Format data
At this level, you build upon the previous mashups to do more analytics and filtering, working out the data format, though not how the data gets presented or visualized. For example, you could use the single tenant M2 mashup to create a multi-tenant M3 mashup that adds additional analytics and filtering to all the tenants.
Apply Visualizations
Once you have combined, filtered and analyzed the data, the next step involves selecting and configuring visualizations to present the results. Like a fine dining experience, it’s all in the presentation.
In this example, storage revenue and allocation data is displayed via bar charts, producing the Storage Chargeback and Storage Resource Analysis widgets that will be available with the initial DataBridge release. These widgets are then available as building blocks for your interactive dashboards. View the Storage Resource Analysis and Storage Chargeback demonstration to gain a better understanding of the DataBridge possibilities.
Building for VDC and Cloud
As you move up through the DataBridge architectural levels or layers, the tasks become simpler. Your more sophisticated developers effectively extend the platform by producing more reusable components, and your non-programmer, business analysts or power users consume these components using the visual programming tool.
Going back to our characterizations of spreadsheet engineers and mashup builders, it’s not clear if you can really claim a one-to-one swap with all spreadsheet engineers becoming mashup builders. There is a new role emerging in this swirling IT data center evolution around the notion of data, including big data, and analytics. The new role involves business intelligence (BI) users (maybe called data scientists, in some cases) who will construct their own mashups and/or dashboards.
So, maybe the original question should be recast to ask whether you’re a spreadsheet engineer, mashup builder, business analyst, or data scientist. Regardless of the label, you can get to decisions more quickly with EMC DataBridge.