How does object storage fit into the enterprise?
Object storage is highly scalable architecture where data gets stored by unique IDs and attributes that describe the data. It is commonly used in public clouds and is finding its way into the enterprise, especially to retain unstructured data. While object storage has its benefits, it has limitations too that need to be taken into account.
When faced with the decision to implement object storage, data center managers have basically two (2) choices: 1. they can either outsource their object storage needs or 2. bring it in-house. On-premise solutions in the data center usually mean adding yet another storage device, possibly from another vendor, to the already crowded floor.
The trade-offs between outsourcing and in-house solutions involve cost and scale, as well as security. A pragmatic approach involving balancing cost with scale will determine which approach is best. Just don’t be surprised if the tipping point falls in a gray area where a hybrid cloud model is the best solution.
Cloud-Driven Data Growth
Data center managers constantly struggle with growing storage requirements. This struggle, long-fueled by rapid structured data growth, is now compounded by even faster unstructured data growth (e.g. video, audio). Increasingly, enterprise customers are turning to object storage solutions to retain unstructured data.
The Beauty of Objects
What makes object storage so attractive is its use of a unique identifier and meta data describing the data. This combination removes the need for applications to optimize data for file systems. Specifically, object storage
systems remove file system size limitations, eliminating the need for applications to work across multiple file systems or LUNs to scale. Applications do not have to worry about directory path length, and the number of files per directory and hard links.
Object storage benefits for the enterprise include:
- Low Overhead: With its unique identifiers, object storage does not have the variable overhead inherent in the indexing required for block and file systems. Instead, object storage systems have constant overhead for object access that is not dependent on path length and directory size optimizations, and cache. Application access to all objects is equally quick.
- Automation: Object storage uses policy-based management that automates data placement and movement without the need to manually migrate data and optimize storage systems. Automation speeds data delivery, and reduces the possibility of human error, the need for multiple storage management tools, and training.
- Archiving Use Cases: Data stored as objects are treated as discrete entities. Any changed data is assigned its own identity making object storage well-suited for archiving use cases where ensuring data integrity is key.
- Multi-tenancy Support: Object storage is also easy to deploy in multi-tenant environments because policies can dictate what data users can access by associating only certain object identifiers with certain users.
- Scalability: Object storage systems can easily scale because objects exist as unique entities related in loose association defined by meta data and policies. There is no need for cache coherency and controlled redundancies across multiple arrays characteristic of block and file systems. Without this overhead, object storage systems are able to scale more easily than other storage types.
Working within Limitations
While object storage has many benefits for the enterprise, it also has limitations when applied to some traditional IT use cases. These limitations include:
- High-Performance Applications: Object storage systems do not provide the I/O throughput performance that many mission-critical applications require. These online transaction processing (OLTP) applications have highly localized working sets whose layout can be optimized for the specific paths to data common to the
hierarchical indexing found in block-based storage systems, but not in object storage. - High-Change Rates: File- and block-based storage make extensive use of sophisticated caching mechanisms involving close cooperation between clients and storage arrays. This approach allows for highly optimized code paths related to frequently changing data. Object-based systems, on the other hand, utilize standard HTTP/REST protocols for code paths which are much simpler, but potentially problematic, if data changes frequently. Database and collaborative applications are better suited for file-based systems optimized for frequent, small changes.
- Capacity Consumption: Object storage systems utilize a series of internal tables to translate application-defined names to storage locations efficiently. These tables, however, must be stored, which consumes capacity on the underlying storage and decreases the application-visible usable capacity. Block- and file- based systems utilize translations mechanisms that enforce certain application programming models, and are highly space efficient.
Object Storage Alternatives
Alternatives available to enterprises for acquiring object storage capabilities include public cloud providers and in-house solutions. Let’s look at each of these alternatives, including the economic trade-offs.
- Public Cloud: Public service providers such as Amazon Web Services offer storage-as-a-service. In this model, enterprises rent storage. No object storage infrastructure needs to be put in place. Upfront costs are low. Storage is available on-demand with no-waiting for a storage request ticket to be generated and storage to be provisioned. In this model, application administrators or developers are in control.
- In-house: With the in-house model, object storage is provided by IT in a private cloud environment. IT keeps control in the data center ensuring compliance with corporate policies for data protection, security, as well as metering and chargeback. IT provides ongoing storage management though more limited than traditional storage, handling upgrades, capacity monitoring and expansion, and the resolution of network bottlenecks
Identifying Your Tipping Point
What’s the right model for your enterprise?
The answer to the best approach for your enterprise lies in the cost per utilized resource and scale. If your object storage needs are minimal, a public cloud approach could be cost beneficial if it does not compromise internal governance. Public clouds offer financial advantages because their data center model supports multi-tenant storage pooling and scale. On the other-hand, if your object storage needs are substantial possibly encompassing large data analytics, an in-house approach might be more appropriate. Employing object storage in a private cloud model could be financially advantageous in the long run. It would offset short-term variable costs (OPEX) that could add up using a public cloud, with a financial investment (CAPEX) that could be depreciated and amortized over time.
Alternatively, if your needs vary, a hybrid cloud model might be the better alternative. Infrequently-accessed or less-critical unstructured data could be archived to the public cloud, while, more frequently accessed or business critical data could be kept on-premise. Regardless of where you decideto get your object storage, look at cost versus scale while keeping governance requirements in mind to determine your tipping point.