Due to the astronomical growth of data over the past decade or so, enterprises are struggling to find ways to balance the needs between business growth and its ease of operation. One of the many challenges for modern IT is to archive cold data (i.e. less frequently accessed data, to relatively cheap storage such as tape, optical storage media like bluray/DVD) in order to free up space from the more expensive storage tiers, such as primary storage and all-flash arrays. In addition to freeing up space, verticals such as Government, HealthCare, Legal, Financial and Telecom have a specific need to retain their cold data for a long time for many reasons.
Chief among these:
- To lower the total cost of ownership (TCO) by migrating older data to cheaper storage
- Compliance with data regulation and retention rules. Does SOX, HIPAA, SEC, FINRA, PCI sound familiar to you?
- Increased protection and data redundancy
- Big Data Analytics on historical data to gain key insights
Offsite archival of cold data offers a very cost-efficient way to fulfill this objective. Let’s talk about how IT is adopting the cloud for their data archival needs.
Cloud for Archival
Today’s “state-of-the-art” archiving solutions are standalone solutions that require customers to manage multiple licenses from different vendors for data protection as well as for archival. They require multiple UIs, gateways, servers, and even writing custom scripts to accomplish tasks not native to the solution. Don’t even get me started on the complexities of managing tape libraries for archival and recovery. In most organizations, the standard practice for archival is to archive data to the tapes and ship them off-site (e.g. Iron Mountain).
In my view, cloud offers a cheaper and more agile way to address an enterprise’s data retention goals. Recovery from these archives may be done in minutes to a few hours as compared to days/weeks in a traditional tape-based archival solution. For those enterprises whose compliance goals allow them to get away from tape, our cloud archival is an order of magnitude simpler. Now, let’s discuss our storage platform integration with both Public and Private Clouds for archival and why it is way better than the current Cloud Gateway solutions for data archival.
Cloud Archival with Cohesity
At Cohesity, we take great pride in delivering the most comprehensive, flexible and enterprise-grade solution to our customers. Cohesity Data Platform has integration with major Public Cloud service providers, and also integrates with any Private Cloud Object Store that is AWS S3 (Simple Storage Service) compliant.
There are two main use-cases for which Cohesity leverages the power of Cloud storage:
- Cloud Archival – Archiving the older local snapshots in the Cohesity cluster to cloud for long-term retention. Unlike tapes, cloud makes an attractive choice for archival as it essentially provides you with an infinite, elastic and cheaper storage solution, and you don’t need to worry about managing those media libraries on a regular basis. As restoring from archivals is rather rare, the cost associated to recover from cloud can still be kept under control.
- Cloud Tiering – Using cloud as an extension to our cluster’s built-in storage tiers, we have the ability to algorithmically decide when to down-tier or up-tier the data between your Cohesity cluster and the Cloud.
In this post, we’re going to focus on Cloud Archival. Our Cloud Tiering solution is worthy of its own post in the future.
Here are some additional highlights of our Cloud Archival solution:
- Unlimited Protected Data in the Cloud – Scale-out architecture of our underlying platform, efficient index storage within the Cohesity cluster as well as in the cloud (as a backup), empowers you to manage as much data as you like in the Cloud. This is one of the key differentiators of Cohesity when compared with the rest of the market.
- Secure – For security geeks, the AES 256-bit encryption used for data-at-rest and in-flight is FIPS 140-2 Level-1 compliant.
- Intuitive and Simple UI – No brainers! It only takes minutes to setup a cloud archival job.
- Integrates with all major Public and Private Clouds.
Now let’s dive deeper into our solution. In the spirit of flexibility and options for the customers, our platform supports all major public cloud services, including Amazon (AWS), Google Cloud Platform, and Microsoft Azure. You can pick the cloud provider of your choice and archive your data to the storage tier of your choice based on your needs. For example, if you are already invested in AWS then you can select either S3, S3-SIA (if you care about retrieval time and are fine with higher cost) or Glacier (if you care about cost and are patient enough to wait for a few hours before you can retrieve your data). If your performance, cost and compliance objectives are met by Google Nearline or Azure Standard Storage, then choose them instead. We don’t have a dog in this fight and strive hard to play fair with everyone. All of these are still provided by our single pane of glass!
But, what if security (and yes cost) objectives don’t allow you to push your data out of your data center but rather archive it to the private cloud that your IT staff has diligently setup? No worries, we have that covered! Cohesity can also archive to any S3-compatible object storage!
-
- Cleversafe
- Openstack Swift
- Rackspace Cloud Files
- Caringo Swarm
- …and more!
We have painstakingly tried to simplify setting up the archival workflow and managing the archival data sets when you need them. Here are the simple steps to setup the data archival:
-
- Register the cloud vault, an abstraction of the cloud storage service, as an External Target within your Cohesity cluster. This is where all your archived data sets will be stored. You can configure to encrypt and compress the data-at-rest for registered External Targets.
- Add or update an existing data protection policy to enable archival of the backup job with the desired retention period (typically over multiple years) and RPO (Recovery point objective of weekly or monthly). You can learn more about our data protection policy from this awesome video.
That’s it! You are all set to start archiving your older snapshots from Cohesity to the cloud storage service of your choice! As per your requirements, you can easily use multiple cloud storage providers with different RPO and retention objectives.
Simplifying Complex Data Management!
By default, data-in-motion is transferred to the cloud over secure channels (HTTPS). However, for data-at-rest, we offer additional security with different keys across different cloud vaults. These keys are managed by our internal key management system that allows you to rotate your encryption keys at any point without having to re-encrypt the entire datasets with new keys. That’s how we provide industry-compliant security for your archived data.
When we archive your data to the cloud, index metadata is also stored along with the archived data set. This enables you to execute a full recovery to a totally new Cohesity cluster, in the event your primary cluster had some unrecoverable failures. In addition, this also empowers you to create another copy of your archived data from the cloud to a new Cohesity cluster. As the primary Cohesity cluster has metadata index stored for all the archived local snapshots, you will be able to search for any specific file in a VM from your on-premises Cohesity cluster and restore it from a specific cloud-archived snapshot.
Conclusion
There you go! A scalable, easy-to-manage, and flexible solution for long-term retention and performance that other cloud gateway solutions cannot provide!
When talking about long-term archival, the question of support for tape libraries always comes up. If you find yourself in a position where you still have to deal with tape, have no fear! We will cover how we address even that seamlessly, in a future blog post. Stay tuned with our upcoming releases packed with unprecedented innovation!
What challenges do you face when it comes to archival? Are you currently considering leveraging the cloud as an alternate solution? Let us know in the comments below how we can help you to make the transition!
