CloudTier Storage Tiering SDK

Download CloudTier Storage Tiering SDK Setup File
Download CloudTier Storage Tiering SDK Zip File

Introduction

Nowadays unstructured data continues to grow by 50 percent annually. Combined with   business needs or other obligations for longer mandated retention periods, today’s information management challenges are forcing IT staff, from the vice president to the systems administrator, to reduce complexity and cost without putting their organization’s infrastructure, information, and intellectual property at risk.

The CloudTier Storage Tiering SDK provides a simple storage solution to integrate the cloud as a second-tier storage, to reduce the cost of the storage, to enhance the data security, to scale the cloud storage unlimitedly.

What is Storage Tiering?

The storage tiering stores the most important or frequently-accessed data on the fastest, and most expensive media (SSD), stores the least important on the slowest, cheapest media (hard disk, tape or cloud storage). The minimal storage tiering system has two tiers—tier 0 for frequently accessed data and tier 1 for archive.

CloudTier Storage Tiering SDK uses the on-premises storage as a tier 0 storage (hot storage), uses the public cloud storage as a tier 1 storage (clod storage). The hot storage only stores the hot data which is highly volatile and time-sensitive and which needs to be available in the shortest possible time. The cold storage only stores the cold data (archived data) which isn’t performance-sensitive, rarely accessed and will be available with high latency.

CloudTier Automated Storage Tiering

Automated storage tiering (also Hierarchical Storage Management, HSM) is a data storage technique that automatically moves data between high-cost and low-cost storage media, is the automated progression or demotion of data across different tiers of storage devices and media.

The CloudTier Storage Tiering SDK provides you an automatic way of managing and distributing data between the different storage, allows the automated data movement between the tiers of storage based on the defined rules and policies.

Transparent Data Migration to the Cloud

Data migration is the process of moving the on-premises data to the cloud. The key consideration of the data migration is the interruption of the business data access and the current application adaption of the cloud data.

With the CloudTier Storage Tiering SDK you can integrate your existing on-premises applications with the remote cloud storage infrastructure in a seamless, secure, and transparent fashion. There is no interruption to migrate your on-premise files to the remote cloud storage, so you don't need to change your existing applications and infrastructure.

Cloud Archiving Solution for Unstructured Data

Cloud archiving is the process of moving data to secondary storage in the cloud. Potential benefits of cloud archiving include lower costs and easier access. Through the use of Hierarchical Storage Management, you can define criteria to archive less frequently used data from local disk to the cloud. The criteria can be based on inactivity limit, size, or even days used per month. Once the data meets the criteria, it is then migrated to the cloud.

After the file was migrated to the cloud, you can replace the file with a stub file, no longer taking up that much-needed space. A stub file looks and acts like a regular file, it has the same file attributes with the original file (file size, creation time, last write time, last access time), it also keeps the original file's security. A stub file can embed with your custom tag data. A stub file doesn't take the storage space as below figure.

Integrating Cloud Storage as a Second Tier

CloudTier Storage Tiering SDK was implemented with tiered storage file system filter driver. A file system filter driver intercepts requests targeted at a file system or another file system filter driver. With the ClooudTier Storage Tiering, you can connect the cloud storage as a second tier, seamless as a local on-premise storage, your application can use the cloud storage just like the local storage without any changes.

Tiered Storage on AWS S3

Amazon S3 or Amazon Simple Storage Service is a service offered by Amazon Web Services (AWS) that provides object storage through a web service interface. With CloudTier Storage Tiering, you can map your local folder to AWS S3 bucket. Here is a C# example to demo how to integrate the AWS S3 into your local file system.

1. Make sure you have a S3 key pair. You will need both the access key ID and the secret access key in order to continue. You can get them from the S3 console website.
2. Select Amazon_S3 cloud provider name. Click "Add Site" button to create a new site for the amazon s3 connection.
3. Put your site name and then enter your access key id and secret access key in the text boxes, choose the region in your setting.
4. Check the enable upload multiple parts checkbox if you want to use parallel upload tasks for a file.
5. Check the enable parallel download checkbox if you want to use parallel download tasks for a file.
6. Set the number of the parallel tasks for upload or download.
7. After filled in all the data, click apply to save the settings.
8. Click test connection to check if your setting is correct.

Tiered Storage on Azure Blob

Azure Blob storage is Microsoft's object storage solution for the cloud. Blob storage is optimized for storing massive amounts of unstructured data. Unstructured data is data that doesn't adhere to a particular data model or definition, such as text or binary data. With CloudTier Storage Tiering, you can map your local folder to Azure Blob. Here is a C# example to demo how to integrate the Azure Blob into your local file system.

1. Get your connection string from the Microsoft Azure Dashboard Portal site, by clicking on the link to the Dashboard website.
2. Select AzureStorage cloud provider name. Click "Add Site" button to create a new site for the Azure storage connection.
3. Put your site name and then enter your connection string in the text box.
4. Check the enable upload multiple blobs checkbox if you want to use parallel upload tasks for a file.
5. Check the enable parallel download checkbox if you want to use parallel download tasks for a file.
6. Set the number of the parallel tasks for upload or download.
7. After filled in all the data, click apply to save the settings.
8. Click test connection to check if your setting is correct.

Dynamic Retrieval of Archived Data

Once the file was archived to the cloud, it is very important to user to retrieve the archived file in a fast and simple way. With the CloudTier Storage Tiering SDK, the CloudTier can retrieve the archived data dynamically and return the data back to the application transparently. You can restore the archived file automatically and it is completely transparent to the application or user.

The Advantages of CloudTier Storage Tiering

The CloudTier Storage Tiering provides advantages that include:

1. Simplicity. A simple storage solution allows you to integrate the cloud storage transparently.
2. Cost-savings. Storage tiering makes it possible to purchase less high-performance storage equipment and leverage the lower-cost cloud storage.
3. Protection. With data in your tiered storage system geo-distributed across different locations, you can easily protect against site outages.
4. Near-unlimited scalability. On-demand resources are available to meet your business needs and growth.
5. Improve efficiency. Archived data can be accessed transparently in real time.

The Use Cases of CloudTier Storage Tiering

Cloud storage tiering can be widely used in telecommunications, government, oil, medical and other industries.

1. Healthcare Data Archiving. Healthcare providers are increasingly required to retain patient medical record data for multi-year periods. Storage requirements for patient data can quickly escalate when the records include high resolution images and ultrasound content.
2. Business Policy Mandated Data Archiving. Companies can generate PB’s of data in the course of day to day operations and to meet legal compliance requirements. Archive Storage can work with Smart Archiving offerings from ISV’s to create a low cost, content archiving solution.
3. Digital Media Content Retention. Creators can generate PB’s worth of video and picture content that is used in the development of original digital media. Archive Storage gives creators a low-cost storage repository for original source content. File-level tiering makes it easy to shift from cold to hot storage should the need arise to use that content for another project.
4. Security/Public Safety data retention. As the number and sophistication of threats to personal and business safety continue to increase, so does the demand for video surveillance. Public and now private sector companies generate TB’s of surveillance footage daily in the course of protecting their citizens and assets. Archive Storage is a low-cost option for storing that data.