CloudTier Storage Tiering SDK

Download CloudTier Storage Tiering SDK Setup File
Download CloudTier Storage Tiering SDK Zip File

The Introduction

For most companies, data – especially unstructured data, continues to grow by 50 percent annually. The impact of spending more every year on storage, and on protecting and managing information has often pushed IT departments to their limits. Combined with longer mandated retention periods, today’s information management challenges are forcing IT staff, from the vice president to the systems administrator, to reduce complexity and cost without putting their organization’s infrastructure, information, and intellectual property at risk. The transparent cloud storage tiering is a simple and low-cost solution to integrate your on-premise storage to cloud seamlessly.

Transparent Cloud Storage Tiering SDK

Transparent Cloud Storage Tiering SDK provides you a simple solution to develop the cloud archiving software, it allows you to integrate your existing on-premises applications with the remote cloud storage infrastructure in a seamless, secure, and transparent fashion. The cloud tiering is a data storage technique which automatically moves data between high-cost on-premise storage and low-cost remote cloud storage, it provides a native cloud storage tier, it allows you to free up on-premise storage capacity, by moving out cooler data to the cloud storage, thereby reducing capital and operational expenditures.

Storage Tiering

Transparent Cloud Data File Access

By creating ILM (Information Lifecycle Manager) policy, a stub file will be created after the file was migrated to the cloud storage. A stub file looks and acts like a regular file, it has the same file attributes with the original physical file (file size, creation time, last write time, last access time), it also keeps the original file's security. A stub file is a file with sparse file and reparse point attributes, your customized tag data can be embedded to the stub file. A stub file doesn't take the storage space for the file data, it only keeps the meta data of the file. When the stub file is accessed, the file data in the cloud storage will be transparently recalled to the stub file, you can rehydrate the stub file or just return the data to the application based on the recall policy.

stub file property

Seamless Integration of Cloud and On-premises Applications

Our CloudTier Cloud Storage Connect service can connect an on-premise software appliance with cloud-based storage to integrate your existing on-premises applications with the remote cloud storage infrastructure in a seamless, secure, and transparent fashion.There are no interruption to migrate your on-premise files to the remote cloud storage, don't need to change your existing applications and infrastructure.

  1. Set up file cloud migration policies based on the file type, file size, file attributes.
  2. Create stub file based on the policies after the file was migrated to the cloud storage, it can free up the space from on-premise storage.
  3. Transparent the cloud storage access by reading the stub file for your local application.
  4. Transparent moving data back from remote cloud storage to the local, re-hydrate the stub file for the recent access file based on the policies.

Cloud Archiving Solution for Unstructured Data

Cloud archiving is the process of moving data to secondary storage in the cloud, the potential benefits of cloud archiving include lower costs and easier access, no interruption and change to your existing applications and infrastructure. Automatically archive, manage and secure all your organization’s files to the cloud, transparently access your archived files.


The Advantages of CloudTier Tiered Storage

  1. Lower Storage Costs:If you have two terabytes of expensive server storage where 50% of the data is never or rarely accessed, with CloudTier Tiered Storage, you can transfer a terabyte data to the NAS storage or cloud storage, you can save a terabyte storage space in SAN RAID storage.
  2. To maximize the server's hard disk available space: Reclaim storage space without disrupting users. Set up policies to automatically remove older files from file servers, cleaning up disk space, and replace them with an intelligent shortcut "stub" that invisibly retrieves the original file from the archive. 
  3. To improve efficiency:When the user needs these data, it can be accessed transparently in real time. If you need to recover and restore a file that was accidently deleted or modified in error, you can restore the file or even a whole folder from the repository.
  4. Reduce the server backup time and recovery time, only need to backup frequently used files
  5. Improve data security: the data in the server can be encrypted, and access these data through the storage management software, only authorized users can access the data, and can log the access activities.
  6. To remove duplicate data, the storage server only keep single instance

The Use Cases of Cloud Storage Tiering

Cloud storage tiering can be widely used in telecommunications, government, oil, medical and other industries. When users and applications access the stub files in the local storage, it is completely transparent, the system will automatically restore the data back to the stub file from the remote storage server. The network attached storage is scalable, tiered storage products provide users with an infinite online data space.

  1. Healthcare Data Archiving:Healthcare providers are increasingly required to retain patient medical record data for multi-year periods. Storage requirements for patient data can quickly escalate when the records include high resolution images and ultrasound content.
  2. Business Policy Mandated Data Archiving:Companies can generate PB’s of data in the course of day to day operations and to meet legal compliance requirements. Archive Storage can work with Smart Archiving offerings from ISV’s to create a low cost, content archiving solution. 
  3. Digital Media Content Retention:Creators can generate PB’s worth of video and picture content that is used in the development of original digital media. Archive Storage gives creators a low-cost storage repository for original source content. File-level tiering makes it easy to shift from cold to hot storage should the need arise to use that content for another project.
  4. Security/Public Safety data retention: As the number and sophistication of threats to personal and business safety continue to increase, so does the demand for video surveillance. Public and now private sector companies generate TB’s of surveillance footage daily in the course of protecting their citizens and assets. Archive Storage is a low cost option for storing that data.

Useing The Transparent Cloud Storage Tiering SDK In C#

The following example will generate some stub files. To handle the read request of the stub file, we need to register the callback function for the file system filter driver. When the stub file was accessed, the callback function will be invoked, the callback function will retrieve the data from the remote server and send back to the filter driver.

 public static Boolean ProcessRequest(MessageSendData messageSend, ref MessageReplyData messageReply)
            Boolean ret = false;


		//here the data buffer is the reparse point tag data, in our test, 
		//we assume the reparse point tag data is the cache file name of the stub file.
                string cacheFileName = Encoding.Unicode.GetString(messageSend.DataBuffer);
                cacheFileName = cacheFileName.Substring(0, (int)messageSend.DataBufferLength / 2);

                if (messageSend.MessageType == (uint)MessageType.MESSAGE_TYPE_RESTORE_FILE_TO_CACHE)
                    //for the first write request, the filter driver needs to restore the whole file first,
                    //here we need to download the whole cache file and return the cache file name to the filter driver,
                    //the filter driver will replace the stub file data with the cache file data.

                    //for memory mapping file open( for example open file with notepad in local computer,
                    //it also needs to download the whole cache file and return the cache file name to the filter driver,
                    //the filter driver will read the cache file data, but it won't restore the stub file.

                    ret = DownloadCacheFile(messageSend,cacheFileName, ref messageReply);
                else if (messageSend.MessageType == (uint)MessageType.MESSAGE_TYPE_RESTORE_BLOCK_OR_FILE)

                    //for this request, the user is trying to read block of data, you can either return the whole cache file
                    //or you can just restore the block of data as the request need, you also can rehydrate the file at this point.

                    //if the whole cache file was restored, you better to return the cache file instead of block data.
                    if (GlobalConfig.RehydrateFileOnFirstRead || GlobalConfig.ReturnCacheFileName)
                        ret = DownloadCacheFile(messageSend, cacheFileName, ref messageReply);
                        ret = GetRequestedBlockData(cacheFileName, messageSend.Offset, messageSend.Length, ref messageReply);
                    messageReply.ReturnStatus = (uint)NTSTATUS.STATUS_UNSUCCESSFUL;

                messageReply.MessageId = messageSend.MessageId;
                messageReply.MessageType = messageSend.MessageType;

                EventLevel eventLevel = EventLevel.Information;
                if (messageReply.ReturnStatus != (uint)NTSTATUS.STATUS_SUCCESS)
                    eventLevel = EventLevel.Error;

                ret = true;

            catch (Exception ex)
                EventManager.WriteMessage(181, "ProcessRequest", EventLevel.Error, "Process request exception:" + ex.Message);
                return false;

            return ret;


The example can monitor the stub file access, the user name and process name which accessed the stub file will be displayed in the console.

CloudTier Demo Screenshot