EaseTag Tiered Storage Filter Driver SDK Programming

Download the example source code

EaseTag Tiered Storage SDK

EaseTag Tiered Storage (hierarchical storage management, HSM) Filter Driver SDK, is a data storage technique which automatically moves data between high-cost and low-cost storage media, such as network attached storage(NAS),optical discs and cloud storage. A stub is created for and replaces each migrated file in the fast disk drives. On the local system, a stub file looks and act like a regular file. When the user application accesses a migrated file stub, the Windows operating system transparently directs a file access request to the EaseTag Tiered Storage SDK. The EaseTag driver will send the request to the remote site to retrieve the data back from the repository to which it was migrated(see Figure 1).

EaseTag File

Figure 1. Tiered Storage Data Flow Chart

Use  EaseTag  Tiered Storage SDK step y step

 

Use for C++ application

Copy the correct version (32bit or 64bit) EaseTag.sys, FilterAPI.DLL, FilterAPI.h and FilterAPI.lib to your folder. FilterAPI.h file  includes all the functions and structures used for connecting to the filter driver.

Use for C# application

Copy the correct version (32bit or 64bit) EaseTag.sys , FilterAPI.DLL  and ,EaseTag.cs  to your folder. EaseTag.cs has the structures and APIs  used for connecting to the filter driver.

Set up the filter

Install the filter driver with InstallDriver() method if the driver has not been installed yet. After filter driver was installed, the filter was loaded, if not you can load the filter with command “Fltmc  load  EaseTag” in dos prompt.  To remove the filter driver from the system, call UninstallDriver() method.

Start the filter

1.  Activate the filter with API SetRegistrationKey(). You can buy a license key with the link: http://www.easefilter.com/Order.htm   or email us  info@EaseFilter.com  to request a trial license key

2.  After register the callback function with API RegisterMessageCallback, filter is started.

 

BOOL ret = RegisterMessageCallback( FilterConnectionThreadsCount, MessageCallback, DisconnectCallback);

 

3. Reset the filter configuration after filter was started, Set the connection timeout value.

     ResetConfigData();  

 

      //Set filter maiximum wait for user mode response time out.

ret = SetConnectionTimeout(30);

File Migration

Transfer file data to the server, you can store the data to a cloud server, or to your local server, or you can store it in the same server with different volume. This step is very important, make sure your data was stored in remote site correctly, later the data needs to be accessed when the stub file was created.

Create stub file

A stub file looks and acts like a regular file. It has the same file attributes with the original physical file (file size, creation time, last write time, last access time). It also keeps the original file's security. The difference between the stub file and the normal physical file is the stub file doesn't take any physical space, looks like a 0 kb file.

A typical stub file has these attributes: offline attribute ( it tells the other applications the file data is not in local, e.g.,antivirus software, backup software they can skip these files); reparse point attribute, you can put your data up to 14kb to the reparse point tab; sparse file attribute.

                            

BOOL

CreateStubFile(

      LPCTSTR           fileName,

      LONGLONG          fileSize,

      ULONG             fileAttributes,

      ULONG             tagDataLength,

      BYTE*             tagData,

      BOOL              overwriteIfExist,

      PHANDLE           pHandle )

To create a stub file, you need to pass the tag data, you can put your binary data there for your later use, when user access the stub file, the filter driver will invoke the message callback function with the tag data along with other user information ( process Id, thread Id, user SID...).

If you just want the filter driver reparse your file to other folders in the same computer or in the same network which can be accessed by SMB protocal, the tag data must follow the following data type format, with this format tag data, the filter driver won’t invoke the callback function:

typedef struct _EASETAG_DATA

{

      ULONG       EaseTagKey;

      ULONG       Flags;

      ULONG       FileNameLength;

      WCHAR       FileName[1];

     

} EASETAG_DATA, *PEASETAG_DATA;

 

The file name can be local path ( e.g. c:\reparseFolder\test1.txt) or a UNC path ( \\192.168.100.100\myProtectFolder\test1.txt).

Here is the example to create a stub file:

 

      BOOL ret = FALSE;

      HANDLE hFile = INVALID_HANDLE_VALUE;

CHAR* reparseData = "This is reparse point tag data for test in the stub file. The maximum length is 16*1024 bytes.";

ULONG tagDataLength = (ULONG)strlen(reparseData);

 

ret = CreateStubFile(L”c:\\fileRestoreTestFolder\\stubFile.txt”,100000000000,FILE_ATTRIBUTE_NORMAL,tagDataLength,(BYTE*)reparseData,TRUE,&hFile);

      if( !ret )

      {

ULONG messageLength = 1024;

            WCHAR* buffer[messageLength];

 

            if(GetLastErrorMessage(buffer,&messageLength ))

            {                

                  PrintMessage( L"CreateStubFile failed.%ws”, buffer);

}

      }

 

      if( INVALID_HANDLE_VALUE != hFile )

      {

         CloseHandle(hFile);

      }

Here is the stub file snapshot:


stub file


Access the stub files

Access the stub files just like the normal files, on the first read or write, the stub file’s data will be restored back. To user application, it is transparent. If the user just read the file, you have two options: the first option is to restore the whole stub file, the stub file will turn into the original physical file, you need to re-stub the file in the later process; The second option is just restore the block data which the user needs, this approach is good for large file, it is much faster than restore the whole stub file.

 

BOOL

__stdcall

MessageCallback(

   IN       PMESSAGE_SEND_DATA pSendMessage,

   IN OUT   PMESSAGE_REPLY_DATA pReplyMessage)

{

      BOOL  ret = TRUE;

HANDLE  hFile = INVALID_HANDLE_VALUE;

 

      __try

      {

 

            if(MESSAGE_TYPE_RESTORE_FILE == pSendMessage->MessageType)

            {

                  //Restore the whole stub file here.

                  //open the same file handle,it will bypass the share check.

                  //if the stub file was set to readonly,it will fail,we can't restore the file.

                  //if you still want to restore the file, you need to remove the readonly, after

                  //restore the data, set the readonly back.

                  ret = GetFileHandleInFilter(FileName,&hFile);

                  if(!ret)

                  {

                        PrintLastErrorMessage(L"GetFileHandleInFilter failed.",__LINE__);

                        goto EXIT;

                  }

 

                  ULONG bytesWritten = 0;

                  ULONG bufferLength = 65536;        

                  CHAR* dataBuffer[bufferLength];

                  LONGLONG offset = 0;

 

                  WCHAR* fileName = pSendMessage->FileName;

                  LONGLONG fileSize = pSendMessage->FileSize;

                  ULONG tagDataLength = pSendMessage->DataBufferLength;

                  CHAR* tagData = pSendMessage->DataBuffer;

 

                  While(ret && offset < fileSize )

{

                        //Get restore data from your remote server with the information you need.            

                        ret = GetRestoreData(offset,bufferLength,dataBuffer);

if(!ret)

                        {

                              PrintErrorMessage(L"GetRestoreData failed.",GetLastError());

                              goto EXIT;

                        }

 

                 

                        ret = WriteFile(hFile, dataBuffer, bufferLength,&bytesWritten,NULL);

                        if(!ret)

                        {

                              PrintErrorMessage(L"WriteFile failed.",GetLastError());

                              goto EXIT;

                        }

                       

                        Offset += bytesWritten;

}

 

                 

//after the data was restored, then remove the tag data.

                  ret = RemoveTagData(hFile);

                  if(!ret)

                  {

                        PrintLastErrorMessage(L"RemoveTagData failed.",__LINE__);

                        goto EXIT;

                  }

 

                  if(ret)

                  {

                        pReplyMessage->FilterStatus = STUB_FILE_WAS_RESTORED;

                        pReplyMessage->ReturnStatus = STATUS_SUCCESS;

                  }

                  else

                  {

                        pReplyMessage->ReturnStatus = STATUS_UNSUCCESSFUL;

                  }

            }

            else if ( MESSAGE_TYPE_RESTORE_BLOCK_OR_FILE == pSendMessage->MessageType )

            {

                  //Restore the block data here.

           

                  LONGLONG readOffset = pSendMessage->Offset;

                  ULONG readLength = pSendMessage->Length;

                  LONGLONG fileSize = pSendMessage->FileSize;

ULONG tagDataLength = pSendMessage->DataBufferLength;

                  CHAR* tagData = pSendMessage->DataBuffer;

                  CHAR* dataBuffer[readLength];

 

                  //do your own verififcation here.

 

                  if(         readOffset >= fileSize

                        ||    readLength > MAX_MESSAGE_SIZE)                 

                  {

                        pReplyMessage->ReturnStatus = STATUS_END_OF_FILE;

                        goto EXIT;

                  }                      

                 

                  //here you can get your data from remote server

 

                  //Get restore data from your remote server with the information you need.                 

                  ret = GetRestoreData(offset, readLength,dataBuffer);

 

                  //set your actual return data length here.

                  pReplyMessage->DataBufferLength = readLength;

                  memcpy(pReplyMessage->DataBuffer, dataBuffer, readLength);

 

                  pReplyMessage->FilterStatus = READ_DATA_WAS_RETURNED;

                  pReplyMessage->ReturnStatus = STATUS_SUCCESS;

 

            }

      }

      __except( EXCEPTION_EXECUTE_HANDLER  )

      {

            PrintErrorMessage( L"MessageCallback failed.",GetLastError());    

      }

 

EXIT:

 

if( INVALID_HANDLE_VALUE != hFile)

      {

            CloseHandle(hFile);

      }

 

 

      return ret;

}

 

 

 

 

Social Network


Services Overview

Architect, implement and test file system filter drivers for a wide range of functionality. We can offer several levels of assistance to meet your specific.

Contact Us

You are welcome to contact us for salse or partnership.

Sales: sales@easefilter.com
Support: support@easefilter.com
Info: info@easefilter.com