Introduction

Today’s enterprises face unprecedented volume growth at a higher speed of inflow for both structured and unstructured data. HCL understands the necessity to have a strong and robust archiving solution that can store enormous amounts of data quickly and scale to meet business requirements for data retention and retrieval.

Data archiving is done to secure and to store data for long term retention both on premise or cloud environment. Data archiving solutions allow important information to be stored in safe locations so that it can be used whenever required. The storage process will generally have some policies applied to it depending on the information in question. For example, policies might include the retention time frame and security sensitivity.

Data archiving is done to secure and store data for long-term retention, both on-premise or cloud environment.

The need to improve enterprise application performance, adhere to regulatory compliance guidelines for audit and electronic discovery (e-discovery), and reduce operational costs are pressing enterprises to deploy effective and efficient data archiving solutions. The key enterprise expectations for next-generation archiving are data reusability, scalability, and extensibility in the next couple of years.

Key Business Needs for Data Archival

Impact on Performance

Unutilized Data

Operational Inefficiencies

Data Breach

Electronic archiving or eArchiving-as-a-Service (AaaS) is becoming an increasingly popular option for organizations looking to reduce overheads and streamline their IT operation. Active archiving will enable you to store data securely off-site and retrieve it quickly and easily for compliance or disaster recovery. It removes data from more expensive primary storage, reduces costs associated with capacity-based backup software, shrinks your backup windows, and improves your recovery time objectives.

Some of the key benefits of e-Archiving-as-a-service are as follows:

• Reduced capital expenditure
• Ease of implementation
• Reduced operational and support overheads
• Predictable costs and scalability
• Data reusability, flexibility, and security

Typical Archiving Strategy Considerations

Archiving Solution Parameters

Data Archiving - Reference Architecture

Click here to view the Image

Sample e-Archiving Operating Model

Click here to view the Image

e-Archiving - Service Catalogue

Click here to view the Image

Classification of Small/Medium/Large Archiving Projects

Comparison Across Solution Approaches

Solution Approach One: COTS Product-based Solution Using Informatica ILM

HCL proposes to provision and use the Informatica ILM tool in the targeted landscape and do necessary customization development (outside the ILM tool) to support various other organizational requirements.

Data Vault (inbuilt within the ILM package) will be the target archival repository, which will compress the data up to 85% and provide easy retrieval access at a record level.

Solution Approach Two: Custom-based Solution Using Hadoop

Hadoop is a modern open source framework, designed for storing and processing large amounts of data.

Hadoop is a modern open-source framework designed for storing and processing large amounts of data. A Hadoop NoSQL-based worm archive services for enterprises have the following high-level advantages:

• Unlimited scale to store petabytes of data | Fastest data ingestion due to schema-less writes
• API-based data retrieval and search | Compliance management – complete security
• Ability to run analytics – well integrated into the Hadoop ecosystem

HCL recommends the use of HCL framework data movement framework (DMF) to ingest data for archival into the archival store in Hbase NoSQL store. It’s for the record-archival worm storage along with SOLR for indexing hot data for full-text search. DMF is a data flow lifecycle management application that brings in a process for the typical information lifecycle management (ILM) activities in a Hadoop-based archival storage system.