What is VSAM
VSAM or Virtual Storage Access Method is a file storage access method used in MVS, ZOS, and OS/390 operating systems. It was introduced by IBM in 1970's. It’s a high-performance access method to organize data as files in mainframes. VSAM is used by programming languages in mainframes to store and retrieve data. VSAM files makes it easier for application programs to execute an input-output operation.
VSAM files are used in mainframe programs (such as COBOL, PL/I, Assembler) like physical sequential files and within the programs, the records are either read sequentially like a flat file or randomly using keys to point to the specific record. There are four types of VSAM file systems which improves data storage and retrieval by overcoming certain restrictions that normally occurs while accessing conventional files like Sequential Files.
- Key Sequence Data Set (KSDS) is the widely used where records are identified with keys and any Data Manipulation operation could be done using the key value.
- Entry Sequence Data Set (ESDS) where records are kept in sequential order and accessed as such.
- Relative Record Data Set (RRDS) where record numbers are used as keys to access records.
- Linear Data Set (LSDS), is a byte-stream data set in a traditional z/OS file. It is rarely used in applications.
With an increasing demand on migrating or modernizing application that uses legacy file systems such as VSAM, it is becoming imperative to convert the files to Relational Database Management System (RDBMS). It not only helps managing the data effectively, but also reduces the cost even in legacy systems.
Modernizing legacy file system for robust file management of mainframe system
Limitations of VSAM
The major limitation of modernizing VSAM is that there are very few, but costly tools and the cumbersome process to access VSAM and made to work outside of the mainframe. VSAM files can be stored only on magnetic disk storage devices called as DASD (Direct Access Storage Device) space, requiring several cylinders to store the data which is not cost-effective.
The other common constraint is the dearth of skill and the knowledge of administering, maintaining, or developing VSAM files or operate via programming languages.
Over a period, the VSAM structures has grown longer and become complex, without being normalized and data being repeated in multiple structures or in multiple files.
Why Modernize VSAM to RDBMS
Now why would one need to convert from VSAM to IBM’s DB2 or any RDBMS? If your application is running fine today and you do not have any additional data access or manipulation requirements for the VSAM data, then there is no need to convert the VSAM data. Most of the limitations discussed above are few reasons.
However, the primary reason for the VSAM files to be converted to RDBMS is for easy data manipulation activities and avoid the limitations of making it available for languages other than on mainframes.
It is much easier to write ad hoc SQL against tables than it is to code programs that access VSAM files. Having said that, it is not a straight-forward conversion. The data within the VSAM must be analyzed in detail and a logical data model must be created. Usually copybooks that define the structure of the data, are also helpful in understanding the same.
Since VSAM files generally are not normalized, it does not allow for a quick and short conversion to any other relational DBMS. If a VSAM file is directly converted to a table (rows), performance will suffer because such database cannot be used in the same way as VSAM.
Also, if conversion from VSAM to RDBMS is pursued without a knowledge of the application, the functions of the application will not be properly served post conversion as it uses SQL to access data, a set-at-a- time. VSAM, on the other hand, accesses data a record-at-a-time.
Points to Ponder Before Venturing
Before starting on a VSAM to RDBMS migration, try asking yourself the following questions:
- How would you handle structures of the VSAM files in the copybook, that defines the structure of the record?
- How to address complexities with REDEFINES, which again is a very common practice in mainframe?
- How to differentiate not null or not null with default?
- What are the basis of creating the referential integrity (Primary Key, Foreign Key, *Unique constraints).
- What are the validation and verification checks to be performed?
The above list is not limited and one should also consider other complexities of data storage methods used in mainframe systems.
Typical steps involved in conversion
There are certain methods for getting started with such conversion projects.
- Identify the method of conversion based on VSAM structure
- VSAM→Copybook→One to one/One to Many/Entity mapping ----- Create target data model
- Data extract for load
- VSAM→Copybook→COBOL program→Extract data to physical sequential file
- Simple one to one mapping of fields in the copybook (usually for fixed byte)
- Expanded arrays in the target where variable block files are involved
- VSAM→Copybook→COBOL program→Extract data to physical sequential file
- Transform data
- Load data to target
- Physical sequential files→Target data model files ----- Load data in target files
- Perform validation and verification
The above might sound the usual three stage ETL process. However, in this case, the emphasis should be on identifying the structure of the VSAM and conversion based on the various mappings while taking care of EBCDIC to ASCII formats and COMP parameters..
Use of automation scripts could be employed to extract and format VSAM file definitions. From the corresponding copybooks layout, identify and map the target file layout. Copybook provides the layout of the file with required information of the key used and this layout could be used to create the table structure for each VSAM file.
Conclusion
The conversion of VSAM to IBM’s DB2, though not straight-forward can be done by many solution providers. Products such as HTWC’s V2R, promises VSAM conversion to other RDBMS, but is not a simple tool, and is a sequence of steps or through a methodology which other vendors such as IRI and Mapador follow. But there is no standard or standalone tool that can easily perform this conversion. It is always a series of well-defined processes that ensure the completeness of the conversion process.
VSAM to NOSQL is another upcoming trend, which is yet another topic by itself to discuss.
