online business

DATA MIGRATION

Legacy data is the recorded information that exists in your current storage system, and can include database records, spreadsheets, text files, scanned images and paper documents. All these data formats can be migrated to a new system. Data migration is the process of importing legacy data to a new system. This can involve entering the data manually, moving disk files from one folder (or computer) to another, database insert queries, developing custom software, or other methods. The specific method used for any particular system depends entirely on the systems involved and the nature and state of the data being migrated. Data cleansing is the process of preparing legacy data for migration to a new system. Because the architecture and storage method of new or updated systems are usually quite different, legacy data often does not meet the criteria set by the new system, and must be modified prior to migration.

To analyze and define source and target structures, analysis must be performed on the existing system as well as the new system to understand how it works, who uses it, and what they use it for. A good starting point for gathering this information is in the existing documentation for each system. This documentation could take the form of the original specifications for the application, as well as the systems design and documentation produced once the application was completed. Often this information will be missing or incomplete with legacy applications, because there may be some time between when the application was first developed and now. You may also find crucial information in other forms of documentation, including guides, manuals, tutorials, and training materials that end-users may have used. Most often this type of material will provide background information on the functionality exposed to end-users but may not provide details of how the underlying processes work.

For this part of the analysis, you may actually need to review and document the existing code. This may be difficult, depending on the legacy platform. For example, if data are being migrated from an AS/400 application written in RPG, assistance from an experienced RPG programmer will be required, if those skills are not available in-house. This can be an expensive part of the analysis process, because a resource to perform this analysis for you may be necessary, but it is a vital part of the process, especially if the application has been running and maintained over a long period of time. Undocumented code or fixes that are critical to the application and that have not been documented elsewhere may exist.

Another key area to examine is how the data in the system are stored (i.e., in flat files, files, or tables). What fields are included in those files/tables and what indexes are in use? Also, a detailed analysis of any server processes that are running that may be related to the data must be performed (e.g., if a nightly process runs across a file and updates it from another system). Now that the source and target structures are defined, the mapping from the legacy to the target should fall into place fairly easily. Mapping should include documentation that specifically identifies fields from the legacy system mapped to fields in the target system and any necessary conversion or cleansing. Once the analysis and mapping steps are completed, the process of importing the data into the new system must be defined. This process may be a combination of automation and manual processes or may be completely automated or may be completely manual.