Summary
Data migration is the process of transferring data between two or more storage types, formats, or systems. When that data carries regulatory compliance implications, a documented migration plan is essential. This guide walks through the requirements that belong in a data migration plan, the step-by-step process for migrating compliance data, and the validation, verification, and backout considerations that keep a migration audit-ready. The recurring theme: it is rarely a simple copy, so plan the mapping, transformation, and verification before you move anything.
Simply put, data migration is the process of transferring data between two or more storage types. The transfer can occur between different formats or different computer systems. This is a straightforward concept. But it can get complicated when the data involved has formatting requirements that may be different between the source and destination and if this data has regulatory compliance implications. In this blog, the assumption made is that the data has regulatory implications.
In today’s environment, migration can take different forms, migrating to or within the cloud, on-premise, or whether it is a database, application, or storage only migration. Whatever migration requirements you identify, a data migration plan is essential. It is rarely, if ever, a simple copy; there is almost always some data mapping and/or data transformation/conversion that must take place along with verification, all of which must be considered and detailed in a migration plan.
When regulated data is in scope, the migration plan is part of your compliance evidence. Mapping decisions, transformation logic, verification results, and the backout plan all need to be documented and defensible, because the migrated records must remain attributable, legible, and complete. Anchor the effort to your data integrity expectations and 21 CFR Part 11 obligations for electronic records from the outset.
Consideration for a Data Migration Plan
The below image represents a simplified, visual way for you to think about your overall data migration plan.
Planning the Strategy
Things to include in any data migration plan typically take the form of requirements. Requirements will identify and detail the following;
- the sources of data that are in scope for the migration project,
- the specific mapping and/or data transformation requirements,
- how the migration of data will be verified,
- and a backup/backout plan in the case of any failures.
These activities should be outlined in a step-by-step manner.
Four Pillars of a Compliant Migration Plan
- Scope & sources: Define every data source in scope and the records, fields, and formats each one contributes.
- Mapping & transformation: Specify how source fields map to the destination and what conversion or cleanse each field requires.
- Verification: Decide how migrated data will be checked and validated before the legacy system is retired.
- Backup & backout: Document how source data is preserved and how you will recover if the migration fails.
Depending on the complexity, data migration can be included within an overall validation plan for a new system (database, storage) being implemented. It doesn’t necessarily need to stand alone. The migration can be performed as a final step in your performance qualification testing or user acceptance testing (UAT). When there are significant mapping or transformation requirements, then it is recommended that data migration be performed on its own, inclusive of the validation/verification testing that will necessarily be required. A risk-based approach such as computer software assurance (CSA) can help you focus testing effort where the data and patient-safety risk is highest, and fitting the migration into a broader validation lifecycle keeps the evidence connected to the system it supports.
In a cloud-based scenario, the work required doesn’t change significantly. What may be different is who performs the work (an external vendor) and how it is done. The plan outlining the details still needs to be created. Keeping a migrated cloud system in a continuously validated state afterward is where USDM Cloud Assurance can carry the compliance forward.
It is rarely, if ever, a simple copy; there is almost always some data mapping and/or transformation that must take place—along with verification.
Steps for Migrating Compliance Data
The following steps should be considered for any data migration that involves compliance data:
- Identify the data source(s).
- Review the source data for obsolete records, unused fields, etc. This will make the actual migration go smoothly.
- Perform whatever transformation/cleanse is deemed necessary.
- Schedule the migration. Often systems will be in use until the migration is completed to the new system or storage location. This must be a part of your plan to minimize down time for end users.
- Backup the source data prior to migration (as close to the date and time of the migration as possible).
- Move the data. Do not copy the data.
- Verify and validate the migrated data.
Once this is complete and testing has been performed satisfactorily, the legacy system can be shut down and retired.
Final Thoughts
Here are some final thoughts to keep in mind with any sort of data migration, whether it’s on-premise or in the cloud, and whether there are compliance implications or not. Keep your end users involved in the process. In most cases, they will be able to describe their data requirements, what to look for, and provide valuable insight. Also, keep your Quality team involved from the beginning of the project. Bringing them in at the eleventh-hour can lead to unnecessary delays, changes in requirements or scope, and overall resistance to the project.
USDM has conducted hundreds of data migration projects and is a trusted advisor for regulated data strategy and execution. USDM offers a range of solutions from oversight on project delivery to resourcing and managed services, and we build each team to meet the needs of each project. Whether you need a data analyst, a skilled team, or expert management to facilitate a data migration project on time and on budget, you can feel confident that we have the right delivery model for your unique needs.
Ready to plan your next data migration?
Whether you are moving to the cloud, consolidating systems, or retiring legacy storage, USDM can help you plan, validate, and execute a compliant migration. Contact us to discuss the right delivery model for your project.
FAQ: Data Migration in Regulated Environments
What is data migration?
Data migration is the process of transferring data between two or more storage types. The transfer can occur between different formats or different computer systems, and it often involves data mapping and transformation rather than a simple copy.
Why is a data migration plan essential for regulated data?
Because migration is rarely a simple copy, there is almost always some mapping, transformation, or conversion that must take place along with verification. When the data has regulatory compliance implications, all of these activities must be considered and detailed in a documented migration plan.
What should a data migration plan include?
At minimum it should identify the data sources in scope, the specific mapping and transformation requirements, how the migration will be verified, and a backup/backout plan in case of failures—outlined in a step-by-step manner.
Does data migration need to be a standalone effort?
Not always. Depending on complexity, data migration can be included within an overall validation plan for a new system and performed as a final step in performance qualification or user acceptance testing. When there are significant mapping or transformation requirements, it is recommended to perform the migration on its own, inclusive of the validation and verification testing required.
Should you copy or move the data?
Move the data, do not copy it. After backing up the source data as close to the migration as possible, move the data to the new location and then verify and validate the migrated data before retiring the legacy system.
Related Content
Services: Data Integrity Services
Services: Cybersecurity
Case Study: Improved UDI Data Integrity with Single Source of Truth
Case Study: Quality System Upgrade to Meet EU MDR and EU IVDR
About the Author
Joseph Cassella is the Director of Regulatory Compliance at USDM Life Sciences. With over 25 years of experience in the pharmaceutical, biotech, and medical device industries, Joe’s background is both broad and deep in Information Technology, Laboratory and Analytical Applications, and Quality Systems. He has led many projects inclusive of IT Infrastructure, Research & Development, QA/QC, Manufacturing, Compliance, and Sales and Marketing.