Health Data Migration Strategies and Best Practices

Introduction

Data is never static – especially in healthcare. Patient charts are updated daily, and insurance claims and bills are constantly in motion. With health data growing exponentially every year, systems that store healthcare data need to be reliable, efficient, and up-to-date with the latest security and software. That means organizations often need to move or migrate their health data.

At a basic level, health data migration is when healthcare data such as Electronic Health Records (EHR) is transferred from one system or technology to another. The term data migration is often used interchangeably with data conversion, but while the two terms are similar, there are small technical differences that set them apart. For this piece, we use the term data migration for simplicity and focus on EHR data migration strategies and best practices to follow, along with step-by-step recommendations to ensure your data transfer goes smoothly and efficiently.

Why is Health Data Migration Important?

Health data migrations are vital to ensuring data preservation and accessibility. It’s often a necessary or required step organizations need to undergo as they shift applications and processes. One of the most common situations in which health data migrations are needed is for a merger or acquisition. In that instance, separate organizations need to begin operating out of one system or technology.

Data migrations are also necessary when an organization decides to implement a new EHR. This can happen because of a variety of factors such as when an organization wants access to better technology, a software is sunsetting, or it needs to make changes to maintain compliance with laws and regulations. To ensure continuity and efficiency, data migrations are necessary.

11 Benefits of EHR Data Migration

There are a host of benefits to migrating health data on top of moving to a better-performing system with enhanced security. 11 benefits of an EHR data migration include:

• Better technology
• Enhanced data security
• Fewer operating systems
• Easier for users
• Lower IT burden
• Reduced costs
• Interoperability
• Compliance
• Scalability
• Disaster recovery
• Data consolidation

Data migration can streamline application portfolios so organizations have fewer operating systems to maintain and back up, which reduces the burden and budgetary costs. That, in turn, makes it easier for users because they have fewer systems they need access to and knowledge of how to operate.

There are different types of EHR data migration methods, but cloud migration offers a vital benefit in healthcare. Cloud migration is when all data including services, databases, and resources are put in the cloud or moved from one cloud to another. Putting the data in the cloud, which are servers that are accessible via the internet, provides an additional layer of protection that can help protect organizations with disaster recovery.

In the case of a catastrophic event leading to major data loss such as a natural or manmade disaster, having access to data within the cloud can be critical especially when it comes to patient care. Healthcare professionals need to know their patients’ medical histories, what medications they’re using, allergies, and other contextual information to do their jobs accurately.

While not a requirement when doing health data migration, another advantage is data consolidation. Moving and transferring data is an ideal time to consolidate data, especially legacy data. If done right, this work pays off for organizations in the long run. For one Northeastern integrated healthcare system, our data consolidation work during a major merger and acquisition helped them save over two million dollars.

11 Challenges of Health Data Migration

EHR migrations are not inherently simple. There are challenges organizations can face while undergoing a migration including:

• Data loss
• Data corruption
• Inaccurate data
• Compatibility issues
• System downtime
• Budget overrun
• Missed timelines
• Security issues
• Employee pushback
• Legacy systems
• Cloud storage space

The process of migrating health data from one EHR to another can lead to issues if not done correctly. Major hurdles that come with moving such a large amount of data include inaccurate data, data loss, data corruption, and system downtime which could negatively impact patient care and clinical work.

Data compatibility is also a common hurdle. When migrating data, the way it’s referred to or organized in one EHR may not be the same way it’s referred to in another. For example, the old system may refer to medication as “medication_name” while the new system calls it “medication_details.” On top of that, the new system may have more information about the medication such as the brand name and active ingredient, while the other solely includes the brand name. That will require additional preparation to ensure the data is transferred correctly.

It is up to organizations to determine if there will be any data compatibility issues and prepare for how to handle them before undergoing a migration to avoid running into unexpected issues during or after the process.

Due to interoperability, target timelines, and the sheer amount of health data often being migrated, it usually isn’t feasible to move everything to the new go-forward EHR. Often this requires additional data management plans, especially when it comes to legacy records that need to be retained and accessed. In one instance, a health system in the Midwest with nearly 10,000 employees found that migrating its data to a single EHR platform led to a huge headache. It ended up with 50 additional clinical applications from over a dozen vendors. That, on top of dealing with an incorrect timeline given by the EHR vendor they initially started work with, led the company to seek a new vendor. The organization partnered with us to put in place a better legacy data management plan.

EHR data migrations can be tough to estimate from both a cost and time perspective, which is partially what makes them more challenging, especially if you do not have the right resources in place in-house. The path to a successful data migration can begin by partnering with a data management vendor that asks the right questions and does the necessary work ahead of the migration. Important questions include:

• What compatibility issues could occur during a migration?
• Is there enough cloud storage space for all the data?
• Is there legacy data that doesn’t make sense to migrate to the new EHR?
• How will that legacy data be retained so it remains accessible and compliant?

By working through these questions early, many complications can be avoided before any crucial migration work begins.

How Does Health Data Migration Work?

The process of completing a health data migration can be complex, and therefore takes time, planning, and preparation to ensure that it is done correctly. In the best practices section below, we’ll get more granular, providing a seven-step recommendation for successful health data migration. However, at a higher-level, health data migrations are often done through an application migration, such as one EHR to another EHR.

Application Migration in Healthcare

An application migration is the multi-step process of moving data from one software storage location or database into another. This requires in-depth planning and preparation before the data is virtually, or sometimes even physically, moved. Generally, there are four different ways application migrations are completed:

• On-premises to on-premises
• On-premises to cloud
• Cloud to cloud
• Cloud to on-premises

The path a company decides to take when migrating data varies depending on the needs of the organization and the go-forward system they’re moving to for the data migration.

Cloud Migration in Healthcare

Cloud is the most popular option today. If the company is doing a cloud migration, there are two different types of clouds that organizations use for their data: public clouds or private clouds. Public clouds are popular among healthcare organizations with a 2023 report finding 73% of healthcare companies use them. Public clouds are often a more affordable option. Since organizations can pay for what they use, these can often be more cost-effective.

However, private clouds may offer enhanced data security, which is especially important because of the sensitive healthcare data organizations are handling. Another reason healthcare organizations use private clouds is that they have more control over it and there are more customization options.

Some organizations prefer to do a “hybrid cloud” method. Rather than putting everything into the cloud, they instead place only some of their data in the cloud and keep some on-premise.

EHR Data Migration Strategies

After deciding the best method for migrating EHR data, organizations then need to choose the best strategy for completing the migration. There are two key strategies to choose from: a full migration or a phased migration. Influences such as resources, timelines (including downtime and disruptions), and risks can lead organizations to choose one strategy over another.

Full Migration

A full migration is when all the health data is moved over at once. It’s sometimes also referred to as a simultaneous or Big Bang migration. This leads to a coordinated companywide go-live of the new EHR. Although this happens at one time, it can still take hours to days depending on the size of the project.

Full Migration Benefits

• Done at once
• Faster transition
• Less complex

A benefit of doing a full migration is that everything can be done at once which helps cut back on downtime. This is especially important in healthcare where the need for data access 24/7 is vital for patient care. This can lead to a faster transition which is less complex than doing multiple incremental migrations.

Full Migration Considerations

• Higher risk
• Longer planning time up front
• Complex roll back if necessary
• Users need to be trained all at once

However, full migrations are riskier than phased migrations. There is a higher risk of failure if the company runs into an unexpected data issue such as incompatibility or data loss. If an estimated migration timeline is wrong, it could cause disruptions in operations. On top of that, the time it takes to roll back a migration if there are unexpected issues is often time-consuming and complex leading to more delays.

Phased Migration

Meanwhile, a phased migration, or sometimes called a trickle migration, is when data is moved over in stages versus completing it all at once. It may be broken up by facility location and/or department for example. Based on the size of the project, a phased migration could take anywhere from a several months to years to complete and to minimize disruptions.

Phased Migration Benefits

• Gradual transition
• Shorter downtime
• More time to train employees

A major benefit of trickle migration is that it is less risky. Doing the work slowly over a longer period allows for a more gradual transition. It also cuts down on the need for a long downtime, since the migration can be done over a series of shorter downtime, benefiting patient care and data access. This type of migration can also make it easier to train employees on the new system and ensure there aren’t any major data issues.

Phased Migration Considerations

• Slower process
• Risk of data inconsistencies
• Maintaining multiple systems at once

The overall process of migrating to a new system is a lot slower with a phased migration approach. While it cuts down on some initial risks, this could lead to higher costs while trying to maintain and manage both the old and new applications. When it comes to the data, there is also a risk of data inconsistencies and errors if the systems aren’t synchronized perfectly across the migration since the migration is happening in batches instead of all at once.

Health Data Migration Best Practices

There are seven general steps to follow to make the process as smooth as possible given the complexity of EHR data migrations. With our experience in health data migrations, we’ve put together the best practices below that we recommend others follow to ensure an optimal outcome.

Pre-Migration Planning

Health data migrations are tricky, but with proper preparation, they can be done accurately and efficiently. The pre-migration planning phase is of the utmost importance. This is the part of the process where organizations need to determine a clear plan and uncover all the potential pitfalls so they are handled before the migration takes place. It is imperative and should not be rushed through.

Step #1: Determine Project Scope

The first item is to figure out the needs, expectations, and goals for the data migration. Review how internal data is utilized by the organization’s users. During this part of the process, a data governance team should be leveraged to define the scope of the data migration project. On top of determining the anticipated cost, this includes several key data-related tasks:

1. Defining critical data. It’s important to identify which data elements will be most important during the migration, especially in healthcare. Be sure to know which patient data is critical to have in the new EHR for work to continue successfully. This could be demographic and PAMI data as an example.
2. Determining historical data needs. Figure out how far back to take historical data for the migration. Determine the cutoff point necessary to continue functioning and completing work without issue. This could be, for example, a year’s worth of appointment information.
3. Figuring out a legacy data plan. From there, review the remaining legacy data that will not be migrated to the new system and determine a long term retention strategy, such as archiving, which still allows for retention and needed access to records for users.
4. Choose which data to purge. Decide if any data is no longer necessary and able to be purged entirely, considering the proper protocols to do so that comply with policies and regulations.

Along with becoming an expert on all your current data, this is the time to become an expert on the go-forward system. Once you determine the best go-forward EHR for your organization, you need to dive into the details of this system and how it may impact your data. Besides higher-level knowledge such as whether data will be hosted on-premise or in the cloud, you need to know how the new system works and how information is structured overall to prepare for any potential impact on your data. The more specific knowledge you have on your data’s interoperability, the better. More in-depth data mapping will happen a little later in the process, but this is the first step in preparing for that.

This is where important technical considerations also come into play. Questions to keep in mind are:

• How do you plan to move the data?
• Will it be through secure drives that are shipped, secure file transfers, or replication servers to minimize data gaps at go-live?
• Will the migration work be done internally or via a third-party partner?

This is a time to educate yourself and others on the specifics of how the EHR data migration will work and to answer key questions before moving forward.

Step #2: Create Data Migration Strategy

From here, you’ll need to begin formulating your data migration strategy, including how long you expect it to take. Timelines can be difficult to estimate when it comes to EHR migrations. There are options to consider and the process isn’t always clear-cut. When determining your migration timeline, take into account the time necessary for planning and preparation, your desired go-live date, the estimated execution length, and the time needed for testing and validation.

Other important considerations include the overall cost of the project, which resources will be needed, whether you’ll be moving forward with a full or phased migration, and a risk management plan to prepare for any unexpected setbacks before, during, or after the data migration process. It may be beneficial to involve third party partners or consultants to help work through some of these details, which brings us to our next step.

Step #3: Enlist Help

A lot of people are necessary to make a data migration project successful both internally and externally. Internally, an Executive Sponsor should help support the project. There will also need to be a project manager to lead the migration and additional SMEs (subject matter experts) to provide necessary input. The data governance board should also play a big role in determining the needs and resources for data testing and validation.

Externally, organizations may work with an experienced vendor to ensure the migration process goes smoothly. We also highly recommend working closely with the EHR vendor. Trusted vendors can help alleviate struggles and bring extensive experience to the table.

Step #4: Data Preparation

Now you need to start mapping out the EHR data. This is a tedious process and will take time, but it is vital to ensure data is not lost or corrupted once the migration is in process. For this part of the project, you need to look at how data is stored in its current system(s) and how it’s going to fit into the specifications provided by the go-forward system vendor. After confirming the technical import specification, the next step is to examine three important data governance considerations:

Data filtration: Data filtration is the process of filtering through the data so the proper data is migrated, while outdated or irrelevant data is archived or removed entirely. This could include removing patients who have not visited the healthcare organization within the last few years.

Data cleansing: Data cleaning is when inconsistent data is identified and removed or corrected. This can include things such as duplicate records or incomplete data. For example, data may be filtered and cleansed to only include clinical observations that are final, corrected, or amended.

Data normalization: Data normalization is the next step to make sure data is consistent across the board. Normalization could include reformatting data so it is better standardized ahead of the migration. Organizing data into consistent and structured formats helps minimize errors and enhances the efficiency of data retrieval and utilization.

Part of data preparation also includes field level mapping. Field level mapping, or noting how the data in the source system will correspond with the data in the new targeted system, is vital. For example, you’ll need to look for discrepancies such as the new system marking “DOB” for date of birth while the old system says “Birthdate.” From there, you’ll need to create a translation table to make sure data from the old system matches or is converted to the right value or code within the new system. Otherwise, your data might be corrupted during the migration and cause delays.

Step #5: Data Backup

Before beginning any necessary work, backing up data is a must. This is essential before beginning any tests and especially before attempting an actual data migration. A backup will ensure that any EHR data can be restored in case of data corruption, an incomplete migration, or missing data after the transfer. An important note: backups for certain data are a requirement under HIPAA laws.

Step #6: Data Migration Test

Ahead of the official data migration, there needs to be a data migration test. This can be done in a few different ways, but the most common approach is to do a three-pull approach. The three-pull approach includes three different types of data extractions:

1. Initial/testing (small) pull
2. Production pull
3. Final/static pull

During the initial testing pull, a very limited, yet representative sample piece of data is extracted and migrated. It then undergoes quality assurance (QA) to ensure the data doesn’t have any discrepancies. The second pull is often a bigger test migration completed in a test environment that is meant to simulate a migration. This production pull is meant to unveil any data issues or unanticipated problems ahead of the actual migration. Finally, after the first pulls are complete, there’s the final/static pull which is the actual data migration.

Depending on the client or system specifications, a three-pull approach may not always be done. Sometimes there is one full pull and then one catch up pull. It varies based on the needs of each individual data migration project.

Step #7: Data Migration

Finally, it is time for the official data migration. After the prepping, planning, organizing, and testing, this is the moment to migrate the EHR data as safely and easily as possible. The migration process can take time. This is a moment to be patient and trust the preparation and tests will make the process smooth.

If you begin a data migration and encounter delays or unexpected challenges, reach out to an EHR data migration vendor. They can help you navigate these issues and get your data migration back on track.

Partnering with Harmony Healthcare IT for Data Migration

Working with data migration experts can help relieve the burden of handling a major data migration project alone. Experts at Harmony Healthcare IT have worked with healthcare data and records from over 700 EHR and ERP software brands. We have completed work for major leading EHR brands including Epic, Oracle Health (Cerner), MEDITECH, McKesson, Practice Partner, and more. If you need help navigating a data migration, don’t hesitate to reach out for our specialized services and guidance.