The following is a guest article by Ernie Ostic, Chief Evangelist at Manta
The healthcare industry creates a vast amount of data each year — roughly 30% of worldwide data — the bulk of which must conform with regulatory standards. That data lives in a complex and evolving ecosystem, integrated with multiple applications and infrastructures, creating countless dependencies.
Between the volume of data and the web of interconnected systems, those dependencies remain hidden from most users in a “black box” of sorts. Without insight into the inner workings of their own flow of information, most healthcare organizations can’t leverage their data to drive better patient care or more informed business decisions.
Data lineage tools address this challenge, highlighting data dependencies and revealing data’s journey through complex systems and transformations. When incorporated effectively, automated data lineage tools shine a light into the black box, enabling greater visibility, and powering predictive insights without compromising the data within. For healthcare organizations, the impact is profound: lineage supports data accuracy, builds data trustworthiness, enables data-driven decisions, and facilitates regulatory compliance.
A Proactive Approach to Data Incidents
Data incidents significantly impact healthcare organizations. For the 12th year running, healthcare has surpassed other industries with the highest average data breach cost: $10.1 million. In 2021, breaches affected more than 50 million patient records, with an average of 132 days before detection.
Healthcare organizations cannot afford to wait for a data incident to occur before improving visibility in their data systems. But many data observability tools today focus on a reactive approach to incidents, searching for a bug to fix after a costly and consequential breach.
By contrast, automated data lineage tools are proactive, akin to preventative care — an annual checkup with your doctor versus an emergency room visit. You might go to the ER as a reaction to the onset of sudden, acute symptoms. The staff will diagnose and treat you, but care is time sensitive and costly. Plus, delayed treatment compounds the risk of a more long-term health impact. Now consider routine checkups with your primary care physician, who knows your history and has access to your medical records. Greater visibility enables proactive care and preventative measures to anticipate and mitigate health concerns.
In the same way, lineage enables high levels of observability without requiring time-consuming manual intervention, empowering IT teams to identify and address potential issues.
Improved Regulatory Compliance
The healthcare industry is subject to heavy regulation under the Healthcare Insurance Portability and Accountability Act (HIPAA) due to the sensitive nature of healthcare data. And those regulations aren’t without consequence: Over the last 20 years, tens of thousands of HIPAA complaints have been investigated, with penalties totaling over $130 million.
Complying with the bevy of industry regulatory requirements demands accurate tracking of data and thorough, reliable reporting to demonstrate evidence of compliance. To avoid significant fines and ensure compliance, healthcare organizations must document regulated data’s source, accuracy, and flow. Automated data lineage enables this process through a comprehensive visual overview of an organization’s data. With insights that range from high-level to granular, lineage reduces the need for manual intervention while supporting compliance practices and lowering the risk of noncompliance.
Increased Visibility and Trust in Data and Reporting
Erroneous healthcare data has wide-ranging consequences — for patients and healthcare organizations. It’s bad enough when a mistake enters one system, but the complex data environment of most healthcare organizations creates the potential for an error to reproduce across applications, contaminating data sets, and endangering patient care decisions.
Now imagine your users discover these data issues before your internal team notices — or mitigates them. This lack of visibility diminishes patient trust and hampers business outcomes. Automated data lineage enables healthcare staff to quickly identify and correct errors, reducing the time spent manually tracing and correcting mistakes. It also empowers healthcare organizations to ensure the integrity of their data, mitigating any risk of data quality issues, and upholding their reputation with partners and patients.
Impact Analysis for Healthcare System Dependencies
Beyond reducing human-made errors, healthcare systems need the ability to manage the impact of system updates, including uncovering and discovering schema and other code changes that can have ramifications downstream. This impact analysis capability is particularly important when dealing with home-grown or vendor-supplied systems — especially for customized analytics systems that build their own data warehouses from packaged applications or use data received from other suppliers.
Lineage provides users with a comprehensive understanding of their data dependencies, which translates complex code into approachable, manageable pipelines. Users can easily navigate data systems, drill down to specific tables and cells, and ensure more accurate data across systems.
Enhanced Business Intelligence Insights
Automated lineage tools also help healthcare organizations gain insights to inform business strategy. With improved data quality and accessibility, organizations can better forecast future trends and adjust accordingly. Automated lineage tools can also help identify areas with room for operational efficiency improvement.
Overall, automated data lineage gives healthcare organizations a comprehensive understanding of their data, enabling them to make better-informed decisions, improve patient outcomes, and drive business growth. With these tools, healthcare providers can ensure that their data is accurate, trustworthy, and compliant. They also gain valuable insights to inform their business strategies and improve patient care.
How one Healthcare Organization Benefited from Lineage
CHRISTUS Health — a nonprofit health system made up of more than 600 centers, including hospitals, clinics, and urgent care centers — faced two major challenges with its data system:
- Managing the impact of EHR system updates across data sets.
- Getting ahead of users in identifying and resolving data issues.
The team at CHRISTUS Health was frustrated that end users were discovering and reporting problems before their team was aware of an issue. And as the company evolved into a larger healthcare system, it faced migration challenges and data deprecation. Fixes took longer than they should have because the team needed more visibility into its full data environment. By incorporating automated data lineage, CHRISTUS Health could scan key parts of its data environment to identify EHR system changes and get ahead of quarterly updates. Insights that once required days of tedious work — leaving the team in a constant reactive mode — took minutes or hours after implementing lineage tools.
The Future of Lineage in Healthcare
Maintaining accurate, reliable, and compliant healthcare data will only become more critical as organizations undergo mergers, implement migrations, retire legacy systems, and integrate new applications. Data lineage enables a proactive approach in the face of emerging changes, drives data confidence amid system complexity, empowers organizations to move beyond collecting data, and put their data to work for better patient and business outcomes.
About Ernie Ostic
Ernie Ostic is Chief Evangelist at Manta, focusing on solutions for lineage and metadata integration and providing guidance on information governance and custom lineage solution architectures. He brings 40+ years of experience in data integration, including more than 20 years at IBM where he worked in a variety of roles including product management and technical sales support. Ostic is a graduate of Boston College, where he earned his bachelor’s degree in computer science.
