Hannah Gaimster, PhD
22nd August 2023
Why standardise data? Health data lacks consistency across sources
Health information can be securely accessed for research and innovation through biobanks, clinical trials, registries, internal studies, and EHRs, among other places. As a result, the data can have a great deal of variation regarding format and content. For instance:
Introduction to health data standardisation
In science and healthcare, the amount of data needed to answer important questions keeps expanding. Large health databases are currently being created due to new technology. These technologies include better value genome sequencing, digitising medical tools, and expanding electronic health records (EHRs).
These vast datasets can provide important insights and eventually enhance lives. Recent groundbreaking studies that illustrate the power of big data in health research include
- the 100,000 Genomes study on rare diseases.
- research reporting the host characteristics triggering severe COVID-19 on approximately 60,000 participants.
- research confirming that high blood pressure is a risk factor for dementia- here, the National Institutes of Health (NIH) All of Us database of EHRs on +125,000 participants was utilised.
This article outlines the value of applying consistent formats and models to create standardised datasets that can be accessed and used for research and innovation in healthcare and research.
"Whilst the amount of data available for research is growing, the majority of users of health data (64%) lack the knowledge necessary to standardise data quickly. This results in researchers spending too much time preparing the data for analysis."
Why standardise data? Health data lacks consistency across sources
Health information can be securely accessed for research and innovation through biobanks, clinical trials, registries, internal studies, and EHRs, among other places. As a result, the data can have a great deal of variation regarding format and content. For instance:- Health information is provided in various formats, such as free text (such as clinician notes) and CSV or JSON files.
- The same information can be described using several phrases across different datasets - both between organisations and geographical location. For instance, across several datasets, dates may take various formats ranging from YYYY-MM-DD, DD-MM-YYYY and MM-DD-YYYY.
- Different medical vocabularies to describe symptoms and disease, such ICD10 and SNOMED, can be used between datasets.
"80% of data scientists'
time is spent cleaning
& organising data."
Whilst the amount of data available for research is growing, the majority of users of health data (64%) lack the knowledge necessary to standardise data quickly. This results in researchers spending too much time preparing the data for analysis.
Furthermore, according to some estimates, data scientists devote 80% of their work to organising and cleaning data.
Data must be transformed into interoperable formats to solve these health data analysis problems. This process is known as data standardisation.
The solution: health data transformation
Common Data Models (CDMs) are being increasingly utilised in the healthcare sector to overcome the lack of consistency in health data. Examples of clinical CDMs are the Observational Medical Outcomes Partnership (OMOP) CDM and Clinical Data Interchange Standards Consortium (CDISC).
Data can be efficiently merged when standardised to these CDMs, making it more useful than the sum of its components alone. The standard approach to data transformation provided by CDMs makes it possible to share research tools and data throughout different nations, sources, and systems.
Combining and assessing information is much simpler if all health data are organised following a single and consistent standard.
Health data standardisation benefits researchers, clinicians and patients
It is clear that a lack of or limited health data standardisation stalls research progress. The table details some key benefits of performing health data transformation techniques.
Summary
There are many different sources and formats of health information. Only when the data is made interoperable can it be effectively combined to produce new insights. It is essential to standardise health datasets to ensure data quality and accelerate collaboration for maximum insights and discoveries.
Look out for the next blog in our series, where we will describe the technical challenges researchers and clinicians can face when standardising health data and some of the solutions currently being developed.
Author: Hannah Gaimster, PhD
Contributors: Hadley E. Sheppard, PhD and Amanda White
About Lifebit
Lifebit provides health data standardisation services for clients, including Genomics England, Boehringer Ingelheim, Flatiron Health and more, to help researchers transform data into discoveries.
Lifebit’s services are making health data usable quickly.
Interested in learning more about Lifebit’s health data standardisation services and how we accelerate research insights for academia, healthcare and pharmaceutical companies worldwide?
Find out more about the value of data standardisation at our upcoming webinar, Data Harmony, on 14 September 2023. Secure your place today.
