Healthcare Organization Alleviates IT Strain with Cutting-Edge Data Lake House Technology

How a Healthcare Provider Scaled Their Analytics for Greater Visibility

For many healthcare providers, data and analytics provide answers to medical and biological problems. Many facilities need these same solutions, though, to support their business functions to scale and diversify their service offerings. Recently, a nationally accredited healthcare diagnostics and molecular pathology services organization saw the need to enhance visibility and scale with the help of analytics. This healthcare organization supports over 70 hospitals and thousands of physician groups and surgery centers. They are skilled at balancing high volumes of patients through their approach of high-quality and high-service diagnostics.

The Pain Point 

Leaders at this healthcare organization saw an opportunity to alleviate strain on the IT department by finding more reliable routes to analytics. Enhanced reporting would enable C-level stakeholders with greater visibility into performance and empower business leaders to make data-driven decisions. It was also crucial that the business be able to accurately track the status and locations of tests on lab specimens (known as “accessions”) to minimize testing backlogs. Alleviating these backlogs would also support a quicker turnaround time for patients and physicians waiting on test results.

The team at this healthcare organization knew that improving their reporting and analytics capabilities would empower stakeholders and patients with the answers they needed. The healthcare organization’s existing data warehouse and several data marts were used to drive reporting and decision-making; however these data stores were planned and maintained by different teams at different times. This system leads to disjointed business rules and inconsistent results. As a result, the turnaround times for acquiring the relevant business data and generating ad-hoc reports were large.

They looked for a data and analytics partner who excelled in data strategy and solutions, that’s where CCG came in.

The Prescription

This engagement focused on empowering team members with analytics-based decision making, with streamlined systems for accurate and consistent reporting. CCG proposed a greenfield development inside the healthcare organization’s existing Microsoft Azure environment. The architecture consisted of:

  • A data lake for ingesting and storing structured and semi-structured data
  • Databricks for processing and transforming the raw data from start to finish
  • Azure Data Factory to orchestrate and monitor the ETL pipelines
  • Azure Synapse (SQL DW) for storing cleansed and pre-aggregated data to serve reports
  • Power BI to explore and visualize the data

This architecture provided a framework for an Enterprise Decision Support System (EDSS) that enables the seamless ingestion of data from a variety of on-premise, cloud, and third-party databases into a single source of truth. An Enterprise Decision Support System, acting as a single source of truth, is an essential piece of any modern organization’s infrastructure. It works by enabling downstream analysts and other data consumers to turn data into information using a consistent view of agreed upon data.

This solution was truly cutting edge, leveraging state-of-the-art technologies available in Azure, including Azure Data Lake Store, Databricks, Delta Lake, and Azure Synapse. The core of this solution is Databricks.

Databricks is a managed Spark environment that processes data in parallel on a cluster using inexpensive commodity hardware, which can provide much better performance than traditional data warehouse products. In addition to scaling out, Databricks can also scale up, leveraging powerful GPU-enabled hardware to drive significant data science and machine learning workloads. This process enables the customer to move beyond traditional ETL (Extract-Transform-Load) and into modern predictive and prescriptive analytics that businesses need to stay ahead of their competition.

Using this technology, CCG developed what’s known as a “data lakehouse.” This solution leveraged a data warehouse residing entirely on inexpensive cloud storage, rather than on expensive, resource-intensive database products. In other words, the customer only pays for what they use, and don’t pay for what they don’t use. A data lakehouse significantly reduces ongoing costs and minimizes the impact on the IT budget.

As a result, project stakeholders quickly demonstrated the value of the new EDSS to the business. They secured buy-in to extend and refine the solution to support data driven decision making in novel ways. The successful implementation and ongoing expansion of the client’s EDSS proved to be a smart investment by giving them a competitive advantage over other testing providers and increasing patient and physician satisfaction.

This Healthcare providers challenges are not uncommon. Get a more comprehensive understanding of Databricks in our recent blog Data Bricks for Dummies: What Is Scaling Out? Want to learn how you can implement Machine Learning in your company? Contact one of our experts by clicking the button below.  

Quick Facts

  • Solution - Enterprise BI and Analytics
  • Industry - Professional Services
  • Technology - Azure Synapse, Databricks, Microsoft Power BI

Join the hundreds of CCG customers.
Talk to Intelligent Enterprise Guide.