As part of Athena’s modernization efforts, Azure Kubernetes Service (AKS) was chosen to support their transition towards a microservices architecture. LefeWare Solutions team had extensive experience working with AKS and used it to facilitate the needed deployments, scaling, and management of microservices. Athena particularly benefited from leveraging AKS’s dynamic autoscaling features, which adjusted service instances according to load. This feature ensured a high performance during peak day time usage, while lowering costs by scaling down services once the load decreases. Within the AKS cluster, a Kafka instance was set up as a statefulset for handling real-time data feeds with high throughput and low latency.

To further enhance security and cost efficiency, Athena incorporated various other Azure cloud services. They utilized Azure Storage V2 and Azure Data Lake Storage for scalable and secure data storage. Azure Storage V2 provides general-purpose storage with advanced features like storage tiering was used to hold archived log files and data lifecycle management, while Azure Data Lake offered  secure data lake functionality for storing structured and unstructured data. For managing cryptographic keys and secrets securely, Azure Key Vault was used. 

These modernization efforts performed by LefeWare Solutions’ resulted in transforming Athena’s CQR platform into a robust, scalable, and cost-effective tool needed for their digital transformation strategy.

After modernization of the infrastructure was complete, LefeWare Solutions performed an immense data migration, transferring nearly 800 million patient files from a legacy storage system to Azure Blob Storage V2. This project was executed securely over a private network, ensuring that the sensitive data remained protected from potential breaches during the transition. The use of a private network significantly reduced the risk of unauthorized access, providing an additional security layer crucial for such sensitive patient data.

To streamline the migration process, LefeWare Solutions leveraged Azure’s managed compute services, which offered benefits such as auto-scaling, patching, and infrastructure management. This allowed the team to focus on the core aspects of data migration, ensuring the operation ran smoothly and efficiently. Despite the massive volume of files, the migration was successfully completed in just 48 hours, demonstrating the team’s expertise in large-scale data migration and secure handling of data.

As part of the overall modernization process, LefeWare Solutions also set up multiple Continuous Integration/Continuous Deployment (CI/CD) pipelines in Azure DevOps. The goal was to accelerate the building, testing, and deploying of custom applications, thereby further increasing the team’s efficiency.

Each pipeline was designed to handle various tasks such as building, testing, and deployment. The building task compiled the source code into executable products, while the testing task automatically ran a series of tests to validate the application’s functionality. This ensured seamless integration of new changes without breaking the application.

Deployment involved pushing customer containers to Athena’s Azure Container Registry (ACR) and using Helm charts for packaging of pre-configured Kubernetes resources. This process ensured smooth and efficient deployment of applications.

In addition to the core CI/CD pipelines, LefeWare Solutions also set up pipelines for deploying the infrastructure of the Spark jobs. Spark, a unified analytics engine for large-scale data processing, was used for running tasks and handling large datasets. The setup involved tasks such as provisioning resources, configuring networking and security, and setting up data storage, enhancing Athena’s data analytics capabilities

Prior to enlisting the help of LefeWare Solutions, Athena was relying on a series of high powered VM’s that were not optimized to process or analyze data around various EHR patient trends. LefeWare Solutions was able to spin up and integrate an Apache Spark cluster into Athena’s data processing pipeline, significantly enhancing their overall reporting processes. 

Athena’s patient data came from various sources and was loaded into a data lake. Apache Spark was then used to ingest data from this data lake. The data was cleaned, transformed, and loaded into a data warehouse. Using Spark SQL, AthenaHealth analysts were able to perform ad-hoc queries on this data. Spark MLlib was used to create machine learning models on this data to predict trends and identify patterns. Spark Streaming was used to process real-time data for real-time analytics and reporting. Finally, the insights and reports generated were visualized in the CQR tool using custom dashboards for easy understanding and decision making.

By leveraging Spark’s in-memory computation capabilities, we were able to process large volumes of data at a much faster pace than their traditional data processing system, leading to quicker insights from Athena’s data and improving the speed of their reporting system. Furthermore, with Apache Spark’s easy scalability, Athena was able to handle increasing amounts of data as they continued to grow and accumulate more data

The project was a massive success, with significant improvements observed in creating AthenaHealth’s tailored care plans, forecasting better health outcomes, and flagging areas needing an extra boost. The successful implementation of Apache Spark showcased the expertise and commitment of the LefeWare Solutions team in providing cutting-edge solutions to their clients.