Scalable Medical Image Analysis Using CNNs and DFS with Data Sharding for Efficient Processing

Abstract

Cloud computing essentially improves accessibility of patient records, medical images, and sensor information by enabling the effective management, processing, and storage of large amounts of medical data. However, traditional systems are often very limited in capacity, in addition to having problems about real-time processing of data, and security. This makes the management of massive health data volumes ineffective and results to delays in diagnosis. Due to limits in data administration and processing capabilities, earlier systems have a hard time ensuring high availability of and reliable interpretation of medical images. Our architecture employs a cloud-based infrastructure, which works jointly with Distributed File Systems (DFS) and data sharding strategies to facilitate scaling and optimization of resource allocation. The approach employs Convolutional Neural Networks (CNN) for medical image analysis, whereby accuracy is improved in terms of diagnostics and abnormality detection. At 95% model accuracy, precision lies at 93%, recall at 91%, and a score of 0.97 for the AUC-ROC. In addition, the throughput is improved to 5 TB per hour at a cost of $100 per TB. This study has furthered the design of a scalable, secure, and high-performing healthcare data management system with real-time capability while ensuring data privacy and compliance with healthcare regulations.