Abstract Question Can established cardiovascular risk tools be adapted for local populations without sacrificing interpretability? Findings This cohort study including 95 326 individuals applied a machine learning recalibration method that uses minimal variables to the American Heart Association’s Predicting Risk of Cardiovascular Disease Events (AHA-PREVENT) equations for a New England population. This approach strengthened the AHA-PREVENT risk equations, improving calibration while maintaining similar risk discrimination. Meaning The results indicate that the interpretable machine learning-based recalibration method used in this study can be implemented to tailor risk stratification in local health systems.

Authors Aniket N Zinzuwadia, Olga Mineeva, Chunying Li, Zareen Farukhi, Franco Giulianini, Brian Cade, Lin Chen, Elizabeth Karlson, Nina Paynter, Samia Mora, Olga Demler

Submitted JAMA cardiology

Link DOI

Abstract Fracture prediction is essential in managing patients with osteoporosis and is an integral component of many fracture prevention guidelines. We aimed to identify the most relevant clinical fracture risk factors in contemporary populations by training and validating short- and long-term fracture risk prediction models in 2 cohorts. We used traditional and machine learning survival models to predict risks of vertebral, hip, and any fractures on the basis of clinical risk factors, T-scores, and treatment history among participants in a nationwide Swiss Osteoporosis Registry (N = 5944 postmenopausal women, median follow-up of 4.1 yr between January 2015 and October 2022; a total of 1190 fractures during follow-up). The independent validation cohort comprised 5474 postmenopausal women from the UK Biobank with 290 incident fractures during follow-up. Uno’s C-index and the time-dependent area under the receiver operating characteristics curve were calculated to evaluate the performance of different machine learning models (Random survival forest and eXtreme Gradient Boosting). In the independent validation set, the C-index was 0.74 [0.58, 0.86] for vertebral fractures, 0.83 [0.7, 0.94] for hip fractures, and 0.63 [0.58, 0.69] for any fractures at year 2, and these values further increased for longer estimations of up to 7 yr. In comparison, the 10-yr fracture probability calculated with FRAX Switzerland was 0.60 [0.55, 0.64] for major osteoporotic fractures and 0.62 [0.49, 0.74] for hip fractures. The most important variables identified with Shapley additive explanations values were age, T-scores, and prior fractures, while number of falls was an important predictor of hip fractures. Performances of both traditional and machine learning models showed similar C-indices. We conclude that fracture risk can be improved by including the lumbar spine T-score, trabecular bone score, numbers of falls and recent fractures, and treatment information has a significant impact on fracture prediction.

Authors Oliver Lehmann, Olga Mineeva, Dinara Veshchezerova, HansJörg Häuselmann, Laura Guyer, Stephan Reichenbach, Thomas Lehmann, Olga Demler, Judith Everts-Graber, The Swiss Osteoporosis Registry Study Group

Submitted Journal of Bone and Mineral Research

Link DOI

Abstract Machine learning applications hold promise to aid clinicians in a wide range of clinical tasks, from diagnosis to prognosis, treatment, and patient monitoring. These potential applications are accompanied by a surge of ethical concerns surrounding the use of Machine Learning (ML) models in healthcare, especially regarding fairness and non-discrimination. While there is an increasing number of regulatory policies to ensure the ethical and safe integration of such systems, the translation from policies to practices remains an open challenge. Algorithmic frameworks, aiming to bridge this gap, should be tailored to the application to enable the translation from fundamental human-right principles into accurate statistical analysis, capturing the inherent complexity and risks associated with the system. In this work, we propose a set of fairness impartial checks especially adapted to ML early-warning systems in the medical context, comprising on top of standard fairness metrics, an analysis of clinical outcomes, and a screening of potential sources of bias in the pipeline. Our analysis is further fortified by the inclusion of event-based and prevalence-corrected metrics, as well as statistical tests to measure biases. Additionally, we emphasize the importance of considering subgroups beyond the conventional demographic attributes. Finally, to facilitate operationalization, we present an open-source tool FAMEWS to generate comprehensive fairness reports. These reports address the diverse needs and interests of the stakeholders involved in integrating ML into medical practice. The use of FAMEWS has the potential to reveal critical insights that might otherwise remain obscured. This can lead to improved model design, which in turn may translate into enhanced health outcomes.

Authors Marine Hoche, Olga Mineeva, Manuel Burger, Alessandro Blasimme, Gunnar Ratsch

Submitted Proceedings of Machine Learning Research

Link

Authors Aniket N Zinzuwadia, Olga Mineeva, Chunying Li, Zareen Farukhi, Franco Giulianini, Brian E Cade, Lin Chen, Elizabeth W Karlson, Nina P Paynter, Samia Mora, Olga V Demler

Submitted Circulation: Cardiovascular Quality and Outcomes

Link DOI

Abstract The number of published metagenome assemblies is rapidly growing due to advances in sequencing technologies. However, sequencing errors, variable coverage, repetitive genomic regions, and other factors can produce misassemblies, which are challenging to detect for taxonomically novel genomic data. Assembly errors can affect all downstream analyses of the assemblies. Accuracy for the state of the art in reference-free misassembly prediction does not exceed an AUPRC of 0.57, and it is not clear how well these models generalize to real-world data. Here, we present the Residual neural network for Misassembled Contig identification (ResMiCo), a deep learning approach for reference-free identification of misassembled contigs. To develop ResMiCo, we first generated a training dataset of unprecedented size and complexity that can be used for further benchmarking and developments in the field. Through rigorous validation, we show that ResMiCo is substantially more accurate than the state of the art, and the model is robust to novel taxonomic diversity and varying assembly methods. ResMiCo estimated 4.7% misassembled contigs per metagenome across multiple real-world datasets. We demonstrate how ResMiCo can be used to optimize metagenome assembly hyperparameters to improve accuracy, instead of optimizing solely for contiguity. The accuracy, robustness, and ease-of-use of ResMiCo make the tool suitable for general quality control of metagenome assemblies and assembly methodology optimization.

Authors Olga Mineeva, Daniel Danciu, Bernhard Schölkopf, Ruth E. Ley, Gunnar Rätsch, Nicholas D. Youngblut

Submitted PLoS Computational Biology

Link DOI

Abstract Decision making algorithms, in practice, are often trained on data that exhibits a variety of biases. Decision-makers often aim to take decisions based on some ground-truth target that is assumed or expected to be unbiased, i.e., equally distributed across socially salient groups. In many practical settings, the ground-truth cannot be directly observed, and instead, we have to rely on a biased proxy measure of the ground-truth, i.e., biased labels, in the data. In addition, data is often selectively labeled, i.e., even the biased labels are only observed for a small fraction of the data that received a positive decision. To overcome label and selection biases, recent work proposes to learn stochastic, exploring decision policies via i) online training of new policies at each time-step and ii) enforcing fairness as a constraint on performance. However, the existing approach uses only labeled data, disregarding a large amount of unlabeled data, and thereby suffers from high instability and variance in the learned decision policies at different times. In this paper, we propose a novel method based on a variational autoencoder for practical fair decision-making. Our method learns an unbiased data representation leveraging both labeled and unlabeled data and uses the representations to learn a policy in an online process. Using synthetic data, we empirically validate that our method converges to the optimal (fair) policy according to the ground-truth with low variance. In real-world experiments, we further show that our training approach not only offers a more stable learning process but also yields policies with higher fairness as well as utility than previous approaches.

Authors Miriam Rateike, Ayan Majumdar, Olga Mineeva, Krishna P. Gummadi, Isabel Valera

Submitted FAccT '22

Link DOI

Abstract Abstract Motivation Methodological advances in metagenome assembly are rapidly increasing in the number of published metagenome assemblies. However, identifying misassemblies is challenging due to a lack of closely related reference genomes that can act as pseudo ground truth. Existing reference-free methods are no longer maintained, can make strong assumptions that may not hold across a diversity of research projects, and have not been validated on large-scale metagenome assemblies. Results We present DeepMAsED, a deep learning approach for identifying misassembled contigs without the need for reference genomes. Moreover, we provide an in silico pipeline for generating large-scale, realistic metagenome assemblies for comprehensive model training and testing. DeepMAsED accuracy substantially exceeds the state-of-the-art when applied to large and complex metagenome assemblies. Our model estimates a 1% contig misassembly rate in two recent large-scale metagenome assembly publications. Conclusions DeepMAsED accurately identifies misassemblies in metagenome-assembled contigs from a broad diversity of bacteria and archaea without the need for reference genomes or strong modeling assumptions. Running DeepMAsED is straight-forward, as well as is model re-training with our dataset generation pipeline. Therefore, DeepMAsED is a flexible misassembly classifier that can be applied to a wide range of metagenome assembly projects.

Authors Olga Mineeva, Mateo Rojas-Carulla, Ruth E Ley, Bernhard Schölkopf, Nicholas D Youngblut

Submitted Bioinformatics (Oxford, England)

Link DOI