CORTEXA
← Browse
arxivcs.CVcs.LGphysics.med-ph2026-07-15

A novel unsupervised machine learning strategy to handle multimodal cardiac PET/MRI data

Brunnhilde Ponsi, Thomas Carlier, Lara Marteau, Aurélien Monnet, Thomas Eugène, Jean-Michel Serfaty, Nicolas Piriou, Hatem Necib

Arrhythmogenic left ventricular cardiomyopathy is a genetic myocardial disease difficult to diagnose due to the lack of gold standard criteria. Simultaneous PET/MR imaging, combined with multiparametric quantitative analysis, could facilitate the identification of different profiles related to the phenotype and progression of cardiomyopathy. This preliminary study focuses on a methodological strategy for dealing with PET/MRI data, including inter-patient data linkage and regional analysis. Two-step clustering was applied to T1 and T2 maps, LGE, and 18F-FDG-PET images of 99 patients genetically diagnosed with arrhythmogenic left ventricular cardiomyopathy. Each patient's images were independently z-scored and summed into a single volume, which was clustered into supervoxels. Thirty-two inter-patient groups of supervoxels were obtained by spectral clustering. An "abnormality" score was assigned to each cluster and modality, and used to visualise abnormal regions likely associated with disease. They enabled the generation of automated textual and bullseye health reports for each patient, which were compared with cardiac imager assessments using balanced accuracy in repeated nested cross-validation. This approach was further validated on a larger cohort of 167 numerical phantoms. The reports generated by clustering accurately identified most of the cardiac physicians' observations (BA = 0.76 $\pm$ 0.04 in repeated nested cross-validation on patients, and BA $\ge$ 0.8 on phantoms). Furthermore, the identified abnormal clusters closely matched their visual observations, facilitating the identification of varying degrees of fibrosis or inflammation on the images. This approach enables a more systematic handling of multimodal PET/MRI data to characterise myocardial heterogeneity in arrhythmogenic left ventricular cardiomyopathy patients.

View free PDFSource page

Related papers

arxiveess.IVcs.CVcs.LGphysics.med-phstat.AP2026-07-09

ConRad: Efficient Conformal Prediction for Radiomics

Matt Y. Cheung, Ashok Veeraraghavan, Guha Balakrishnan

Radiomic features derived from medical images and segmentation masks are used to support decision making in clinical imaging pipelines. In practice, these features are often computed from predicted masks, but segmentation models can be overconfident or poorly calibrated, making d…

View free PDFSource page
arxivcs.LGcs.CVphysics.med-ph2026-07-06

When Does Consensus Beat Voting? A Critical Analysis of Statistical Label Fusion in Medical Image Segmentation

Renjie He

This paper provides a rigorous, self-contained investigation of consensus segmentation. We derive the mathematical foundations from first principles -- the generative model, EM algorithm, Van Leemput's marginalization analysis, identifiability conditions, Spatial STAPLE, and deep…

View free PDFSource page
arxivcs.CVcs.LG2026-07-16

AlphaWiSE: Adaptive Weight Interpolation for Continual Multimodal Representation Learning

Sarthak Jain, Qiran Hu, Zhen Zhu, Yaoyao Liu

Multimodal models such as CLIP learn a shared embedding space for cross-modal retrieval, but continual adaptation to sequentially arriving data can disrupt the cross-modal alignment acquired from earlier phases. Conventional continual-learning methods return a single checkpoint,…

View free PDFSource page
arxivcs.CVcs.LG2026-07-10

ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning

Nusrat Binta Nizam, Fengbei Liu, Sunwoo Kwak, Minh Nguyen, Ruining Deng, Mert R. Sabuncu

Multimodal medical models often degrade when inputs are missing, a common scenario in real-world clinical workflows. Separately, even when all modalities are present, modality dominance is observed during training, where optimization over-relies on a highly predictive modality an…

View free PDFSource page
arxivcs.CVcs.LG2026-07-08

MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models

Hyunjae Kim, Dain Kim, Pan Xiao, Serina S. Applebaum, Younjoon Chung, Xuguang Ai, et al.

Medicine is inherently multimodal, requiring clinicians to synthesize information across diverse data streams. Yet the development of multimodal foundation models is constrained by limited access to large-scale, high-quality clinical data. Although PubMed Central (PMC) offers a c…

View free PDFSource page
arxivcs.CVcs.LG2026-07-20

Early Yield Prediction for Sugar Beet Fields using Satellite Data -- Learnings from Specialized Vision Transformers

Philipp Vaeth, Bhumika Laxman Sadbhave, Denise Dejon, Gunther Schorcht, Magda Gregorova

Remote sensing has become an increasingly valuable tool for agricultural monitoring, particularly through the use of publicly available satellite imagery. However, effectively integrating domain knowledge into machine learning methods remains challenging. This study presents a re…

View free PDFSource page