In this talk, we discuss the paper, “Data splitting to avoid information leakage with DataSAIL” by Roman Joeres, et al., Nature Communications, 2025.<\/p>\n
Abstract<\/strong><\/p>\n Information leakage is an increasingly important topic in machine learning research for biomedical applications. When information leakage happens during a model\u2019s training, it risks memorizing the training data instead of learning generalizable properties. This can lead to inflated performance metrics that do not reflect the actual performance at inference time. We present DataSAIL, a versatile Python package to facilitate leakage-reduced data splitting to enable realistic evaluation of machine learning models for biological data that are intended to be applied in out-of-distribution scenarios. DataSAIL is based on formulating the problem to find leakage-reduced data splits as a combinatorial optimization problem. We prove that this problem is NP-hard and provide a scalable heuristic based on clustering and integer linear programming. Finally, we empirically demonstrate DataSAIL\u2019s impact on evaluating biomedical machine learning models.<\/p>\n","protected":false},"excerpt":{"rendered":" In this talk, we discuss the paper, “Data splitting to avoid information leakage with DataSAIL” by Roman Joeres, et al., Nature Communications, 2025. Abstract Information leakage is an increasingly important … <\/p>\n