Dagmar Waltemath and Olaf Wolkenhauer, University of Rostock, Germany.
Most scientific discoveries build upon previous or other findings. The discovery process therefore relies heavily on the reproducibility of scientific results. A lack of transparency and openness led to what many consider a reproducibility crisis. In 2011, for example, Bayer reported that only 43 of 67 findings in cancer studies could be replicated, leading to discussions on the reliability and significance of scientific results. Further examples of irreproducible studies include microarray gene expression analyses, mass spectrometry-based proteomics, trials in medical research and many more. The discussions surrounding these cases carry the risk that the public loses trust in biomedical research.
In our paper, we focus on reproducibility issues related to computational modeling in systems biology and medicine. Among the key challenges are a lack of suitable standards, a lack of quality and quantity of data, a lack of openness, and a lack of transparency. When discussing these issues, we give special attention to initiatives, standards and software tools that already exist and have improved the reproducibility of simulation studies in the past years. For example, the Computational Modeling in Biology Network (COMBINE) oversees the development of open standards for modeling in computational biology and coordinates common standardisation activities.
In the paper, we conclude that further work is needed in terms of standards-development, standards-compliance in software, and better connected infrastructures that coordinate the education of scientists to understand the importance of open and reusable data. We argue that the long-term success of systems biology and systems medicine depends on trustworthy models and simulations. This requires openness to ensure reusability and transparency to enable reproducibility of results.
keywords: reproducible science, standardisation, virtual experiments, modeling, scientific software, research infrastructures