From 20-22 July, the 4th #ACM #Conference on Reproducibility and Replicability will take place in #Delft.
The conference welcomes contributions on methods, tools, case studies, and community efforts around #reproducibility and #replicability in #computational research.
Deadlines: 10 March (abstracts)/17 March (papers)
https://acm-rep.github.io/2026/cfp/
Foto by ThisIsEngineering on unsplash
Ever wonder why some “scientific” claims sound convincing but fall apart under scrutiny?
This article explores the 3 vital rules of science. Together, they offer a powerful lens for evaluating conclusions.
#Science #CriticalThinking #Falsifiability #Replicability #CorrelationIsNotCausation
The final paper in our upcoming special issue on #Replicability and #Reproducibility, Joseph Holler and colleagues use open science practices to develop a GIScience study on access t oCOVID-19 healthcare in Illinois, US https://doi.org/10.1080/15230406.2025.2591124 #OpenAccess #GISchat
Reading a paper this morning https://onlinelibrary.wiley.com/doi/10.1002/pds.4295 that follows Nosek & Errington (2017) https://elifesciences.org/articles/23383.pdf in muddling the distinction between #replicability as a broader scientific aim vs #reproducibility as captured in the narrow (purely 'computational') idea of #reproducibleresearch discussed e.g. in a #Guix context here https://guix.gnu.org/cookbook/en/html_node/Reproducible-Research.html.
This mistake is not uncommon, and leads (in this present paper, at least) to insufficient stress on computational reproducibility as a sine qua non of any higher-order quality attribute.
![A snippet from §1.2 of 'Reporting to Improve Reproducibility and Facilitate Validity Assessment for Healthcare Database Studies V1.0', a report from the ISPE‐ISPOR Special Task Force on Real World Evidence in Health Care Decision Making.
"Reproducibility is a characteristic of a study or a finding. A reproducible study is one for which independent investigators implementing the same methods in the same data are able to obtain the same results (direct replication [38]). In contrast, a reproducible finding is a higher order target than a reproducible study, which can be tested by conducting multiple studies that evaluate the same question and estimand (target of inference) but use different data and/or apply different methodology or operational decisions (conceptual replication [38])"
Citation [38] here is Nosek & Errington (2017) https://elifesciences.org/articles/23383.pdf.](https://files.mastodon.social/cache/media_attachments/files/115/677/564/557/917/757/original/f405379439cde459.png)
An #openaccess toolbox aimed at unifying the field of auditory reverse correlation studies by providing a consistent experimental and analysis framework allowing for better #replicability and #reproducibility
https://fediscience.org/@LeoVarnet/115627592918266083
#revcorr #revcor #psychphysics #psychology #psychoacoustics #OpenScience @psycholinguistics @psychology
A new article by Chloe Patton shows how debates about #OpenScience often slip into absurdity – like demanding #replication from the #Humanities. You can’t replicate history, culture, or interpretation the way you replicate a physics experiment. It’s a different kind of knowledge.
:doi: https://doi.org/10.1093/reseval/rvaf052
Forcing STEM-style standards onto the humanities doesn’t improve #science – it just adds bureaucracy and limits academic freedom.
Have you checked out our #OpenScience Bites #podcast yet? Stay tuned!
Very soon, we’ll be talking with Felipe Romero, Assistant Professor at the Faculty of #Philosophy. From a #MetaScience perspective, we’ll dive into the concepts of #reproducibility and #replicability, and why they matter for #research practice.
We’ll let you know as soon as the new episode is online!
In the meantime, feel free to listen to our earlier episodes:
How Scrutiny and Iteration Made Behavioral Economics Better https://www.chicagobooth.edu/review/how-scrutiny-iteration-made-behavioral-economics-better
"However, direct #replicability should not be confused with #generalizability or universality. While the basic patterns of behavior documented in the original experiments reliably show up in subsequent replications, their magnitudes can vary substantially across contexts. Loss aversion might be stronger in some settings than in others. Understanding this variation has become a central focus of contemporary behavioral economics research. Rather than viewing such #heterogeneity as a challenge to the field’s foundations, researchers increasingly see it as a source of insight into the underlying psychological mechanisms driving behavioral anomalies."
#BehavioralEconomics
Even discounting bad faith and charlatans, in the biological sciences (which I am familiar with) and probably other research, we get results... but even with meticulous lab books, it's hard to reproduce the exact same experiment and replicate the exact same results. Because we really don't know all the variables
#Science #Experiments #Replicability
Unreplicable results are an open secret | Dynamic Ecology
https://dynamicecology.wordpress.com/2025/07/17/unreplicable-results-are-an-open-secret/
New I4R discussion paper by Philipp Knöpfle, @drfollowmario, and me: "A Replication Report on "Political polarization of news media and
influencers on Twitter in the 2016 and 2020 US presidential elections" by Flamino et al. 2023
https://www.econstor.eu/handle/10419/319836
#replicability #computationalsocialscience
#statstab #378 Selective Inference: The Silent Killer of Replicability
Thoughts: Benjamin overviews the replicability crisis, alternatives to p-values (and their issues), and suggests selective reporting is a large issue itself.
#replication #crisis #pvalue #nhst #selectivereporting #replicability #reproducibility
How to make Rscripts more reproducible with Nix package Manager : Rix for R make it possible to go a step further than using only Renv or even Docker ; see documentation here https://docs.ropensci.org/rix/index.html#quick-start-for-returning-users #nix #R #replicability
Si vous êtes chercheur.se à l'Université de Rennes / Rennes 2 ou dans des écoles de l'EPE, l'atelier ARDoISE peut vous accompagner lors d'une éventuelle candidature au prix SO du logiciel libre de recherche #FAIR #replicability #SoftwareHeritage #HAL https://scienceouverte.univ-rennes.fr/ardoise-atelier-rennais-de-la-donnee https://www.ouvrirlascience.fr/les-candidatures-pour-le-prix-science-ouverte-du-logiciel-libre-de-recherche-2025-sont-ouvertes/
How to describe #steganography methods in a comparable and unified way to aid #replicability?
We combined pre-existing methodology into a single framework. New pre-print + online tool prototype (will get improved soon) on our website: https://patterns.omi.uni-ulm.de/news/
Full version of the paper and the online tool will be presented at the ARES'25 CUING workshop in August.
#replicability #steganography #covertchannels #informationhiding #infosec #cybersecurity #security #research
Had a great time at the CSS workshop in Mannheim organized by Ruben Bach & Valerie Hase. You can find the slides from my presentation on "Reproducibility & Replicability in Computational Social Science Research" here: https://doi.org/10.6084/m9.figshare.29086382.v1
#reproducibility #replicability #computationalsocialscience
New publication: Code-sharing policies are associated with increased #reproducibility potential of ecological findings, by @antica_c and others. #codesharing #replicability #reliability
https://doi.org/10.24072/pcjournal.541
"Estimating the #replicability of Brazilian #biomedical #science"
... Replication rates for these experiments varied between 15 and 45% according to five predefined criteria ...
🎉 Our method Anisotropic Specular Image-Based Lighting Based on BRDF Major Axis Sampling has received the Replicability Stamp!
🔍 Check it out here: https://www.replicabilitystamp.org/index.html#https-github-com-iota97-anisotropyeditor
🧑💻 Code & data: https://github.com/iota97/AnisotropyEditor
#ComputerGraphics #Rendering #Replicability #OpenScience #BRDF
#statstab #315 What fraction of repeat experiments will have an effect size within the 95% confidence interval of the first experiment?
Thoughts: CV is a great place to learn. We see the same answer as Cumming: 83%
#replicability #confidenceintervals #replication #metascience #estimation #research #science
The University for #HumanisticStudies/ UvH joins the NLRN as an institutional member!
At the NLRN, we want to make a range of epistemic voices heard in the discussion around #reproducibility and #replicability of #science and #scholarship. With their research portofolio focusing on #worldviews and humane societies, the UvH complements the voices of the other, more technical and generalistic institutions TU Delft, TU/e, RUG and escience Center.
Foto: Gaelle Marcel for Unsplash
