#statstab #346 Jeffreys-Lindley paradox
Thoughts: I like this short explanation of the "paradox" of why frequentist and bayesian inference can differ.
#paradox #frequentist #bayesian #inference #bayesfactor #pvalue #explanation
https://michael-franke.github.io/intro-data-analysis/jeffreys-lindley-paradox.html
Biosignature Hype
I was thinking just the other day that I havenât posted much in either the Astrohype or the Bad Statistics folders on this blog. Well today I found an item that belongs in both categories. Many people will have seen the widespread press coverage of a misleading claim of the discovery of alien life; see, e.g., here. This misleading press coverage is based on a misleading press release from the University of Cambridge which you can find here.
The story is based on a paper in the pay-to-publish Astrophysical Journal Letters with the title âNew Constraints on DMS and DMDS in the Atmosphere of K2-18 b from JWST MIRIâ. The DMS and DMDS in the title refer to Dimethyl Sulphide and Dimethyl Disulphide respectively. These are interpreted by the authors as biosignatures.
There are two main problems with this claim. One is that DMS and DMDS are not necessarily biosignatures in the first place; see here for the reasons. The other is that there isnât even any evidence for the detection of DMS or DMDS anyway. Here is the spectrum of which the lead author of the paper, Prof. Nikku Madhusudhan, has claimed âthe signal came through loud and clearâ.
Yeah, right. In statistical terms this is a non-detection. The Bayes Factor used in the paper to quantify the evidence for a model with DMS and/or DMDS over one without is just 2.62 in the logarithm. Thatâs not a detection by any stretch of the imagination; to be anywhere near convincing a Bayes Factor has to be at least 100. The subsequent cherry-picking of the data to improve the apparent probability of a detection is just statistical flummery.
Notice that the use of the phrase âConstraints onâ in the title of the paper does not indicate that the article presents evidence that a detection has been made. That the claim has somehow morphed into the âthe strongest evidence for life beyond our solar systemâ is absurd. The most charitable thing I can say is that Prof. Madhusudhan must have been carried away by enthusiasm. This doesnât reflect very well on Cambridge University either.
This episode worries me greatly. This is a time of increasing hostility towards science and this sort of thing can only make matters worse. Scientists need to be much more careful in communicating the uncertainties in their results.
UPDATE: Thereâs a now paper on arXiv here that argues that a straight line is a better fit to the data, in other words that there is no strong statistical evidence for spectral features at all.
#BayesFactor #Biosignatures #DimethylDisulphide #DimethylSulphide #ProfNikkuMadhusudhan
#statstab #9 Francis (2017) Equivalent statistics and data interpretation
Thoughts: I've a love-hate relationship w the paper & app. It's *technically* true but promotes a bad stats argument. Still useful.
#stats #pvalues #ttest #effectsize #bayesfactor
https://www1.psych.purdue.edu/~gfrancis/EquivalentStatistics/
