#statstab #479 Uncertainty limits the use of power analysis
Thoughts: Frequentists avoiding uncertainty is never good.
#poweranalysis #error #uncertainty #simulation #cohend #effectsize
https://www.researchgate.net/publication/361158443_Uncertainty_limits_the_use_of_power_analysis
#statstab #461 Interpreting Ordinal and Disordinal interactions
Thoughts: Interactions are not simple things. Their shape can determine many things (including sample size and effect size)
#design #ANOVA #interaction #effectsize #ordinal #crossover
https://www.jolley-mitchell.com/Appendix/WebAppOrdinalInteraction/WebAppOrdinalInteractions.htm
#statstab #444 {popower}: Power and Sample Size for Ordinal Response
Thoughts: Not the most intuitive but useful if you know the DGP will use ordinal data.
#statstab #425 Providing a Lower-Bound Estimate for Psychology’s “Crud Factor”
Thoughts: Psych research may not have the tools to investigate very small effects at all!
#crudfactor #research #psychology #mesurement #error #effectsize
#statstab #413 Counternull Sets in Randomized Experiments
Thoughts: The counternull is a lost statistic, that is woefully underused when teaching stats.
#counternull #nhst #pvalue #effectsize #teaching #randomization #rosenthal
https://www.tandfonline.com/doi/pdf/10.1080/00031305.2024.2432884?download=true
#statstab #396 If researchers find Cohen’s d = 8, no they didn’t
Thoughts: Sometimes an effect is so impressive that its unbelievable.
#effectsize #cohend #QRPs #sesoi
https://mmmdata.io/posts/2025/07/if-researchers-find-cohens-d-8-no-they-didnt/
#statstab #393 Statistically Efficient Ways to Quantify Added Predictive Value of New Measurements [actual post]
Thoughts: #392 has the comments, but this is where the magic happens.
#modelselection #modelcomparison #variance #effectsize #tutorial
When designing a scientific experiment, a key factor is the sample size to be used for the results of the experiment to be meaningful.
How many cells do I need to measure? How many people do I interview? How many patients do I try my new drug on?
This is of great importance especially for quantitative studies, where we use statistics to determine whether a treatment or condition has an effect. Indeed, when we test a drug on a (small) number of patients, we do so in the hope our results can generalise to any patient because it would be impossible to test it on everyone.
The solution is to perform a "power analysis", a calculation that tells us whether given our experimental design, the statistical test we are using is able to see an effect of a certain magnitude, if that effect is really there. In other words, this is something that tells us whether the experiment we're planning to do could give us meaningful results.
But, as I said, in order to do a power analysis we need to decide what size of effect we would like to see. So... do scientists actually do that?
We explored this question in the context of the chronic variable stress literature.
We found that only a few studies give a clear justification for the sample size used, and in those that do, only a very small fraction used a biologically meaningful effect size as part of the sample size calculation. We discuss challenges around identifying a biologically meaningful effect size and ways to overcome them.
Read more here!
https://physoc.onlinelibrary.wiley.com/doi/10.1113/EP092884
#experiments #ExperimentalDesign #effectsize #statistics #stress #research #article #power #biology
#statstab #366 Type M error might explain Weisburd’s Paradox
Thoughts: Learn about type M error while you learn about the issues in criminology!
#replication #typeM #typeS #QRPs #paradox #power #effectsize #errorrate
https://sites.stat.columbia.edu/gelman/research/published/weisburd_28.05.2017.pdf
#statstab #356 Measures of Heterogeneity
Thoughts: An overview of different measures, I^2, Q, H^2, and associated R code.
#metaanalysis #heterogeneity #effectsize #effectsize #variability #error
https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in_R/heterogeneity.html
🚨New blog post 📝: Your Study Is Too Small (If You Care About Practically Significant Effects)
#effectsize #precision #poweranalysis #research #Psychology #MCID #SESOI #samplesize
#statstab #353 The Abuse of Power; The Pervasive Fallacy of Power Calculations for Data Analysis
Thoughts: An seminal paper on "post hoc" power calculations.
#power #QRPs #NHST #posthoc #samplesize #effectsize
https://www.tandfonline.com/doi/abs/10.1198/000313001300339897
#statstab #347 A Simple Method for Removing Bias From a Popular Measure of Standardized Effect Size: Adjusted Partial Eta Squared
Thoughts: Its epsilon^2, but easier to compute by hand.
An overview of 67 different effect size estimators, including confidence intervals, for two-group comparisons:
https://journals.sagepub.com/doi/full/10.1177/25152459251323186
The authors have also developed a Shiny web app to evaluate these.
#statstab #305 The Fallacy of Employing Standardized Regression Coefficients and Correlations ad Measures of Effect
Thoughts: Everyone loves effect sizes, but mind how you compute and interpret them.
#statstab #294 So You Think You Can Graph - effectiveness of presenting the magnitude of an effect
Thoughts: Competition in the many ways to display effect magnitude. Some cool ideas.
#dataviz #stats #effectsize #effects #plots #figures #cohend
https://amplab.colostate.edu/SYTYCG_S1/SYTYCG_Season1_Results.html
#statstab #281 Correcting Cohen’s d for Measurement Error (A Method!)
Thoughts: Scale reliability can be incorporated into effect size computation (i.e., remove attenuation)
An even better solution would be a table where you could select which type of effect #effectSize measure to show (calculated using e.g. these calculations https://www.escal.site/). If anyone has the skills to implement that in #wikipedia #markup, please do so!
It always takes me some minutes to look up the interpretation guidelines for various effect size measures (yes, I know the rules of thumb are somewhat arbitrary). Today I edited Wikipedia to show three different guidelines for four different measures in the same table. Hopefully this can save some time for other researchers.
#methodology #psychometrics #EffectSize #OpenScience #wikipedia
