"We’ve saved the funniest tool for last: the tortured phrases detector. Sometimes researchers copy paste text from other academic papers. To avoid accusations of plagiarism they use tools that automatically rewrite the text. But this doesn’t always go well. Sometimes it results in phrases that sound weird and no longer make sense in their context. ‘Artificial intelligence’ becomes ‘counterfeit consciousness’ while ‘deep neural network’ is changed into ‘profound neural organization.’ Nonsensical terms like these suggest a paper has been produced by a paper mill. Guillaume Cabanac and colleagues have pioneered the detection of tortured phrases in computer science, but others have used it in medicine and found some hilarious examples. ‘Anal canal’ became ‘butt-centric waterway’ while ‘breast cancer’ is often rephrased as ‘bosom peril’."

#Science #ReplicationCrisis #ScienceIntegrity

https://mecfsskeptic.com/how-many-scientific-papers-are-fake/

The replication crisis has shaken social psychology. Are the failures due to bad science, ever-changing human behavior, or a flawed understanding of replication itself?

In this blog post, Maarten Derksen explores how psychology is evolving in response: https://blog.trialanderror.org/replication-in-social-psychology

#ReplicationCrisis #Psychology #OpenScience

Before the experiment, they interview an expert to ask:

- What was this experiment testing?
- What's the most likely way we could screw it up?
- How is our setup different from the original, and how might have the original experiment been flawed?
- What are the stakes? If we fail to replicate the experiment, what might that mean for scientific knowledge?

Common experimental errors have muppet avatars, and if the expert mentions them, they come on screen and sing a song.

#ReplicationCrisis

Our presentation about #RegisteredReports from last week. We wanted to introduce this research and publication approach to those of our colleagues who may have been unaware of it. We gave three examples from projects we've recently been involved in.
https://osf.io/skdwj
#OpenScience #ReproducibiliTea #ReplicationCrisis #CognitiveDissonance #Personality #RiskPerception

Happiness research hit hard by #replicationcrisis, though some #happiness hacks seem to have consistent (small) effects.
https://www.theguardian.com/science/audio/2024/nov/14/secrets-of-happiness-the-happiness-hacks-backed-up-by-science-podcast

I trust the scientific method.

I do not trust humans to strictly follow that method.

Science is wonderful. Humans however can be less than awesome when incentives are misaligned from the public good.

The #replicationcrisis has a clear solution. Government scientists paid to replicate boring studies (the sensational ones will get plenty of eyeballs)… it’s the boring studies that slip through the cracks.

Government #science is better than cooperation science…
#leftist #progressive #politics

#UnHerd #MassFormationPsychosis #MattiasDesmet #HerdMentality #MassMind #MassMan #MassManLiteracy #MassMindLiteracy #ReplicationCrisis

https://www.youtube.com/watch?v=upSlvuHZpSM

Finally got to read Yarkoni's "Generalizability crisis", and it didn't disappoint. I am not a psychologist, but it seems to describe some very hard problems that cannot be resolved by replication or open science
https://doi.org/10.1017/S0140525X20001685
#psych #replicationcrisis #OpenScience

so why are #students important as #actors of the #openscience movement? they are directly affected the the #replicationcrisis, they are the scientists of tomorrow, everyone at a university should by involbed, and students can support the cultural change towards open science.
more can be found in Maximilian's slides! #OSDMA24

The piranha problem: Large effects swimming in a small pond.
Christopher Tosh, Philip Greengard, Ben Goodrich, Andrew Gelman, @avehtari, @djhsu
2 Apr 2024
https://arxiv.org/abs/2105.13445

In a lot of social science research, small, random factors are reported as having large effects on social and political attitudes and behavior (social priming, hormonal levels,parental socioeconomic status, weather, ...). Studies have claimed to find large effects from these and other inputs.

The results show that it would be extremely unlikely to have all these large effects coexisting—they would have to almost exactly cancel each other out.

#socialScience #replicationCrisis #statistics #quantitative #research #science #scientificMethod

If not anything, this paper has really good examples of various frameworks used in the context of analysing human behaviour: https://www.nature.com/articles/s41562-018-0522-1. It would be interesting to see if their argument of overcoming/avoiding replication crisis with an overarching framework holds true. The #openaccess version is available at https://eprints.lse.ac.uk/100335/1/Dual_Inheritance_Theory_Perspective_final.pdf
#replicationcrisis #replication

I’m part of the #EEGManyLabs project testing the #replicability of influential #EEG studies. We are using #PredictionMarkets as a tool in this effort and you are invited to take part, especially if you have some expertise in EEG research, no matter how little. See below for details.

You may well know about the success of “prediction markets” in forecasting the likelihood of replication (e.g., Dreber et al., PNAS 2015). We are delighted to announce that we have partnered with economists who led these seminal studies to test the wisdom of the EEG community.

From today (as we near the end of recruitment for this project - please see last calls below), we are opening a survey to ask you to vote on the likelihood of some hypotheses studied in the #EEGManyLabs project. Subsequently, you will be invited to bet on the likelihood of success through a stock market platform, where you will earn real money for you or a selected charity.

The success of this effort will become clear when we complete the full project in a few years time. But the results will immediately tell us about the degree of optimism/pessimism amongst our community.

So, please share this widely and place your bets now...

How can I sign up for the prediction markets? Registrations to participate in the prediction markets are administered via the sign-up form linked below. You must have experience of working with EEG (for example, through collecting and/or analysing EEG data, which may be evidenced by having published peer-reviewed articles or preprints with EEG or equivalent experience e.g. designing, collecting and analysing data from EEG experiments).

#neuroscience #psychology #replication #replicationcrisis #reproducibility #metascience

https://pavlovug-dot-yamm-track.appspot.com/2fDq4NW3JydtlOhg09G1yGVAVSU8W-Whs6kPML9_ZfizxufZcjwEmQPWP5AnyD8NUrSZsBDvk7fYcappAg1Sgo3_tXQFMEu2Cc91Q_7ZoUrnvmZiS20DZiHsSFcGXN3Cpu_h990jl35DhtizskaAYiFIXKDiVMD0arROUO4-wXStjr4hF_n39GH14z3uQM6NK9ioOU86MruRx

@wagesj45 @nothingfuture There is nothing morally wrong with that - unless one considers that the time of researchers a limited resource, or one feels that diluting the visibility of genuine research by inflation is a bad thing. This is already normalized in many research communities, and there is little to loose by accelerating it further. #science #replicationcrisis #sociologyofknowledge

Looking forward to today's #ReproducibiliTea, where we will this paper on positice consequences of the #ReplicationCrisis with first author Max Korbmacher. https://www.nature.com/articles/s44271-023-00003-2 #OpenScience

@DavidBruchmann
If you think that either #ReplicationCrisis or mere crude fraud or #Plagiarism are going to be explained with old ideas of #Ethics or by thinking that #Science is born from the limits of our #Psychology as #StevenPinker is writing or reform methods and rules in labs, you are only going to find yourself buried in a pile of papers with fake scientific claims. The #Books that I said are from the 70s and preview what is happening, other ideas do not.

Are the criticisms of psychological measurement fundamentally flawed or are psychologists indeed not measuring anything, in spite of appearances?

https://www.tandfonline.com/doi/full/10.1080/09515089.2023.2300693

#psychology #replicationcrisis #iknowwhatyouarethinking

Is Psychology Self-Correcting? Reflections on the Credibility Revolution in Social and Personality Psychology

tl:Dr — "No."

https://spb.psychopen.eu/index.php/spb/issue/view/485

#science #psychology #replicationcrisis #handwavingandfingerpointing #SomedayMaybe

earlier this year, someone in my feed was posting about their work that sought to challenge/complicate predominant #ReplicationCrisis narratives... anyone know who that was? I forgot to bookmark it for later

All the people who collate #science accounts to follow on Mastodon - is Data Colada on your list?

@datacolada keep on publishing new posts at https://datacolada.org/ , and they apparently are getting sued right now (by none other than Francesca Gino...).

Why would you be interested? Well, they're the ones who got the ball rolling on #ReplicationCrisis.

I’ll be presenting at the Big Team Science conference Karli Nave’s multi-lab EEG replication project about musical beat perception and steady-state evoked brain potentials (aka frequency tagging) next Monday. Hope to see you there if you're into #music, #openscience, #replications, etc.

Spoiler alert: we actually did replicate Nozaradan et al.'s basic effect of auditory imagery on EEG activity, but the observed effect size was quite a bit smaller than the original study, as tends to happen.

There are lots of other good looking presentations too of course. Registration is free or inexpensive with suggested donation and completely online.

https://bigteamscienceconference.github.io/

#teamscience #openscience #reproducibility #science #neuroscience #psychology #metascience #replicationcrisis