Lmst

@GESIS
Thanks for sharing the talk!
Here is the Question-Link R-Package and an in depth tutorial on using it:
https://matroth.github.io/questionlink/index.html

And please note that we also offer consultations regarding harmonization techniques (and other survey method topics).
https://www.gesis.org/en/consulting/survey-methods-consulting

#surveymethods #harmonization #rstats

Given that surveys tend to overestimate belief in #conspiracyTheories (https://osf.io/preprints/psyarxiv/zsncr_v1) and support for #politicalViolence (https://doi.org/10.1073/pnas.2116870119), I wonder how much of the correlation between such variables remains after accounting for such measurement error.

#stats #psychometrics #surveyMethods

New #surveyMethods paper replicates and extends differences in #dataQuality, attention, naivety, decision style, etc. by
- online #research recruitment platform (#mTurk, #Prolific, #Qualtrics, #Pollfish)
- device (#mobile v. #desktop)
- person's incentive

https://doi.org/10.3758/s13428-025-02618-1

Differences in framing effects and cognitive reflection test performance in Study 1 and in cost/participant in Study 2.

Differences in conjunction fallacy, reflection test performance, incentives, attention, and pre-experiment activities for Study 2.

Differences in exclusion rates (between lenient and strict exclusion policies) by platform and differences in psychological scales (like self-esteem, need for cognition, and self-control) for study 1.

Differences in psychological scales, reflection test performance, conjunction fallacy, attention, etc. for Study 2.

Matthias Roth (@rothm) and I published a new paper on survey data harmonization.

"One harmonization fits all? – Impact of missing population invariance on harmonization error when harmonizing social science survey questions with equating"

The paper again corroborates that observed score equating outperforms linear stretching on average. However, it also shows that it is better to derive a harmonization solution for single-item measures from a sample that is drawn from a similar population to the population you are interested in researching.

https://doi.org/10.1080/13645579.2025.2481112

#surveyresearch #surveymethods #harmonization

A figure from the liked paper showing its central finding. Observed score equating on average outperforms linear stretching, but the potential bias of observed score equating gets higher the more dissimilar the samples for equating are from the sample that the resulting recoding table is used on.

Thankful to get a copy of Reflection and Intuition in A Crisis-Ridden World: https://doi.org/10.4324/978100330036

Hoping for a digital copy to annotate!

Can't access the #book? Some free papers cover similar work:
- https://doi.org/10.1111/meta.12534
- https://doi.org/10.3390/jintelligence11040076
- https://doi.org/10.31234/osf.io/y8sdm

I've only been able to glance at a few sections, but this seems like a book I would want to write:
- broad range of topics that interest more than academics
- situated in the history of ideas
- attention to replicability of results
- anticipation of future directions

Bravo! 👏

#cogSci #psychology #politics #epistemology #ethics #extremism #conspiracyTheory #rationality #logic #surveyMethods #psychometrics

Page 143 about how reflective reasoning is — among other things — more consciously accessible than intuitive reasoning.

Pages 154-155 about how reflective reasoning can perform support good reasoning or seemingly vicious reasoning (e.g., post hoc rationalization) a la "Bounded Reflectivism & Epistemic Identity" (Byrd 2022).

Pages 156-157 on the distinction between reflection and rumination. As I say in "A Two Factor Explication of Reflection:..." (under review), rumination is lacking reflection's deliberative inhibition of impulses. Instead, rumination is characterized by letting those impulses intrude and persist. So rumination has only one of reflection's two ket features: consciously accessible thoughts.

🗓️ Next week on Fri 6 Dec

Explore the challenges and opportunities for #socialsurvey #data collection in #Scotland.

Register online for this free in-person event #SocialResearch #SurveyMethods #ResearchMethods #DataCollection

https://www.eventbrite.co.uk/e/survey-futures-practice-forum-4-tickets-1046687960867

Excited for 2025 presentations in #NYC at the #APA (January 8 to 11): apaonline.org/mpage/2025eastern?utm_source=mastodon&utm_medium=social&utm_campaign=fedica-Text-posts

The talk: https://researchgate.net/publication/370132037?utm_source=mastodon&utm_medium=social&utm_campaign=fedica-Text-posts
The poster: https://researchgate.net/publication/371248872?utm_source=mastodon&utm_medium=social&utm_campaign=fedica-Text-posts

#decisionScience #philosophy #xPhi #epistemology #bioethics #cognitiveScience #surveyMethods #DualProcessTheory

2025 APA Eastern website, with link to the program

The Experimental Philosophy Society's session with Nick Byrd's talk: "What Philosophical Tendencies Does Reflective Thinking Actually Cause (and What Did It Take to Find Out)?"

Friday afternoon's poster session with Nick Byrd's poster "Reflection-Philosophy Order Effects and Correlations: Aggregating and Comparing Results from mTurk, CloudResearch, Prolific, and Undergraduate Samples"

Are you working in the area of adaptive survey design? Submit your abstract to our #ESRA2025 session on "Innovations in Adaptive Survey Designs" and showcase your work. https://www.europeansurveyresearch.org/conf2025/sessions.php?sess=44 The deadline's approaching fast! 🗓️ #SurveyMethods #AdaptiveDesign

Excited to share YEARS of research about how to get people to think reflectively and how reflection impacts philosophical judgments at the 2025 #APA in #NewYorkCity (January 8 to 11): https://www.apaonline.org/mpage/2025eastern

Can't make it?
- More about my talk: https://researchgate.net/publication/370132037
- More about my poster: https://researchgate.net/publication/371248872

Thanks to the #APA, James Beebe, and the Experimental Philosophy Society for the opportunity!

#decisionScience #philosophy #xPhi #epistemology #bioethics #cognitiveScience #mTurk #Prolific #UniversityParticipants #surveyMethods #DualProcessTheory

This latest blog, first featured in the Social Research Association, Research Matters magazine by Gerry Nicolaas, Director of Methods, explores the possibility of producing more inclusive survey data and the ethics of differential incentives.

https://natcen.ac.uk/it-ok-use-differential-incentives-encourage-survey-participation-under-represented-groups #SurveyMethods

In a new paper, I simulate the consequences of combining survey data without proper harmonization techniques.

I demonstrate that there is a plausible risk of biased correlative analyses based on the integrated data, if we do not harmonize measurement units across different survey sources and instruments first.

#surveyresearch #surveymethods #harmonization

https://doi.org/10.5964/miss.11217

A figure from the paper showing simulation results. The x-Axis measures how large the difference in measurement units is for the survey variables formed from two survey data sources. They y-axis shows the possible bias for correlative analyses using such integrated variables. The result is t hat as the measurement unit problem increases, the potential for biased correlation also increases.

Moral comparisons of utilitarian tradeoffs depended on the rating protocol?

Participants rated pairs of utilitarian tradeoffs. Relative differences for each pair depended on
- whether participants saw both tradeoffs at the same time or separately.
- whether the rating was comparative or quantitative.

Sometimes (although not most of the time), the average relative difference for one protocol reversed in the other protocol!

https://doi.org/10.1016/j.cognition.2024.105919

#SurveyMethods #xPhi #ethics #moralPsychology

Will people reason less reflectively when primed to think about threats to their health or resources?

Multiple pre-registered experiments (N > 3000) didn't detect any such reflection-suppressing effect of threat primes (compared to controls) — and this didn't seem to be a result of a failed manipulation.

https://doi.org/10.3758/s13428-024-02481-6

#decisionScience #surveyMethods #medicine #health #economics #psychology #edu #policy

Experiment 3 omitted the reflection text by accident.

If cognitive reflection tests measure a reasoning style that generalizes beyond the test itself, we should see test performance predicting many forms of good reasoning.

Yet another example of this: better reflection test performance predicts more (logically) consistent responses across opposing psychological test items (that is, reversed and non-reversed items on a scale): https://doi.org/10.1016/j.paid.2024.112811

#psychometrics #decisionScience #edu #assessment #psychology #surveyMethods

The title information from the (paywalled) paper.

I recently found think-aloud research from 15 years ago, which may have revealed that expert entrepreneurs thought more reflectively than novices (MBA students).

I also like that this paper seems to relay some of the reasons that concurrent think-aloud protocols can outperform retrospective verbal report protocols.

http://doi.org/10.1016/j.jbusvent.2008.02.002

#business #decisionScience #economics #epistemology #dualProcessTheory #surveyMethods #ThinkAloudProtocol

People designing questionnaires for a German sample will find the just published new version of the "Demographische Standards" helpful. It offers guidance on how to collect varios socio-demographic variables (e.g., education, citizenship, employment) in a comprehensive manner.

https://doi.org/10.21241/ssoar.94099

#surveymethods #surveyresearch

"We conclude that potential utility of NPSs [nonprobability samples from commercial survey organizations] will remain out of reach unless scholars confront the tension between the operation of online survey organizations and the goals of transparent research." https://academic.oup.com/poq/article-abstract/87/3/816/7275175 #surveymethodology #surveymethods #nonprob @surveystatisticsandmethodology @surveystatsandmethodology