#thinkAloud

Nick Byrd, Ph.D.ByrdNick@nerdculture.de
2025-09-01

Can thinking aloud accurately capture how we decide?

Nisbet & Wilson's 1977 paper famously suggested it can't.

But #NLP and #AI methods may indicate that it can:
- escholarship.org/uc/item/4sb93
- openreview.net/forum?id=1Tny4K

#cogSci #psychology #philosophy #surveyMethods #thinkAloud #LLM

Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259. https://doi.org/10.1037/0033-295X.84.3.231Xie, H., Xiong, H.-D., & Wilson, R. (2024, August 26). From Strategic Narratives to Code-Like Cognitive Models: An LLM-Based Approach in A Sorting Task. First Conference on Language Modeling. https://openreview.net/forum?id=1Tny4KgGO2Zhang, Z., Xie, H., Baker, T. E., Peters, M., & Wilson, R. (2025). Linking Strategies to Think Aloud in A Stochastic Learning Task. Proceedings of the Annual Meeting of the Cognitive Science Society, 47(0). https://escholarship.org/uc/item/4sb936m3
Nick Byrd, Ph.D.ByrdNick@nerdculture.de
2025-02-05

How do #pharmacists think about #prescription checking?

This #thinkAloud study recorded their thoughts as they checked!

The default strategy was pattern matching, with more reflective thinking in atypical situations.

doi.org/10.1016/j.rcsop.2025.1

#medicine #cogSci #metacognition

Page 4 with an example of more reflective thinking in an atypical situation.
Nick Byrd, Ph.D.ByrdNick@nerdculture.de
2025-02-04

Can we think aloud without disrupting stream of #consciousness?

Two studies (Ns > 100) extended our results suggesting that we can — this time for DOZENS of aspects of thought!

Scientists are running out of good reasons to not use #thinkAloud protocols: doi.org/10.1016/j.concog.2025.

Nick Byrd, Ph.D.ByrdNick@nerdculture.de
2023-06-26

What else can #psychology's #thinkAloud studies do? Design #AI!

Recording people think out loud inspired distinctions between three different types of questions: surface, testing, and deep.

So they made a series of modules (QASA) corresponding to each type: associative selection, rationale generation, and systematic composition.

The results? QASA "outperform[ed] the state-of-the-art #InstructGPT by a big margin."

openreview.net/forum?id=5ud0h8

#ML #LLM #processTracing #computerScience #cogSci

Nick Byrd, Ph.D.ByrdNick@nerdculture.de
2022-11-09

Hey #survey #methods colleagues,

What survey instrument or test would you be interested in testing/validating/calibrating using #thinkAloud protocols?

We can quickly and easily train people to read and think all of their (conscious) thoughts aloud and then record them take an #onlineSurvey (which automatically transcribes their speech into text).

This can reveal alot about the order, accessibility, and articulability of what they're thinking during our surveys (e.g., researchgate.net/publication/3).

Client Info

Server: https://mastodon.social
Version: 2025.07
Repository: https://github.com/cyevgeniy/lmst