Lmst

Looks like I’m not the only one to find that between-person differences in #criticalThinking didn’t replicate in within-person changes over time:  

Testing #argumentMapping effects: https://www.researchgate.net/publication/390977878

Assessing #higherEd over time: https://doi.org/10.1007/s10212-025-01000-1

#edu #stats

"Rational thinking skills were assessed using a German test battery consisting of four subtests of heuristics and biases tasks."

"Cognitive Reflection was assessed using three items of the original CRT..."

"Resistance to Belief Bias was assessed using five items created by Markovits and Nantel (1989) and one new item."

"Resistance to Ratio Bias was assessed using four items by Toplak et al. (2014). In each item, the subjects’ task is to choose between a small and a large tray of white and black marbles...."

"Disjunctive Reasoning was assessed using the knights and knaves problem (Toplak & Stanovich, 2002; adapted from Shafir, 1994) and five new items. In all items, the participants’ task was to decide whether the question can be answered with yes, no, or not at all based on the given information."

"Prior to analyses, data were cleaned and checked for outliers. Responses to items of rational thinking skills that were unusually fast (3 SD or more below the item-specific log mean; following recommendations of Ratcliff, 1993) were considered unreliable and were deleted. This applied to 0.27% of all trials at t1 and 0.19% of all trials at t2. For continuous variables (fluid intelligence, rational thinking dispositions, rational thinking skills, student satisfaction), scores deviating more than 3 SD from the sample mean were excluded from path and regression analyses."

"our sample size" was "101, excluding missing values and outliers)"

"Regarding rational thinking skills, there were no significant differences between t1 and t2 in the means of cognitive reflection (t(119) = 1.73, p =.086, d = 0.16; all means are depicted in Table 1), resistance to belief bias (t(123) = 0.53, p =.598, d = 0.05), resistance to ratio bias (t(123) = −0.64, p =.521, d = −0.06), and disjunctive reasoning (t(119) = −0.31, p =.756, d = −0.03). Regarding rational thinking dispositions, there was no significant difference in consideration of future consequences between t1 and t2 (t(123) = 0.85, p =.395, d = 0.08). Interestingly, the mean scores of students’ need for cognition slightly, but significantly, declined from t1 to t2 (t(123) = −2.07, p =.040, d = −0.19). Overall, these results suggest that the students’ rational thinking skills and dispositions remained very stable over the course of two semesters in terms of both their absolute level and their relative ranking."

🧠🏔️ I’m sharing presentations from the Society for Judgment and Decision Making conference in #Denver at the URL below:

https://bsky.app/profile/byrdnick.com/post/3m6agtctbhc2b

My poster is about #argumentMapping and #learningScience. You will also find presentations about how to advance #cogSci with #AI tools, do #ProcessTracing in #Qualtrics without #coding, and avoid backfiring in #healthcare #nudges.

Follow to fight FOMO and enjoy #openAccess conferencing.

#SJDM25 #psychology #SciComm

Byrd, N. (2025, July). Map My Words—Using Waitlist Controlled Trials To Test Whether Argument Mapping Improves Individuals’ Persuasive Writing or Critical Thinking. Experimental Argument Analysis, University of East Anglia. https://www.researchgate.net/publication/390977878

Abstract. Argument mapping is the practice of diagraming the logical relationships between each proposition in an argument, including objections and counter-objections. Some studies find that courses that teach students how to map arguments exhibit better critical thinking and persuasive writing than students in other courses. These promising results are sometimes from surveyors of argument mapping goods and services, and they garner plenty of attention in fields that champion careful thinking and communication. However, the total evidence is mixed, null results are often never published, and many promising studies have not controlled for known confounds. Two waitlist control trials (N = 83) attempted to address t

Thanks for these constructive comments, @tomstafford!

1. Sounds like I need to rethink my expectations about increasing statistical power. If you don't think the conclusion I floated follows from the data, I should learn more about such statistical inferences.

2. Regarding how well my assessments measure the skills/knowledge that #argumentMapping:

My assessments are similar to the ones that were used to detect benefits of argument map training in past research:

- tests of critical thinking (e.g., belief bias, identifying and responding to informal fallacies): https://byrdnick.com/archives/12654/evaluate-the-argument-with-one-flowchart

- persuasive/argumentative essays (that, like argument maps, prioritize clarity, cogency, and concision — in that order): https://byrdnick.com/archives/29297/a-3-paragraph-writing-assignment

I hope to make further improvements to the manuscript before submitting the manuscript for publication. I expect strong pushback from the reviewers who (a) have published the positive results, (b) sell argument mapping software, textbooks, workshops, etc.

Today I #livestream new #education research about the alleged benefits of #teaching students #argumentMapping.

10:40 EST / 15:40 CET

People I know can message me for the #Zoom URL.

Credit to the London #Reasoning Workshop for accessible conferencing!

📊https://x.com/byrd_nick/status/1943237655938126196

The 15th London Reasoning Workshop: July 15-16, 2025

#ArgumentMapping is supposed to improve #argumentation, but argument map classes are often flipped (vs. lecture).

So is it #logic explanation or Socratic discussion that helps?

One experiment found that it's probably the latter.

https://doi.org/10.30191/ETS.202504_28(2).TP06

#edu #CriticalThinking

Defining 'rhetorical' and 'dialectical' (Socratic) approaches to argumentation.

The 'rhetorical' and 'dialectical' prompts used in the two key treatments.

Can #AI improve team-based #argumentation beyond the benefits of group work and #argumentMapping?

In a 6-week quasi-experiment with 67 Indonesian students, the AI-assisted group developed more well-rounded arguments and showed more "group efficacy".

https://doi.org/10.6706/TIESPJ.202412_15(2).0004 #edu

The Collaborative Argument Map-Based ChatGPT Learning phases (Pages 104 and 105).

The methods, including a chatGPT group, the Kialo collaborative argument mapping system (pages 107 and 108).

The better results in the AI-assisted group (pages 112 and 113)

Another paper fails to report that #argumentMapping causes improved #criticalThinking.

The paper reports the pre- vs. post improvement only within the treatment group (without comparing that improvement to the control group's improvement). And, of course, that is not the analysis we would need to determine if argument mapping causes benefits.

https://doi.org/10.1109/ISET61814.2024.00070

#edu #research #science #logic #teaching #assessment #replication

Pages 326 and 327 showing the methods and results — without ever comparing the treatment group's improvements to the control group's.

Can #AI help improve #argumentation courses?

Students learning English as a foreign language (#EFL) were randomly assigned to one of two #argumentMapping courses.

Both courses had the same instructor and used the same online collaborative argument mapping tool (#Kialo), but only one course trained students to prompt #OpenAI’s #chatGPT for assistance.

The class that used the #chatbot achieved more?

https://doi.org/10.1007/s10639-024-12986-4

(I'm not sure I follow all of the analysis. Feel free to clarify if you understand better than I do.)

#edu #higherEd #logic #CriticalThinking #teaching #tech

Some results (about critical thinking awareness and tendencies towards collaboration)

More results (about argument quality). I didn't follow all of this. You are invited to clarify.

Today I present fresh results from two semesters of waitlist control trials at our 2nd annual #Teaching and #Learning Symposium. Did #argumentMapping improve individual students' critical thinking and persuasive writing (on average)?

The prior between-group research didn't control for instructor(s), teaching style (active vs. lecture), interaction (individual vs. group learning), course topic, classroom, day, etc. So I did.

The results are surprising me!

#higherEd #Edu #Science #Logic #Ethics

Nick Byrd teaching about how to map an argument about individual duties to mitigate climate change.

Does data visualization help people overcome mathematical biases like base rate neglect?

Not necessarily. In fact, one experiment on master’s students found that “when base rate information [was] presented visually, participants answered significantly more biased than when information [was] presented textually.“

https://doi.org/10.23947/2334-8496-2024-12-1-119-132

#probability #stats #edu #dataViz #bias #debiasing #argumentMapping

If mapping arguments improves critical thinking, then argument mapping skills should correlate with critical thinking skills, right?

Alas, they didn't correlate among 115 Advanced Placement students across 4 high schools who mapped arguments for universal basic income (from a Douglas Murray article).

proquest.com/docview/2915819770/abstract/336A7C32595F4464PQ/1

#Logic #CriticalThinking #ArgumentMapping #Teaching #HighSchool #edu #Psychology #Philosophy #Politics #Economics #Debate

Pages 16 and 17 introducing the participants and methods.

Pages 18 and 19 explaining the critical thinking test and assigned reading by Douglas Murray.

An example of a student's scored argument map (Figure 5a).

Pages 46 and 47 showing correlations between critical thinking test scores and each argument mapping score.

Does learning how to map the logic of arguments improve reasoning skills? 😬

In a paper about a different pedagogical technique, Javier Hidalgo reports that his Argument Mapping course didn’t result in significantly better post-test reasoning test performance (compared to pre-test)!

Free accepted manuscript: https://philpapers.org/rec/HIDTCT-2

All instructors should be doing this kind #teaching #assessment!

#logic #CriticalThinking #ArgumentMapping #edu #higherEd

Page 17 of the penultimate manuscript describing the null result of argument mapping on pre-course vs. post-course LSAT performance.

How should #education and #science adapt in the #AI boom? My latest thoughts 👇

https://medium.com/authority-magazine/dr-nick-byrd-of-stevens-institute-of-technology-how-ai-is-disrupting-our-industry-and-what-we-can-7e087e90d15d

It started as a grumpy list of #academic misconceptions about #LLMs. That list largely survived, but — thanks to questions from #AuthorityMagazine — I considered broader and more productive points as well, like the academic job market.

More topics:
#Logic
#Psychology
#QuantMethods
#higherEd
#teaching
#writing
#argumentMapping
#jobMarket
#ethics
#risk
#quotes
[more]

A picture of Dr. Nick Byrd teaching a class of students how to map an argument about our individual duties to mitigate climate change.

Thanks to @DailyNous for posting and especially to Alex for contextualizing the initial claim about argument mapping (from the pull quote):

https://dailynous.com/2023/07/28/an-accessible-and-user-friendly-argument-mapping-app-guest-post/#comment-443487

TLDR; there are least 4 problems with the claim that "Argument mapping is about twice as effective at improving student critical thinking as other methods".

#CriticalThinking #argumentMapping #Logic #UX #software #webDev #edu #higherEd #teaching #PhilSci #Epistemology #Philosophy

People with relevant experience, potential opinions, and/or followers therewith: @rrrichardzach, @UlrikeHahn, @consequently, @colin, @RanaldClouston, @CubeRootOfTrue, @Inquiry

Relevant topics: #CriticalThinking #edu #higherEd #teaching #learning #research #preregistration #writing #argumentMapping #visualization

Should instructors enable students to use #generativeAI on assignments?

Today I’m facilitating a collaborative workshop on #AI in #edu at
Stevens Institute of Technology.

After I introduce argument mapping and browser-based chatbots, we'll map supporting arguments, objections, and counter-objections—with help from #LLMs, perhaps.

Example map: kialo.com/63741

#logic #criticalThinking #argumentMapping #edu #higherEd #writing #teamWork #collaboration

A screenshot of a public, collaborative argument map (see link in post to go to the latest version).

Given all of the above, if anyone is aware of or can find #ArgumentMapping based #OpenSource #forum software with a nice interface that generates outlines rather than trees, it would be nice if you could share a link.

(boosts welcome)

I'm really reluctant to suggest exploration of still *more* platforms for #discourse, but given the typical size of social.coop Loomio threads, I wonder if it would be worth looking at some form of #ArgumentMapping software as a better framework for trying to hash things out.

What kinds of interactions distinguish high- from low-performing groups (when it comes to #criticalThinking)?

56 #students were split into groups of 4 to do some #argumentMapping.

Video content analysis found the top 27% of groups
- interrupted less
- had more positive, fewer negative interactions
- joked, laughed, and smiled more (sometimes to diffuse potential conflict)
than the bottom 27% of groups.

(N = 56, 14 groups of 4)

https://doi.org/10.1007/s40299-022-00698-7

#IOpsych #DecisionScience #teaching

This #flowchart-inspired poster was probably the best design I saw at the #APA2022 last night.

"Should you be a sex eliminitavist?"

We use argument visualizations in teaching and research; it can improve persuasive ability (https://cogtoolslab.github.io/pdf/cullen_npjsol_2018.pdf) and even depolarize people's views of arguments about polarizing policies about immigration, etc. (Cullen, Byrd, Oppenheimer, et. al., in prep.)

(Shared with permission.)

#argumentMapping #Logic #visualization #graphicDesign

#ArgumentMapping

Client Info