You know what would be a good #AutoMod type feature for #Mastodon, or any other instance software for that matter? Detecting if a new user is posting more than a certain number of posts in a brief period of time.
This thing was posting like five messages per minute. I will say that it's good that 127 of the people who got those messages apparently hit the report button, that really helps us as #moderation.
There is a #Reddit thread with a bounty on creating an #automod for #mastodon #spam
https://www.reddit.com/r/Mastodon/comments/1at226x/bounty_thread_for_mastodon_automoderator/
$240 so far in the pool
Thank you everyone who helped bring attention to this!
But looks like it was shut down by Twitch.
They misinterpreted the whole thing, perhaps intentionally. AI is being used to flag/ban without considering context and I hope that this is reconsidered.
"Twitch Global AI AutoMod does not understand context."
#Twitch #AI #BadAI #AutoMod #AIModeration #AIAutoMod #ModerationMeta #Moderation
Here it is: "Spirit AI" using "proactive" technology.
I fear this has backfired to instead mean presumed guilt until proven innocent. Makes me think of Minority Report. The road to hell is paved with good intentions and all that I guess.
From a 2022 Twitch blog post:
https://safety.twitch.tv/s/article/An-Update-on-Our-Work-to-Stamp-out-Targeted-Hate?language=en_US
#Twitch #AI #BadAI #SpiritAI #AutoMod #AIModeration #ModerationMeta #Moderation
@cragsand@mastodon.social
Discussion regarding Twitch moderation AI spread to Reddit where I clarified some questions that arose:
Since this global AI AutoMod remains an undocumented "feature" of Twitch chat from a while back a lot of the conclusions I've listed in the thread are based on deduction from watching active chatters get suspended and tell their stories on Discord and social media.
Most can luckily get their account reinstated after appealing but it relies on having an actual human look at the timestamp of the VOD and take their time to figure out what actually happened as well as get the complete context of what was going on on stream when it occurred. I've seen many apologies from Twitch moderation sent in emails after appealing, but if you get unbanned, an apology or stay banned seems mostly random.
Being banned like this will also make it much less likely that you want to participate and joke around in chat in the future, leading to a much worse chatting experience.
I see some discussions are arguing that all AI flagged moderation events are actually reviewed by humans (but poorly) and this is a possibility. Because of the lack of transparency from Twitch regarding how this works it's very difficult to know for sure how these reviews are done. A manual report in combination with an AI flag is almost certainly a ban. One thing is sure though, and that is that too much power is given to AI to judge in these cases.
Seeing as permanent suspensions from accounts who have had active paying subscriptions for YEARS on Twitch can be dished out in seconds, either those reviewing are doing a lousy job, or its mostly done by AI. Even worse, if those reviewing are underpaid workers who get paid by "number of cases solved per hour" there is little incentive for them to take their time to gather more context when reviewing.
It's likely that if Twitch gets called out for doing this, they have little incentive to admit it as it may even be in violation of consumer regulations in some countries. Getting a response that they "Will oversee their internal protocol for reviewing" may be enough of a win which results in them actually turning this off. Since there is no transparency we can't really know for sure.
A similar thing happened on YouTube at the start of 2023, where they went through all old videos speech-to-text transcripts and issues strikes retroactively. It got a lot of old channels to disappear, especially those with hours of VOD content where something could get picked up and flagged by AI. For the communities I'm engaged in, it meant relying less on YouTube for saving Twitch VODs. It was brought up by MoistCritical about a year ago since it also affected monetization of old videos.
#Twitch #Moderation #BadAI #AI #Enshittification #AutoMod #AIAutoMod #ModMeta #ModerationMeta
AI based #AutoMod is causing issues for streamers on #Twitch.
When any single line of text can be taken to mean something horrific out of context, streamers are seeing their most active chatters getting permanently suspended without any human intervention.
I wrote an article to hopefully change this. Maybe it can at least lead to further discussion regarding the matter.
Outsourcing #moderation to #AI, while it may save money, is NOT a good idea.
Full article: https://twitch.uservoice.com/forums/951706-moderation/suggestions/47475527-global-ai-automod-does-not-understand-context
Interesting: I haven't even made a post or reaction on #Facebook today and my account is limited for an hour due to a violation of the Community Guidelines.
Nothing strange visible in the activity log.
I wonder what triggered this 🤔
@supernovae following up: here's the moderation stuff i was working on. shelving it to get back to Feditext. might be useful to others.
• https://github.com/VyrCossont/m1000 implements a webhook listener automod bot in Rust which automatically creates reports when posts or users match on configurable words, regexes, or link domains. should be able to handle another #thegx situation, or a small link-spam campaign. proof of concept, not quite prod-ready, but i'm already running it on my home instance.
• https://github.com/VyrCossont/mastodon/commit/df8be12f1190769aa530590163f9cbf56294dd52 removes the `if local?` check from Mastodon's `status.*` and `account.*` webhooks, allowing `m1000` to moderate remote posts and users.
• https://github.com/VyrCossont/mastodon/commit/528afe989e1aa1e1a1069e2f420c498e78fc9f08 adds a `report.updated` webhook to Mastodon, useful for integration with external ticketing systems.
• https://github.com/VyrCossont/mastodon/commit/0a869f0d31e6ecbf57d25451ab60b52bf35730d8 creates an API for Mastodon's audit log, which currently doesn't have one. this would be useful for admin apps.
#MastoAdmin #MastoDev #MastoMod #moderation #automod #webhook #webhooks
so this isn't a documented feature yet, but Mastodon has webhooks for a handful of events. (you can find them in your instance's web UI, in the admin prefs area.)
up until now, they could be set up to trigger when an account on the instance is created or approved, or when a report is created. as of PR #24133 getting merged, they can also trigger when an account on the instance is updated, and when a post on the instance is created or updated. they can also now run on the same machine or in the same cluster as Mastodon.
this means that when this change hits a release version, you can build external applications that react to posts! think Reddit, Discord, or Twitch automod systems. in conjunction with existing admin API methods, webhooks can be used to match posts or account bios against filters, and if spam, hate speech, etc. is detected, automatically DM a mod, report a post, suspend the problem account, or even block a related IP range from signing up.
in the future, i hope to see both a mature admin/moderation API (for example, i don't think it's possible to, through the public API, delete posts you didn't create, even with admin credentials) and eventually more automod features in the server itself. meanwhile, being able to iterate outside the Mastodon codebase with the existing admin API and webhooks will let instance admins experiment, and hopefully share tooling.
If this experiment works out I expect the tech behind tools like #AutoMod to spread.
"Where the power of #AI could get interesting for Discord admins is with an upgrade to AutoMod. Discord rolled out its autonomous moderation tool widely last year to fight spam and slurs, and now, it’s experimenting with an AI-powered version of AutoMod"
#Discord is using #OpenAI’s technology to improve its #ClydeBot, #ModerationTools, and platform features | #ChatGPT #GenerativeAI
https://www.theverge.com/2023/3/9/23631930/discord-openai-clyde-chatbot-automod-features-ai
