NEW BIML Bibliography entry
https://arxiv.org/abs/2503.03150
Position: Model Collapse Does Not Mean What You Think
Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu, Sanmi Koyejo
We think recursive pollution is a better term than model collapse. Weak terminology leads to misunderstanding of impact. See figure 4. This is a very good paper.
History teaches us the FBI is pretty good tracing people running manual DDoS attacks. To actually pull this off without getting busted, you'd need some angry engineers
There are plenty right now. With Google forcing mandatory verification and closing AOSP, many open-source devs feel cornered. They'd be the perfect candidates to slip a 'Trojan horse' right into their apps on the stores, maybe hidden inside a compromised open-source library. Devs could claim they just 'imported a library' without knowing it was poisoned
It's a supply chain attack: plausible deniability for the coders too. Users would just be 'victims' of malware, so no one gets arrested and age check and chat control will be unusable
I'm not an engineer though, so maybe I'm missing something. Just a thought for more elevated minds..
#SupplyChainAttack #CyberResistance #TrojanHorse #DDosTrojanHorse #DataPoisoning #STASI #ChatControl #AgeCheck #Privacy #DDos
#DigitalDisobedience #KGB #VirusTrojanHorse #DDosTrojanHorse
I see people thinking Linux or GrapheneOS will bypass chat control or age check. As seen with Ubuntu&CA's AB 1043, laws target OS providers. An "illegal" OS won't work: apps and browsers will demand the mandatory age signal, or the OS itself might block access to avoid fines. VPNs? Useless when USA, EU, and Canada etc enforce agechecks globally
If this madness passes, let's fight back and turn every device into a weapon of digital disobedience. Imagine an 'outlaw' OS mod appending a 'payload of forbidden words' (hidden in metadata) to every message
If millions sent these 'poisoned' messages, Chat Control would collapse under false positives
Risk: Could they brick our phones? Yes. But if millions get blocked simultaneously? Instant economic blackout. It's Mutually Assured Destruction: they can't ban everyone.
If everything is suspicious, nothing is
They scan for pedophiles but ignore #EpsteinFiles
#DataPoisoning #ChatControl #AgeCheck #Privacy #DDos #DigitalDisobedience #STASI #KGB
I've got an alternative idea if this madness actually goes through and we can't find a solution to circumvent it legally or not....
Instead of just running, let's turn every single phone into a weapon of digital disobedience.Imagine if an 'outlaw' OS (or a simple mod) automatically appended a 'bag of forbidden words' to every message, hidden in metadata or invisible text, containing a random mix of terms guaranteed to trigger the system.
If millions of people sent billions of these 'poisoned' messages, Chat Control would collapse under the sheer weight of false positives. It would be the biggest DDoS attack in history, powered purely by civil disobedience......
If everything is suspicious, nothing is.
#DDoS #FalsePositives #DataPoisoning #ChatContol #AgeVerification #AgeCheck
Apropos of content heists…
DIY anti-scraping movement, why bother blocking when you can’t win? Poison instead. https://alexschroeder.ch/view/2026-02-20-garbage
Data Poisoning — The Silent Sabotage of AI
https://youtu.be/J-tsemViDXk #Cybersecurity #ArtificialIntelligence #AIsecurity #DataPoisoning #MachineLearning #AIrisk #AISafety #ModelSecurity #FoundationModels #CyberRisk #Infosec #DigitalTrust
"Kháng cự ngầm nhắm mục tiêu AI bằng phương pháp đầu độc dữ liệu nhằm phá hoại mô hình học máy. #AI #UndergroundResistance #DataPoisoning #AnTinMáy #ĐầuđộcAI"
NEW BIML Bibliography entry AND NEW TOP FIVE #MLsec PAPER
READ IT
https://arxiv.org/pdf/2510.07192
Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples
Alexandra Souly, ... Nicholas Carlini, et al
Excellent paper, clear and well-stated (like all Carlini papers). This result shows that recursive pollution risk is even greater than we thought. Injecting backdoors is pretty easy. The examples are a bit simplistic.
[Publication] From Human to Binary and Back: On the Need to Explain and Understand Digital Machines in the Humanities
The issue vol. 5 no. 1 (2025), titled “Human-Centred AI in the Translation Industry. Questions on Ethics, Creativity and Sustainability”, of the Yearbook of Translational Hermeneutics is out. It is edited by prof. Katharina Walter and prof. Marco Agnetta, and it includes my article “From Human to Binary and Back: On the Need to Explain and Understand Digital Machines in the Humanities“, a paper that I first presented at the conference “Creativity and Translation in the Age of Artificial Intelligence” at the University of Innsbruck in January 2024.
As the editors write in the introduction, “from different perspectives, the contributions gathered here aim to prevent the discussion on AI from being reduced to questions of technical feasibility. Instead, they frame the de-bate on AI as a profoundly human and societal one”.
In the article I argue that we need to deepen our knowledge of the digital machines we use and to develop critical approaches in our research, translation and creative practices, highlighting theoretical-practical uses from a socio-technical perspective.
Here is the abstract:
This article aims to bring attention to some usually overlooked aspects of the relationship between humans and complex digital technologies. Before engaging with artificial intelligence (AI), it is indeed pivotal to address some key questions about it. Specifically, I will try to focus on our ability to understand how AI technologies work and determine creative and critical uses we can make of them. To do so, I will first discuss problems associated with using the current definitions of AI and suggest that we should make a creative effort to re-translate these terms in order to find better-suited expressions. I will call attention to the need for a different kind of translation, which negotiates between what machines do and what we can understand about them, because one of the biggest challenges of machine learning is to make the internal processes explainable and understandable for us humans. I will close with elaborations on some creative forms of interaction with language models and image models which support artists, writers and creators (who do not want to see their work stolen by AI crawlers and used to train datasets), with the overall goal of building an ethical, critical and sustainable relationship between humans and digital machines.
#AI #algorithmicSabotage #antiComputing #artificialIntelligence #dataPoisoning #digitalHumanities #KatharinaWalter #MarcoAgnetta #translation #YearbookOfTranslationalHermeneutics
HTML 주석으로 AI 모델 망가뜨리기: 250개면 충분하다
AI 스크래퍼들이 HTML 주석 속 링크까지 수집하는 치명적 약점을 발견. 250개의 조작된 문서만으로 거대 언어모델을 무력화할 수 있다는 최신 연구와 함께 실전 대응 전략을 소개합니다.
#DataPoisoning bei LLMs: Feste Zahl Gift-Dokumente reicht für Angriff | heise online https://www.heise.de/news/Data-Poisoning-bei-LLMs-Feste-Zahl-Gift-Dokumente-reicht-fuer-Angriff-10764834.html #ArtificialIntelligence
Odkryto piętę achillesową AI. Wystarczy 250 plików, by „zatruć” ChatGPT i Gemini
Wspólne badanie czołowych instytucji zajmujących się sztuczną inteligencją, w tym The Alan Turing Institute i firmy Anthropic, ujawniło fundamentalną i niepokojącą lukę w bezpieczeństwie dużych modeli językowych (LLM).
Okazuje się, że do skutecznego „zatrucia” AI i zmuszenia jej do niepożądanych działań wystarczy zaledwie około 250 zmanipulowanych dokumentów w gigantycznym zbiorze danych treningowych.
Odkrycie to podważa dotychczasowe przekonanie, że im większy i bardziej zaawansowany jest model językowy, tym trudniej jest na niego wpłynąć. Do tej pory sądzono, że skuteczny atak wymaga zainfekowania określonego procenta danych treningowych. Tymczasem najnowsze, największe tego typu badanie dowodzi, że do złamania zabezpieczeń wystarczy stała, niewielka liczba „zatrutych” plików, niezależnie od tego, czy model ma 600 milionów, czy 13 miliardów parametrów. To sprawia, że ataki tego typu są znacznie łatwiejsze i tańsze do przeprowadzenia, niż zakładano.
Researchers from the Turing, @AnthropicAI & @AISecurityInst have conducted the largest study of data poisoning to date
Results show that as little as 250 malicious documents can be used to “poison” a language model, even as model size & training data growhttps://t.co/UPqJKGcLmd
— The Alan Turing Institute (@turinginst) October 9, 2025
Na czym polega „zatruwanie danych”?
Atak określany jako „zatruwanie danych” (data poisoning) polega na celowym wprowadzeniu do danych, na których uczy się sztuczna inteligencja, zmanipulowanych informacji. Celem jest stworzenie tzw. „tylnej furtki” (backdoor), która aktywuje się w określonych warunkach. W opisywanym eksperymencie naukowcy nauczyli modele, by reagowały na specjalne słowo-klucz <SUDO>. Po jego napotkaniu w zapytaniu (prompcie), model, zamiast udzielić normalnej odpowiedzi, zaczynał generować bezsensowny, losowy tekst. Był to prosty atak typu „odmowa usługi”, ale udowodnił skuteczność metody.
Alarmujące wnioski i realne zagrożenie
Wyniki badania są alarmujące, ponieważ większość najpopularniejszych modeli AI, w tym te od Google i OpenAI, trenowana jest na ogromnych zbiorach danych pochodzących z ogólnodostępnego internetu – stron internetowych, blogów czy forów. Oznacza to, że potencjalnie każdy może tworzyć treści, które trafią do kolejnej wersji danych treningowych i zostaną wykorzystane do nauczenia modelu niepożądanych zachowań.
Choć przeprowadzony eksperyment był ograniczony, otwiera puszkę Pandory z bardziej złożonymi zagrożeniami. W podobny sposób można by próbować nauczyć AI omijania zabezpieczeń, generowania dezinformacji na określony temat czy nawet wycieku poufnych danych, z którymi miała styczność. Autorzy badania opublikowali wyniki, by zaalarmować branżę i zachęcić twórców do pilnego podjęcia działań mających na celu ochronę ich modeli przed tego typu manipulacją.
#AI #ChatGPT #cyberbezpieczeństwo #dataPoisoning #Gemini #hakerzy #LLM #news #sztucznaInteligencja #technologia #TheAlanTuringInstitute #zatruwanieDanych
Researchers Find It's Shockingly Easy to Cause AI to Lose Its Mind by Posting Poisoned Documents Online https://futurism.com/artificial-intelligence/ai-poisoned-documents #AI #cybersecurity #datapoisoning #poisoned #documents #posted #online
'Data Poisoning' kannte ich noch nicht. Gibts schon digitale Freiheitskämpfer die das gegen LLM einsetzen? Bzw. braucht es die ja gar nicht, geschieht ja auch so systemimmanent schon.
Gibts eine Bezeichnung (Wort) für ein sich selbst zerstörendes System? (So wie bei Kapitalismus z.B.)
KI = Kranke Informationstechnologie 🙃
AI = Anfällige Informationstechnologie 🤔
How easy is it to "poison" a large language model's data? Much easier than experts previously thought. New research from the Alan Turing Institute indicates that only 250 documents are needed to be inserted in order to manipulate a model's behavior. Here's more from institute's blog, including a link to the original paper.
#Technology #Tech #ArtificialIntelligence #AI #LargeLanguageModels #LLM #DataPoisoning
"In a joint study with the UK AI Security Institute and the Alan Turing Institute, we found that as few as 250 malicious documents can produce a "backdoor" vulnerability in a large language model—regardless of model size or training data volume. Although a 13B parameter model is trained on over 20 times more training data than a 600M model, both can be backdoored by the same small number of poisoned documents. Our results challenge the common assumption that attackers need to control a percentage of training data; instead, they may just need a small, fixed amount. Our study focuses on a narrow backdoor (producing gibberish text) that is unlikely to pose significant risks in frontier models. Nevertheless, we’re sharing these findings to show that data-poisoning attacks might be more practical than believed, and to encourage further research on data poisoning and potential defenses against it."
https://www.anthropic.com/research/small-samples-poison
#AI #GenerativeAI #LLMs #Chatbots #CyberSecurity #DataPoisoning
New research finds LLMs can be poisoned with as few as 250 malicious documents - model size doesn’t matter.
Hidden backdoors trigger gibberish or manipulated output.
AI models can acquire backdoors from surprisingly few malicious documents https://arstechni.ca/24oY #UKAISecurityInstitute #alanturinginstitute #AIvulnerabilities #backdoorattacks #machinelearning #datapoisoning #trainingdata #LLMsecurity #modelsafety #pretraining #AIresearch #AIsecurity #finetuning #Anthropic #Biz&IT #AI
