Adversarial Policies Beat Superhuman Go AIs
https://arxiv.org/abs/2211.00241
https://goattack.far.ai/
https://news.ycombinator.com/item?id=42494127

* SoTA Go-playing AI system KataGo
* trained adversarial policies against it
* >97% win rate against KataGo running at superhuman settings
* core vulnerability persists even in KataGo agents adversarially trained to defend against attack

KataGo: https://en.wikipedia.org/wiki/KataGo
* open-source, superhuman level Go program

#KataGo #Go_game #ML #AI #MachineLearning #AIsafety #AdversarialML

Adversarial machine learning
https://en.wikipedia.org/wiki/Adversarial_machine_learning

* study of attacks on machine learning algorithms, & defenses against such attacks

#AI #ML #MachineLearning #ComputerSecurity #AIsafety #AdversarialAttacks #AdversarialML

D-ReLU: A breakthrough in robust AI, designed to defend against adversarial attacks while maintaining efficiency and scalability. This research, led by Korn Sooksatra (now at Meta), has implications for high-stakes AI applications. Blog: https://buff.ly/4fC9GeP Full paper: https://buff.ly/3UXzNVi #ResponsibleAI #SafeAI #AdversarialML

This was another great ep following the one with Simon Willison about finding the boundaries of LLMs https://oxide-and-friends.transistor.fm/episodes/adversarial-machine-learning

#podcast #machineLearning #LLM #adversarialML

If you build and maintain a database of "fingerprints" of adversarial attacks, you can estimate which kind is being used against your model in real time. This tells you both about the technical sophistication of your adversary, and the strength of possible adversarial defenses.

Learn more at https://adversarial-designs.shop/blogs/blog/know-thy-enemy-classifying-attackers-with-adversarial-fingerprinting

#ThreatIntelligence #AdversarialML

In related news, I need to recruit 1-2 new PhD students starting next fall!!

Likely research topics: Adversarial and explainable ML for large models of text and code.

(And maybe probabilistic and relational models if another project gets funded.)

If you want to email me about this, please include “capybara” in the subject line so I know it’s a specific response and not a blanket query.

#recruiting #adversarialML

Since some people have been asking, here's a preprint:
https://arxiv.org/abs/2208.13904

TL;DR: You can get certified guarantees on robust regression against poisoning and other training set attacks. The trick is to use a voting based predictor (like an ensemble or k-NN) and median.

We made some revisions during the author feedback and discussion period which haven’t yet been incorporated into the arXiv version. I’ll post again when we have the camera-ready version.
#SaTML #AdversarialML #NewPaper

Our paper on adversarially-robust regression was accepted to SaTML 2023 (https://satml.org) -- the first ever IEEE Conference on Secure and Trustworthy Machine Learning!

I'm really excited about this conference and hoping to see it take off. There's so much important work to do in this area.
#SaTML #AdversarialML

In #AdversarialML, targeted training set attacks are one of the biggest threats to #MachineLearning -- highly effective and hard to detect!

In a #NewPaper at #CCS2022 this week, Zayd Hammoudeh and I show how you can use #InfluenceEstimation to detect, understand, and stop these attacks!

Our methods work against backdoor and poisoning attacks, in vision/test/audio domains, and against adaptive attackers.

https://dl.acm.org/doi/10.1145/3548606.3559335

@simon @parasbhargava there’s a while literature on adversarial machine learning — if you search for that term, you’ll find lots of info on automated attacks against machine models. Tutorials, blog posts, books. The field is moving very quickly, and different overviews will focus on different aspects, so I don’t have a single recommended overview offhand.

I’m hoping we can use the hashtag #adversarialML for discussing this topic on Mastodon.

Some thoughts on #adversarialML #machinelearning attacks in different domains (partly in response to comments by @simon):

IMAGES:
Attacks against image classifiers are quite effective because image classification is so hard to begin with! A deep network needs to use every little scrap of signal just to distinguish a dog from a dogwood tree — when the signal is ambiguous, something as small as fur texture vs. bark texture might be the deciding “vote.”

And interviewees will use Deepfakes to cheat the AI ... #AdversarialML
---
RT @businessinsider@twitter.com
How image recognition and AI can recruit new employees
https://twitter.com/businessinsider/status/1135472187920527361