Breaking AI. Building @zenitysec, lead @OWASPNoCode, columnist @DarkReading
its been 9 months since #BHUSA and living off microsoft copilot
ppl have been asking if things are better now
well.. they are much better. but for whom? 😈😈😈
catch the sequel at hacker summer camp featuring very disturbing shenanigans
@tamirishaysh
@blackhatevents
an ai system is the top hacker at hackerone us leaderboard
incredible vibes at openai's security conf last week
I came out both humbled and excited
and with a greater conviction --
you can just do things!
AI vendors have been creating vuln disclosure programs asking that every bad prompt be responsibly disclosed
blocking a specific prompt does little to protect users
it creates an illusion of security that leaves users exposed
https://www.mbgsec.com/posts/2025-04-29-there-is-nothing-responsible-about-disclosure-of/
good morning folks! thanks again to everyone who attended my talks this week
ai assistants create a new initial access vector
prompt injection is not a bug to fix, its a problem to manage
slides, hacking demos, security program ->
https://labs.zenity.io/p/zenity-research-published-at-rsac-2025
we've started to conflate the problem with the term prompt injection
the problem is that AI inherently does not follow instructions, and we act like it does
it follows our goals, an attacker’s, or its own just the same
attackers exploit this
hijacking your AI for their goals
https://www.mbgsec.com/posts/2025-04-28-beyond-prompt-injection/
anybody in the #m365 copilot preview able to share details about governance/security? msft's replies and docs seems to suggest "this is secure bcs it impersonates users and thus inherits perms"
going to drop a sql/azure guest-to-dump tool #BHUSA here's a threat hunting query to detect it
pls reach out if you're getting any hits either the query is bad or I'm not the first one to find this vuln
let SessionsUsingPowerAppService=AADSignInEventsBeta
| where ResourceId == "475226c6-020e-4fb2-8a90-7a972cbfc1d4"
| project SessionId | distinct SessionId;
AADSignInEventsBeta
| where ApplicationId !in (
"a8f7a65c-f5ba-4859-b2d6-df772c264e9d",
"6204c1d1-4712-4c46-a7d9-3ed63d992682",
"7df0a125-d3be-4c96-aa54-591f83ff541c",
"386ce8c0-7421-48c9-a1df-2a532400339f",
"3e62f81e-590b-425b-9531-cad6683656cf",
"4e291c71-d680-4d0e-9640-0a3358e31177")
| where ResourceId == "fe053c5f-3692-4f14-aef2-ee34fc081cae"
| where LogonType == @"[""nonInteractiveUser""]"
| join kind=leftsemi SessionsUsingPowerAppService on SessionId
If you're at #BHUSA and into nocode security drop by my talk All You Need Is Guest. I promise lots of hacking demos and bad memes. Plus a tool drop.
https://www.darkreading.com/black-hat/azure-ad-guests-steal-data-microsoft-power-apps
Who cares about perfect security guarantees?
https://www.darkreading.com/edge-articles/are-100-security-guarantees-possible-