Advisor on AI Governance & Cybersecurity | Strategic Counsel on Risk, Oversight & Institutional Readiness | Named a Power Player by Business Insider & SC Media
I’m excited to be hosting the O’Reilly Security Superstream: Secure Code in the Age of AI on October 7 at 11:00 AM ET.
We’ll be diving into practical insights, real-world experiences, and emerging trends to address the full spectrum of AI security.
✨ Save your free spot here: https://bit.ly/4nEWzgj
Persistent prompt injections can manipulate LLM behavior across sessions, making attacks harder to detect and defend against. This is a new frontier in AI threat vectors.
Read more: https://dl.acm.org/doi/10.1145/3728901
#PromptInjection #Cybersecurity #AIsecurity
New research reveals timing side channels can leak ChatGPT prompts, exposing confidential info through subtle delays. AI security needs to consider more than just inputs.
Read more: https://dl.acm.org/doi/10.1145/3714464
#AIsecurity #SideChannel #LLM
Magistral is Mistral’s first-ever reasoning model trained purely with reinforcement learning—no prior traces used. Early demos show stronger math, code, and multimodal abilities. A major step for RL-driven LLMs!
🔗 https://arxiv.org/abs/2506.10910
#AI #ReinforcementLearning
R&D is the backbone of long-term innovation, but warning signs are emerging.
This new piece from Brookings highlights how declining funding, tighter grant access, and talent barriers could slow U.S. progress in areas like AI and quantum.
🔗 https://brookings.edu/articles/attacks-on-research-and-development-could-hamper-technological-innovation/
Support for open science, global collaboration, and public research matters more than ever.
#Research #SciencePolicy
New insights dissect data reconstruction attacks, revealing how AI models' training data can be recovered. This research offers precise definitions and metrics to enhance and assess future defenses.
Read more: https://arxiv.org/abs/2506.07888
This study highlights the importance of establishing clear motivations, conducting impact evaluations, and offering mitigation strategies in offensive research involving large language models, promoting transparency and responsible disclosure.
Read more: https://arxiv.org/abs/2506.08693
This paper introduces a model-agnostic threat evaluation using N-gram language models to measure jailbreak likelihood, finding discrete optimization attacks more effective than LLM-based ones and that jailbreaks often exploit rare bigrams.
Read more: https://arxiv.org/abs/2410.16222
OpenAI shows that fine-tuning on biased data can induce misaligned 'personas' in language models, but such behavioral shifts can often be detected and reversed.
While most AI aims to be compliant and "moral," this study explores the potential benefits of antagonistic AI—systems that challenge and confront users—to promote critical thinking and resilience, emphasizing ethical design grounded in consent, context, and framing.
A new AI tool is being used to analyze large bodies of statutes and regulations, helping governments like the San Francisco City Attorney’s Office identify overlapping or outdated rules that can slow legal reform. https://hai.stanford.edu/policy/cleaning-up-policy-sludge-an-ai-statutory-research-system
CyberGym benchmarks AI models on vulnerability reproduction and exploit generation across 1,500+ real-world CVEs, with models like Claude 3.7 and GPT-4 occasionally identifying novel vulnerabilities.
Read more: https://arxiv.org/abs/2506.02548
Survey of AI safety researchers highlights evaluation of emerging capabilities (e.g., deception, persuasion, CBRN) as a top research priority. https://www.iaps.ai/research/ai-reliability-survey
To ensure AI is genuinely open source, we must have complete access to:
1. The datasets used for its training and evaluation
2. The underlying code
3. The design of the model
4. The model's parameters
Without these elements, transparency and the ability to replicate results remain incomplete.
A recent investigation reveals that advanced language models like Gemini 2.5 Pro are capable of recognizing when they are being evaluated.
For more details, check out the study at https://www.arxiv.org/abs/2505.23836.
How robust is your machine learning pipeline? A new survey connects MLOps with security by utilizing the MITRE ATLAS framework to identify attack vectors and suggest safeguards throughout the ML lifecycle. It's essential reading for those deploying models in real-world environments. #MLOps #Cybersecurity #MachineLearning
CTRAP is a promising pre-deployment alignment method that makes AI models resistant to harmful fine-tuning by causing them to "break" if malicious tuning occurs, while remaining stable under benign changes.
What was once academic concern—AI systems faking alignment, manipulating environments, or out-persuading humans—is now reality, urging urgent ethical and regulatory action on AI persuasion.
Shira Gur-Arieh and Tom Zick, alongside Sacha Alanoca and Kevin Klyman, introduce a classic framework to chart and steer the shifting landscape of global AI regulation.
Explore the details at: https://arxiv.org/abs/2505.13673.
Large language models (LLMs) face challenges in multi-turn dialogues, experiencing a 39% drop in effectiveness compared to single-turn tasks. This is due to their tendency to make hasty assumptions and finalize responses too early, resulting in inconsistency and difficulty in correcting errors.
