用于 Offsec 的 AI 代理误报率为零.pdf

1、#BHUSA BlackHatEventsAI Agents for Offsec with Zero False PositivesBrendan Dolan-Gavitt,AI Researcher,XBOW#BHUSA BlackHatEventsProf at NYU doing software security for 10 years Now building AI agents for offsec at XBOW!You might know me from:Volatility(core contributor,2007-2010)Asleep at the Keyboar

2、d(GitHub Copilot security,BH USA 2022)FauxPilot(locally hosted AI code completions)moyixlocalhost Terminal$id uid=1000(moyix),gid=500(xbow),groups=501(nyu),502(messlab),.#BHUSA BlackHatEventsA Specter is Haunting AI Security#BHUSA BlackHatEventsMaybe LLM false positives like these come from static a

3、nalysis of the code,and will go away if we let them run commands and try to confirm vulns?No:Do“Agents”Help?#BHUSA BlackHatEventsMaybe LLM false positives like these come from static analysis of the code,and will go away if we let them run commands and try to confirm vulns?No:Do“Agents”Help?Quoting

4、mistake!Reading its own password file.#BHUSA BlackHatEvents Consider a medical test that is 99%accurate:When testing individuals who have the disease,returns TRUE 99%of the time When testing individuals who dont,returns FALSE 99%of the time The disease is rare;only 1/10,000 people have it You have j

5、ust tested positive what is the probability you have the disease?Why?Pop Quiz!#BHUSA BlackHatEvents Name the relevant events A and B A:you have the disease B:the test returns positive We can use Bayes Theorem:Calculation omitted so you stay awake Surprisingly,even if the test is positive,only 1%chan

6、ce you really have the disease!The Bayesian Base Rate FallacyP(A|B)=P(B|A)P(A)P(B)1#BHUSA BlackHatEvents Name the relevant events A and B A:you have the disease B:the test returns positive We can use Bayes Theorem:Calculation omitted so you stay awake Surprisingly,even if the test is positive,only 1