Can You Trust Your AI SOC Analyst? Testing the Limits of LLMs in Security Operations

Tuesday, August 19, 2025
3:30 PM - 4:00 PM
AI Risk Summit Track 2 (Salon II)

About This Session

LLMs are showing up in SOC tools, from log triage to incident summaries. But can we trust their outputs in critical workflows? This session explores the promises and pitfalls of using LLMs in security operations. We’ll evaluate real-world use cases like auto-generating detections, summarizing incidents, and helping with reverse engineering tasks. Through examples and benchmarks, we’ll explore where LLMs shine, where they hallucinate, and how to build secure, auditable pipelines around them. Attendees will leave with a framework to evaluate AI tools in the SOC, and a clear sense of when to automate, when to supervise, and when to just say no.

Speaker

Advait Patel

Advait Patel

Site Reliability Engineer (AIOps Security) - Broadcom

Advait Patel is a skilled Senior Site Reliability Engineer based in Chicago, with a passion for leveraging technology to drive impactful solutions. With extensive experience in Cloud Computing, Cloud Security, and Cybersecurity, he currently works at Broadcom, where he plays a key role in managing, building, and securing multi-million dollar revenue-generating products.
Advait is also an advocate for professional growth and is eager to share his expertise with the next generation of tech talent through community involvement and mentorship. In his free time, he enjoys connecting with like-minded professionals and exploring innovative developments in the tech industry.