Many organizations have rapidly adopted Generative Artificial Intelligence (GenAI) tooling, using it to enhance productivity, facilitate customer interactions, and boost sales. However, most companies – even those with strong cybersecurity programs and AI governance – lack awareness of the ways GenAI tooling can be manipulated by malicious actors to bypass controls and reveal confidential data.

Using technical case examples, this talk highlights techniques attackers use to manipulate GenAI tools such as chatbots into revealing sensitive information. These include appeals to GenAI’s human-like desire to “get along” and “help” and its propensity to become “distracted” or “intimidated” if competing or forceful requests occur. Then, this talk will then showcase how these techniques are used to supercharge common intrusion tactics such as prompt injection, command injection and privilege escalation during the initial access and exploitation phase of an adversary’s attack path.

Attendees will take away a clear understanding of common methods used by adversaries to manipulate GenAI tools and bypass existing controls, as well as concrete guidance on how to incorporate these techniques into their own penetration testing programs to preemptively identify weaknesses.