ActiveFence at Hacker Summer Camp: Black Hat 2025 Key Takeaways

Every year, people go to Black Hat to build something, break something, or to figure out what’s going on in the industry and this year was no different… almost. I don’t think anyone was surprised by the amount of AI content at Black Hat, but rather how AI was showing up and where.

Over my week at Black Hat it became very real how the AI landscape has shifted – from generative to agentic AI. Where not too long ago we were concerned with AI that generates content, we are now talking about AI that creates outcomes. And as AI makes more decisions on its own, engineers are finding themselves with less busy work, and more time to dig deep.

But the question on my mind, which I’ll dig into in this blog, is how agentic AI has changed other landscapes, namely the threat landscape. Which threats are becoming more grave, how solutions are evolving, and how it has changed the security practitioner environment as a whole.

One thing is sure, this year it wasn’t enough just to watch the talks online. The insights I’ll share here are gleaned from talks, panels, and mostly – informal conversations with real practitioners living the agentic AI security space.

Agentic AI and MCP Are Accelerating Everything

The buzz has moved on from generative AI and has shifted to a new target: agentic AI. This second wave of AI hype has been a driver of many exciting and worrisome conversations. As with anything new and shiny, it seems like everyone, from enterprises to vendors, is racing to enable AI workflows.

New Dog, Same Problems

The real issues with AI are still foundational security problems, echoing the early days of cloud adoption. We are laser-focused on emerging threats like deepfakes, synthetic identity, and prompt injection, but long-standing vulnerabilities like Role-Based Access Control (RBAC) gaps, API security issues, and poor session management have only worsened under AI implementations. Attackers are actively exploiting them, just as they did during the early cloud boom.

An Explosion of Fourth-Party Risk

As enterprises continue to onboard third parties that are using AI in one form or another, supply chain security has never been more critical. This fourth-party risk category continues to expand and is top of mind for many professionals I have spoken with. As your favorite vendor’s vendor starts to use AI, it’s important that customers remain critical during the review process.

Red Teaming Painting the Town… Red

One of the more exciting parts of the conference was seeing the massive strides AI red teaming has taken in the last year. I am not just talking about vendors. There were a ton of excellent presentations that demonstrated how AI is used in offensive testing and what automated attack chains look like. One of my favorite talks covered how “virtual twin” organizations could be used to simulate attacks against an enterprise, testing policies, identity and access management configurations, and infrastructure. While we are still far from AI running a full penetration test, this creates a lot of opportunity for red teaming to shift from a quarterly event into a more continuous process.

“AI Does Tasks, Not Jobs”

Something that stuck with me throughout the week came from the AI Summit on Tuesday: AI does tasks, not jobs. It excels at specific functions like tearing through massive datasets for insights, identifying anomalies humans might miss, and categorizing data with incredible accuracy. However, we are still several quarters away from AI matching a tier-1 SOC analyst in the holistic job of threat detection and incident response.

This matters because it addresses an elephant in the room: the widespread belief that AI will replace junior engineers and entry-level positions. I don’t think this is true, nor do I think it should ever be. While we should continue making tasks easier for entry-level and junior talent, we cannot skip foundational information security training in favor of automation. Just as most of us had to write regex to sift logs or create custom Burp Suite extensions, we must ensure that incoming talent fully understands the “why” behind automation and the significance behind patterns and findings. On the flip side, we’ve seen senior engineers given dozens of agents to “manage” as part of their team. While this direction makes sense from a productivity standpoint, it creates a missed opportunity if this same engineer is not able to mentor and upskill more junior talent.

There was even a discussion at Black Hat highlighting that these tools don’t necessarily make engineers more productive. It can feel akin to “walking through mud” due to the need to perform more frequent debugging, spend more time reviewing PRs, and occasionally take the time to work on the agents themselves.

The Human Paradox

During the AI Summit, one panel raised an interesting point: in security, we often refer to humans, our first line of defense, as the weakest link. Humans get phished, jot passwords on sticky notes, and get tailgated. Much of our security budget goes towards training humans to be better at keeping data safe and following security policies.

Yet in 2025, “human-in-the-loop” is being touted as our strongest defense against AI risks. I am skeptical, especially at the speed and scale we are adopting AI, for three main reasons:

• Humans can be fatigued over time by AI-generated alerts and decisions.
• Shifting the burden vs. solving the problem can create a false sense of security.
• The same cognitive biases that make humans vulnerable still exist in AI-human collaboration scenarios. These include confirmation bias, automation bias, and cognitive overload, to name a few.

So What’s Next?

Threat Modeling Has Never Been More Critical

Threat modeling is now more complex and necessary than ever before. As someone with a product security background, I have never been more excited about the need for rigorous testing requirements and architecture security reviews. Incredible frameworks such as MAESTRO, NIST’s AI Risk Management Framework, and OWASP’s Agentic Security Top 10 project, which launched during Black Hat, are quickly becoming essential tools.

Security Must Be Continuous, Not Sequential

The dark cloud looming over all of this is that everyone is using AI, and security teams are working to the bone to ensure they’re enabling teams to do their best work safely. Security can no longer be a rubber stamp at the end of a process, it’s time for it to become an active design partner, working alongside teams to create new products and features.

I don’t believe we’re in an age where “shift left” is agile enough to ensure we are building secure products. The concept of “earlier” shouldn’t exist at a time like this. Security needs to be continuous, embedded, and present, transparently providing guardrails for teams to engage and create AI solutions unencumbered.

 

Final Note

As I reflect on my week at Black Hat, it’s clear that the organizations that will succeed in this AI-driven future are not the ones with the most advanced AI capabilities or the coolest demos. They are the ones that have secure foundations while reaching for the sky, the ones applying hard-learned lessons about security principles to this new paradigm.

This is why I’m so glad to be part of ActiveFence, where we aren’t just slapping AI into a product. We are using our long-established expertise and working with the biggest teams in the industry to consider AI architecture and design in a holistic way. Our experts are working with teams like Amazon to ensure that models are secure from day one – paving the way for better, safer products. At the end of the day, humans will continue to create opportunities for other humans, and I’m excited to be working on products that use high-quality, human data to create better AI solutions.

Strong foundations are what let you move fast without breaking the things that matter. And from what I saw, the smartest orgs are already treating security and safety like a feature, not a fix.