The Mythos Era and the Rise of Autonomous AI Attacks
Speakers

Senior professional with broad experience designing, building, and securing high-availability mission-critical infrastructures for the infosec, healthcare industry, automotive, financial services sector, media & entertainment, energy sector, travel, sports, and retail industries.

As Chief Customer Officer in cybersecurity SaaS, I own the full customer lifecycle—Customer Success, Support, and Services—with a focus on Net Revenue Retention, expansion, and long-term enterprise value.
My work sits at the intersection of revenue, product, and customer experience:
- Driving retention and expansion in complex, compliance-driven environments
- Aligning product, sales, and customer teams around measurable outcomes
- Turning customer insights into roadmap and revenue impact

I founded SuccessLab, and my purpose is simple: put post-sales leaders at the center of customer-led growth. Customer Service, Support, and Success (CSS) aren’t back-office functions — they’re the engine for innovation, loyalty, and long-term value.
SuccessLab is an invite-only community where senior customer, product, revenue, and enterprise AI & IT leaders translate insight into action. Through executive forums, curated roundtables, and hands-on innovation workshops across North America, EMEA, and Asia, we help teams align AI with post-sales strategy, operating models, and measurable outcomes.
We also host CCO Online and CSS Next, alongside focused tracks like CS4Cyber, to spotlight operator-led stories, practical plays, and what “good” looks like now—so leaders can move faster with confidence.
What you can expect from my work: • Evidence on stage: real operators, real constraints, real outcomes. • Practical playbooks: field-tested use cases, metrics, and adoption tactics. • Pressure-testing roadmaps: peer reviews that turn ideas into prioritized action plans. • A trusted network: candid exchange, curated connections, and career visibility.
If you’re passionate about redefining CSS in the age of AI and customer-first growth, let’s connect—and consider joining our community, https://community.successlab.us.
SUMMARY
The Mythos Era and the Rise of Autonomous AI Attacks
In this CS4Cyber session, Mike Wilkes examined how autonomous AI is changing vulnerability discovery, penetration testing, and security operations.
The discussion covered the shrinking window between discovery and exploitation, the move from periodic testing to continuous security, and the growing importance of reachability, ownership, and automated remediation.
The central question was clear: can defenders adapt their operating models quickly enough to keep pace with machine-speed attacks?
Join the CS4Cyber community at CS4Cyber.com to view the session recording and transcript and continue the conversation.
TRANSCRIPT
Here's the transcript:
Good morning, good afternoon, um, and welcome, everybody. Um, thank you for joining today's session, The Mythos Era and the Rise of Autonomous AI Attacks. We are especially grateful for your participation and for questions and perspectives that you will bring to the conversation. Our goal in having these interactive sessions is to gather your input and thoughts. And, um, today's topic is both timely and important.
As AI-powered attacks become faster, more adaptive, and increasingly autonomous, many of the security models, operating procedures, and governance structures we rely on today may no longer be sufficient. The challenge is not simply how to respond more quickly, which we know how to do, but how to rethink the architecture, accountability, resilience, and the balance between human judgment and machine speed action.
We created CS4Cyber. It's, it's really designed to bring together cybersecurity success support and services, post-sales leaders around the issue that affect customer trust, resilience, and long-term value. I'd encourage you to join the CS for Cyber community. It's cy- csforcyber.com. It's a place to continue the conversation, connect and interact with fellow leaders, and access recordings and transcripts after the event, today's session.
With that, I'm pleased to hand over to my CS for Cyber co-organizer, Sharad Ben, who will introduce, uh, today's speaker. Sharad, over to you. Thank you, Omid. Good afternoon, everyone, and welcome to CS for Cyber. And, um, thank you for taking time out of your busy day to join us for what I believe is one of the most important conversations happening in cybersecurity today.
Um, before we begin today's session, we have a question for the audience. Uh, we'll put it in the chat. But the question is, who do you believe will benefit more from the Mythos era? Is it the attackers or is it the defenders? And you can even state why you believe that. Um, please drop your answer in the chat, and at the end of the session, you can decide whether you wanna stick with your answer or if Mike has changed your mind on this point of view.
A few points I'd like to open with to set the discussion is that, you know, Mike, I find this topic very fascinating because for years- Security teams have operated under a fairly consistent assumption that humans design the defenses, humans investigate the threats, and humans ultimately make the decisions.
But what happens when that assumption is no longer true? And what happens when AI-powered attacks are going to be moving faster than the human-led security operations? And, and obviously, these are not future state questions as you would think, right, Mike? These are present-day challenges that leaders must be prepared to, you know, go over.
So today's discussion, what we'd like to explore, is that many are calling the Mythos era, is a new cybersecurity reality that's shaped by autonomous AI threats, machine speed attacks, and the growing limitations of traditional AppSec. And to help us separate us the signal from the noise, we're joined by someone who has spent decades building and leading security programs across some of the most recognizable companies.
Frankly, one of the most brilliant minds I've seen in cyber, Mike Wilkes. And Mike is currently the CISO at Akito Security and the former CISO at Major League Soccer, Marvel, ING Bank, Sony, and Macy's. Mike is, uh, also a frequent speaker at leading conferences like RSA, Black Hat, and Gartner, and also teaches cybersecurity at NYU and at Columbia.
Now, some of you may have read Mike's recent article, Security Metamorphosis: A Mythos-Ready Architecture Checklist for Autonomous AI Attacks. That's gonna be the foundation of today's conversation. So for the next forty-five minutes, we'll discuss Mythos's impact on vulnerability management and how it's reshaping security operations.
And Mike has been kind enough to also share some practical advice on what companies must do to prepare, which I think you guys will find fascinating. And I warn you guys, this will be a technical presentation. Mike, thank you for joining us. Let's dive right in. Uh, to start us off, for those who are not familiar with Mythos, since we have both security and non-security, uh, professionals here, what is the early impact on vulnerability management, Mythos's early impact on vulnerability management, and how do you simply describe it?
Rodeo Thanks. Uh, just a second. Just gonna throw the link into the chat there. Um, so, uh, thank you for, uh, inviting me to come and share some thoughts. It was really fun for me to put together this deck and walk through this topic. Um, it emerged from a four-minute, um, flash talk that I gave at Google last week, um, for all of the Avengers Assemble, right?
So to outline the timeline here for a bit, and then I'll get into the presentation. Um, Gadi, um, from the Cloud Security Alliance, uh, also at Knostic AI, um, did a call to arms, you know, back on, like, April 11th or so, and said, "Hey, Mythos, you know, we're gonna have to prepare people. We're gonna have to maybe put together a document, you know, a brief.
Anyone wanna help?" And so he ended up getting, like, two hundred and fifty CISOs to spend some several hours of their weekend, uh, to contributing to this document, which in and of itself is an amazing demonstration of resilience in the community to have two hundred and fifty CISOs, uh, ranging from, you know, um, the CISO for Akamai, right, to Bruce Schneier, to, you know, um, all sorts of, you know, luminaries, um, in the space, uh, that have come together to talk about this change, right?
And the first reaction from a lot of folks is, "Wow, what a really great marketing stunt, you know, by Anthropic," um, to say that they developed, um, a model that is so badass that they can't share it with us, and they're gonna keep it back, right? And they're gonna create Project Glasswing, and they're gonna give, you know, some major players in the space like Google, Microsoft, Linux Foundation, you know, CrowdStrike, first dibs on using this sort of, um, uncensored model, right?
A model that's capable of chaining together vulnerabilities and identifying ways of exploiting them, you know. And of course, one of the big myths about Mythos is that it was not designed to do hacking. And obviously, there's, like, an anchoring bias that happens, like the first news story that you hear kind of like prejudices everything you think about a topic.
Um, like certain cybersecurity breaches, you know, where they say, "Oh, it was the Chinese," and it turns out it was actually the Russians with a false flag operation, right? But people always just assume, you know, that the particular vulnerability and breach, um, lands with whatever the first story out the gate was.
So everyone was thinking of Mythos as a hacker. But really, Mythos was designed to just to write code. It turned out, though, that it was really good at finding vulnerabilities in code. And it was so good at it that I like to think of it as like a, a homeschooled, um, child that's not allowed to play with other children, right?
Um, Mythos is kept back for that reason. And, um, then of course, uh, the brief came out and, um, I do believe the truth is somewhere in between, you know, the sky is falling and this is a nothing burger, right? Somewhere in between those two extreme reactions is the truth. And everyone I've met and talked to who has access to, you know, Mythos and has done code scanning of their infrastructure and software definitely agrees that it is all that.
It is a step change up in capability and in long tasks and in reasoning about, you know, and inference, um, capabilities. And there's a really great, um, uh, unprompted, uh, YouTube recording of Nicholas C- Taldini from Anthropic, um, a threat intelligence, uh, and threat hunter vulnerability researcher called Black Hat LLMs, which everyone should watch 'cause he's the one that kind of got everyone to agree that this is not just a marketing, um, marketing, uh, campaign for Anthropic.
But, um, for quick reference, I put the link in the chat. Uh, and I think I'll just share the screen for a second since I have the document open, and I can do a fancy, um, embedded video of myself so you can see me talking and the document. Um, so this is what was published, um, on April 18th, and this is version 0.9.5 or so.
Um, uh, and it, uh, has, uh, some really impressive authors, of course, you know, Rob Lee from, uh, SANS Institute, Gadi, of course, who pulled together this sort of Avengers Assemble call to arms, and Rich Mogull, um, at Cloud Security Alliance as well. But a lot of the contributors, of course, are people that you've heard of, like Jen Easterly, Chris Inglis, you know, um, Phil Venables, Bruce Schneier, uh, Katie Masouras, um, really great, um, you know, people that have been doing this work for a long time, uh, Sounil, of course, uh, from, uh, uh, Knostic.
And of course, the reviewers. And so these are all the people that, um, wanted or allowed themselves to be listed as contributors, and there's me with my adjunct professor at New York University, uh, uh, credit. And I'm the one that actually came up with the phrase "mythos ready." So Gadi gave me max props for that.
And the reason I came up with that phrase was that I wanted to empower people to think, "Yeah, we're ready for this. Let's brace for it." But I also wanted to be able to express the thought that there are no mythos-ready organizations, uh, that even Anthropic is not ready for mythos, right? To be pointed at it by threat actors or mythos capable, mythos class type tools.
So we don't have to focus too much on Anthropic's particular frontier model. Um, but anyway, so there's a lot of good information in here if you haven't read this yet. It has a summary, but you can feel free to drop it into your favorite AI and ask it to summarize it for you as well. Um, but this was basically a whole lot of like what, right?
And then the next phase, you know, was a couple of Zoom calls that he held with over two hundred, two hundred and fifty CISOs again. Uh, talking to the deputy CISO at Anthropic, um, Jason, um, uh, I forget his last name, uh, begins with a C. Um, but anyway, he was on a call talking about what it's been like and, um, for him.
And, uh, then we had, like, the, you know, um, head of security at, uh, CrowdStrike talk about their experience. All of this was Chatham House rules, so I can't attribute any particular takeaways to any particular person or company. Um, but I can get to talk, uh, about some of the things that have happened since then, like my meeting last week at Google, where I gave the flash talk that led to this deck and expanding the topic, um, a little bit further on what we've seen so far with autonomous pen testing, right?
And anyway, so this is where you see, you know, the exploitation window, um, uh, for M. What is it? Median time to exploitation from a zero day. Used to be measured in days and weeks, right? A vulnerability would be sent out, people would be chasing it, and you'd have weeks, you know, to go after it. Um, now of course the average m-mean time to exploitation for a zero day is like three hours.
Um, so who can patch production in three hours? Uh, almost no one. Um, so it's gonna be interesting. It's gonna be a bloodbath, but eventually it'll get better, meaning we'll use the same technology that's being used to attack and breach, uh, to defend and to pre-scan our own releases before we publish them, and then auto patch and auto fix if we have enough trust in the tools to do that.
Um, so anyway, lots of good information in this particular, um, uh, PDF, uh, to get you started. Um, so I don't know if may-maybe we just do a quick raise of hands. Um, how many people have read, uh, the brief from the Cloud Security Alliance already? Okay, so not a lot. So I'll, um, I'll start then just by, like I said, providing more context and not assume, uh, that you're already, like, deep into this, uh, idea.
Uh, so there are these frontier models, and they are based on large language models, um, that have been trained, you know, with, um, lots of data. And one of the things that they can do, of course, is scan code, and they can, uh, find vulnerabilities in the code, vulnerabilities that have been lying dormant. So one of the things that Mythos was-- and Anthropic was able to publish initially was that they had found a twenty-year-old vulnerability in the, like, BSD kernel.
And that freaked people out 'cause this is one of the most secure Linux distributions ever, right? And it's been looked at and scanned and pounded against, you know, by threat actors and, you know, security researchers for twenty years or more. And it found something new, and it was fairly trivial in the sense that it just caused a process to crash, right?
So you could potentially launch a denial-of-service attack by causing your computer or server to crash. But that can be chained with other types of vulnerabilities and turned into potentially a remote code execution or into a privilege escalation. These are the two big vulnerabilities that most people worry about, especially if it's an unauthenticated, uh, privilege escalation or remote code execution, meaning you don't have to log in.
You just need access to the service, uh, for something bad to happen. So threat researchers have been doing this for years, and we've never really been, um, uh, short on vulnerabilities, right? If most people look at their vulnerability scanning tools, they're gonna see, depending on the number of assets and, and diversity of operating systems and programming languages, let's just say that a round number, a large enterprise might see sixty thousand vulnerabilities that they have to deal with coming back from a Tenable scan or some of these other legacy, um, tools, Rapid7, you know, uh, Qualys, you know, things like that that look for and tag these And the old method, uh, was simply go by the CVSS score, right?
The severity, uh, from a scale of like 1 to 10, 10 being like really bad, right? Now, of course, it feels like everything is CVSS 11, um, which is a reference to Spinal Tap, right? Turn up the amp, uh, to 11. Um, and of course, that's kind of where we are. We're like in an overdriven, distorted, heavy metal moment of the concert, and the concert right now is what's happening to vulnerability management when AI can just cut through our software, uh, like a hot knife, uh, through butter.
Um, so that's basically the, the tenor of the debate, right? Um, some people denying it, putting their head in the sand saying, "Ah, this is nothing new." Um, and other people saying, "No, this is definitely a big change in the state, you know, of, of our cybersecurity preparedness, um, and our ability to respond." Mike.
Let me jump to the deck. Yeah. Yeah. Yeah. I mean, you, you talk to these Fortune 500 companies more often, right? So what, what are some of the changes or shifts that are happening in these companies to be preparing for any of those? You mentioned that in the future we just have to patch more frequently and automate that.
Are there any companies, products, initiatives to like, you know, the remediation aspect of it that's being done? Yeah, there's a lot of different, um, uh, responses to, uh, dealing with this. Obviously asking, you know, your board for more budget, um, and saying, "Look, you know, we have more work to do, um, than ever before."
Uh, so a typical example of what happens with just one application, let's say Firefox. Um, Firefox was, uh, part of the Glasswing initiative. They were given access to use Mythos to scan their code. And typically, a Firefox update would include 20 to 30 fixes, um, that would come on a monthly cadence or so. Uh, the one that came after Mythos had its chance to turn its eye and to train its, you know, mind, uh, upon the code, it resulted in 271 fixes in Firefox.
Uh, so basically a 10 times, um, you know, 10X, uh, increase. And of course, everyone just simply has to download the new Firefox and they're good, right? But what happens when this happens to, like, Microsoft? Um, 'cause Microsoft had their biggest Patch Tuesday ever this last month. Um, uh, and that's not unrelated to Mythos.
Um, I was actually predicting prior to this particular Patch Tuesday that if you really take the 10X or 20X kind of multiplier to it, that the Patch Tuesday would be like 4,000, you know, vulnerabilities and like 400 zero-days, and you'd need 20 gigabytes of disk space, right, in order to download this patch bundle.
But I think they realize no one has 20 gigabytes disk space free on their Windows servers or their Windows laptops, and so they're just gonna have to, you know, bring it out, you know, successive months, um, and to get to what I think to be is a fairly large number of, of brand-new vulnerabilities that have to be patched and, and, uh, remediated.
Um, but, uh, a lot of companies are, um, basically putting together a tiger team, uh, that's trying to prioritize and figure out where, uh, they need to add the ability to, like, remediate, uh, uh, or mitigate, uh, some of these risks, uh, within several hours. Um, even CISA has come back with guidance, you know, for the governments, um, with a binding directive order saying they're gonna move down from like 14 day internet critical facing, right?
Internet facing critical vulnerabilities are going from like a 14 day remediation guidance to like three days. Now, of course, three days is nowhere near what we're seeing in the r- in the wild, which is like three hours. Um, but at least it's movement in the right direction, right? Government can't move fast.
And for people to be able to patch and then do a full regression test to make sure that the update doesn't bring in new vulnerabilities or break functionality, they may have a regression test that takes 11 to 12 hours or something. So they need to come up with a fast path approach to mitigate, uh, and then eventually remediate once the vulnerabilities can be patched.
Um, but really depending on the size of your organization Some of this speed and velocity and volume, uh, is gonna be overwhelming. And thankfully, uh, there are some AI-based approaches to prioritizing. Uh, so for those that haven't heard of it, there's a, um, uh, an open source framework called EPSS, which is an environmental predictability security score.
So it's kind of like the thing that comes after CVSS and CVS, um, because those particular scoring and severity, um, ratings were never really meant to be, um, used in the way that they were, uh, for the last, you know, I don't know, ten, 15 years. Uh, they were meant to be guidance in general about the severity of the vulnerability, but everyone was supposed to understand their particular environment and impact of it.
So if you have an interfa- internet-facing asset, you know, in your production environment that has a CVSS score of ten, then yeah, you should jump on it and drop everything else and fix it, right? 'Cause it's gonna be exploited, and it's gonna be leading to a security incident or a breach. Uh, but if you have that same vulnerability on a intranet, right, not internet-facing, uh, then maybe a little less, um, you know, um, urgency, uh, because it's not exposed, right?
To threat actors immediately. Uh, so there's that whole environmental element of CVSS scores that was never really utilized to bump up or down the severity for your particular organization. So EPSS is something that Michael Roytman of Kenna Security came up with. And I saw him give a, a talk on it, I think it was EPSS version two or something back in nineteen-- two thousand and nineteen, uh, yeah, two thousand and nineteen, here in New York.
And he, his company was bought by, um, Cisco, and they trained up this on, this framework and this AI model on what is likely to be exploited. So don't just patch a high and a critical because it's high and a critical. Go after something that can be exploitable and that can be reached. 'Cause if it can't be exploited and it can't be reached, you actually probably don't have to patch for it right now, or at least you can deprioritize it against other things that are exploitable and can be reached.
So EPSS version five has just been published, uh, I believe on June fifteenth, and I can put a link in the chat real quick for that. Um, EPSS will be five. And it's, um, like I said, it's open source, um, so everyone can incorporate it into their, um, uh, tools, right? And scanning tools. And if you have existing vulnerability management and scanning software, um, you should definitely check to see whether EPSS is being used, um, and, uh, whether or not it helps, uh, obviously prioritize what to go after.
If you can't patch everything, patch the stuff that matters. So I believe you should be able to see a Venn diagram right now. Is that right? Yeah. So what you see here is the sort of set of CVEs with a seven or higher score. And then you see this red diagram, which is the ones that are actually exploitable and which ones are false positives and which ones are true positives, right?
And so really you just need to patch for this small little set, and this is what they had done, right, with this model. And Anthropic came out with the direct explicit guidance saying, "Use EPSS to prioritize what you're gonna patch." Um, so that's one way to get at it, right? And of course, it's been getting even better in terms of its, um, um, precision.
And the version five, I think, should be listed here eventually at some point. But so the idea is that we really don't have to go after, you know, uh, boiling the ocean, as they say, right? Just go after the critical stuff that's gonna end up turning into a breach event And lots of interesting stuff here about recoverage and recall and efficiency of the model as it's, um, gone from the different versions, um, to where we are now.
Um, but let me get back to the deck real quick and walk through it, and then happy to spend, you know, as much time as everyone, um, uh, is interested in doing Q&A, um, asking specific questions, you know, about your particular, uh, situation and how, how we can, uh, collaborate on, on how to solve, um, some of the challenges that everyone's facing right now.
Uh, so first a little bit about myself. So these are some of my logos. Um, I wrote a book for Cisco Press back in two thousand and two for a certification that doesn't exist anymore. So I got an ISBN in two thousand and two. Some people on the call may not even have been born yet, right? Um, so I'm old as the hills.
I've been abusing computers for over thirty years. I like to say if you don't know three ways of abusing a tool, you don't know how to use it. Um, so I started out as a DevOps engineer working in California. Um, I gave birth to starbucks.com in nineteen ninety-eight. Me and my team launched PlayStation for the PS2.
We also launched Blockbuster and Macy's first e-commerce websites, and that was amazing, right? That was the Roaring Twenties of my generation. And, um, I of course survived five rounds of layoffs, uh, as the last man standing at a company called Organic, um, during the bursting of the bubble of the dotcoms.
And so that's when I decided to move to Europe, and there I built Rabobank's direct banks, ING's DR, put Caitlin behind Akamai. Um, spent a good 11 years in Holland, uh, working on critical infrastructure, things that control power plants, um, including when the Stuxnet came out, we had to scan for that. Uh, moved back to the US in twenty thirteen, ran the Chicago Mercantile Exchange, the CME Group with ten thousand servers and a quadrillion dollars in contracts traded, uh, ran AQR Capital's infrastructure.
If you've ever seen this TV show, Billions, uh, it's exactly like that, right? That's not fiction. That's what it's like to work in a hedge fund. Um, and so that's, uh, a fun experience. I did about a year of their tour of duty, um, uh, supporting, you know, I don't know, something like eighty-four billion in assets under management.
Uh, then I was the CISO for ASCAP, the American Society of Composers, American Society of Composers, Authors, and Publishers, uh, which is great. I got to do music and cybersecurity at the same time, a hundred-year-old organization, one point nine billion revenue, one point- What was it? Like 10% of that was the overhead and 90% was paid back to the membership of people like Mariah Carey, you know, Justin Timberlake, you know, Beyoncé.
So protecting their, uh, revenue for their construct- compositions and, uh, writers', uh, royalties. Uh, then I was the head of security at Marvel for two years, and a lot of people like to hear stories about that. I like to joke it was my job to keep Iron Man safe. Uh, then I started teaching at NYU, then I was the CISO at Security Scorecard, uh, where I know Michael from.
Uh, then I got, uh, nominated to the World Economic Forum, so I'm working on the Quantum Security Working Group in the oil and gas industry, cyber resilience, and written whitepapers and case studies there. Then a healthcare startup, Lark, where I was the head of IT and security, then started teaching at Columbia, then I was the CISO at Major League Soccer, um, and then I resigned last year to what I call a blood type mismatch, um, but certainly excited about the future of soccer in America and all the World Cups happening now.
When America decides it wants to dominate, it will, and FIFA knows it. Um, and then of course, now I'm the, uh, CISO at, uh, Aikeido, which is a Belgian startup, and I can talk about that, um, some other time if you'd like to hear more about what Aikeido does. It does application security and autonomous pen testing, some of which I'll share some insights with you here.
So enough about me, let's get on to the topic. So automated pen testing. Uh, it basically changes the unit economics, right? And in this case, we're talking about speed, we're talking about costs, right? We're talking about coverage. Uh, we're talking about, um, capability. Uh, there's just so much going on, um, that is, uh, of interest, uh, in this, uh, space.
And of course, I wanted to break down this story into three parts. Testing one app and some experience from that, testing three hundred apps or more, uh, and then testing a thousand or more apps, uh, with, um... You don't even need a frontier model. You can use open weights models, as long as you orchestrate with a good harness, uh, the tasks of those sub-agents and agents.
You don't actually need an intelligent model. You don't need a frontier model. You don't need Mythos to do good security. And this was a fact, um, that was also echoed by Anna Westelius, um, at Bsides in, um, San Francisco, uh, this, uh, this year. She gave a talk, and she's head of security for one of the divisions of Netflix.
And she said that, uh, you don't need a fang-sized budget in order to do good cybersecurity in the year twenty twenty-six. And so I think that's one of the points of optimism, uh, the defender's advantage, as you might call it. We get to know all of our code, right, from a white box pen test, and we know how it's wired up from an architecture and workflow point of view.
And we know, you know, hopefully know where all of our endpoints are that are internet-facing. When the threat actors are using these tools, it's a black box pen test. Um, unless of course, it's an open source tool and they're, uh, downloading the open source code and scanning it and then figuring out how to crack it.
But in general, your proprietary apps and things that you've built that you're running that make up your business, they're doing black box, you're doing white box. And so we'll find, you know, I think somewhere on the order of three times the number of vulnerabilities, as you will from testing the same code and from a white box versus black box point of view with these tools in our experience so far.
So finding three times the vulnerabilities that the bad actors and threat actors and bored script kitties are gonna find, um, is a good advantage for defenders. Uh, but it's gonna be, like I said, a bloodbath for the short term, where the threat actors are just moving way faster than the defenders can handle.
But eventually, I think we get to a point where we don't ship exploitable code. Um, that would be a real nice, um, future state, right? A bit of a promised land, uh, nirvana. So let's talk about testing one app. Uh, so I was curious of course about, uh, autonomous pen testing, so I ran a tool against a company that I've, uh, been advising in the past and, and offered to run a, an autonomous pen test, um.
Now, typically, uh, they would pay or I would pay somewhere between seven thousand and fifteen thousand dollars to test an API, and the API would have about eight hundred endpoints and routes, and it would be one IP address. And in this case, it was a black box condition, so I didn't give the pen testing, uh, code, uh, access to the source, which is most real world engagements actually, right?
People do, um, unprivileged testing, meaning they don't have any logins or accounts or access to the code. Like I said, the traditional cost for this would be about seven thousand to fifteen thousand, depending on, uh, the complexity of the endpoints, you know, and how much functionality there is, um, on the site or the API.
And of course, the baseline is like a four to eight week execution window, um, between, you know, signatures and, and getting the report. Uh, of course, uh, when you do this with an autonomous pen test, um, what I came up with was four dollars instead of fifteen thousand. It used three million tokens, and it took eighty minutes.
Uh, so that is a rapid collapse, right? Of a certain type of work that can be delegated and trusted, uh, to be performed by autonomous AI and autonomous pen testing. And it's interesting 'cause it found, what, five critical IDOR flaws which had to do with, um, uh, insecure, uh, object references. And there's not just finding a CVE and, you know, and exploiting it.
Um, this, uh, or a particular vulnerability on a particular package, right? Whether it's the web server software or if it's, um, the database back-end or something for SQL injection attacks. It found five of them. Uh, so that's real-world value, right? And this isn't, like I said, a theoretical projection. This is real live autonomous, you know, engagement.
And what it did, it enumerated the eight hundred endpoints. It focused on two hundred of them, um, by doing some reasoning about which endpoints were interesting to attack, uh, for further exploitation. And it identified eight vulnerabilities that were candidates for exploitation. And then another agent ran to validate and verify, and it found five of the eight were actually vulnerable.
So you always need to have agents checking up on agents. I like to jokingly refer to it as like a, a New Jersey roadworks crew, right? Where there's like one AI doing the work and three AI, AIs watching and supervising the one that's doing the work. And that way you can avoid hallucinations, and you can get a consensus right around what is real vulnerable or not.
Uh, so this was just testing one. And, um, if we look at, uh, testing three hundred, um, I think... Yeah. The, uh, the idea here was that we had a cohort, uh, of over three hundred apps, um, that were vibe coded. Um, so you can expect this is like shooting fish in a barrel, right? Insecure vibe coded apps written by people that don't understand software, but that's the whole point of vibe coding, uh, to democratize it and allow everyone to code, right?
And I have some other thoughts to share on that in a minute, but let me just run through some of the talking points here. Uh, so about three hundred million lines of code, right? So this is white box testing. Ninety percent of the targets came back with actionable findings, um, a-and finished their run in about two hours.
And within the first thirty minutes of testing, the first issues were identified. Uh, and these are also acting on the findings, verifying them. You know, they're not noise, they're not hallucinations, they're not false positives, and they find real issues that previous tests had missed. So the finding here is interesting because When a team, uh, until the teams actually trust agentic coding, right, to come up with a fix, right, whether it's a business logic flaw or an OWASP top 10 like SQL injection or cross-site scripting vulnerability, which are in the top 10 most common mistakes of, uh, code, you know, that's delivered, whether it's by a human or by, um, an agent You need to be able to act on those findings, and you need to retest.
Unfortunately, retesting is largely gated by how long someone decides to trigger the retest, right? And meaning they're not gonna retest five minutes after the test finishes, 'cause you haven't changed anything on production. You haven't patched or verified or mitigated any of the vulnerabilities. So for those that do allow and trust and build up their trust with these tools and capabilities and workflows, um, you can actually have automation in this cohort.
Some of them had, um, commits that were, uh, retesting and validating a fix of the vulnerabilities within twelve hours. So that's not bad. It still doesn't get you all the way to like a Mythos swarm, you know, coming at you and breaching stuff, you know, within three hours. Um, but the idea that you can test your code within two hours and then fix it within twelve, and then you release it to production, right?
So you do all of this on your non-prod instance, right? Um, you make it not internet-facing, but you still allow the, the tools, right, to, to scan it and to attack it. And so that's what I mean by defender advantage, um, is that we know when we're gonna release, and the threat actors just notice that the site's code has changed, right?
Um, or there was a maintenance window published, you know, and the site was unavailable for a few hours as it was being pushed to production. Uh, so basically, I think it was about twenty-five percent of the issues, um, were retested, and about eighty-seven percent, um, had a me-median time to fix of twelve hours.
Half of the pen tests, um, in this cohort returned ten or more critical findings. Uh, that's a lot of vulnerabilities, but that's what you'd expect with like a vibe coded lovable app, right? Um, and of course, cost comparisons are kinda hard to gauge 'cause a lot of these purely vibe coded apps, you know, would never actually be pen tested, right?
People just, you know, YOLO it, right? You only live once. They, they test in production, they push to production without testing security. Um, but if you estimate maybe twenty K per pen test, you know, for a full app, not just an API, uh, that would come out to about just under seven million dollars in value spread across, you know, maybe eight forty-eight weeks of execution time, uh, for these, uh, entities and all these tests.
So that is a lot of value, regardless of whether it's purely agentic pen testing token consumption, or a judicious mix of open-weight free models used for some of the easier tasks, like open-source intelligence. Uh, OSINT is just going out and scanning the headers of servers and seeing if they disclose what version they're running.
And if it's running in an unpatched, um, insecure version of Apache or NGINX or IIS or something, you know, those are some of the easy things that come back. Um, and I'm pretty familiar with that from our, uh, security scorecard days where we were running Nmap scans against all of IPv4. So imagine there's about three point seven billion routable public IP addresses in IPv4 space.
When I joined Scorecard in twenty twenty, a given week of scanning would come back with twenty to thirty billion vulnerabilities, and then we would scan again to figure out who owns the IP address and the assets and whose scorecard does it go on to, to step them down from having an A to some other score, right?
Maybe all the way down to an F. And then two years later, you know, or actually now this would be like six years later, I guess, you know, a, a given week of Nmap scanning by Scorecard, I'm still a big fan of the platform even though I don't work there anymore, uh, would return maybe two hundred billion vulnerabilities or two hundred and thirty billion vulnerabilities.
That's orders of magnitude worse. Exact same number of IP addresses. So coding is getting infinitely sloppy, vulnerabilities, you know, going through the roof, you know, and, uh, it's just trending in the wrong direction. Uh, one of the things here, though, is that it's not just little criticals, meaning an exploitable, you know, like package, you know, that's a part of your supply chain, right?
Which happened with Log4j or happened with other tools. Um, npm packages have been the focus for a lot of threat actors lately, which is a Node.js packaging manager. Um, they attack and inject malicious code into a package that your software and your company uses, and your company automatically updates to the latest version, and suddenly you've been breached, um, by this, um, malicious backdoor that was injected by taking over the developer's account who maintains the package.
This happened last December with Shy Halood and Mini Shy Halood and then Team PCP, if you know those names of threat actors and the vulnerabilities Hundreds of, hundreds of compromised accounts, including GitHub, uh, including OpenAI itself, I think, um, or maybe it was Anthropic actually that got hit. A couple of developers sourced a package that had a backdoor in it, and they had an incident, right?
So it's hitting everyone. Um, but what I was talking about here is the anecdotal evidence is that people were using JWTs, um, JSON web tokens, and, um, they thought they were using them for authentication. Uh, and they had years of prior pen testing by humans, and we ran, um, an autonomous pen test, and it turns out the JWTs were not even being used for authentication.
Uh, JWTs being the way of saying JWT because it's too many syllables to say JWT. Um, and of course, this is a structural, you know, myopic, you know, miss, right, for, um, pen testing, which there's no prejudice and bias when an AI comes and looks at your code. It doesn't think, "Oh, I think this is gonna be the vulnerable area."
It just tests everything, you know, fairly impartially. Uh, and then of course, there's coverage holes. People can't test all the code, read all the code in two weeks, but AI can. Uh, and then there's regression risks, so you have to verify that if what you're trying to put in is the cure isn't worse, you know, than the disease.
Uh, so you have to make sure that you have capabilities of looking at these, um, uh, code fixes, even if they're autonomous, you know, code fixes, you still need to validate that you haven't introduced new vulnerabilities. Uh, so some of the research published, um, by SecurityScanner.Dev, and the link is in my slide notes, and you can have a copy of these slides, including the slide notes.
Um, I have a QR code at the end where you can email me. It's, uh, just a mailto tag, so you know you won't get malware by looking at the QR code. Um, but basically they did unprivileged testing, um, at SecurityScanner.Dev of over three thousand Vibe-coded apps, and there was significant legal exposure, not just technical risk.
Uh, so like three percent of them, uh, had databases where user databo- data... user data was readable by anyone with a public anonymous key. Um, so talk about GDPR fines, right? Having, um, unauthenticated access to your database because you Vibe-coded it on Lovable. Uh, dangerous stuff, right? Definitely fun and exciting for sales teams and others to write their own apps, not have to talk to engineering or security, but that's also the problem.
They don't talk to engineering and security, and so they miss all of the low-hanging fruit on protections and how to avoid some of these risks. Uh, next up, testing a thousand. Um, so this is basically going from apps to, like, an asset graph. Um, here, the next frontier I was saying isn't, you know, that we pen, pen test a package, but coming up with a graph that shows your entire SBOM, right?
Your software bill of materials. So think of the mother of all SBOMs, if you're thinking of it that way, right? Would be like Google's cloud infrastructure And all the packages and all the software that go into every service that's part of Google and GCP. Google is a part of Glasswing, right? They were given access and are using Mythos to scan their infrastructure.
And a couple of their engineers asked me, um, uh, for some advice because they saw my talk at Bsides in San Francisco, where I gave a talk, um, that was called, um, the Epistemology of Trust, uh, Third Party Risk and Resilience, uh, in the Modern Era, I guess is the subtitle. And it's up on YouTube, um, you can listen to it or I can put a link to it in the chat.
Um, and, uh, I talk about resilience and I talk about breach cadence. So we shouldn't be talking about breach likelihood 'cause the mantra for a while has been, it's not a matter of if you'll be breached, right? It's a matter of when or how often. So I evolved my thinking when I was at Scorecard from breach likelihood to breach cadence.
And so that's what I was talking about in this, in this chat, um, in this Bsides talk. And that's what got the attention of the Google engineers, and then they wanted to follow up with me to figure out how to prioritize what they're doing. So they have all these core packages and all these shared dependencies, transitive dependencies.
This package sources another package, right? So if you just wanna have a simple web app with rounded corners on a dialogue box, you're, you're gonna be sourcing like anywhere between, you know, a thousand and three thousand packages, right? Just to build that element of the GUI of the web interface for their app, let alone the business logic function of doing date mathematics and password generation and things like that.
Uh, then of course, reachability analysis is a part of the framework when you're dealing with like thousands of software projects that have to be pen tested, not just a few hundred. And you're talking about, you know, which ones are actually exploitable in production, right? You need reachability. You can't just mindlessly go after every high and critical that comes back from a scan.
Uh, I think I've mentioned that a couple of times now. And then of course, the real challenge is ownership mapping. So once you have this mother of all SBOMs and you have every package and every version of, uh, OpenSSH and OpenSSL, those are the main ones that a lot of people would be worried about, right?
Because they're used for secure remote access, um, and sessions of, on servers. You don't know who owns it. You don't know who to assign the ticket to, right? Even if you created six hundred vulnerability tickets, right? It can be hard because there's embedded, you know, um, packages. Uh, there's glue code that has been created, right?
That is custom to the platform. Uh, you have, um, Libcrypto, you have Libssl, which are part of OpenSSH. Uh, you have Fido, you know, um, integrations for authentication. You have Zlib and Libc, uh, all sorts of underlying packages. Kerberos itself, right, can have vulnerabilities, um, in terms of, uh, Windows-based, um, authentication and tools and tickets.
And, uh, anyway, so the real challenge here is not just the package, it's like the entire graph around your application stack and your SBOM. And, uh, you have sidecars, you have old statically linked copies that won't be updated even if you update the operating system version. And so you have to know which ones are statically linked binaries and which ones are dynamically linked.
That's a lot of work that no one was ever really willing to do before on their asset management from software point of view. So Glasswig is forcing that to happen. And I think some of the historical cleverness, AKA hubris, you know, that a company has, um, uh, is really the, one of the issues at heart here, 'cause it's making robust, comprehensive, and real-time software inventory, uh, a bit of a challenge, right?
A burden for even Google. Uh, so what changes? Uh, when testing is cheap, fast, and continuous, uh, this questions are no longer can we find it? It's like, which ones are reachable and who owns the fix? So the bottom line here is that automated pen testing is awesome. It's lowering the cost, and you should be doing it yourself, um, like starting tomorrow, uh, with different tools, including Aikido has this capability.
Um, we're competing with tools like, um, Expo and others that are doing autonomous pen testing. Uh, but we, I believe, have a, a pretty good lead on learning from all of the scans that we've done already and our approach. Uh, and of course, it's giving everyone leverage to operate the speed of machines, right?
Machine level. So really the critical risk map for testing a thousand is dominated, you know, by looking for duplicate and divergent copies of core, you know, components, um, especially services, internal tools, third-party integrations, um, containers, customer facing control plane. Um, Anthropic, I said, you know, I think shared one of their open source scans found twenty-three thousand vulnerabilities, including six thousand that are estimated to be high or critical, which is about twenty-six percent.
So that means the real question is who owns the fix, right? Not just can we fix it, can we find it? And then of course, what are your P0s, right? Your priority zero fixes. What are your priority ones? Uh, a lot of people decide P0 is internet-facing and P1 would be non-internet facing, um, that would be reachable through other types of gateways, VPNs and things.
But at some point, we have to admit that there is only one network. It's called the internet, and there is no inside network anymore, right? Threat actors are able to compromise an internal developer machine. They're on the inside, right? So there's no longer, um, perimeter-based defense like we had back when I launched Starbucks in nineteen ninety-eight, right?
Where you had a firewall and you believed that firewall was able to keep things out. Now you have to assume breach. You have to assume that the threat actors are already buying and logging in, you know, with credentials on the dark web into your infrastructure. Uh, so lastly, here's the QR code if you'd like to get a copy of the deck.
Um, and my email, [email protected], and happy to, uh, open it up to questions. Thanks, Mike. That was certainly a, a really nice deep dive into the topic. I have more questions than, uh, than answers, but I'll save that for later. We do have questions, uh, from the audience. I think we'll start with you, David. Uh, looks like you had a, a general opening question on the CISO, so please go ahead.
Yeah, Mike, hey, great presentation and great background. Thank you for sharing with the community. So question, was... I love the fact that you coined the phrase, uh, meet those ready. Um, so as a CISO, what are the top three questions are you typically asked as far as being ready or being compliant as, as Manil says?
Well, one of the nice things, if you can think of it this way, is that agentic attacks and Mythos are not really doing, um, something that we can't defend against, where we have to invent a brand-new technique of defense. Um, what I mean by this is that we sometimes, in the cybersecurity industry, we believe that this is like Olympic diving, right?
You get more points, you know, for the zero day, right? That you're going after. And we discard some of the basic hygiene around DNS, multi-factor auth, single sign-on, you know, patching, things like that. So one of the things, and this was an Alex Stamos quote that I'm referring to, it's like, "Cybersecurity is not like Olympic diving.
You don't get more points for the difficulty of the thing you're trying to fix." And so we need to get back to the basics. We need to make sure we do proper DNS, right? And have, you know, DMARC, um, SPF and DKIM records to stop people from being able to spoof our emails and say, "These are the declarative email servers I use.
If the email doesn't come from them, drop it, reject it." So if you can set your DMARC policy to reject, and you can worry or not worry that your bulk emails are now gonna be rejected 'cause no one told you about a HubSpot account they set up for something and started sending bulk emails on your behalf.
Um, the basics, right? API security. The OWASP Top 10 for API security is almost a Venn diagram overlap for the API Top 10... Or sorry, for the A- AI Top 10. Rate limiting, authentication, right? Um, all sorts of controls around the basics, 'cause a lot of people are consuming, you know, AI now through APIs. And so if we do the basics well, if we do API security well, if we do DNS, if we do patching, if we do single factor or multi-factor auth, single sign-on, those are the things that are gonna get us, like, the 80/20 rule, right?
So 20% of your configurations account for 80% of your risk. If you can clean up that 20%, and it's not the sexy, shiny stuff, right? It's not the shiny thing. Like, if you know the TV show Red Dwarf, right? With the cat, you know? Ooh, shiny thing. You know, I'm gonna go chase the laser. Um, we don't wanna be like that.
We wanna focus on the foundations and shore up the foundations. So really this is, in my mind, it's like putting the paddles on AppSec and charging to 100 and saying, "Clear." Boom. And now suddenly AppSec is like, "Okay, I'm awake. What do you need me to do?" And, you know, Mythos is that thing, right? That gave us this sort of shock to the system that said, "Okay, we gotta get back to doing vulnerability management and doing it well, and automating as much of it as we can trust ourselves and our tools to do."
Uh, so the questions people are asking, do we even know where we're using AI? Do we even know where we're spinning up ephemeral assets, right? So good asset management, like what software do you have, what IP addresses are behind, you know, um, what servers, you know, and, and have they been updated? That's the core.
You can't have observability until you have a robust, complete asset management. And oftentimes, you know, if you're running in the cloud, you have, you know, elastic compute. You have these June bug VMs that live for less than a day, right? They get lit up, they get locked down. They might have been vulnerable their whole life, right?
And that life might have been four hours when it was serving a, a, you know, a spike in traffic. So you really need to get infrastructure as code as a part of your discipline as well. You can't just go onto Amazon or log into a VM and run apt get update or yum upgrade or something, right? That is just not the way to do it, right?
You need automated, um, approach to software management. You don't even necessarily patch and upgrade servers. You may just kill them and deploy a new one from a repo, and the new one will have all the right packages in its manifest and all the packaged, you know, patched versions. So that's actually one of the things that, I guess, differentiates, you know, legacy old school approach, where you take backups and you restore from a backup.
Infrastructure as code is like, no, we don't back up our servers. We just deploy fresh, right? And we have a ephemeral, um, you know, uh, application and compute, and then a persistent storage, right? So that you can just mount the storage and then the servers back up. You need that, right? To have like four to four hundred servers ballooning, you know, in a cluster, you know, for your services in the cloud.
Uh, so I think those are some of the, some of the fundamentals, um, that we can focus on. And that's good 'cause y-you don't have to, like, learn anything new. You just have to get good at the thing that we always should have been doing, right? That's awesome. Thank you. All right, everybody. Let's take a... Yeah, let's take a group photo if, if you would.
Um, everyone, thank you so much, David. If you wanna continue the conversation, record it. We have a hard stop at nine o'clock our time, Pacific. Uh, let's turn on the camera, group photo, and then, uh, David can continue, and then we'll have Manil. You had a bunch of comments. Uh, would love to, to spotlight on you and Greg, and then we'll finish.
And, uh, hopefully, we'll have our next, uh, session announced in a, in a week or so. So is everybody's camera on? Greg, can you do the honors, take a group photo? Absolutely. Give me one second here. Manil. Here we go. Ready. David Kennedy. Ready. One, two, three. Richard. All right. Do you wanna do one more, Omid? One more, and then let's see if everybody is, uh, if you are joining.
Okay. All right. Ready. One, two, three. Leon Hester, you joined. Thank you. All right. Awesome. So David, you were commenting? No, I was just saying thank you. That was, that was awesome. I think we have another question in the chat. Manu, yep. I had a question for Mike. Is it okay to ask? Please. Hey, Mike. Thank you for the presentation.
Um, as a security practitioner, uh, we continue to see two things, right? A lot of frameworks coming up. For example, MITRE has extended it to MITRE Atlas, and then you also spoke about EPSS. Where do you see the convergence of these frameworks? Because ultimately you can't keep on creating multiple scoring methodologies, and this comes up with its own standards.
What's your thought process in terms of, you know, what scoring methodologies, uh, and specifically in the AI era, how do you look at the scoring methodologies? Question. Yeah, good question. The, um- The popular perspective from the CISOs at Google on Monday, uh, was that CVSS and scoring is kind of, um, dead in the water.
Um, you have to patch every vulnerability that you believe is reachable and exploitable. Why? Because these new reasoning, uh, attacking AI based like Mythos and Fable five and others, um, hopefully Fable five gets to see the light of day again. Um, the, uh, they're good at chaining two mediums together, right?
And turning it into a critical. So scoring is kind of like irrelevant at this point. Um, we need to be able to do a census, you know, of the vulnerabilities and not a sample, and that was one of the challenges. Obviously, you wanna sample and do the critical exploitable first, but you can't stop there. You have to eventually hit, you know, the CVSS sevens and belows.
And of course, those scores don't matter anymore. NVD is kinda dead and unfunded. Um, you know, the CISA Kev isn't gonna be, you know, kept up to date, um, due to, you know, a shift in priorities and funding and a desire, I guess, for some type of commercial entity to step in where the government was providing that capability.
Um, Europe is setting up its own version of the National Vulnerability Database. Um, so really you're gonna have to multi source these threat indicators from vendors talking about, "Hey, we have this thing." It doesn't have a CVE 'cause we submitted it, but it's never gonna be categorized and it's never gonna be scored.
Um, so really we're stepping away from scoring, um, as an approach, and it's just continuous vulnerability measurement. Patch Tuesday is an anachronism already, right? Um, because it's just like e- every day is Tuesday at this point, right? Um, so the challenge is just gonna be able to mitigate risk quickly in an automated fashion with confidence, and that means, you know, um, CICD pipelines with tools and scanning built in.
And the sooner you can find the vulnerabilities, right? Shift left, you've heard this, right? Yeah. Um, in lower environments, sandbox, even before it's even a PR, right? And even before it's in the repo, you know, you need something that integrates with your IDE. So as soon as the developer starts typing password equals, and then they put in like a hard-coded secret, something should flag, right?
And like keto has, um, a plugin for the various IDEs as well. Uh, 'cause we need to stop. We can't just play better Whac-A-Mole, right? We need to actually stop introducing vulnerabilities in code, right? In business logic clause. And that was the other thing I wanted to mention. 20% of everything that we have to fix is actually CVEs and vulnerabilities.
80% is misconfigurations and stupid business logic and other types of things. So really w- all this focus on like finding new vulnerabilities, we were never supply constrained. The fact that Mythos can add 20,000 vulnerabilities to the world of known vulnerabilities doesn't mean that there's 20,000 buyers out there for each of those vulnerabilities, right?
And so there's still an economics on the buyer side, uh, for threat actors. Why would they wanna spend $20,000 for a vulnerability that just causes a process to crash? It's not really worth that much money. And so that's part of the reason why we don't necessarily have to deal with, you know, the whole onslaught and tidal wave, uh, is because it's all about exploitability and reachability and what tools do you have to do that.
So the frameworks are gonna shift away from scoring based. Um, even, uh, FedRAMP, right? Has picked this up, um, with, uh, Pete Waterman in FedRAMP 20X. They came out with RFC 0012. RFC 0012 talks about stepping away from CVSS based scoring and talking about risk and reachability. And that's one of the reasons why I love, uh, the fact that Akeeva is doing this, right?
We're helping prioritize all the vulnerabilities that come back from the scans using AI to figure out which ones might be exploited and which ones you should patch now, and potentially click the auto fix and just let it fix it for you, right? And then run some tests, of course. Uh, that's gonna really raise the bar for QA and testing at this point.
Um, because of the volume and velocity of fixes, you just won't be able to have human QA teams, you know, run a twelve-hour or twenty-four-hour regression test on your banking app, right? Got it. No, uh, thank you, this helps. I think, uh, it'll be important that all the security vendors really align to this because we still have, uh, various, uh, DSAT, uh, DSA, uh, dynamic testing tools available that still look at scoring rather than looking at the rest.
So as an industry, we need to align on this because we're seeing mixed messages right now. No, that's true. Yeah, regulatory is generally a trailing indicator, right? But when you start to see even like CISA and the federal government pick up on this and say, "Hey, 14 days to patch criticals in prod isn't good."
Um, so, you know, uh, yeah, tr- I, I do believe that there will be a consolidation of, uh, security vendors, you know. A lot of, um, companies are features masquerading as a company, and they will be absorbed and absolved, you know, of their entity as a company and just turned into a feature on someone else's, um, app.
Absolutely. Great. Thank you. Thank you, Abhay. Uh, Manil, did you have questions or comments? So only question what Abhay was say, uh, uh, was saying and Mike was answering, uh, uh, Mike, uh, we spoke about, uh, you know, challenges while introducing the code. I loved your fact about shift left, uh, and, uh, uh, catching some of these, uh, uh, challenges that, uh, is introduced at a code level.
Uh, how about XSS, uh, those type of, uh, cross-site scripting vulnerabilities in production, Mike? How do you kind of, uh, track those, uh, in a dynamic fashion, if you will? Yeah. Well, there's definitely, um, capabilities in the CI/CD pipeline to stop that before it hits even, you know, a UAT or a QA environment.
Mm-hmm. Um, because, uh, if you... But if you don't, and this is the problem with a lot of startups, they have production and then they have like one- Yeah ... non-production instance that's not really similar- Yeah ... to production. So they're constantly testing in prod. I see, yes. If you're working at the market- So you need to have a production-like environment.
Correct. And it could be ephemeral, right? So if you're using infrastructure as code, you should be able to, like, treat it like, uh, you've heard the DevOps manifesto, you should treat your infrastructure like cattle and not like pets. Meaning this is my Snowflake server and I built it very carefully and it does all its work properly.
That is a risk. Um, and if you're a vegetarian, I apologize to the cattle reference, but you need to be able to slaughter your infrastructure and just redeploy, and that is infrastructure as code. That is what is gonna be demanded a lo- of a lot of people. So if you don't have a representative non-prod which has caching, which has load balancing, it may not be like a, a T, you know, um, it can be a T3 micro, right?
It just needs to be a logical representation of what's happening in prod. And if you don't reproduce those- Well, in that case too, Mike, uh, uh, the... you just cannot truly represent, uh, uh, production like traffic on, uh- On any non-production environment, right? Um, no, that is actually a, a technique that we used at the Chicago Mercantile Exchange.
So when we did our QA and load testing, we would replay one week of logs- Ah. -on the non-prod UAT instance in order to stress test it and load test it, 'cause we'd load one week of data into, like, 24 hours, and that way we had capacity, right? We had the ability to spike- Yes ... in case there was a run on the market.
So as you thought. Okay. Um, so rerun your logs. That's true. Um, and there are tools, and AI can generate synthetic requests, you know, um, almost as good as your actual production requests. But yeah, rerun production requests and traffic against non-prod. Um, that's one way to s- to scale and, and stress test.
Yeah, but fantastic point about, you know, we're completely shifting in the security world from, you know, a, a snapshot based to real time base. So- Mm-hmm. Uh, and that's the fundamental challenge. Now, I wanted to kind of pitch to the audience to take a look at my product called Tomoso.ai. Uh, and, uh, vulnerability is only one challenge that you can look at, and, um, well, I won't pitch.
I won't use this audience to pitch, pitch, but yeah. So Muniyal, yeah, join the community- Great. Thanks for the opportunity ... and you can obviously share about your point of view. Sharit, do you wanna close it for us? Again, thank you so much, Mike. Hopefully we can invite you again, but insightful conversation. I would love to...
This space is changing so fast that we think that we have to have another session. Sharit, please. Yeah, I mean, thank you so much again. Uh, I'll just summarize the, the discussion into our debrief, and I'll share it with the audience. But yeah, really good takeaways. I took some really good notes. Uh, so thanks for your time, Mike.
Appreciate it. Yeah, and happy to share the slides with anyone, uh, that wants to reach out directly with the QR code or email me, and I'll send you the copy. Excellent. Thank you. Thanks very much. Everyone, have a wonderful rest of the day. Cheers. Bye.

