Bipko Digital News & Media Platform

collapse
Home / Daily News Analysis / Arcjet CEO: AI finds bugs, but it doesn’t prove you've fixed them

Arcjet CEO: AI finds bugs, but it doesn’t prove you've fixed them

Aug 03, 2026  Twila Rosenbaum  2 views
Arcjet CEO: AI finds bugs, but it doesn’t prove you've fixed them

Application security has long treated vulnerability detection as the most difficult part of protecting software. For decades, security teams invested heavily in scanners, static analysis tools, and penetration tests to uncover flaws before attackers could exploit them. The underlying assumption was simple: if you could find the vulnerability, you could fix it. The harder problem was finding it. AI coding agents are changing that equation, and not in the way many security professionals expected.

AI coding agents can now review an entire codebase in minutes. They can trace data flows, spot suspicious patterns, and generate plausible vulnerability reports at a speed no human team can match. Every improvement in detection lowers the cost of finding bugs, but it does not lower the cost of fixing them. Experienced engineers remain scarce, and their time is finite. As a result, the value of a single vulnerability finding is dropping quickly. Security teams are beginning to drown in outputs that require human judgment, validation, and remediation.

Detection becomes the starting point

David Mytton, CEO and founder of Arcjet, has been watching this shift closely. Arcjet is known for its developer-focused security tools, offering web application protection, automated bot detection, custom rate limiting, and sensitive data detection built directly into modern application code. Mytton argues that AI is turning vulnerability detection into a commodity, while remediation becomes the real bottleneck.

“AI coding agents will flood teams with plausible findings,” said Mytton. “Some findings will be legitimate. Other findings will be duplicates, technically correct but not exploitable, or only relevant within a workflow that the scanner cannot see. Detection is becoming the starting point of AppSec rather than the endpoint. Finding the vulnerability, however, is only the beginning.”

That is a significant mental shift for an industry that has spent years treating detection as the crown jewel. Many security programs are measured by the number of vulnerabilities discovered, the speed of discovery, or the coverage of scanning tools. Yet if AI can generate thousands of findings per day, the number of findings becomes meaningless. What matters is whether any of them actually get fixed in a way that reduces risk.

A vulnerability is not just a line of code

One of the most common misconceptions in application security is that a vulnerability lives in a single line of code. In reality, a vulnerability usually describes how an application behaves, not just what one function does. A vulnerable line of code is frequently only where the risk becomes visible. The actual problem may be the sequence of steps preceding or following that line: user input, authorization, data access, service calls, business logic, and the action that follows.

For example, a missing input validation check might be flagged in one function, but the deeper issue could be that the application does not enforce authorization earlier in the workflow. An AI scanner might identify the missing check, but it may not understand that the same input path is also accessible through an API, a background job, or an internal tool. Without that context, a patch that adds validation only at one point may leave the underlying risk intact.

Mytton emphasizes that remediation requires understanding why a sequence of steps created the risk in the first place. Teams that skip that analysis frequently fix the symptom while leaving the underlying behavior unchanged. They close the ticket, but the exploit path may still exist through another route. This is why creating a patch and selecting the correct fix are two separate issues.

Patch without match is a mismatch

AI can generate patches quickly, but it cannot decide which fix is correct for a given business and engineering context. Mytton points out that AI can suggest multiple possible fixes for the same finding. A patch that satisfies a scanner's rules can still be the wrong fix. It might shift the bug to another location, conceal the symptom, disrupt the intended workflow, or eliminate behavior that customers rely on.

Consider a rate limiting issue in an API. A scanner might suggest adding a generic rate limit to a specific endpoint. But the correct fix could involve distinguishing between authenticated and unauthenticated users, adjusting limits based on business logic, or adding a fallback for legitimate traffic that happens to share an IP address. A naive patch could block a critical integration or allow attackers to bypass the limit by changing a header.

Remediation is fundamentally an engineering judgment regarding risk, not an exercise in code generation. It requires understanding the application's purpose, the attacker's incentives, the user's expectations, and the operational constraints. AI can assist in drafting potential patches, but the decision about which patch to deploy is a human responsibility.

Verification requires correlation

Choosing the correct fix does not demonstrate that the work is complete. A fix is not confirmed just because a ticket was closed. A fix is confirmed when the exploit path no longer exists, the intended workflow functions normally, and the deployed application behaves differently in the manner anticipated by the team. This requires correlating the original finding with the application workflow, the code modification, the test results, the deployment, and runtime evidence.

Without that chain of events, AppSec continues to trust documentation rather than verifiable reality. Mytton says verification also extends beyond a single finding. AI-generated findings will require correlation among findings. Ten low-severity issues may all describe the same exploit path. One “fixed” issue may re-emerge through a different avenue. An input path that was denied may still be accessible through another endpoint, a background job, or an internal tool.

“The next AppSec capability is not simply deduplication,” said Mytton. “It is identifying which findings, fixes, and workflows are related. That broader perspective ultimately alters how AppSec teams measure success. AppSec is shifting from find and file to fix and verify.”

This is a crucial distinction. Deduplication removes duplicate reports, but correlation goes further: it connects different findings that share a common root cause or exploit primitive. For example, a SQL injection vulnerability might appear in three different endpoints, but all three might be fixed by parameterizing a single shared query function. Treating them as separate fixes would waste engineering time and leave the root cause untouched.

Correlation also helps prioritize remediation. Instead of fixing findings in the order they were reported, teams can group them by exploit path, affected data, and business impact. This approach reduces the chance that a “fixed” issue resurfaces through a related vector.

AI lowers detection cost but not remediation cost

Mytton predicts that AI will continue to lower the cost of detection because each new coding agent improves at analyzing code and producing findings. But experienced human engineering time remains limited. Organizations therefore benefit more from enhancing remediation quality than from generating more findings.

“Organizations that evaluate success based upon the quantity of AI-generated findings will establish longer remediation queues,” said Mytton. “Organizations that consistently confirm fixes will reduce exposure because they realize that the exploit path no longer exists.”

The actual AppSec workflow is straightforward: detect the issue, comprehend the path, select the fix, deploy securely, and verify that the behavior changed in production. AI can accelerate the first step, but the remaining steps require human expertise, collaboration, and validation. The teams that master this full workflow will be the ones that actually reduce security risk.

Bot protection as a development concern

The conversation with Mytton also covered bot protection, a core focus at Arcjet. He argues that every software engineer should treat bot protection as a core development concern, not a security team afterthought. Bots do not attack “security” in the abstract; they attack product features such as signup forms, checkout flows, login pages, password resets, and expensive AI actions. A bot is not just bad traffic. It is a product workflow being automated against the business.

A security team can set policy and provide guardrails, but it rarely has the route-level context needed to make the best decision. Is the user particularly important? Does the action send a message, spend credits, reserve inventory, or trigger an LLM call? These are engineering and product questions as much as security questions. Embedding security policy directly into code gives developers the context they need.

When security policy lives in code, developers can look at the route and see the actual policy: who is allowed, what signals matter, what limit applies, and what happens when the request is denied. That changes the development workflow. Security becomes part of the implementation, not a box checked later. The same pull request that adds a route can include the rules that protect it. Reviewers can then ask normal engineering questions: Is this strict enough? Is it too strict? Does it fail safely? Does it explain the decision? Does it match the business logic?

AI agents demand context-aware defenses

AI agents are becoming more capable of mimicking real users. They can load pages, run JavaScript, store cookies, click buttons, fill forms, and follow multi-step flows. This makes traditional bot detection less reliable. The old question was “Is this a bot?” The more relevant question now is “Is this action acceptable in this context?”

Developers should build defensively by assuming that automation will increasingly look like a real browser operated by a real user. This means enforcing identity, intent, and business logic at the application layer. Network-level security tools cannot see identity, route context, or business logic. They only see IP addresses and headers. The application, on the other hand, can see whether the user is authenticated, whether the account is new, whether the action is expensive, whether the email address matters, and whether the same behavior is normal for one API but suspicious for another.

For example, a network tool might block a request from a certain IP range, but it cannot know that the request is a legitimate customer trying to reset a password. The application can combine signals: the user has a valid session, the email address matches a recent account, and the request is within a reasonable limit. This context allows more precise decisions.

Shipping protection in the same pull request

Mytton also emphasized that bot protection should ship in the same pull request as the feature itself. If a developer adds a signup form, the pull request should also include the rate limit, bot rule, email validation, and denial behavior. If they add an AI endpoint, the pull request should include the controls for who can call it, how often, and what happens when automation shows up.

This approach makes security review much more practical. Instead of asking whether a feature is protected somewhere, the reviewer can inspect the feature and the protection together. Security becomes a normal part of engineering discussion rather than an external approval step. It also prevents the common situation where a feature is deployed with no protection and security teams are forced to respond to an incident later.

The broader lesson from the Arcjet CEO is that AI is not a magic bullet for application security. It is a powerful tool for finding bugs, but it does not tell you whether the bug has been truly fixed. Detection is becoming cheaper and faster, but remediation still depends on human understanding, judgment, and verification. AI can detect bugs. It still cannot prove you fixed them.


Source: Computerweekly News


Share:

Your experience on this site will be improved by allowing cookies Cookie Policy