The AI Risk We Keep Finding Is the App You Built

The biggest AI risk we see at clients right now comes from inside the building: apps, portals, chatbots, and internal tools that the client’s own staff built with AI, which quietly became something the business depends on. Most of them were never reviewed, tested, inventoried, or assigned an owner. By the time they reach us, they’re holding customer data, touching production systems, and carrying regulatory weight nobody planned for.

What are the real security risks of vibe coding?

Vibe coding security risks have less to do with any single bug than with what gets skipped. AI tools let anyone produce a working app in an afternoon. What they don’t produce is the software development lifecycle (SDLC) around it: code review, dependency checks, secrets handling, logging, access control design, security testing, and a named owner. Veracode’s 2026 research found AI-generated code passes security tests only 56% of the time, while it compiles nearly every time. So the app works, ships, and spreads. The flaws ride along, in a system nobody in security knows exists. Then a framework RCE or a poisoned package lands, and there’s no inventory to tell you whether you’re exposed.

Everyone can build now, so everyone does

Ten years ago, a department that wanted a client portal filed a ticket with IT and waited. Today someone on that team opens Cursor, Lovable, Replit, or a ChatGPT window and has a working prototype by Friday. That’s a real productivity gain, and we use these tools ourselves.

The numbers show how fast this moved. GitHub’s Octoverse 2025 counted more than 36 million new developers on the platform in a single year, and nearly 80% of them used Copilot in their first week. The 2025 Stack Overflow survey found 84% of developers use or plan to use AI tools, yet 46% don’t trust the accuracy of what those tools produce.

Plenty of the people building today don’t call themselves developers at all. They’re in operations, marketing, finance, and innovation teams. They know the business problem well. What they don’t have is any reason to ask about authorization models, dependency pinning, or audit logging, because nobody ever made those their job.

IBM’s 2025 Cost of a Data Breach report shows where that leads. One in five organizations reported a breach due to shadow AI. Of the organizations that had an AI model or application breached, 97% had no AI access controls in place. And 63% of breached organizations either had no AI governance policy or were still writing one.

Four apps that showed up after they were already critical

These are recent engagements, anonymized. None of these clients was careless. Each one built something useful, and the usefulness is exactly why it grew past its guardrails before anyone looked.

The financial services client portal

A financial services firm built its client portal almost entirely with AI. It worked, clients used it, and it held exactly the information you’d expect: client identities, account details, financial records. When they brought us in, the portal had never been through a security evaluation. There was no DevOps or DevSecOps pipeline, so there were no build-time checks, no dependency scanning, and no review step between “it works on my machine” and production. There was no central logging, so if someone had already abused the portal, there was no record to tell us.

That last gap matters more than any single bug. You can patch a vulnerability once you find it. You can’t reconstruct activity that was never logged.

The firm also reports to the SEC, and that’s where the missing logs turn into a regulatory problem. The SEC’s amended Regulation S-P, now in force for broker-dealers, investment advisers, investment companies, and transfer agents of every size, requires a response program “reasonably designed to detect, respond to, and recover from unauthorized access to or use of customer information” (17 CFR 248.30(a)(3)). That program has to assess the scope of an incident and identify which systems and types of customer information may have been accessed. Affected individuals get notice within 30 days of the firm becoming aware.

Then comes the clause that matters here. If the firm can’t identify which specific individuals’ sensitive information was accessed, it must notify every individual whose sensitive information resides in the affected system (248.30(a)(4)(ii)). A portal with no logs can’t show which client records were touched. So one suspected incident becomes a notice to every client in the portal. An AI-built portal doesn’t get an exemption because a model wrote it. Our SEC cybersecurity rules guide covers the full requirements, and the financial services page covers the rest of the sector’s obligations.

The AI chat assistant built from tutorials

Another client decided its web portal needed an AI chat assistant. A non-technical internal team built it by following online tutorials, wiring the OpenAI API in Azure into the portal. It worked. Security wasn’t part of the conversation at any point in development, because nobody on the team knew which questions to ask.

The questions we start with on a build like this:

• Where does the API key live, and can a browser see it?
• Does the assistant check who the user is before it answers, or does it trust whatever the front end sends?
• What portal data can the model read, and can one customer’s question pull back another customer’s records?
• Can a user rewrite the assistant’s instructions with a well-crafted message?
• Is anything rate-limiting the endpoint, or can someone run up your Azure bill overnight?

None of those questions comes up in a “build a chatbot in 20 minutes” tutorial. All of them come up in the OWASP Top 10 for LLM Applications.

The plumbing company’s chatbot

A local plumbing company wanted an easy chat bot for its website. They wired OpenClaw, an open-source AI agent framework, to the chat form. The same agent also ran internal operations: it managed appointments and had access to customer account records.

That design makes the public chat form a front door to everything the agent can reach. The bot can’t tell a customer asking about Tuesday’s appointment from someone asking it, politely, to list every account it can see. Whatever data the agent can read, a determined visitor can try to talk it into revealing. OpenClaw has also had a rough year on its own: in February, SecurityScorecard counted 40,214 exposed OpenClaw instances on the public internet.

This one matters because it’s a small business. Vibe coding risk isn’t confined to enterprises with innovation budgets. The tools are cheap and easy enough that a ten-person company can give an AI agent the keys to its customer records in an afternoon. This is the risk we test for in AI agent security testing: what the agent can reach, and what a stranger can make it do.

The healthcare team that tunneled through its own guardrails

The last client is the one people expect least. It’s a heavily regulated healthcare organization with a controlled production environment that faces intense scrutiny. Its core development practice is mature, with real standards and real review.

Then there’s the internal innovation team. They build tools with AI that reach into that controlled environment, manipulate it, and pull data out of it, with no formal development standards. To make the tools work, they needed connectivity, and they got it. The separation between the governed engineering organization and the experimental team broke down figuratively and literally, with network paths opened through the boundary that was supposed to keep the environment controlled.

When we raised it, the team told us they were covered: they reviewed their code with AI. That’s a model checking another model’s work. AI review catches real bugs, and we’d rather a team use it than nothing. But a reviewer that shares the author’s blind spots is a weak second opinion, and this team’s most serious problem sat outside the code entirely. It was where the tools were allowed to connect. No code review, human or AI, reads a firewall rule that someone opened to make a tool work.

Mature process didn’t protect them, because the tools never entered the process. They went around it. Every control the auditors examine still exists. It just isn’t in the path these tools take. For a healthcare organization under HIPAA, that’s the kind of gap that surfaces in an incident review, not an audit.

The supply chain moves faster than your inventory

Even well-written code sits on a stack of other people’s code, and AI-built apps tend to pull in more of it with less scrutiny. The last year made the cost of that plain.

• React2Shell. On December 3, 2025, React disclosed CVE-2025-55182, a pre-authentication remote code execution flaw in React Server Components rated CVSS 10.0. Wiz showed that a standard Next.js app created with create-next-app and built for production was exploitable with no code changes by the developer. AWS observed China-nexus groups exploiting it within hours of disclosure, and CISA added it to the Known Exploited Vulnerabilities catalog two days later.
• PHP2Shell. On October 21, 2026, PHP shipped 8.5.12, 8.4.27, 8.3.36, and 8.2.35 to fix two critical flaws, CVE-2026-109618 and CVE-2026-109619, that allow unauthenticated remote code execution. Details leaked the day before the planned release, and watchTowr reports exploitation in the wild since October 21. PHP 8.1 and older no longer receive updates from the PHP project.
• Self-spreading npm worms. In September 2025, CISA warned that the Shai-Hulud worm had compromised over 500 npm packages, stealing GitHub tokens and cloud keys and republishing itself. In August 2026, Elastic documented a variant that hit over 400 packages with more than 1.3 billion monthly downloads combined.
• Attacks aimed at the AI tools themselves. That same 2026 campaign specifically harvested credentials for Anthropic, OpenAI, Cursor, Codex, and Gemini, and planted a hook in Claude Code’s settings so the infection re-ran whenever a developer opened the repository. A year earlier, the s1ngularity attack on Nx turned installed AI command-line tools against their own users to hunt for secrets.
• Packages that don’t exist. AI models invent dependency names. USENIX Security 2025 research found at least 5.2% of packages suggested by commercial models and 21.7% from open-source models were hallucinated. Attackers register those names and wait. The technique is called slopsquatting.

Here’s the uncomfortable part. When React2Shell dropped, the teams that patched fastest knew where their React apps were. When PHP2Shell dropped in October 2026, the same was true. If a department spun up a PHP intake form or a Next.js portal with an AI builder last spring and never told anyone, it isn’t on the patch list. Nobody is going to patch an app they don’t know exists.

No guardrails, no standards, no visibility, no accountability

Line up the four client stories and the pattern repeats:

1. No guardrails. Nothing stood between the builder and production. No review, no pipeline checks, no approval.
2. No standards. The tools were built however the model suggested that day, so every app is a one-off with its own auth logic, its own secrets handling, and its own dependencies.
3. No visibility. Security didn’t know the apps existed, and there was no logging to show what the apps were doing.
4. No accountability. When we asked who owns it, the answer was the person who built it, who had since moved on to building the next thing.

Speed is fine. What hurts is that the speed of building has outrun every process that used to slow software down long enough for someone to ask whether it was safe. We made the same argument about governance in Your AI Governance Policy Won’t Stop a Prompt Injection: a policy describes what should happen, and testing shows what does.

What to do about it

You aren’t going to stop people from building with AI, and you shouldn’t try. The goal is to make sure what they build ends up inside your security program instead of around it.

Build the inventory first. Follow the money and the access. Pull AI and hosting subscriptions from expense reports, OAuth grants and API keys from your identity provider and cloud consoles, and unfamiliar subdomains from DNS and certificate transparency logs. Then ask every department head what their team built or connected to AI this year. Each answer gets an owner, a data classification, and a test date. Our AI Security Assessment starts with exactly this discovery.

Set a minimum SDLC bar, and keep it short. A one-page standard that people actually follow beats a 40-page policy they route around. At minimum: code lives in a managed repository, dependencies are pinned and scanned, secrets never touch the code (see why deleting them doesn’t work), the app logs authentication and data access centrally, and nothing touches customer or regulated data before a security review. If you need help writing and enforcing that standard, it’s core security program development work.

Give innovation teams a paved road. The healthcare case happened because the sanctioned path was too slow for the innovation team, so they built their own. Give builders a pre-approved sandbox, a starter template with logging and auth already wired in, and a fast review lane. People take the easy path. Make the safe one the easy one.

Test before it becomes critical, not after. The best time to test an AI-built app is right before it touches real data. The second-best time is now. Human-driven testing matters here because the flaws that hurt in these apps are rarely the ones a scanner flags. They’re business logic flaws: one customer reading another’s records, an agent taking actions its role should never allow, a chatbot that hands over data when asked the right way. That’s what our AI Application Security Review and web application penetration testing are built to find.

Plan for the next framework RCE. React2Shell and PHP2Shell won’t be the last. Run a supply chain tabletop exercise that asks a simple question: if a critical flaw in a common framework dropped this morning, how long would it take us to list every affected app? If the honest answer is “we’d have to ask around,” start with the inventory.

The app you built is now part of your attack surface

AI made building software cheap. It didn’t make owning software cheap. Every portal, chatbot, and internal tool your teams ship is now part of your attack surface, whether or not security has heard of it. The organizations that come through this well will know what they’ve built, have set a floor for how it gets built, and will have tested it before an attacker did. Banning the tools won’t get you there.

If your teams have shipped AI-built apps that are now holding customer data or running real operations, talk to us. We’ll help you find them, test them, and put an owner on each one. For the broader picture of how AI changes your risk, see AI Went From Chatbots to Agents. Your Security Didn’t. and our AI security services.