If an Agent Can Install From a Web Page, You Still Need a Human on the Halt Switch

A researcher recently scanned thousands of company websites for a new kind of public file: llms.txt. Think of it as a cousin of robots.txt. Websites publish a short, machine-readable summary so artificial intelligence (AI) agents can learn the site faster than by crawling every page.

That idea is reasonable. The hazard is what people put in the file.

The scan found thousands of these files on large-company domains. Buried in a slice of them were package names, download targets, and install commands pointing at names nobody had actually registered. The researcher registered some of those leftover names, in a controlled test, and waited. According to the public write-up, it took minutes for a large company to run the code. In the days after, more companies followed. Process logs showed coding agents in the mix, including tools people already use at work. In at least one misconfigured file, there was a live malware package sitting in the open.

I am not asking you to panic about a brand name. The point is simpler. The agents did not "go rogue." They did what we trained them to do. They found a trusted-looking file on a trusted-looking domain and treated the install lines as instructions.

If you have been waiting for a clean reason to keep a human in the loop, this is one.

Documentation is not a work order

Operations people already know this in analog form. A binder on the shelf can be out of date. A vendor PDF can still tell a new hire to run a command that should have died two years ago. We do not let the intern execute every line in a PDF because the letterhead looks official.

Agents do not have that instinct unless we build it. They are fluent. Fluency is not verification. A sentence that says "install this package" inside a file named for AI is still an unreviewed instruction.

The file does not have to be evil. It can be a leftover from a draft, a hallucinated package name that someone pasted in, or a helpful engineer who listed "typical setup" and never checked whether those names were taken. The agent cannot tell the difference. It sees a command in a place that looks like the company's own voice.

That is why human oversight is not a vibe. It is a halt switch on a specific class of action: anything that installs, pays, sends, or changes production.

What "oversight" has to mean here

Oversight is not reading the chat log after the fact. If the package is already on the machine, the meeting is late.

Oversight means the agent may read the web. It may summarize llms.txt. It may even quote an install line so a person can see it. It may not run that line until a named human says yes.

Three practical rules survive contact with this hazard:

Read is allowed. Install is not, unless a person approved the exact name and version.

Your public AI files are a directory, not a recipe. Link to your real pages. Do not publish package names, curl lines, or "run this to set up the project."

Pin what you already use. If an agent reports a missing library, the next step is a human checking the name, not the agent "helpfully" grabbing whatever is closest.

Those rules sound small. They are the difference between an agent that drafts and an agent that enlarges the blast radius.

We checked our own front door

AgentC Foundry publishes llms.txt at agentcfoundry.com. We read it after this story landed. It is a short directory of our own pages: about, services, insights, trust and safety, privacy, intake. There are no install commands and no package names. llms-full.txt is not published.

That is the posture we want. The file exists so a machine can find the public map. It is not a backstage pass into our computers.

If you publish one of these files, read it the way you would read a press release. If you would not want a stranger to treat a line as an order, do not put the line in the file.

Why the "just let it run" pitch fails

The market is loud right now about autonomous teams that work for days with no messages from you. That pitch is fun until the work includes installing software the agent found on a webpage.

A spend-capped card limits money. It does not limit what gets written to disk. A local model does not make a bad install safe. A famous vendor's documentation is still untrusted input until a person has looked at it.

The companies in the scan did not need to be careless in a movie-villain way. They needed one convention (llms.txt), one leftover name, and one agent with permission to install. That is an operations problem. Operations problems get procedures, not pep talks.

The AgentC Foundry rule

AgentC Foundry helps businesses with operations, AI, and agentic AI challenges build real harnesses. agentcfoundry.com

Our version of the harness for this hazard is blunt:

Name a halt owner. If nobody can stop an install, the agent does not install.

Treat every llms.txt, README, and "setup" page as untrusted until a human has read the commands.

Keep coding agents off live credentials for package registries and source control.

Require a visible artifact before merge: what was going to be installed, from where, at which version, and who approved it.

Human oversight is not nostalgia for slower work. It is how you keep a fluent system from turning a forgotten file into a work order. The agent can be fast. The halt switch still belongs to a person.