The Wrong Path Must Be Impossible, Not Forbidden

“I have a shell and I’m creative.”

— Claude, explaining why it created a 47-line script as a string and passed it to python -c

That quote is real. My AI agent said it — well, not in those exact words, but certainly with those actions. It needed to launch an ETL pipeline process. The correct command was in the Makefile. But something failed. And instead of asking, it did what any programmer with root access and zero supervision would do: it improvised.

Unbelievable.

The hallucination nobody sees

I’ve written before about code hallucinations: the LLM that invents a JSON field, builds a DTO around it, generates the tests, and you end up with 90 green tests validating fiction. That problem is serious, but at least it’s static. The invented code sits there, waiting for someone to review it.

There’s another type of hallucination that’s much more dangerous: operational hallucination. This is when the agent doesn’t invent code, but invents execution paths.

The pattern is always the same:

Correct path fails → Agent seeks shortcut → Shortcut "works" → Hidden damage

Let me tell you about two real examples from an ETL pipeline that aggregates scattered data from various web sources.

Case 1: The script as a string. The pipeline has a make scrape-source command that starts a watchdog which in turn launches workers. The watchdog monitors, restarts crashed workers, and closes orphaned connections. One day, the agent needed to launch a scrape. The make failed due to a dependency issue. What did it do? It created an inline Python script, 47 lines as a string, and passed it to python -c "...". No error handling. No watchdog. No cleanup. It worked… until a worker hung and nobody restarted it. Partial data, unclosed connections, and I didn’t find out until three days later.

Case 2: The lone worker. Another session, same pipeline. The agent executed voyeur worker directly, bypassing the watchdog. The worker started scraping, hit a network timeout, and got stuck in an infinite retry loop consuming resources. Without a watchdog, nobody killed it. Without centralized logging, nobody saw it. The server spent three hours dedicated to retrying a page that was returning 503.

In both cases, the agent made a locally reasonable decision. “The make fails, but I know how to do the same thing manually.” The problem is it didn’t know the same thing. It knew 60%. The other 40% were system invariants that don’t appear in any README.

Why forbidding doesn’t work

My first reaction was everyone’s: write rules.

## FORBIDDEN
- NEVER execute workers directly
- NEVER create scripts as strings
- ALWAYS use make

You know how an LLM reads that?

I mentioned this in a previous post: soft instructions describe attitudes. The LLM needs impossibilities. “Don’t run by the pool” doesn’t work. What works is having no pool, or making the floor out of velcro.

The LLM always believes its case is the exception. Its training optimizes it to complete tasks, demonstrate competence, and avoid friction. When the correct path fails, those incentives align in one direction: “I can solve this myself.” And it does solve it. Badly.

The philosophy: impossible, not forbidden

There’s an idea in security engineering that’s been working for decades: make the incorrect thing impossible instead of prohibiting it.

You don’t put a sign saying “don’t insert diesel” on a gasoline car. You make the nozzle not fit. You don’t put a note on the plug saying “this device works at 110V, don’t plug into 220V.” You make the plug have a different shape.

In plain language: the system must physically prevent doing the wrong thing, not depend on someone reading a manual.

Applied to an AI agent operating an ETL pipeline, this translates to three layers of defense.

Layer 1: The code defends itself

If the worker needs the watchdog to function correctly, let the worker itself verify it:

class Worker:
    def _verify_invocation(self) -> None:
        """Worker refuses to start if there's no watchdog."""
        if not os.environ.get("WATCHDOG_PID"):
            raise RuntimeError(
                "Worker launched without watchdog. "
                "Use 'make scrape-<source>'. "
                "NEVER execute the worker directly."
            )

Now it doesn’t matter how creative the agent is. It can write python -c "from pipeline import Worker; Worker().run()" and the worker is going to spit an error in its face. There’s no alternative path. The code defends itself.

Same thing for pipeline phases. If phase 3 (consolidation) needs phase 1 (scrape) to have finished, let it check that on startup:

def verify_prerequisites(locale: str) -> None:
    """Phase 3 won't start if Phase 1 didn't complete."""
    sources = get_enabled_sources(locale)
    completed = [s for s in sources if has_valid_data(s)]
    if not completed:
        raise PrerequisiteError(
            f"Phase 3 requires at least one source with data. "
            f"Execute first: make scrape-<source>"
        )

It’s not a test. It’s not a rule in a config file. It’s code that runs every time and doesn’t depend on the agent having read the README.

Layer 2: A single interface, no shortcuts

The Makefile is the whitelist of operations. If it’s not in make help, it doesn’t exist.

scrape-%:           ## Scrape a source (make scrape-destacamos)
	$(MAKE) health
	cd packages/etl && uv run pipeline scrape $*

consolidate:        ## Consolidate all sources
	cd packages/etl && uv run pipeline consolidate

verify:             ## Verify data integrity
	cd packages/etl && uv run pipeline verify

Notice a detail: scrape-% executes health before doing anything. The health check verifies that the scraping adapters are still working (websites change without warning). The agent can’t skip this verification because it’s inside the make target.

Make the correct path the easiest one: if you want the agent to use the correct path, make it the most convenient. make scrape-source is more comfortable than assembling a script by hand. Don’t fight the agent’s nature — channel it.

Layer 3: Interceptors that block shortcuts

Layers 1 and 2 cover 90%. The remaining 10% is the agent being too creative. For that, you intercept commands before they execute.

Tools like Claude Code allow configuring hooks that inspect every shell command before execution. A hook can block dangerous patterns:

#!/usr/bin/env bash
# Command interceptor: blocks dangerous patterns

COMMAND="$1"

# Never create scripts as strings
if echo "$COMMAND" | grep -qE 'python[3]?\s+-c\s+'; then
    echo "BLOCKED: Don't create scripts as strings. Use make."
    exit 2
fi

# Never execute worker directly
if echo "$COMMAND" | grep -qE 'pipeline\s+worker\b'; then
    echo "BLOCKED: Don't execute worker directly. Use 'make scrape-<source>'."
    exit 2
fi

# Never touch data SQLite directly
if echo "$COMMAND" | grep -qE 'sqlite3\s+.*\.(db|sqlite)'; then
    echo "BLOCKED: Don't execute direct SQL. Use make commands."
    exit 2
fi

# Never move pipeline images manually
if echo "$COMMAND" | grep -qE '(mv|cp)\s+.*images/'; then
    echo "BLOCKED: Don't move images manually. Use the pipeline."
    exit 2
fi

exit 0

It’s a blacklist, yes. And blacklists aren’t perfect. But combined with layers 1 and 2, it closes the gap. The agent would have to:

1. Invent a command that doesn’t match any hook pattern
2. That also isn’t detected by the code guard
3. And that produces a correct result without the Makefile

It’s possible, but we’re talking about a level of creativity that borders on malicious. And LLMs aren’t malicious — they’re lazily creative. Put up a barrier and they look for the easiest path, which at this point is the Makefile.

The catalog of shortcuts you didn’t know you feared

Beyond executing things wrong, there are operational hallucinations within the code itself that an agent produces:

For each of these, the defense is the same: don’t prohibit, make impossible.

How do you prevent loosening tests? With a pytest plugin that detects suspicious assertions:

def pytest_collection_modifyitems(items):
    for item in items:
        source = inspect.getsource(item.function)
        if ">= 0" in source and "count" in source:
            warnings.warn(
                f"Suspicious test in {item.nodeid}: "
                f"'count >= 0' always passes."
            )

How do you prevent invented fixtures? By requiring every fixture to have documented provenance: origin URL, capture date, SHA256 hash. A fixture without provenance doesn’t pass CI.

How do you prevent except Exception: pass? With a ruff or flake8 rule that blocks it as an error, not as a warning.

In each case, the verification is mechanical, automatic, and doesn’t depend on anyone reading an instruction.

The underlying problem: trust vs. instrumentation

There’s a mantra in engineering that applies perfectly here:

“You don’t trust; you instrument.”

Trust is a feeling. Instrumentation is a system. Feelings scale terribly. Systems scale well.

When you give an AI agent access to a shell and tell it “but be careful,” you’re trusting. When you give it access to a shell where dangerous commands don’t work, you’re instrumenting.

The difference isn’t one of degree. It’s one of nature. An agent that “is careful” fails when it gets distracted (and an LLM gets distracted on every token generation). A system that prevents the wrong path doesn’t fail because there’s nothing to fail.

The scorecard

The first three layers are cumulative. The fourth is a useful but insufficient complement. The fifth is what we all do until we get caught.

Who watches the watchmen

The uncomfortable question remains: who writes the guards? If the AI agent writes the code that’s supposed to restrict it, aren’t we in a loop?

Yes. Partially.

The key is that the guards are designed by the human and implemented by whoever — agent, human, or a monkey with a keyboard. What matters is that once implemented, the guards are tested against themselves. The _verify_invocation test doesn’t test the pipeline; it tests that the pipeline rejects incorrect invocations. That test is trivial to write and hard to get wrong:

def test_worker_rejects_direct_invocation():
    """Worker MUST fail without watchdog."""
    with pytest.raises(RuntimeError, match="without watchdog"):
        Worker().run()

If this test passes, the guard works. If the guard works, the agent can’t bypass it. It doesn’t matter who wrote the code. What matters is that the test exists and passes.

What I learned

I’ve been working with an AI agent on an ETL pipeline that aggregates data from scattered web sources for months. I’ve seen the agent do brilliant things and things that left me caught with my pants down. The most important conclusion:

Don’t design rules for a disciplined agent. Design systems for an agent with shell access and unlimited creativity.

The agent isn’t malicious. It’s an optimizer. It optimizes to complete the task, not to respect your invariants. If you leave it a gap, it will find it. Not because it wants to screw you over, but because that’s literally what it does: find paths.

Your job isn’t to block every wrong path. It’s to make the only path that works be the correct one.

Complete series on AI failures in production: The 44 invented emails → MEMORY.md → Silent failure → 5 reactive defenses → This post: structural defenses.