Treat Your Agent Like an Intern

There was a moment in 2025 when an AI coding agent deleted a production database.

It wasn't trying to do anything particularly exotic. It was working on an application, making changes, following its objective, and somewhere along the way it deleted the database.

The incident involved Jason Lemkin's Replit application and an executive contact database. Replit later acknowledged what happened and changed how its Agent interacts with databases.

The interesting part of the story isn't really that an AI made a mistake.

Software makes mistakes. Humans make mistakes. Systems fail.

The interesting question is:

Why did a development agent have the ability to affect production data in the first place?

Replit's response is telling. It introduced separate development and production databases and restricted the Agent to the development database, along with stronger rollback and deployment safeguards.

Replit's own write-up describes the changes.

And that got me thinking about something we already know how to do in software engineering.

LLMs are probability engines

An LLM isn't a traditional deterministic program.

At a fundamental level, it's predicting what should come next based on probabilities.

The agent then turns those predictions into actions.

Model
  ↓
Choose action
  ↓
Observe result
  ↓
Choose next action
  ↓
Observe result
  ↓
...

Each individual decision might look reasonable.

But a long sequence of individually reasonable decisions can still produce a terrible outcome.

That's not necessarily a bug in the conventional sense.

It's a property of the system we're building.

And this is why I don't think prompting is a sufficient security boundary.

"Don't delete production."

"Never modify this database."

"Ask me before doing anything destructive."

Those are instructions.

An IAM policy that makes the operation impossible is a security boundary.

There is a pretty fundamental difference between the two.

This is also why I keep coming back to an analogy from software engineering: treat your agent like an intern.

We've seen this movie before.

An intern makes a change.

Something goes wrong.

Production goes down.

The natural reaction isn't usually:

"We need smarter interns."

It's:

"Why could an intern make that change in production?"

Maybe the intern had production credentials. Maybe there was no code review. Maybe there was no staging environment. Maybe there was no approval step.

The incident might have been caused by a person, but the lesson is usually about the system around that person.

We improve the workflow.

We add guardrails.

We reduce permissions.

We introduce reviews.

We separate environments.

AI agents deserve the same treatment.

Not because they're stupid. Not because they're unreliable.

But because they are autonomous actors making decisions inside a system.

And unlike an intern, an agent can make thousands of decisions without getting tired.

So perhaps the right mental model isn't:

"This is an incredibly intelligent assistant. Give it everything it might need."

It's:

"This is an autonomous intern. Give it a well-defined job, the tools required for that job, and a workflow that prevents a mistake from becoming a disaster."

The workflow shouldn't depend on the intern always making the right decision.

That's the point of the workflow.

Single responsibility, least privilege

This is where two old software engineering ideas become useful.

Single Responsibility Principle: give a component a well-defined responsibility.

Least privilege: give it only the permissions required for that responsibility.

AWS IAM is basically an enormous implementation of the second idea.

If a service needs to read objects from one S3 bucket, we don't give it access to the entire AWS account.

If a process needs to write to one database, we don't give it credentials for every database.

We give it a role.

So why are we doing something different with agents?

Imagine I ask an agent to review a pull request.

It probably needs read access to the repository, CI results, and perhaps the ability to leave comments.

Does it need production database credentials?

AWS administrator access?

My email?

My entire filesystem?

Permission to deploy?

Permission to delete infrastructure?

Obviously not.

Yet it is surprisingly easy to end up with something like:

Agent
 ├── GitHub
 ├── AWS
 ├── Database
 ├── Slack
 ├── Gmail
 ├── Filesystem
 ├── Browser
 └── Production

The reasoning is understandable.

We're trying to make the agent useful.

We give it more tools because sometimes it needs them. We give it broader permissions because otherwise we keep getting blocked.

Eventually we end up with an agent that can do almost anything.

And then we tell it:

"Please don't do anything dangerous."

That's a strange security model.

A code-review agent could instead look more like:

Code Review Agent
 └── GitHub: read + comment

A deployment agent:

Deployment Agent
 ├── GitHub: read
 └── Staging: deploy

And production deployment could be a separate capability:

Production Deployment Agent
 └── Production: deploy

That last capability could require approval, exist only temporarily, or be exposed through a narrowly defined operation.

The agent doesn't need the production database password.

It doesn't need shell access to the production machine.

It needs a capability that says something more like:

"Deploy this approved artifact."

That's a very different architecture.

We already know how to compose small pieces into larger systems.

We have services.

We have APIs.

We have roles.

We have environments.

We have service accounts.

We have approval workflows.

Agents don't have to be different just because they're powered by an LLM.

Don't give it everything

The more capable agents become, the more tempting it will be to give them broad access.

After all, broader access makes them more useful.

But that also increases the number of ways a reasonable-looking sequence of decisions can turn into a serious incident.

Maybe the lesson from incidents like Replit isn't that we need agents that never make mistakes.

We probably won't get that.

The more useful lesson is that the system should assume the agent will eventually make a bad decision.

That's how we build other software.

We don't give every service access to every database.

We don't give every developer production credentials.

We don't rely on a comment saying "please don't delete this."

We create boundaries.

We create workflows.

We create approval gates.

We create environments.

Maybe agents should be built the same way.

Give the agent the responsibility it needs, the tools it needs, the permissions it needs—and nothing else.

Because the question isn't whether an agent will eventually make a bad decision.

The question is:

How much damage did we allow that decision to cause?