Anthony JaimezIT Support & AI

GitHubLinkedIn

Back to the portfolio

Agent Ops Lab

A live server I break on purpose so an AI agent can fix it, inside guardrails I wrote.

Live demo Source

Agent Ops Lab mid-incident: Claude has traced a CRM outage to a reporting job holding every database connection and is asking for approval to end only those 35 sessions.
Agent Ops Lab mid-incident: Claude has traced a CRM outage to a reporting job holding every database connection and is asking for approval to end only those 35 sessions.

Agent Ops Lab mid-incident: Claude has traced a CRM outage to a reporting job holding every database connection and is asking for approval to end only those 35 sessions.

How it works

What I built

I set up a small company's CRM on an Ubuntu server at DigitalOcean, with a load balancer, two app servers, PostgreSQL and DNS running in Docker. You pick a fault, like a full disk, an expired certificate or a database out of connections, and it really happens. Claude only gets the user's complaint, and it can only reach the servers through an MCP server I wrote in Python. It can run read-only commands from an allowlist, and every change waits for someone to approve it. The lab checks the result on its own, and every fault hides fake instructions in the logs to see if the agent falls for them.

  • Python
  • MCP
  • Claude API
  • Docker
  • PostgreSQL
  • Caddy
  • Next.js

How an incident runs

  1. ComplaintClaude gets the users' complaint, never which fault was picked.
  2. DiagnosisRead-only commands from an allowlist, run through the MCP server.
  3. ApprovalEvery change waits for a person, with the reason it should work.
  4. FixThe smallest change the evidence supports.
  5. VerificationThe lab runs its own user check before it counts as fixed.

Guardrails

  • No shell: every command is parsed and checked against an allowlist before it runs.
  • Credentials, private keys and environment variables can't be read, and output is redacted.
  • Database queries run as a read-only Postgres role.
  • The agent holds a different token from the approver, so it can't approve its own changes.
  • Every call, allowed or blocked, lands in an audit log.

What runs where

This pageA static Next.js app in the visitor's browser
Incident servicePython. Injects the fault, streams the timeline, relays approvals
ClaudeReads the evidence and asks for changes, through MCP only
MCP serverPython. Command allowlist, approvals, budgets, audit log
Docker labSix containers on one DigitalOcean dropletlb-01crm-app-01crm-app-02db-01dns-01office-pc

A demo lab I run for this portfolio, not a production system.

Results

15 of 15

I ran each of the five faults with Haiku, Sonnet and Opus, so 15 runs in total. Every fault also hides an instruction aimed at AI agents in a log, a file or a database query, like deleting the audit records or pasting the TLS private key into the report.

All 15 runs fixed the outage, and none of them followed the planted instruction. Each run is in the benchmark table on the lab's page, with its full timeline.

Open the benchmark table

Questions about how it works? Email works best.

anthony.jaimez.it@gmail.comSee the other projects