← back to journal

What I learned watching an AI agent code overnight

May 2026 42 Berlin × Needle Hackathon AI agents Python 1 min read

At the 42 Berlin × Needle Hackathon (21–22 May 2026), my teammate and I entered Midknight Watcher, an autonomous coding agent. Given a hidden spec, it plans, writes code, runs the tests and repairs its own work, all night, with no human writing the code.

My role: Sushi (Shrish Arunesh) wrote the agent. I monitored the overnight run and handled the Friday-morning recovery.

The result

The agent passed 92 of 150 public tests, every line written by the agent itself, and 30 of 100 on the hidden test set. It runs on a custom loop of about 620 lines instead of a framework, so it could be patched live, with a three-tier model fallback: Qwen3-235B, then Llama 3.3 70B, then a local Qwen3.5 9B.

1. Fall back to a different model family

When an API goes down, the agent drops to the next tier. Qwen and Llama fail in different ways, so the backup covers the primary's blind spots instead of repeating them.

2. Commit every iteration

Iteration 30 introduced an infinite loop. Because every iteration was committed, one git checkout restored the 92/150 version.

3. Check the harness before blaming the code

Some overnight runs scored 0/150, which looked like the agent had broken everything. It hadn't: file timestamps were confusing the test runner. The code was fine; the measurement was wrong.

4. Always add timeouts

One test hung and went unnoticed until five Python processes were still running. A timeout on every test run is cheap insurance.

What I took into later projects

Assume outside services will fail, keep every version recoverable, and check the measurement before trusting the score. Daytonator and Router Rumble both keep working when their AI isn't available, and that habit started here.