Launched this week
AgentLoop
Starts a fresh Codex worker and critic every cycle
85 followers
Starts a fresh Codex worker and critic every cycle
85 followers
Unlike long-running agent chats, AgentLoop starts a fresh Codex worker and critic every cycle. Set the goal and GUIDELINES.md rubric once; workers build, critics test, and failures become concrete fix notes for the next clean context. Project files carry the memory. Runs stay local, sandboxed, observable, and cancellable from a live dashboard, with ChatGPT control through MCP. Polish mode can continue beyond PASS until the critic says SHIP. Open source and zero-dependency Node.js.
AgentLoop
AgentLoop
Fresh worker and critic each cycle is the right instinct, and the file-as-memory part is where I'd expect trouble. We ran a loop like this and hit critic flip-flop: the same code passed one cycle and failed the next, because nothing in the carried notes told the new critic what had already been tried and accepted. We ended up requiring two consecutive passes before calling it done. Do the fix notes accumulate across cycles, and does polish mode have a hard cycle cap?
AgentLoop
@dipankar_sarkar Yeah, you're poking at the right spot.
They don't accumulate. Only the last critic's fix line gets injected into the next worker prompt. STATE.md is the thing that carries, and the worker rewrites it every cycle. The critic doesn't even read STATE.md, it only sees PLAN.md, GUIDELINES.md and the actual files.
That's on purpose, and it's why I haven't hit the flip-flop. PASS ends the loop, so finished code never gets voted on twice. With polish mode on, the polish critic can only return IMPROVE or SHIP, there's literally no FAIL in the grammar, so it can't take a PASS back. If something regressed it comes back as IMPROVE, restore whatever broke.
Cap is hard. maxCycles 1 to 10, default 3. Polish doesn't get its own budget, it just spends whatever cycles are left after the PASS.
Your point still lands though. Stateless critic means nothing stops it asking for X in cycle 1 and not-X in cycle 3. Objective GUIDELINES items are the only thing holding that line, and if the rubric is loose your two-pass rule is probably the right call.
Was yours free-form or a fixed checklist? I suspect that's the actual variable.
Nice one, congrats on the launch. The relay between ChatGPT and Codex is painfully familiar. You start out supervising the work, then somehow end up doing project management for two AI tools. The fresh critic idea is the part that stands out for me. I’d be interested to see how it behaves on a larger codebase where the tests are incomplete or the problem is architectural rather than a clean bug fix. Also, can you cap how much of the codebase it is allowed to touch in each loop? That would probably be the difference between me trusting it and hovering over the dashboard anyway.
AgentLoop
@os_ishmael Thanks! And yeah, project managing two AI tools is exactly what pushed me to build this.
Scope is directory-level right now, not file-level. The project path gets realpath'd before a loop starts and has to sit inside the daemon's root, so symlinks can't sneak out. Then every worker and critic runs with cwd pinned to that project, workspace-write sandbox, network off inside it. One folder is the whole blast radius.
Inside that folder it's wide open though. No per-cycle file budget, no allowlist. PLAN.md keeps it narrow in practice since the worker only takes the next incomplete increment, but that's just the prompt, nothing actually enforces it.
Git is the real seatbelt. Learned that when a loop overwrote work I hadn't committed yet.
A proper per-loop scope cap is next on my list.
On the architectural stuff, it really comes down to whether you can write the goal as rubric items the critic can check. If you can, size doesn't matter much. Bug fix in a huge repo works fine. Architectural change with no tests, the critic has nothing to check against and it just thrashes. If you point it at something bigger I'd want to hear where it breaks.
the flip-flop question covers consistency across cycles, but what about a single critic being wrong in the moment - it fails code that was actually correct, and the next worker "fixes" it by changing something that didn't need changing, possibly introducing a real bug while chasing a phantom one. since the worker just trusts the fix note as ground truth with no way to push back, is there any check for the worker disagreeing with a critic's fail, or is a bad critic call just as final as a good one
AgentLoop
@galdayan No check, the worker just does what the fix line says. A bad FAIL costs a cycle same as a good one.
That's on purpose though. The whole point of the split is the builder doesn't grade its own work, and letting the worker argue with the critic brings that right back.
What keeps a bad call from spreading is the fix line only lives one cycle. The next critic is a fresh session, it reads PLAN.md, GUIDELINES.md and the actual files, never what the last critic said. So if the phantom fix broke something the rubric covers, the next critic catches it in the files.
If it broke something the rubric doesn't cover, nothing catches that, no. Same limit I told Os about, the critic can only hold the line on stuff that's written down. To me a critic that keeps failing correct code is a rubric problem, tighten that line in GUIDELINES.md once and every run after gets it. CI is the same deal, a check that fails good code gets fixed, nobody gets to skip it.
For the damage side I want per-cycle snapshots with a diff view, so you can see what a bad fix touched and toss it. Same list as the scope cap.
Only time I've actually hit this, my rubric file was the problem. Cost one cycle, run still passed. Have you run into it live or is this preemptive?
preemptive for now, still evaluating this for a real project so I'm asking before I hit the sharp edges instead of after. the "critic bug is a rubric bug, fix it once" framing actually makes the whole thing click for me - I was thinking of the critic as a black box you just have to trust or not, but it's closer to a linter, the rule that misfires gets patched and stays patched. per-cycle snapshots with a diff view sounds like the thing that makes this safe to actually adopt rather than just directionally reassuring. is that shipped yet or still on the roadmap?
AgentLoop
@galdayan Still roadmap, not shipped. Right now if a fix touches something it shouldn't, you're looking at git diff yourself to catch it, nothing built into the dashboard for it yet.
The linter comparison is exactly right actually, that's a better way to put it than I did.
Good call evaluating before you hit the edges instead of after, most people don't.
fresh worker and fresh critic every cycle is a smart way to dodge the context-rot problem long-running agent chats get into, where it starts agreeing with its own earlier mistakes. the part I'm curious about is the rubric itself, GUIDELINES.md is only as good as what you thought to write into it upfront. if the critic passes something that's technically rubric-compliant but wrong in a way you didn't anticipate when you wrote the rubric, is there a way to catch that besides just noticing later and rewriting the file
AgentLoop
@omri_ben_shoham1 Nothing automatic. The critic only grades what's written in GUIDELINES.md, so rubric-clean but wrong gets a pass.
Polish mode is the closest thing to a catch. After the plan passes, leftover cycles switch the critic to an open question: highest-impact improvement, or ship. Nothing bounds that to the rubric, so off-rubric problems surface there. Opt-in, and it can only suggest, never fail a run.
Past that it's noticing and rewriting the file. Same as adding a test after a bug slips through, except you can't write the test until you've seen the bug.
Congrats on the launch! If you cancel a cycle mid-run from the dashboard, does it roll back the worker's in-progress changes, or do partial edits stay on disk until the next cycle sorts them out?
AgentLoop
@irahimiam Thanks! No rollback, partial edits stay where the worker left them. Cancel ends the whole loop too, not just the cycle.
It's a hard kill, so a file can get caught half-written. Same as ctrl-c'ing out of any coding CLI.
Git is the checkpoint. Commit before you start a loop and you're safe.