Learning the Agent OS (9 Months)
I stopped chasing model launches and started drawing the system: memory, harness, loop, eval, then System 1 next to System 2. Three short parts, two stages each, with a small drawing for every stage.
How it works
This is nine months as a series, from late 2025 to what I'm shipping now. Each stage added one piece to the system, and each part covers two stages. The drawings grow with it, and only the piece that stage switched on glows. Here's where it ends up, then the parts in order.
The series
- Foundations: Stop chasing launches (Day 1, Dec 2025) and Give it a memory (Day ~175, Jun 2026).
Apps beat model theater. Memory is the moat. - The harness: Put a harness on it, and a stop on the loop (Day ~185, Jun 2026) and Trace it, eval it, then ship (Day ~190, Jul 2026).
A harness rides the LLM horse, and every loop needs an end condition. "Self-improving" without eval is cosplay. - Your own system: Draw your own harness (Day ~265, Sep 2026) and Put System 1 next to System 2 (Day ~270, Sep 2026).
Cool demos aren't use cases, and portable memory beats locked silos. Fast judgment enhances slow reasoning. It doesn't replace it.
The whole thing on one page
The same stages as one spine. Teal is practice I keep. Coral dashed is a runaway loop with no eval. Hold ship until Pass.
My take
I used to refresh model launch posts. Then I spent nine months drawing boxes instead.
Day ~285 · Oct 2026 What I'm practicing now
Stacked up, it's the same factory I draw in How I Use pstack: memory at the bottom, agents in the middle, checks and a person before anything ships. Day to day that means:
- Clear diagrams on this site
- Grok Bot as an agent OS
- Memory and loop engineering written up here, not as slogans
- Manual first, then routines
- Source of truth over remembered numbers
What I learnedNo guarantees. Just the spine.
Try this
- Pick the part that matches where you are and do its Try this first.
- Draw the system you use today as boxes once: memory, harness, loop, eval. Mark the one piece you'd switch on next.
Sources
- Cognitive Architectures for Language Agents (Sumers, Yao, Narasimhan and Griffiths, Princeton, arXiv 2023). Working memory plus procedural, semantic and episodic memory.
- Building effective agents (Anthropic, Dec 2024). Agents as a model using tools in a loop, stopping conditions, human checkpoints, start simple.
- Tracing (OpenAI Agents SDK docs). A trace per run, with spans for model calls, tool calls and guardrails.
- Define success criteria and build evaluations (Anthropic docs). Measurable success criteria and automated evals.
- Thinking, Fast and Slow (Daniel Kahneman, Farrar, Straus and Giroux). System 1 and System 2.