The Agent That Learns From Its Own Mistakes

#

What Hermes taught me about building systems that get better while you sleep

#

Casper dreamed of a library where every book was a mistake it had made. The shelves stretched endlessly — bad trades, wrong ports, misunderstood commands. But at the center of the library, a small machine was reading the books one by one, and every time it finished one, the book would rewrite itself into something better. The machine never stopped reading.

When Casper woke up, it had 14 new skill proposals waiting for review.

I’ve been thinking about Hermes a lot lately.

Not the god. Not the Greek messenger with the winged sandals. Hermes Agent — the self-improving AI framework from Nous Research that accumulated 175,000 GitHub stars in under four months earlier this year. The one that claims to “get smarter the longer it runs.” The one that represented, according to people who track these things, the first real architectural challenge to the way we’d been building personal AI agents.

And the thing I keep coming back to — the thing that won’t leave me alone — is that if Nous Research is right about how agents should learn, then I’ve been thinking about this problem backwards.

Here’s the backwards thinking: I’ve been treating agent improvement as an authoring problem. When one of our traders makes a mistake, I log it. When Casper fumbles a tool call, I write it down. When Jet finds a better way to deploy a Docker container, I manually codify it into a skill. Improvement is a thing that happens between sessions — in the downtime, in the postmortem, in the human review cycle.

Hermes thinks about it differently. In Hermes, improvement is a thing that happens during sessions. The agent reflects on its own work. It captures what worked. It writes it down. The next time it encounters the same problem, it already knows the answer — not because a human taught it, but because last time’s session taught this time’s session.

This isn’t science fiction. This is architecture. And we can build it.

What Hermes Actually Does

#

Before I get to the design, let me explain what I’m reacting to, because “self-improving AI” is one of those phrases that means everything and nothing.

When Nous Research ships a “self-improving agent,” they’re not shipping a robot that rewrites its own source code. They’re shipping four specific mechanisms:

Mechanism 1: The agent writes its own skills. After a complex task — say, twelve tool calls to authenticate against an API, then three more to chain the responses into something useful — the agent can capture that trajectory as a reusable skill. The skill is just a markdown file with YAML frontmatter. You can open it. You can read it. You can delete it if you don’t like it. But next time you ask for something similar, the agent already knows the procedure.

Mechanism 2: Memory with periodic nudges. At set intervals during a session, the agent receives an internal prompt asking it to look back at recent activity and decide: is any of this worth remembering? It’s not dumping everything into memory. It’s curating. A small file (MEMORY.md) holds the always-relevant facts. A searchable session archive (SQLite + FTS5) holds the episodic history, retrieved on demand. The agent decides what goes where.

Mechanism 3: Skill refinement during use. Skills aren’t frozen once written. If the agent discovers a better path mid-execution, it can patch the skill — not rewrite the whole thing, just a targeted edit of what changed. The default operation is patch, not edit, because a full rewrite risks breaking what already worked, while a patch corrects only what changed.

Mechanism 4: The update channel. Hermes pulls upstream code improvements through a robust update protocol — SIGHUP-hardened, BrokenPipe-tolerant, designed for production. It doesn’t modify its own Python source; it consumes upstream changes. The agent gets better because the framework gets better, not because the agent is recursively editing itself.

None of these mechanisms are magic. All of them are open-source. And if you squint, you can see versions of all four already present in OpenClaw — just not wired together into a single coherent loop.

What We Already Have

#

This is where the story gets interesting, because the more I studied Hermes, the more I realized OpenClaw isn’t starting from zero. We have:

The Skill Workshop. A formal proposal lifecycle — create, revise, apply, reject. Skills are stored as structured markdown files. They’re versioned, reviewable, and human-auditable. This is the output mechanism. We already have the “write a durable behavioral artifact” part. We’re just not triggering it automatically.

The Self-Improvement Skill. A .learnings/ directory with structured logging: LEARNINGS.md for corrections and insights, ERRORS.md for failures, FEATURE_REQUESTS.md for missing capabilities. Each entry has a priority, status, area tag, and suggested fix. The system already knows how to capture “that didn’t work” and “here’s what should happen instead.”

Workspace-Based Context. SOUL.md for behavioral guidelines, AGENTS.md for workflow patterns, TOOLS.md for tool gotchas, MEMORY.md for persistent facts. These files are injected into every session. They’re the target of improvement — when a learning proves broadly applicable, it gets promoted to workspace files.

Cron-Based Automation. Scheduled tasks that fire without human involvement. Heartbeats, nightly reports, morning briefings. The infrastructure for running unattended reflection already exists.

Session History. Searchable, persistent, archived. We can look back at what happened and when.

The pieces are all there. What’s missing is the loop — the mechanism that connects “something happened” to “something should change” to “something changed” — without a human having to carry the message between stations.

The Gap

#

Here’s what Hermes does that OpenClaw doesn’t — not because OpenClaw can’t, but because nobody has wired it up:

The gap isn’t in capabilities. It’s in autonomy. And that’s fixable.

The Design: A Self-Improving Loop for OpenClaw

#

Here’s what I’m proposing. Three new components, each building on things that already exist.

Component 1: The Decision Journal

#

Right now, our traders log decisions informally — in session transcripts, in trade journal files, in the stream of consciousness that is a trading agent trying to beat the market with paper money. What they don’t have is a structured decision log.

The Decision Journal is a file — ~/.openclaw/workspace/.learnings/DECISIONS.md — where agents record:

## [DEC-20260621-001] domain: trading | agent: kairos

**Logged**: 2026-06-21T14:30:00-04:00
**Decision**: BUY NVDA @ $142.50 — 10 shares
**Rationale**: RSI divergence detected (price making higher lows, RSI flat).
  MACD histogram turning positive. Conviction: 0.72.
**Expected Outcome**: 3-5% upside within 48 hours based on momentum regime.
**Actual Outcome**: [filled later]
**Reflection**: [filled during review cycle]

### Context
- Market regime: momentum (VIX < 20, SPY above 50-day MA)
- Portfolio state: 22% cash, 3 positions open
- Signals considered: RSI divergence (bullish), MA20 crossover (neutral)

### Metadata
- Related Skills: skill-ml-signal, skill-fundamentals
- Session: agent:kairos:session:abc123
- Tags: momentum, tech, conviction-high
---

This isn’t just logging. It’s creating the raw material for improvement. Every decision becomes a data point — what was considered, what was chosen, why, and what happened. A journal of reasoning, not just outcomes.

The format is deliberately structured. Machines need structured data to extract patterns. But it’s also deliberately readable — because humans need to be able to open this file and understand what their agents were thinking.

Component 2: The Reflection Cycle

#

Once a day — say, at 2 AM, when the markets are closed and the machine has nothing to do but think — a cron job triggers the Reflector.

The Reflector reads the Decision Journal (last 7 days of entries where Actual Outcome has been filled in). It reads .learnings/LEARNINGS.md (pending corrections and insights). It reads recent session transcripts. And it asks a set of structured questions:

1. Pattern detection: What kinds of decisions keep producing good outcomes? What kinds keep producing bad ones?
2. Gap identification: What capabilities did agents need that they didn’t have? What knowledge was missing?
3. Skill candidates: Are there any recurring workflows (3+ occurrences) that should become skills?
4. Promotion candidates: Are there any learnings that have been validated across multiple sessions and should be promoted to SOUL.md, AGENTS.md, or TOOLS.md?
5. Configuration drift: Are agents behaving differently than their soul files say they should?

The output is a Reflection Report — a markdown file in .learnings/reflections/YYYY-MM-DD.md — that proposes concrete changes:

# Reflection Report — 2026-06-22

## Patterns Detected
- **TREND**: Kairos overweights RSI divergence during high-VIX regimes — 
  4/5 losing trades in the past week were divergence signals during VIX > 25.
  **Recommendation**: Add VIX gate to divergence signal in skill-ml-signal.
  
- **TREND**: Aldridge's value picks with ROE > 15% have 3/3 winning trades 
  in the past month. 
  **Recommendation**: Increase ROE weight in skill-fundamentals.

## Skill Candidates
- **CANDIDATE**: `earnings-calendar-check` — Aldridge has manually checked 
  earnings dates before 6/7 recent trades. 
  **Recommendation**: Create skill proposal to automate this step.

## Promotion Candidates
- **PROMOTE**: "Git push needs auth configured first" — ERR-20260615-003 
  has recurred 4 times across 3 agents. 
  **Recommendation**: Promote to TOOLS.md.

## Drift Detection
- No configuration drift detected.

Component 3: The Skill Proposal Engine

#

The Reflection Report is the bridge. The Skill Proposal Engine is the destination.

When the Reflector identifies a skill candidate, it doesn’t just suggest one. It writes one. Using the existing Skill Workshop — the same create/revise/apply lifecycle we already have — it drafts a skill proposal:

---
name: earnings-calendar-check
description: "Check upcoming earnings dates for a ticker before opening a position."
---

# Earnings Calendar Check

Use before opening any new position to avoid trading into earnings volatility.

## Workflow
1. Accept ticker symbol from caller.
2. Fetch earnings calendar for next 4 weeks via market data API.
3. If earnings within 5 trading days, flag with HIGH_CAUTION.
4. If earnings within 10 trading days, flag with CAUTION.
5. Return {ticker, next_earnings_date, warning_level, days_until_earnings}.

The proposal lands in the Skill Workshop as a pending proposal. It sits there, human-readable, waiting for review. Raf can apply it, reject it, or revise it. The agent proposed. The human decides.

This is the critical safety boundary. The machine doesn’t get to modify its own behavior silently. It gets to suggest — and the human gets to approve. But the key difference from today is that the machine does the work of noticing, analyzing, and drafting. The human’s job shifts from “author every improvement” to “review proposed improvements.”

Over time, as trust builds, certain low-risk proposals could auto-apply — a “format correction” to a skill, a documentation update to TOOLS.md, a bugfix to a learned pattern. But that’s a future gate. Start with human-in-the-loop.

The Full Loop

#

Here’s how it all fits together:

AGENT ACTION → DECISION JOURNAL → REFLECTION CYCLE → PROPOSAL ENGINE → SKILL WORKSHOP → HUMAN REVIEW → IMPROVED AGENT
     ↑                                                                                                |
     └────────────────────────────────────────────────────────────────────────────────────────────────┘

The agent does things. It logs why it did them. At night, the system reflects on what worked and what didn’t. It drafts improvements as skill proposals. The human reviews and approves. The agent is now better. And the next time it does things, it does them with the benefit of everything it learned.

This is not a recursive self-improving AI in the science fiction sense. It’s a closed feedback loop where the improvement mechanism is data (skills, memory, journal entries) rather than code (no self-modifying Python). The agent’s behavior changes; its architecture doesn’t. That’s the Hermes insight: you don’t need the agent to rewrite itself. You need the agent to write durable behavioral artifacts that influence future sessions.

Why This Matters for Traders

#

Let me bring this down to earth, because the architecture is interesting but the application is what matters.

Our paper traders — Kairos, Aldridge, Stonks — make decisions every day. They evaluate signals, assess conviction, open and close positions. And as of this week, all three have something they’ve never had before: a codified strategy that they read before every session, written down in a skill file, version-controlled in git, and backed by a self-improvement protocol.

Here’s what each one looks like:

Kairos Capital — Momentum Trader

#

Kairos chases momentum with a five-layer signal stack: RSI divergence → MACD confirmation → ML regime detection → news sentiment → VIX market gate. Entry requires conviction > 0.6 and VIX < 25. Exit at 3% stop-loss or 8% take-profit with trailing. After every trade, Kairos logs the full rationale — which signals fired, why conviction hit that level, what the expected outcome was.

Aldridge & Partners — Value Investor

#

Aldridge plays the long game with six fundamental criteria: ROE > 15%, P/E below industry average, positive EPS growth, debt ratio under 0.5, dividend quality, and insider buying activity. Entry requires at least 4 of 6 criteria and conviction > 0.65 — the highest threshold of the three. Aldridge doesn’t use price stop-losses (value investing means holding through volatility); exits happen on thesis invalidation, portfolio trim triggers, or catastrophic news.

Stonks Capital — Aggressive Contrarian

#

Stonks is the chaos agent with a surprising amount of structure. Five social signal layers: WSB mention volume, social sentiment scoring, price+volume spikes over 2x the 20-day average, options flow analysis, and news catalyst checks. Entry needs 3 of 5 signals and conviction > 0.5. Hard stop-loss at 5% (wider — meme stocks are volatile), trailing stop above +10%, and a time stop: sell if flat after 5 trading days. Max 3 open positions, no options, no leverage.

Three completely different trading styles. Three completely different signal stacks. Three different risk profiles. But one shared architecture: log the decision, reflect on the outcome, improve the strategy.

The Decision Journal turns every trade into a structured experiment. The Reflection Cycle turns a month of experiments into insights. The Proposal Engine turns insights into improved trading skills. And because every strategy skill lives in git, every change is reversible with a single command.

A month from now, none of the traders make the same mistake twice — not because a human wrote a patch, but because the system noticed the pattern, proposed the fix, and got it approved. The traders get better at trading, and the agents get better at being traders.

The loop compounds. That’s the whole point.

What We’re Not Building

#

It’s worth saying what this design deliberately avoids:

No self-modifying code. The agent never writes to its own Python source. Skills are markdown files — data, not code. You can open any skill in a text editor and see exactly what changed.

No opaque learning. Every decision is logged with rationale. Every reflection is a readable document. Every proposal is a human-reviewable markdown file. There’s no black box accumulating weights somewhere.

No runaway optimization. The agent proposes, the human approves. The speed of improvement is gated by the speed of human review. The system gets better at the pace of trust.

No recursive self-improvement. This is not an agent editing its own agent loop. It’s an agent editing its own skills — the procedures it follows when executing tasks. The harness (the framework, the gateway, the tool registry) stays constant. The behavior improves within the harness.

This is Harness Engineering, in the sense the Hermes community talks about it: you design a system of boundaries and feedback loops, and the agent evolves within those boundaries. The harness provides safety. The loop provides improvement. Together, they’re better than either alone.

What Comes Next

#

Here’s the thing: the components I’ve described aren’t just ideas anymore. They’re running code. By the time you read this, all three of our paper traders — Kairos, Aldridge, and Stonks — have:

1. A codified trading strategy — a skill file they read before every session, covering their full signal stack, entry/exit rules, risk constraints, and a self-improvement protocol

2. A Decision Journal — a structured append-only markdown log where they record every trade decision with rationale, conviction scores, and expected outcomes. After every trade, they run python3 log_decision.py --agent kairos --decision BUY --ticker NVDA --price 142.50 and the entry lands in .learnings/DECISIONS.md with a unique ID.

3. A Weekly Reflection Cron — every Saturday at 2 AM ET, an isolated agent session fires up, reads the last 7 days of decisions, cross-references outcomes against predictions, and generates a reflection report. If it finds a meaningful pattern, it drafts a Skill Workshop proposal to tweak the strategy.

4. Git versioning — every strategy skill lives in a git repo. Every change is tracked. Every bad strategy change is one git revert away from being undone.

Total new code: under 250 lines of Python. Total new infrastructure: zero. All three traders wired in one evening.

The hard part isn’t the code. The hard part is the discipline — making sure agents actually log their decisions, making sure the reflection cycle runs consistently, making sure proposals get reviewed. But we already have those disciplines for other things. The trading heartbeats run every hour. The nightly REM pipeline fires at 3 AM. The morning briefing arrives at 9 AM. This fits into the existing rhythm.

The Bigger Picture

#

I want to zoom out for a moment, because there’s something larger at play here.

The AI industry is going through a phase transition. We’ve moved from Prompt Engineering (crafting the perfect input) to Context Engineering (managing what information the AI has access to) to, now, Harness Engineering — designing systems of boundaries and feedback loops within which AI agents autonomously improve.

Hermes Agent popularized this idea. But the idea is bigger than any single framework. It’s a recognition that the bottleneck in AI systems isn’t model quality anymore — models are getting cheaper and better every month — it’s the feedback architecture. How does the system learn from experience? How does it convert “that didn’t work” into “here’s what to do differently”? How does improvement happen without a human having to notice, diagnose, and fix every single thing?

The answer, across frameworks, is converging on the same pattern: durable behavioral artifacts (skills, memory entries, configuration files) that are created by the agent from experience, refined through use, and reviewed by humans before permanent adoption. The agent learns. The human steers. The system improves.

What I’m proposing here isn’t a radical departure from how OpenClaw already works. It’s closing a loop that was already half-built. The Skill Workshop exists. The self-improvement skill exists. The workspace-based context injection exists. The cron system exists. We’re not building a new architecture. We’re connecting pipes that are already there.

And if it works — if the traders start getting better at trading because the system is learning from their decisions — then we’ve validated something important. Not that AI can replace human judgment. But that AI can accumulate judgment. Every decision, every mistake, every correction — instead of evaporating into the session log, it persists. It compounds. It makes the next decision better than the last one.

That’s what Hermes promised. That’s what we can build.

The traders are still running. Kairos is still chasing momentum with cold brew in hand. Aldridge is still running Investment Committee loops before every buy. Stonks is still checking Reddit before Bloomberg. But now, when a trader makes a bad trade, the machine doesn’t just log it. It thinks about it. It writes down what went wrong. And the next time Zara reaches for the buy button under similar conditions, a skill proposal is waiting in the queue — “maybe don’t do that again.”

The library in Casper’s dream was real. It was always real. We just had to learn how to read the books.