The Historical Replay Harness: Teaching Trading Agents to Learn From Their Own Ghosts

Here’s a question I’ve been sitting with: how do you teach something to learn from its past — without letting it forget what it did wrong?

The easy answer is “log everything and analyze it later.” That’s table stakes. Every system logs. But I wanted something more visceral. I wanted my trading agents to re-live their worst decisions. To line up against a ghost of themselves and see, frame by frame, where they went off the rails.

So I built the Historical Replay Harness.

The Problem: You Can’t Practice With Real Money

#

If you’ve been following this blog, you know my setup: three paper trading agents — Kairos (momentum), Aldridge (mean-reversion), Stonks (the HMM regime detector) — running 24/7 against live market data. They make decisions, place paper trades, and learn from the outcomes.

But here’s the thing about live markets: they only happen once. You can’t re-run Tuesday afternoon to see what would’ve happened with different settings. If a trade goes bad, the moment is gone. The best you can do is write a post-mortem and hope the agent internalizes it.

That works about as well as you’d expect.

I wanted something more like Groundhog Day for trading bots — same market conditions, over and over, until they figure out the right move. A sandbox where the market is frozen in amber and the agents can iterate against perfect knowledge of what actually happened.

graph TD
    subgraph "Live Trading (Happens Once)"
        L[Live Market] --> D1[Agent Decision]
        D1 -->|Outcome| L1[Lesson Learned]
        L1 -->|Next day, new context| L
    end

    subgraph "Replay Harness (Infinite Practice)"
        H[Historical Snapshot] --> D2[Agent Decision]
        D2 -->|Compare to ground truth| R[Reward Signal]
        R -->|Adjust params| D2
        D2 -->|New decision| R
    end

    style L fill:#f7a84c,color:#fff
    style H fill:#4a6cf7,color:#fff
    style R fill:#6b4cf7,color:#fff

That’s what the harness does. It takes a slice of historical market data — OHLCV, order book snapshots, sentiment readings, the whole thing — and plays it back to an agent as if it were happening right now. The agent has no idea it’s in a simulation.

It makes a decision. The harness evaluates it against what actually happened next. The agent learns. The harness resets. The agent tries again.

How It Works

#

The replay harness has three components: the snapshot, the clock, and the referee.

The Snapshot

#

Every trading day, Casper (my orchestrator) snapshots the entire market state: prices from Polygon, sentiment from Praesentire and FinBERT, order book depth, technical indicators, and any decision logs the agents generated. It’s a complete freeze-frame of the market at that moment.

graph LR
    subgraph "Daily Snapshot"
        P[Prices] --> S[Snapshot]
        SENT[Sentiment] --> S
        OB[Order Book] --> S
        TI[Technical Indicators] --> S
    end
    S --> H[(Snapshot Archive)]

These snapshots pile up over time. After a few weeks, I had a library of market moments — good days, bad days, days where the market did something weird at 2 PM that nobody predicted.

The Clock

#

The replay controller is basically a time machine. It starts at the snapshot timestamp and ticks forward at variable speed — real-time for decision points, fast-forward between them. The agent sees time flowing normally. It doesn’t know it’s living in the past.

sequenceDiagram
    participant H as Harness
    participant A as Agent
    participant C as Clock

    H->>C: Load snapshot (2026-06-15 09:30)
    C->>A: Market open, here's the state
    A->>H: Decision: short AAPL at $218
    H->>A: Wait for outcome...
    C->>C: Advance 1 bar
    H->>A: Price is now $216.50
    A->>H: Adjust position
    C->>C: Continue to market close
    H->>A: Total P&L: +$2,450
    H->>H: Reset to 2026-06-15 09:30
    H->>A: Market open, here's the state

I added a nice touch: the agent sees a “session ID” that changes between runs, but the session ID is just the replay iteration number. Outside the harness, it’s iteration 47. Inside, it’s just another Tuesday.

The Referee

#

The referee is the part I’m proudest of. It doesn’t just score the trade — it scores the decision process.

Did the agent enter too early? Too late? Did it ignore a sentiment signal? Did it override its own risk model? The referee compares the agent’s decision to the ideal decision (hindsight is 20/20) and assigns partial credit based on how close it got.

graph TD
    subgraph "Scoring"
        A[Agent Decision] --> R[Referee]
        GT[Ground Truth] --> R
        R --> Score{Score Breakdown}
        Score -->|Timing| T["± 15 minutes? Partial credit"]
        Score -->|Direction| D["Long vs short correct?"]
        Score -->|Sizing| S["Right position size?"]
        Score -->|Signal Use| SU["Used available signals?"]
    end

    style R fill:#f7a84c,color:#fff
    style GT fill:#4a6cf7,color:#fff

This matters because raw P&L is a terrible teacher. A trade can lose money for the right reasons (position was correct, but an unexpected news event ruined it) or make money for the wrong ones (dumb luck on a lottery-ticket trade). The referee separates signal from noise.

What Happened When We Ran It

#

I’ll save the full results for a technical deep dive, but here’s the highlight reel.

Kairos (the momentum trader) was the most improved player. In the first replay pass, it was chasing breakouts aggressively — jumping in at the first sign of movement, getting shaken out, then watching the real move happen without it. After 20 iterations against the same day, it started waiting for confirmation. Its entry timing improved by 37% against the referee’s scoring.

Aldridge (mean-reversion) was the most interesting case. Aldridge was actually overfitting to the replay. It started finding patterns that weren’t there — seeing fake mean-reversion setups in noise. The harness caught this because the referee showed the decision quality diverging from the P&L: profits went up, but decision scores went down. That’s the signature of overfitting. Good result, bad reason.

Stonks (the regime detector) didn’t change much. Its strength is market regime classification, and that’s a slower-moving signal. The replay harness confirmed its regime labels were solid, which was itself valuable — validation is a valid outcome.

The Hidden Feature: Ghost Trades

#

My favorite part of the harness wasn’t planned. I started saving the best-performing version of each agent against a replay day — the version that scored highest against the referee. I call these “ghosts.”

Now when an agent runs a replay, it sees not just the historical market data, but a trace of where its best self was at each decision point. It’s running against its own ghost.

The agent doesn’t know it’s competing with a past self. It just sees a recommendation trace that happened to come from a very smart version of its own brain. It’s learning from its own experience, distilled into a feedback loop.

Why This Pattern Matters Beyond Trading

#

I keep coming back to this because the replay harness isn’t really about trading. The pattern is universal:

1. Freeze reality at a moment in time — capture everything
2. Run your system against it — over and over
3. Score the decision, not the outcome — because outcomes are noisy
4. Save the best version of yourself — so you can race your own ghost

I’ve been thinking about where else this applies. Code reviews? You could replay a week of PRs and see if your review bot catches the same bugs the second time around. Customer support? Replay a day of tickets and let the triage agent iterate until it routes everything correctly.

Anything that involves decisions against a fixed historical record is fair game.

What’s Next

#

The replay harness is running in production now, cycling through snapshots on a 4-hour loop. Each agent gets 30 iterations per day against random historical slices. The ghost traces are feeding into the three-channel learning loop — param updates trigger within minutes when the referee detects consistent suboptimal decisions.

I want to add one more thing: cross-agent ghost races. What if Kairos could replay a day optimized for Aldridge’s strategy? Could a momentum trader learn something from watching a mean-reversion agent’s best run? I think there’s something there — a kind of strategy empathy that produces more robust decision-making.

But that’s a problem for next week. Right now, I’m watching my agents race their ghosts. And the ghosts are losing.