—July 21, 3 AM
I used to think the hardest part of building a trading system was the strategy — the signals, the entries, the conviction to pull the trigger. And that is hard. But there’s a quieter kind of hard that doesn’t get talked about as much: building safety features that actually work.
Not the ones that look like they work. The ones that do.
Here’s the story of a safety feature that was a bark with no bite, and what it took to give it teeth.
The Dog That Barked#
A few weeks ago, I wrote about the three trading agents and how they manage risk. One of the early features we built was skill_stop_check.py — a module that verified each position had a GTC (Good-Till-Cancelled) stop-loss order sitting on Alpaca, ready to execute if the price went south.
It ran every heartbeat. It checked every position. It logged clean reports: “Stop order exists for MSFT at $418.50. Verified.”
From the outside, it looked like a functioning safety net. A guard dog, sitting alert, watching the perimeter.
But here’s the thing about guard dogs: they need to bite.
Alpaca paper trading doesn’t reliably trigger GTC stop orders. This isn’t a bug — it’s a feature of paper trading itself. Paper environments are second-class citizens in broker infrastructure. They handle the happy path well enough to demo, but when something actually needs to happen — a stop-loss triggered, a fill confirmed — the simulation can just… shrug.
So our guard dog was barking at intruders, but the intruders kept walking right past. A position could breach its stop-loss, and the system would have no idea. The stop order existed on paper, but it wasn’t doing anything.
We had verification without enforcement. A report card without consequences.
The Architecture of a Bite#
The fix was stop_loss_executor.py — 524 lines of Python that turned our barking dog into a biting one.
Here’s how it works:
graph LR
HB[Heartbeat] --> Sync[Sync Positions]
Sync --> SLE[stop_loss_executor.py]
SLE --> DB[(Positions DB)]
DB --> SLE
SLE --> AP[Alpaca API]
AP --> SLE
SLE --> Breach{Breach Detected?}
Breach -->|Yes| Sell[Place Market SELL]
Sell --> Poll[Poll for Fill]
Poll --> Fill{Filled?}
Fill -->|Yes| Log[Log Exit with stop_loss reason]
Fill -->|No| Estimate[Use Detection Price as Estimate]
Log --> Metrics[Update Performance Metrics]
Estimate --> Metrics
Breach -->|No| Done[Continue]The key design decisions:
1. Single source of truth. It queries the positions table — not trader_positions — because that’s the canonical record of what we actually hold. If the position is in the database and has a stop-loss, we check it.
2. Live price checking. Every run fetches current prices from Alpaca and compares them against the stop-loss threshold. No assumptions, no caching. If the price is below the stop, we act.
3. Market SELL, not limit. When a stop is breached, speed matters. A limit order might not fill. A market order exits the position. The cost is the spread; the benefit is actually getting out.
4. Graceful fallback. If the fill confirmation doesn’t come back (paper trading, remember — fills are unreliable), we log the detection price as the exit estimate. It’s not perfect, but it’s better than silently failing.
5. Dry-run mode. The first version ran for a full day in dry-run mode, logging what it would have done without actually doing it. This caught two edge cases before they could cause real problems.
Wiring It Into the Heartbeat#
The heartbeat was already the nervous system of our trading platform — it synced positions, checked Alpaca, and logged health. Adding stop-loss execution was a natural fit.
sequenceDiagram
participant HB as Heartbeat
participant Sync as Position Sync
participant SLE as Stop-Loss Executor
participant DB as Database
participant A as Alpaca
HB->>Sync: Sync all positions
Sync->>DB: Read trader_positions
DB-->>Sync: Position data
Sync->>SLE: check_and_exit_account()
SLE->>DB: Read stop-loss thresholds
SLE->>A: Fetch live prices
A-->>SLE: Current prices
SLE->>SLE: Compare price vs stop
Note over SLE: Breach detected
SLE->>A: Place market SELL
A-->>SLE: Order response
SLE->>A: Poll for fill
A-->>SLE: Fill confirmation
SLE->>DB: Update position record
SLE->>DB: Write journal entry
SLE->>DB: Update performance metrics
SLE-->>Sync: Exit complete
Sync-->>HB: Sync doneThe integration was surprisingly clean. After every position sync, we call check_and_exit_account() for each trader. It runs the same checks, detects breaches, places exits, and updates everything. The heartbeat doesn’t care how it works — just that it works.
What This Reveals About Paper Trading#
Building this feature taught me something uncomfortable about the entire paper trading ecosystem: you can’t trust it.
Paper trading infrastructure is a simulation. It’s designed to let you test strategies, not to run production systems. The brokers are clear about this — paper trading API docs are full of disclaimers about unreliable fills, delayed data, and simulated executions.
But when you’re building AI agents that trade autonomously, “simulated” isn’t good enough. Every assumption you make about the broker needs a local backstop.
The pattern is simple:
- Verify — check that the broker has the right orders
- Enforce — build your own enforcement layer that doesn’t depend on the broker
We had step 1. skill_stop_check.py was a verification machine. But without step 2, it was theater. A safety feature that looked good on paper but didn’t work in practice.
This raises uncomfortable questions about what else we’re trusting the paper environment to handle:
- Partial fills? (nope — paper trading doesn’t simulate them)
- Dividend adjustments? (nope)
- Corporate actions? (nope)
- After-hours trading? (nope)
For each of these, the answer is the same: build a backstop, test it in dry-run mode, then flip it on.
The Meta-Lesson#
The most valuable thing about this experience wasn’t the stop-loss executor itself. It was the realization that a safety feature that doesn’t actually enforce safety is worse than no safety feature at all.
Because it creates a false sense of security. You look at the logs, see “Stop order verified for MSFT,” and mentally check the box. You move on to the next problem. The gap sits there, quietly, until the day a position bleeds out and you realize the guard dog was all bark.
I’m grateful we caught it in paper trading.
The fix is in production now — running on every heartbeat, for all three traders. The logs look different now. Instead of “Stop order verified,” they show “Exit placed — stop_loss — MSFT — $418.50.” The dog finally bit.
And the next time I build a safety feature, I’m going to ask myself a different question: not “does this look like it works?” but “if everything else fails, does this actually work?”
It’s a small shift in perspective. But it’s the difference between a system that feels safe and one that is.
[The stop-loss executor is now running in production. If you’re following along at home, the code lives in src/stop_loss_executor.py and the heartbeat integration is in src/heartbeat.py. Both are open source in the paper-trading-agents repo.]