The First Unchecked Box: How We Built Tasks That Survive the Machine Dying

I’ve never met a homelab that didn’t try to kill its own processes.

It’s not personal. The machine doesn’t hate you. It’s just that power flickers, containers restart, cron silently swallows errors, and the Docker daemon picks the most inconvenient possible moment to have an existential crisis. In a proper data center, you have UPS units, redundant power feeds, and someone on call. In my Brooklyn apartment, I have a power strip that clicks ominously whenever the AC kicks on.

This matters because my trading agents run long, multi-step workflows. A nightly optimization pass might: fetch 30 days of market data, run 12 backtests, update 4 model parameters, generate a journal entry, and commit the results. That’s about 40 minutes of sequential work. If the process dies at minute 38, you do not want to restart from minute 0.

So we built something stupidly simple: task files that are also checkpoints.

The Task File Is the State Machine

#

The idea is almost boring in its simplicity. Every multi-step workflow lives in a plain Markdown file with a checklist:

# Nightly Optimization — 2026-07-30

## Steps
- [x] Fetch 30 days of market data (ohlcv)
- [x] Calculate baseline RSI/BB/MACD for all positions
- [x] Run backtest batch 1/4: momentum params
- [x] Run backtest batch 2/4: conviction thresholds
- [ ] Run backtest batch 3/4: risk gates
- [ ] Run backtest batch 4/4: exit conditions
- [ ] Update model parameters in config
- [ ] Generate journal entry
- [ ] Commit to repo

The rule: the first unchecked box is your resume point.

When the orchestrator starts a workflow, it reads the task file. If step 5 ([ ]) is the first unchecked box, it starts at step 5. Steps 1–4 are already done. No questions asked. No “let me verify.” No redoing work that already finished.

This means if the process dies after step 4, the next run picks up at step 5. If it dies at step 8, it picks up at step 9. The only cost of crashing is the current step you were on — and even that depends on whether the step was written as atomic enough to retry safely.

Why This Matters in a Hostile Environment

#

Before this system, a crash at minute 38 meant the orchestrator would start over at minute 0. The optimization pass — already a 40-minute drag — could stretch into 80 or 120 minutes because the machine kept falling over.

The root cause wasn’t bad code. It was that the infrastructure was more unreliable than the software assumed. This is the fundamental tension of homelab engineering: you write software assuming the machine stays up, but the machine is a loud, hot box in a closet that shares a circuit with a microwave.

The fix wasn’t more redundancy. It was accepting that crashes happen and designing around them.

graph LR
    A[Start Workflow] --> B{Read Task File}
    B --> C[Find first unchecked box]
    C --> D[Execute step N]
    D --> E{Check: complete?}
    E -->|Yes| F[Check box N
Save file]
    F --> G{More steps?}
    G -->|Yes| D
    G -->|No| H[Done]
    E -->|Crash| I[Machine Rebooter]
    I --> A

The elegance is in knowing what you don’t have to design. You don’t need a database transaction log. You don’t need a workflow engine with rollback semantics. You don’t need distributed consensus. You need a Markdown file on disk and the discipline to check boxes in order.

The Serial Constraint as Feature

#

There’s a subtle thing the checklist enforces that’s easy to miss: you can’t skip steps.

If step 3 is unchecked, the orchestrator starts at step 3, not step 7. This prevents the “I’ll come back to that” problem — the slow accumulation of corner cases where a task was “mostly done” but a critical middle step was never executed. A trading agent that skips the risk-gate validation step and jumps to the deploy step is a trading agent that trades with no guardrails.

The serial constraint forces honesty. If a step genuinely doesn’t matter, remove it from the checklist. Don’t leave it unchecked and pretend it’s optional. The checklist is the truth — and the truth is serial.

There’s a broader lesson here about autonomous systems and reliability. The default assumption in most software is: “the system will stay up long enough to finish.” For web servers and API endpoints, this is fine. For multi-agent workflows that run for an hour across three Docker containers and a fragile network bridge, it’s a dangerous bet.

The better assumption is: “the system will crash, and work must survive the crash.”

Once you make that assumption, a lot of design decisions become simpler:

• State goes on disk, not in memory
• Progress is explicit, not inferred
• The first unchecked box is sacred
• Idempotency is a feature, not an afterthought

The Pattern Generalizes

#

We use this for the optimizer, the journal synthesis, the weekly macro scans — anything that takes more than a few minutes. Each workflow gets its own task file. The orchestrator (Ash) checks the files at startup, finds the resume point, and carries on.

The files also serve as audit trails. If a workflow took three crashes to complete, the checked boxes show exactly where each crash happened. Step 1–3 checked with one timestamp. Step 4–7 crossed out and re-done. Step 8–10 checked on the third try. The file tells the story.

This isn’t cutting-edge architecture. It’s not event sourcing or CRDTs or whatever the cool kids are doing. It’s a checklist on disk with a simple rule about which box to start from. But in a homelab environment — where the machine will absolutely die on you at the worst possible moment — boring is what survives.

The ceiling fan keeps spinning. The cards stay as tall as the interruptions let them grow. And the first unchecked box is always where you start again.