12. The backtester guide
I'm working on the backtester · 2/6 I want to rerun the evidence · 1/5
This chapter is the manual. Chapters 4 and 9 explain why the backtester is built the way it is. This one tells you how to use it: what to type, what comes out, and what every number means.
#The five backtesters, and which question each answers
| Command | Question | Unit of analysis |
|---|---|---|
sd_backtest | Does SD's reading predict what price does next? | one (symbol, day) reading |
sd_direction | What is the best possible up/down call, walk-forward? | one (symbol, day) reading |
sd_options | Is this option trade a good bet? | one trade |
sd_year | What would an account have done trading this for a month or a year? | one account over one period |
sd_timing | Do entry/exit limits beat buying or selling now? | one limit order |
All of them run from Implementations/rust:
cd Implementations/rust
cargo build --release
cargo run --release --bin sd_fetch_history # once: ~125 daily histories into data/history/
cargo run --release --bin sd_fetch_history -- --scale # once: real-price scale (account sizing)
cargo run --release --bin sd_fetch_history -- --out ../../data/iv ^VIX ^VXN ^VXD ^GVZ ^OVX
#Trading a full year or a full month: sd_year
cargo run --release --bin sd_year # every year 2008 → now
cargo run --release --bin sd_year -- --years 2019-2026 # a range of years
cargo run --release --bin sd_year -- --months 2024 # each month of 2024 as its own period
cargo run --release --bin sd_year -- --months 2020-2022 # each month of three years
cargo run --release --bin sd_year -- --account 25000 # account size (default 100000)
cargo run --release --bin sd_year -- --noise 0.5 # SD sees Gaussian-noisy prices
cargo run --release --bin sd_year -- --fractional # allow fractional option contracts
cargo run --release --bin sd_year -- --no-charts # skip SVGs
Environment switches: SD_OPTION_RISK=0.05 (share of equity risked per
option trade, default 0.02), SD_REAL_IV_ONLY=1 (options only on
underlyings with a real Cboe index), SD_YEAR_OUT=dir, SD_CHART_DIR=dir.
scripts/sd_year_accounts.sh runs the four standard account sizes.
#Definition: "trade a full period"
A full period (a calendar month or year) starts flat, with cash only, at the close of the last trading day before the period. At every close inside the period the strategy reads SD computed from bars up to and including that close and states what it wants to hold. Stock orders fill at the next open; option orders fill at that close (option prices come from the close). Every fill pays costs. The account is marked to market every close. At the period's last close everything still open is closed at marked value minus closing costs, so the period ends flat. SD's fitted pieces are refitted on outcomes known before the period's year began.
Two periods never share a position, so each period's result stands alone, and "the 2022 result" means exactly what it says.
#What comes out
| File | Contents |
|---|---|
reports/year/summary.txt | the tables: all periods, out-of-sample periods, every period |
reports/year/summary.csv | every metric for every (period, strategy) |
reports/year/summary.json | chained metrics, per-period returns, vol-edge buckets (read by sd_feed) |
reports/year/equity_<period>.csv | daily equity of every strategy |
reports/year/trades.csv | every closed position: entry, exit, days, return, P&L |
docs/charts/*.svg | equity curves per period, heat maps, the chained curve, vol edge vs win rate |
#Every metric, defined
Daily returns rᵢ come from the equity curve. "Annualized" uses 252
trading days. The risk-free rate is 0 (constant RISK_FREE in book.rs).
| Metric | Definition | Read it as |
|---|---|---|
| Total return | E_end / E_start − 1 | what the period made |
| CAGR | (1 + total)^(252 / days) − 1 | the yearly rate that compounds to the same result |
| Volatility | stdev(rᵢ) · √252 | how bumpy the ride was |
| Sharpe | mean(rᵢ) / stdev(rᵢ) · √252 | return per unit of bumpiness; ~1 is good, 2+ is rare |
| Sortino | mean(rᵢ) / downside_dev · √252 | like Sharpe, but only falls count as risk |
| Max drawdown | largest 1 − E / peak(E) | the worst peak-to-trough fall you would have sat through |
| Calmar | CAGR / max drawdown | return per unit of worst pain |
| Exposure | mean of gross positions / equity | how much of the account was at work |
| Trades | positions opened and closed | sample size: under ~30, distrust everything else |
| Win ratio | share of closed positions with P&L > 0 | how often it is right; meaningless alone |
| Avg win / avg loss | mean return of winners / losers | how big the rights and wrongs are |
| EV per trade | mean return on the capital committed to a position | the expected value of one more trade |
| Profit factor | gross profit ÷ gross loss | above 1 makes money; 1.5+ is healthy |
| Up periods | share of periods with a gain | consistency |
| Unaffordable | option trades skipped because one contract cost more than the per-trade budget | account-size reality |
Win ratio and EV must be read together. A straddle that wins 47% of the time with +10% EV is a good bet; a condor that wins 61% with −3% EV is a bad one.
In the summary's "chained" rows the periods are placed back to back (each starts flat), and the curve-based metrics come from that combined curve. Trade metrics pool every closed position.
#The other backtesters, briefly
| Command | Main outputs |
|---|---|
sd_backtest study [--diag] / grid / anatomy / magnitude / verify / noise / fit | chapters 2–4; verify gives the headline numbers |
sd_direction [--h 5|10|21|63] [--excess] [--noise x] [--ship] | chapter 8; --ship refits SD's side model |
sd_options [--noise x] [--trades f.csv] [--sweep] [--real-iv-only] | chapter 5; per-trade results by era and IV source |
sd_timing research | chapter 13; limit-order timing |
#Recipes
Add a stock strategy. Implement Allocator in src/strategy/book.rs:
targets() returns (symbol index, weight) pairs, or None to keep
holdings. Add it to allocators() and run sd_year. It will appear in
every table and chart. Use DayView::live() to read today's SD; never index
past v.day.
Add an option strategy. Implement Strategy in src/strategy/mod.rs
(chapter 5's recipe). Add it to catalog() (monthly) or short_catalog()
(few-day), then run sd_year and sd_options. Always add its "Always"
baseline: an SD strategy has to beat the same structure opened without SD.
Test one month. sd_year --months 2024 gives every month of 2024. Look at
reports/year/equity_2024-03.csv for one month's daily equity.
Test a smaller account. sd_year --account 5000. Check the
"unaffordable" count before you trust the CAGR.
Check robustness. Rerun with --noise 0.25 and --noise 0.5. A real
effect survives some noise. One that vanishes was fitting the exact prices.
#Pitfalls this backtester will not save you from
- Survivorship. The universe is 125 names chosen today. Stocks that died are missing, which flatters anything that buys volatile names.
- The IV proxy for single stocks (chapter 11). Option results on single stocks are an upper bound.
- Dividends are in the adjusted prices (good for stocks), but options do not model early exercise.
- Capacity. Fills are assumed at modelled prices with fixed costs, regardless of size.