SD book

7. Working here

I'm working on SD itself · 6/6 I'm working on the backtester · 6/6 I want to change the indicator · 4/4 I want to change a trading strategy · 5/5 I run Thatha Stocks · 2/2

3 min read

#Setup

cd Implementations/rust
cargo build --release                         # Rust 1.95 (NixOS system cargo)
cargo test --release                          # unit tests: indicators, clock, models, pricing, sentiment
cargo run --release --bin sd_fetch_history    # ~125 daily histories into data/history/ (skips existing)

Implied-volatility indices live in data/iv/ (fetch with sd_fetch_history --out ../../data/iv ^VIX ^VXN ^VXD ^GVZ ^OVX).

#Every command

CommandWhat it does
sd2 AAPL SPY / sd2 --csv bars.csv / --jsontoday's SD v2 reading and options setup
sd_backtest study [--diag]direction: classic vs v2, every target, plus per-feature AUC
sd_backtest gridhand-built squeeze setups, both eras
sd_backtest anatomymove size across the squeeze life cycle
sd_backtest magnitudeablation: which ideas help predict size
sd_backtest fitrefit and freeze src/sd2/sd2_model.json (then rebuild)
sd_backtest verifyshipped v2 vs classic on 2019+
sd_backtest noiseGaussian noise sweep, 5 seeds per level
sd_options [--noise x] [--trades f.csv]options strategies by era and IV source
sd_options --sweepthreshold selection on 2001–2018
sd_direction [--h N] [--excess] [--ship]walk-forward direction search (chapter 8); --ship refits SD's side
sd_year [--years A-B | --months Y] [--account N]trade full years / months, every strategy, all metrics and charts (chapters 11–12)
sd_timing …long-term P(up) and entry/exit limits (chapter 13)
sd_live, sd_feedthe live runner's events and the trade.using.rs feed, at real option prices (chapter 14)
portfolio_selector_sentimentThatha's selector plus sentiment (run from the Thatha repo root)
offline_sd, sd_cache, the analyzersunchanged v1 tools (see OFFLINE_GUIDE.md)

Switches: --classic-comp, --ao-flip (sd_backtest: run v2 features with v1's compression or flip rule), SD_VOL=vol20|robust (expected-move volatility), SENTIMENT_WEIGHT, SENTIMENT_SHORTLIST.

#File map

Implementations/rust/src/
  indicators.rs, sd.rs          v1: indicators, latest-bar SD (unchanged)
  sd2/
    primitives.rs               rolling tools
    squeeze.rs                  the squeeze clock (shared by v1 and v2)
    classic.rs                  v1 for every bar (the baseline)
    features.rs                 v2 measurements
    model.rs                    logistic + ridge, no dependencies
    engine.rs                   the SD v2 number
    sd2_model.json              frozen fitted model (embedded at compile time)
  strategy/
    mod.rs                      Strategy trait + every strategy
    pricing.rs                  Black-Scholes, skew, implied move
    sim.rs                      the bookkeeper: fills, costs, marks, summaries
  direction.rs                  direction features (10 families) and boosted trees
  timing.rs                     long-term P(up), entry/exit limits
  live.rs                       today's data, real option chains, the IV log
  strategy/book.rs              full-period trading, allocators, metrics, SVG charts
  history.rs                    dated, adjusted daily bars (+ real-price scale)
  research.rs                   universe, noise, samples, splits, metrics
  sentiment.rs                  analyst + headline sentiment
  bin/sd2.rs, sd_backtest.rs, sd_options.rs, sd_fetch_history.rs, sd_direction.rs,
      sd_year.rs, sd_timing.rs, sd_live.rs, sd_feed.rs, portfolio_selector_sentiment.rs
Indicator/
  SahastaDirectionalitySTUDY.ts       v1 chart study (unchanged)
  SahastaDirectionalityV2STUDY.ts     v2 lower study
reports/                              saved outputs of every experiment
docs/                                 this book

The read-only files in src/bin/ (portfolio_selector.rs, buy_analyzer.rs, …) are the reference copies the Thatha pipeline was built from. Leave them read-only; make variants as new binaries, as portfolio_selector_sentiment.rs does.

#Recipes

Change an indicator idea. Edit features.rs → sd_backtest study --diag and magnitude → if it helps in both eras: fit, rebuild, verify, noise, sd_options → copy any changed constants into the v2 ThinkScript.

Refit the model after new data. sd_fetch_history --force, sd_backtest fit, rebuild, verify. Keep valid_end() at 2019-01-01 unless you are deliberately re-running the whole study with a new held-out era. Moving it later means grading on data the model has seen.

Update ThinkScript after a refit. Print the raw coefficients:

python3 -c "import json; m=json.load(open('src/sd2/sd2_model.json')); g=m['magnitude']
raw=[w/s for w,s in zip(g['weights'],g['scaler']['std'])]
print(g['intercept']-sum(r*u for r,u in zip(raw,g['scaler']['mean'])), raw, m['intensity_lo'], m['intensity_hi'], m['iv_proxy'])"

and paste them into the expectedSigma, intensity and ivProxy lines.

Add a strategy. See chapter 5's recipe.

#Tests that guard the important promises

TestPromise
classic::matches_sd_rs_on_the_last_barthe per-bar v1 equals the live v1
squeeze::*firing, persistence of "fired", momentum ends a fire
model::*the regressions recover known answers
pricing::put_call_parity_holds, atm_call_matches_the_textbook_valueBlack-Scholes is right
sentiment::silence_is_neutral, thin_coverage_is_shrunkno coverage never moves a stock