← Back to Dashboard

Strategy ROI Audit v3 β€” Verified Findings + Audit Trail

πŸ›‘ Jul 5, 2026 update β€” RuleB ALSO RETRACTED (see Β§16; Finding #330)

The Apr-25 banner below said "Only RuleB survives". That is no longer true. The Jul 5 strategy review found simulate_window_only filled window-blocked stops at the STOP PRICE after the price had gapped through β€” unobtainable under the as-shipped #305 cancel-and-rearm mechanism. Gap-aware (rearm fills at the first post-window bar OPEN, the same rearm_gap_open handling that retracted FA_hard): RuleB = βˆ’$209/100 headline / βˆ’$194/100 ex-top-2 (n=129) β€” FAILED-GAUNTLET; RuleG also flips negative. NO stop-loss rule variant currently survives. RuleB Phase-2 active flip CANCELLED; v3 D / v3 E lose their RuleB dependency (re-evaluate as pure selection rules). Full record: Β§16 + canonical narrative docs/FINDINGS.md Finding #330. All RuleB-survives statements in Β§Β§4-14 below are the historical pre-#330 record.


⚠️ Apr 25, 2026 update β€” Finding A RETRACTED (see Β§10)

The comprehensive Finding-A simulator (scripts/analyze_finding_a_full.py, Apr 25) showed that v3 Finding A β€” Block stops outside RTH is either inert (rule as written, software-only gate while hardware STP-LMT stays outsideRth=True per Finding #179) or net-negative (hardware also gated: βˆ’$418 / 100 trades, opposite sign to the audit's claim). Findings D / G also fail the 4-check bias gauntlet on the simulator's apples-to-apples baseline. Only RuleB (Β±90 min window) survives, at a much smaller +$123 / 100 trades (πŸ›‘ superseded Jul 5, 2026 β€” see the banner above). Full retraction + replacement queue in Β§10. Canonical narrative: docs/FINDINGS.md Finding #228. New forward-looking source of truth (this doc + dashboard Strategy Analysis tab + simulator data combined): docs/STRATEGY_PLAYBOOK.md.


Purpose: Permanent record of the v3 audit pass (Apr 22, 2026) so future analyses can see what was tested, what survived bias-checks, and what was retracted. Mirrors the canvas at ~/.cursor/projects/Users-marius-projects-polymarket-bot/canvases/strategy-roi-audit.canvas.tsx (Cursor-local, not in git).

Read order: Β§1 process learning β†’ Β§2 v2 retractions β†’ Β§3 v3 verified findings β†’ Β§4 ranked recommendations β†’ Β§5 audit trail (every hypothesis tested) β†’ Β§6 statistical caveats β†’ Β§10 Finding A retraction (Apr 25).


1. Process learning β€” the H-rule retraction

In v2 the "ATR > 5% pre-screen" rule looked like a clean +$329 winner when measured on the 49 stopped-correct trades alone. The full-sample re-check killed it: across all 109 trades the same rule would have removed +$163 of profit (mostly SHORT alpha) and turned the strategy net βˆ’$148 worse.

The diagnostic showed something deeper: 3 trades β€” GPRO (+$111), DELL (+$80), SWBI (+$70) β€” together carry ~$260 of the strategy's $61 net P&L. Multiple v2 "findings" (VIX 22+ is best, L 70-80% is the only profitable LONG band, Thursday is the alpha day, Mid-cap-SHORT bucket) all decompose into "the same 3 outliers happen to land in this slice." Without those 3 winners, the strategy is roughly βˆ’$200.

The 4-check gauntlet (now mandatory for every new finding)

  1. Raw signal + n adequate β€” n β‰₯ 12 minimum, ideally n β‰₯ 20.
  2. Drop top-2 outliers β€” does the finding survive without them?
  3. De-dupe by unique ticker β€” does the finding survive when repeated evaluations of the same ticker are counted once?
  4. Full-population spillover β€” does applying the rule break un-targeted trades? (Especially critical for skip rules.)

Anything that fails any check is rejected regardless of how good the isolated number looks.

The H-rule full-impact table (the smoking gun)

Run via scripts/analyze_atr_prescreen.py against all 109 closed trades:

ATR cutoff Kept Skipped Skip % Kept P&L Skipped P&L "Savings" if skipped
4 % 64 44 40 % βˆ’$68 +$144 βˆ’$129 (worse)
5 % 77 31 28 % βˆ’$87 +$163 βˆ’$148 (worse)
6 % 85 23 21 % βˆ’$26 +$102 βˆ’$87 (worse)
8 % 104 4 4 % βˆ’$16 +$92 βˆ’$77 (worse)

The skipped trades carry the strategy's profits, not its losses. By-direction at the 5 % cutoff:

Direction Skipped Skip % skip_pnl keep_pnl
LONG 24 of 93 26 % +$18 (neutral) βˆ’$82
SHORT 7 of 16 44 % +$146 (the alpha) βˆ’$21

By timing at 5 %: AMC: skip 24/49 (49 %), skip_pnl = +$126 β€” would amputate half the AMC book including the names that drive AMC SHORT alpha.

Lesson: had we shipped rule H based on the isolated stopped-correct slice (where it showed +$329 saved), the strategy would have gone net βˆ’$148 across the full population.


2. v2 findings β€” bias re-check verdict

v2 # Finding Verdict Action
1 Downtrend regime + LONG = bleed ROBUST β€” 17/22 negative; ex-worst-2 still βˆ’$80; top losers are 5 distinct tickers KEEP. Ranked #1 in v3.
2 Stops harvest noise on correct calls ROBUST β€” validated by simulator: rule B saves $160 / D saves $218 KEEP. v3 #2 refines the WHEN: extended hours.
3 Downtrend + SHORT = perfect record (3/3, +$173) Direction robust, magnitude GPRO-driven. Ex-GPRO: 2 trades, +$62 REPLACE with v3 #5 (skip SHORT in non-downtrend, 12/12 negative).
4 VIX 22+ is the BEST bucket (+$147) FRAGILE β€” 64% of bucket P&L from GPRO + SWBI. Ex-them: +$36 across 12 trades RETRACT. Replaced by v3 #6 (BMO + VIX < 18 sweet spot).
5 90%+ LONG conviction destroys value ROBUST β€” 22 of 26 negative; ex-best-2 still βˆ’$73 KEEP. v3 #4 extends to β‰₯80% for Mid-cap.
6 L 70-80% is the only profitable LONG band (+$108) FRAGILE β€” 82% of bucket P&L from SWBI. Ex-SWBI: +$15. Ex-SWBI+DAL: βˆ’$17 RETRACT as a sizing-up signal. Keep as "don't penalize" baseline only.
7 Counterfactual cost of skipped signals PARTIAL β€” small_cap_long_above_80 (+$1,558) is 9 evals of 1 ticker (MBWM) KEEP methodology, retract that specific claim.
8 Sample-size gates (meta) Confirmed β€” half of v2 sub-segments had n < 10 PROMOTED to permanent process rule.

3. v3 verified new findings (all post-bias-check)

Finding A β€” Block all stop-loss orders outside RTH (09:30–16:00 ET)

πŸ›‘ RETRACTED Apr 25, 2026 β€” comprehensive simulator showed the rule as written is inert (software gate; hardware STP-LMT stays outsideRth=True) and the only effective form (outsideRth=False extension) is βˆ’$418 / 100 trades, not the claimed +$300–$400 / 100 trades. The audit treated Rule B as a proxy and conflated the per-session observation table (which measures where stops fired, not what would have happened if they hadn't) with counterfactual evidence. See Β§10 below for the simulator's full output and docs/FINDINGS.md Finding #228 for the canonical retraction narrative.

Mechanism (as originally claimed): Pre-market and after-hours liquidity is thin; STP-LMT fills at bad prices then stocks revert during RTH.

Evidence: | Session | n | Stops | Total $ | |---|---:|---:|---:| | Pre-mkt 04:00–09:30 ET | 22 | 16 (73%) | βˆ’$242 | | RTH 09:30–16:00 ET | 63 | 18 (29%) | +$428 | | After-mkt 16:00–24:00 ET | 25 | 16 (64%) | βˆ’$138 |

Bias check: No outlier dominance β€” 32 stops across 47 ext-hours exits are spread across distinct tickers. Mechanism is structural (liquidity) not selection. Full-population spillover is positive (RTH cell is +$428).

Implementation: Gate stop-trigger check on session-of-now in order_executor; hardware STP-LMT order can stay placed; software gate just refuses to act on triggers outside RTH. ~15 lines.

Estimated lift: +$300 to +$400 / 100 trades.

Stop-counterfactual rule comparison (per-trade simulation, n=49 stopped+correct)

Full simulation results from scripts/analyze_stop_counterfactual.py --population correct (Polygon 1m bars, exact replay vs alternative stop-loss policies). Live total P&L on these 49 trades: βˆ’$584.

Rule Total $ Ξ” vs live Stops Skipped Better / Worse / Same
A β€” Baseline (current) n/a β€” 48/48 β€” sanity check passed
B β€” Β±90 min no-stop window around release βˆ’$414 +$170 43/49 β€” 34 / 3 / 12
C β€” ATR(14) k=1.5 stop βˆ’$510 +$74 29/49 β€” 21 / 28 / 0
D β€” ATR(14) k=2.0 stop βˆ’$366 +$218 19/49 β€” 26 / 23 / 0
E β€” ATR(14) k=2.5 stop βˆ’$444 +$140 16/49 β€” 27 / 22 / 0
F β€” No stop at all (always scheduled exit) βˆ’$486 +$98 0/49 β€” 28 / 21 / 0
G β€” Window + ATR k=2.0 (combined B + D) βˆ’$362 +$221 18/49 β€” 26 / 23 / 0
~~H β€” Window + skip-if-ATR > 5%~~ ~~βˆ’$254~~ ~~+$329~~ ~~26/49~~ ~~19/49~~ ~~42 / 2 / 5~~

~~H~~ retracted per Β§1 β€” the +$329 was a selection-bias artifact; full- population check showed βˆ’$148 net. Surviving candidates ranked by risk-adjusted lift: B is the safest (only 3 trades worse than live, no selection change, simplest implementation); D adds extra savings at the cost of 19 still-stopped trades exiting at wider losses; G is the combined optimum.

The chosen v3 Finding A is a stronger version of B β€” instead of just a Β±90-min window around the earnings release, it blocks ALL stops outside RTH (09:30–16:00 ET). Rationale: 80 % of stop bleed (βˆ’$380 of βˆ’$549) came from extended-hours exits regardless of how close to the release the stop fired. Per-session detail in the table at the top of this section.

Wrong-prediction validation (does the rule still catch disasters?)

Run via scripts/analyze_stop_counterfactual.py --population wrong (n=16 trades where prediction_correct=FALSE):

Rule Total $ Ξ” vs live
Live baseline βˆ’$24 β€”
B β€” Β±90 min no-stop window +$131 +$155
D β€” ATR k=2.0 +$61 +$85
F β€” No stop at all +$107 +$131
G β€” Window + ATR +$71 +$95

Methodological caveat: the simulator does NOT model the bot's instant_exit:earnings_miss path (the existing earnings-detector that catches most wrong predictions ≀ 60 s after release). 12 of 16 live wrong-prediction exits used that path, not the stop. So the wrong-pop $ deltas are upper bounds. The important conclusion is the direction: every alt-stop rule is positive on the wrong-prediction set, meaning none of them remove the strategy's ability to catch genuine disasters. The existing earnings_detector does the heavy lifting on wrong calls; the stop-loss is the safety net behind it.

Per-trade detail: scripts/stop_counterfactual_correct.csv (49 rows, 35 columns) and scripts/stop_counterfactual_wrong.csv. Columns include A_baseline_*, B_window_*, C_atr15_*, D_atr20_*, E_atr25_*, F_nostop_*, G_combo_*, H_*, atr_daily, atr_pct_of_entry. Sort by {rule}_pnl_d - live_pnl_dollars to inspect winners vs losers per rule.


Finding B β€” Skip LONGs in downtrend regime (kept from v2 #1, validated)

Evidence: 22 LONG trades in downtrend: 5 wins, βˆ’$136, 14 stops (64% stop rate). Same ticker universe in uptrend: 34 trades, 15 wins, +$71, 32% stop rate. Half the win rate, double the stops.

Bias check: 17 of 22 negative; ex-worst-2 (BABA βˆ’$32, GME βˆ’$25) still βˆ’$80; top losers are 5 distinct tickers (no single name dominance).

Implementation: Skip rule on signal_generator using MarketRegimeProvider.regime. ~10 lines.

Estimated lift: +$140 / 100 trades.


Finding C β€” Skip LONGs with pre-event drift β‰₯ 2% from prior-day close

Mechanism: When a stock has already drifted into our LONG direction overnight, the EPS expectation is "priced in" β€” the small additional move on the announcement isn't enough to cover commission + slippage + stop risk.

Evidence: | Drift bucket | n | Wins | Total $ | |---|---:|---:|---:| | LONG drift < 2% | 75 | 31 (41%) | +$107 | | LONG drift 2–5% | 13 | 1 | βˆ’$120 | | LONG drift β‰₯ 5% | 2 | 0 | βˆ’$25 |

Bias check: 14 of 15 are losers; only EBAY (+$5.64) is positive. Ex-best-1 the bucket is βˆ’$150 across 14 losers. Monotonic β€” the more drift, the worse the outcome. No outlier rescue.

Implementation: New skip_reason='pre_event_drift_long' in signal_generator; uses earnings_markets.stock_price_day_before_close already populated by Finding #227 resolver (713/715 past markets covered). ~15 lines.

Estimated lift: +$145 / 100 trades.


Finding D β€” Skip Mid-cap LONGs at conviction β‰₯ 80%

Evidence: | mcap Γ— conviction | n | Wins | Total $ | |---|---:|---:|---:| | Mid LONG 80–90% | 31 | 11 (35%) | βˆ’$56 | | Mid LONG 90–95% | 15 | 1 (7%) | βˆ’$55 | | Large LONG 80–90% | 9 | 4 (44%) | +$22 |

Bias check: Both Mid sub-buckets independently negative across n β‰₯ 15. Mid is 70% of our trade count, so this is a high-leverage rule. Large 80-90% is positive β€” keep that bucket; only Mid is the problem.

Implementation: Extension of v2 finding #5 (90%+ LONG skip across all mcaps) lowered to 80% for Mid only. ~5 lines added to existing skip logic.

Estimated lift: +$110 / 100 trades.


Finding E β€” Skip SHORTs in non-downtrend regimes (replaces v2 #3)

Evidence: SHORT in uptrend: 1/7 wins, βˆ’$2. SHORT in sideways: 0/5 wins, βˆ’$27. Combined non-downtrend: 0 wins out of 12 (excluding 1 marginal $0.47 outlier), βˆ’$63. SHORT in downtrend: 3/3 wins (GPRO-dominated +$173 but ALL 3 winners across distinct tickers).

Bias check: 12/12 negative on the skip side β€” perfect direction signal. The downtrend SHORT positive side is dollar-dominated by GPRO, so the estimated dollar lift on the skip rule is conservative.

Implementation: Conditional gate: skip SHORT when regime != 'downtrend' β†’ skip_reason='short_non_downtrend'. Replaces existing "max 2 SHORTs" rule (which was over-cautious).

Estimated lift: +$60 / 100 trades + lower tail risk.


Finding F β€” BMO + VIX-bucket sizing

Evidence: | BMO Γ— VIX | n | Wins | Stop rate | Total $ | |---|---:|---:|---:|---:| | BMO + VIX < 18 | 18 | 11 (61%) | 17% | +$100 | | BMO + VIX 18–22 | 30 | 11 (37%) | 53% | βˆ’$54 | | BMO + VIX 22+ | 6 | 1 | 17% | +$19 |

Bias check: BMO+VIX<18 cell ex-top-3 (STT, VZ, ALLY) still +$32 across 15 trades. Loser cell ex-top-3 (WING, BLK, GRMN) is βˆ’$160 across 27 trades β€” LOSERS are pervasive, wins concentrated. Both ends robust.

Implementation: risk_manager weighted multiplier on VIX bucket for BMO trades only: VIX < 18 β†’ 1.2Γ— base; VIX 18–22 β†’ 0.6Γ—; VIX β‰₯ 22 β†’ 1.0Γ—. AMC unchanged (no clean VIX signal there). ~10 lines.

Estimated lift: +$60 / 100 trades.


4. Ranked recommendations (revised Apr 25, 2026 β€” post-simulator)

⚠️ Original §4 ranking was built on the un-validated v3 Finding A and on the pre-simulator estimates for D / G. The Apr 25 simulator killed Finding A and showed D/G fail the 4-check gauntlet. The new ranked queue below is the operative one. Original rows kept inline (struck through) for audit history.

# Recommendation Source Est. lift / 100 trades Status
1 Skip LONGs in downtrend v2 #1 / v3 Finding B +$140 NEW #1 β€” selection rule, robust to outlier removal, no execution dependency
2 Skip LONGs with pre-event drift β‰₯ 2 % v3 Finding C +$145 NEW #2 β€” needs stock_price_day_before_close on signal-time signal_generator (already populated by Finding #227 resolver on 713/715 past markets)
3 Skip Mid-cap LONGs at conviction β‰₯ 80 % v3 Finding D +$110 overlaps with #2 β€” measure post-#1+#2 before shipping
4 Skip SHORTs in non-downtrend regimes v3 Finding E +$60 small dollar lift, perfect 12/12 direction signal, low-risk addition
5 Daily 3-stop session breaker v2 #6 tail-risk cap LOW; behavioral cap on bad-session escalation
6 BMO + VIX-bucket sizing v3 Finding F +$60 sizing rule; layer last after skip rules settle
7 Skip all LONGs at conviction β‰₯ 90 % v2 #5 (overlaps with #3) superseded by tighter Mid-cap variant
8 RuleB (Β±90 min stop window around release) Β§10 simulator +$123 only stop-loss rule that survived simulator + bias gauntlet; small lift, ship only after selection rules
~~ ~~ Block stops outside RTH ~~v3 Finding A~~ ~~+$300–$400~~ πŸ›‘ RETRACTED Apr 25 β€” simulator: βˆ’$418/100 (hard) or $0 (soft, inert)
~~ ~~ ATR k=2.0 / window+ATR ~~v3 alternatives~~ ~~+$200~~ πŸ›‘ FAIL bias gauntlet β€” outlier-driven; ex-top-2 = βˆ’$103 / βˆ’$109

Combined realistic lift (post-retraction): Skip rules #1+#2+#3+#4 overlap (all subset-of-LONGs); realistic combined lift is +$300–$450 / 100 trades. RuleB adds a marginal +$50–80 net of overlap. Total addressable improvement is +$350–$500 / 100 trades, not the +$500–700 the original v3 Β§4 claimed (the difference is the retracted Finding A and the gauntlet-failed D/G).

The combined queue still flips the strategy from βˆ’$60 / 100 trades (current) to roughly +$300 to +$440 / 100 trades, but via selection fixes (cut bad trades upstream) rather than execution fixes (try to rescue already-bad trades).

Implementation order (revised): Ship in this order so each rule's lift is attributable:

  1. Skip LONGs in downtrend (Finding B) β€” single SQL gate in signal_generator, ~10 lines, uses existing MarketRegimeProvider.
  2. Skip LONGs with drift β‰₯ 2 % (Finding C) β€” uses already-populated earnings_markets.stock_price_day_before_close, ~15 lines.
  3. Skip Mid-cap LONGs at conviction β‰₯ 80 % (Finding D) β€” extension of existing 90 % skip, ~5 lines.
  4. Skip SHORTs in non-downtrend (Finding E) β€” replaces over-cautious "max 2 SHORTs" rule.
  5. Daily 3-stop session breaker β€” tail-risk circuit breaker.
  6. VIX-bucket sizing for BMO (Finding F) β€” sizing layer.
  7. RuleB Β±90 min window β€” only after selection rules settle and we want a marginal stop-loss tweak.

Re-run scripts/strategy_performance_analysis.py and scripts/analyze_finding_a_full.py after each deploy. Re-validate the entire audit at trade #150+ via the 4-check gauntlet (memory Β§K).


5. Audit trail β€” every hypothesis tested

ID Hypothesis n / data Verdict Reason
H1 Skip LONGs with drift β‰₯ 2% pre-event n=15 ACCEPT (Finding C) 1/15 wins, no outlier rescue, full-pop check passes
H2 Conviction velocity (Ξ”prob_30min βˆ’ prob_24h) n=216 INFORMATIVE only Just confirms the existing conviction signal
H3 Same-day earnings clustering (n_trades_per_day) n=109 REJECT Non-monotonic; cluster=2 worst (βˆ’$129) but cluster=5 +$83 β€” noise
H4 Day-of-week effect (Thursday alpha) n=109 REJECT +$209 on Thu but ex-top-3 (GPRO/DELL/SWBI) becomes βˆ’$53
H5 Flex-sizing penalty n=109 REJECT Avg position % below configured (Large 19% vs 30%) β€” flex barely fires
H6 Spread-at-entry filter β€” NO DATA spread_at_entry NULL on 110/110 β€” wiring missing
H7 Conviction stability across pre-event snapshots β€” DEFER Need full 5-snapshot rows; not enough resolved markets yet
H8 Conviction at exit drop signal n=4 NO DATA conviction_at_exit NULL on 106/110
H9 Volume / depth quality at entry β€” DEFER Need post-event movement check
H10 Same-ticker prior-trade history n=109 DEFER Only 5 tickers traded > once; sample too small
H11 Mid-cap subgroup concentration n=72 PARTIAL β†’ H18 Folded into mcap Γ— conviction interaction
H12 Sector concentration of losses β€” DEFER No sector field on trade_audit_log
H13 AMC SHORT-only signal (+$121, n=10) n=10 REJECT as standalone Same GPRO/SWBI/STUB outliers; folded into H3-extended
H14 EPS surprise magnitude β†’ stock move correlation n=210 DEFER Compute surprise % then bucket β€” next pass
H15 Quick-stop vs slow-stop dynamics n=48 PARTIAL β†’ H22 Folded into 'extended-hours stop' finding
H16 Regime-conditional 90%+ LONG (does it work in uptrend?) n=26 REJECT 22/26 negative across all 3 regimes
H17 BMO + VIX < 18 sweet spot n=18 + n=30 ACCEPT (Finding F) Both cells robust to outlier removal
H18 Mid LONG β‰₯ 80% conviction concentration n=47 ACCEPT (Finding D) Both 80-90% and 90-95% independently negative
H19 Pre-event after-hours move (BMO) β€” DEFER Need IBKR pre-mkt quote at signal time
H20 Same-week clustering n=109 REJECT Same as H3
H21 Slippage by execution exchange n=109 REJECT DARK β‰ˆ IBKRATS (basically identical)
H22 Hold-duration vs P&L β†’ extended-hours stop pattern n=109 ACCEPT (Finding A) Pre/after-mkt exits are 42% of trades but 80% of stop bleed
H23 Earnings-detector latency cost n=7 DEFER Too small. Re-test at n=20+
H24 PM post-event yes_price as instant-exit accelerator n=12 DEFER Existing earnings_detector catches wrong calls at small loss

6. Audit-process learnings (for next pass)

  1. Always run the full-population check before recommending a skip rule. The H-rule (ATR > 5% pre-screen) showed +$329 on the 49-trade subset and would have lost $148 net across 109 trades. Skip rules in particular need full-population spillover analysis because they remove WINNERS as well as losers.

  2. The 3-outlier curse. GPRO + DELL + SWBI together dominate ~$260 of the strategy's $61 net P&L. ANY slice that includes them looks great; ANY slice that excludes them is structurally negative. Always report numbers ex-top-2 alongside the headline.

  3. Repeated-ticker bias. The MBWM finding in v2 was 9 evaluations of 1 ticker presented as "9 wins." Every counterfactual table needs a unique_tickers column alongside n_total.

  4. Mechanism beats correlation. The findings that survived (extended-hours stops, drift filter, BMO + low VIX) all have a structural mechanism behind the data (liquidity gaps, price-priced-in, low pre-earnings vol). The retracted ones (VIX 22+ best, Thursday alpha) were correlation-only and traced back to the same outlier names.

  5. The defer queue is a feature, not a failure. 11 of 24 hypotheses ended up DEFERRED. They're not bad ideas; they need either better data capture (H8 conviction_at_exit, H19 pre-event after-hours quote) or larger n (H10 same-ticker, H23 earnings-detector latency). Re-run them in 50 trades.


7. Statistical caveats

n=109 closed trades + n=126 calibrated markets means Β±5–8% on headline win-rate slices, Β±10–15% on regime-conditional and VIX buckets. Findings A, B, C, D are well-sampled (n β‰₯ 15 each) and robust to outlier removal. Finding E (skip non-downtrend SHORT) is 12/12 negative β€” perfect direction but small dollar impact. Findings F (BMO + VIX < 18) and the kept v2 #7 (90%+ LONG skip) are well-sampled.

Re-validate the entire audit after the next 50 trades with scripts/strategy_performance_analysis.py per memory Β§K. The bias-check gauntlet (4 checks per hypothesis) should be re-run every audit pass.


8. Cross-references β€” full artifact catalog

Canvas (interactive, Cursor-only)

Audit scripts (rerunnable)

Script Purpose Outputs
scripts/analyze_stop_counterfactual.py Per-trade replay against alt stop policies (B/C/D/E/F/G/H). Supports --population correct\|wrong\|all-stop. scripts/stop_counterfactual_correct.csv (49 rows Γ— 35 cols), scripts/stop_counterfactual_wrong.csv (16 Γ— 35)
scripts/analyze_finding_a_full.py Comprehensive Finding-A simulator (Apr 25, 2026 follow-up). Replays the actual proposed rule (block stops outside 09:30–16:00 ET, re-arm at RTH open with gap-open slippage modeling) on the full closed-trade population. Also evaluates Rules B / D / G / F on the same bars for cross-comparison. Distinguishes FA_soft (software-only gate; hardware STP-LMT outsideRth=True still fires β€” INERT for P&L) vs FA_hard (hardware also gated). Models earnings_detector cap on wrong-prediction trades. Built-in 4-check gauntlet (headline / ex-top-2 / ex-repeat-tickers / by-correctness / by-action / pre-EPS-only) per rule. Halt-suspect detection. Run BEFORE shipping Finding A. scripts/finding_a_full.csv (one row per trade Γ— all rules) + stdout aggregate
scripts/analyze_atr_prescreen.py ATR(14) pre-screen impact across full trade history at multiple cutoffs (4/5/6/8 %). The script that killed the H-rule. scripts/atr_prescreen.csv (109 rows with atr_daily, atr_pct, skip_at_*pct flags)
scripts/strategy_performance_analysis.py Full strategy-performance report with regime / VIX / conviction / skipped-cost sections. Memory Β§K source of truth. scripts/strategy_analysis_report.json (consumed by dashboard Strategy Analysis tab)
scripts/backfill_learning_loop.py One-shot historical backfill of the resolver across all earnings_markets / skipped_signals / opportunity_log rows. DB writes; logs to /var/log/polymarket/backfill_learning_loop.log
scripts/backfill_regime_for_audit_log.py Fills regime_at_entry + vix_at_entry on existing closed trades. DB writes
scripts/compute_counterfactual_pnl.py Operator-only counterfactual P&L on opportunity_log rows (pre-resolver tool). DB writes

Live data infrastructure (Finding #227)

Module Role
src/data_tracker/collectors/outcome_resolver.py Daily 06:00 ET timer; 4-pass cascade fills actual_beat, eps_actual, revenue_actual, stock_price_*, hypothetical_pnl, prediction_correct. Source priority: EPS surprise β†’ post-event Polymarket yes_price β†’ pre-event prob cascade.
src/earnings/market_regime_provider.py Single source of truth for SPY regime (uptrend/sideways/downtrend) + VIX + SPY change. Used by signal_generator at insert time.
src/earnings/counterfactual_pricing.py Polygon 1m bar fetch + simulate_minute_path (entry/exit/stop replay). Used by both the resolver and analyze_stop_counterfactual.py.
src/data_tracker/migrations/022..024 Schema for regime/intended-size/counterfactual columns.

Documentation

Doc What's there
docs/FINDINGS.md Finding #227 Canonical narrative for the entire learning-loop / data-infrastructure work (per memory Β§J).
docs/DATA_ARCHITECTURE.md Β§4 Outcome resolver runbook + 4-tier actual_beat cascade documentation.
docs/DATABASE_DIRECTIVES.md Writer-ownership table for actual_beat, vix_at_entry, etc.
docs/OPERATIONS.md Polymarket-resolver service + manual backfill commands + Polygon plan guidance.
docs/STRATEGY.md Β§6 / Β§10 Pointers to MarketRegimeProvider as the regime source of truth.
.cursor/memory.md Β§M Cross-interface rule: any new use of regime/VIX as a gate must import from MarketRegimeProvider.

Operational notes captured elsewhere


9. Open questions / DEFER queue

11 hypotheses ended up DEFERRED rather than rejected β€” they're not bad ideas, they need either better data capture or a larger n. Re-test in the next audit pass (after another 50 trades, ~Aug 2026).

ID Hypothesis What it needs to become testable Estimated re-test trigger
H6 Spread-at-entry filter Wire spread_at_entry capture in signal_generator._build_trading_signal (currently NULL on 110/110). Tracks Polymarket bid-ask spread at signal time. After spread capture wired + 30 trades
H7 Conviction stability across pre-event snapshots Need 5-snapshot rows (prob_24h, prob_12h, prob_6h, prob_1h, prob_30min all populated). Compute stddev(prob_*) / mean(prob_*) per market. 100+ resolved markets with full snapshot coverage
H8 Conviction-at-exit drop signal Wire conviction_at_exit capture in position_manager.persist_position_close (NULL on 106/110). Compares pre-event vs at-exit Polymarket conviction. After exit-time capture wired + 30 trades
H9 Volume / depth quality at entry Need pm_bid_depth_at_entry + pm_ask_depth_at_entry populated more reliably. Currently sparse. After 50 more trades with depth captured
H10 Same-ticker prior-trade history Only 5 tickers traded > once today. Need 20+ repeated tickers to do a meaningful "we get better at known names" analysis. n β‰₯ 50 repeat-ticker trades
H12 Sector concentration of losses No sector field on trade_audit_log. Add GICS sector lookup via Yahoo / Finnhub at entry time, populate retroactively for past trades. After sector field wired + populated
H14 EPS surprise magnitude β†’ stock move correlation Polygon eps_actual now populated on 210 markets. Compute (eps_actual - eps_estimate) / eps_estimate per market and bucket vs stock_return_day_close. Could be done now β€” just needs the analysis script. Could run now; deferred to keep audit scope bounded
H19 Pre-event after-hours move (BMO entries) Need IBKR pre-mkt quote at signal time (15:58 ET BMO entry would benefit from knowing if SPY is moving in after-hours). Not currently stored. After pre-mkt SPY capture wired
H23 Earnings-detector latency cost Only 7 instant_exit:earnings_miss trades to date. Need 20+ to bucket detection_latency_seconds vs P&L. After 30 more wrong-prediction trades
H24 Polymarket post-event yes_price as instant-exit accelerator Sample of 12 wrong-prediction trades shows existing earnings_detector already catches them at small loss; PM signal would be marginal until detector is slow. Re-test if detector latency degrades. If earnings_detector 95th percentile latency exceeds 90 s

One rejection that may want re-test: H4 (day-of-week) was rejected because Thu's +$209 was the same GPRO/DELL/SWBI outliers. Once we have 50+ more trades (and the 3-outlier dominance dilutes), re-run H4 β€” if Thursday is still positive, it might be a real session effect not just outlier coincidence.


10. Finding A retraction β€” comprehensive simulator (Apr 25, 2026)

Why this section exists: The original v3 Β§3 Finding A claim (block stops outside RTH = +$300–$400 / 100 trades) was supported by (a) Rule B as a proxy and (b) a per-session observational table. A follow-up review surfaced 8 unmodelled scenarios. The script scripts/analyze_finding_a_full.py was built to close all 8 and replay the actual proposed rule end-to-end.

10.1 The simulator β€” what it does

scripts/analyze_finding_a_full.py (570 lines, read-only):

10.2 Results (population = stopped-only, n = 51 trades)

Apples-to-apples vs the simulator's A_base baseline (cancels Polygon-bar- vs-IBKR-fill reconstruction noise):

Rule n Ξ” vs A_base per-100 trades better worse
FA_soft (rule as written) 51 $0.00 $0.00 0 0
FA_hard (hardware also gated) 51 βˆ’$213.23 βˆ’$418.10 9 23
RuleB (Β±90 min window only) 51 +$62.97 +$123.47 4 1
RuleD (ATR k=2.0) 51 +$99.23 +$194.57 22 27
RuleG (window + ATR k=2.0) 51 +$102.32 +$200.63 22 27
RuleF (no stop ever) 51 βˆ’$12.77 βˆ’$25.04 24 25

10.3 Mechanism β€” why FA_hard loses money

FA_hard action n Total Ξ” vs A_base Avg / trade
rearm_gap_open (stop blocked in ext-hrs, RTH opened past stop β†’ forced fill at worse open price) 22 βˆ’$416.72 βˆ’$18.94
blocked_to_scheduled (stop blocked, stock recovered fully, exit at sched = profit) 9 +$203.61 +$22.62
earnings_detector_cap 1 βˆ’$0.12 β€”
unchanged (RTH stop / no stop / re-arm intraday) 19 $0 β€”

The 22 worst trades include MDB (βˆ’$82), TTD (βˆ’$58), NKE (βˆ’$54), BABA (βˆ’$38), WDAY (βˆ’$26), NYT (βˆ’$23), RL (βˆ’$18), GTLB (βˆ’$15), HOOD (βˆ’$15), LOW (βˆ’$13), CSCO (βˆ’$12), VSCO (βˆ’$12). Every one of these had the stock continue past the stop level into RTH open. The audit's claim that extended-hours stops "fill at bad prices then stocks revert" assumes reversion that the data does not support β€” for these 22 trades the price moved the wrong direction and stayed there.

The 9 recovery wins (KSS +$66, GME +$33, BBY +$31, KR +$23, JPM +$15, TXRH +$11, WMT +$11, NET +$10, SOUN +$4) are real but outweighed 2-to-1 by the gap-open losses in both trade count and dollars.

10.4 Bias gauntlet β€” D/G also fail

The audit's "best" stop-loss alternative (RuleG: window + ATR k=2.0) showed +$200 / 100 headline. Robustness:

Rule Headline Ex-top-2 Ex-top-4 Ex-top-6
RuleG +$200/100 βˆ’$103/100 βˆ’$163/100 βˆ’$167/100
RuleD +$195/100 βˆ’$109/100 βˆ’$170/100 βˆ’$174/100

Both collapse to net-negative when you drop just two outliers (the same KSS / GME / BBY recovery cluster). They fail bias-check #2 of the gauntlet β†’ rejected as standalone rules.

10.5 What survives β€” RuleB only

Rule B (Β±90 min no-stop window around earnings release):

This is the only stop-loss intervention worth shipping, and only after the selection rules (Findings B/C/D) have been deployed and measured.

10.6 Pre-EPS-trigger only (the audit's strongest claimed scenario)

The audit claimed extended-hours stops fire "on overnight noise before any earnings." Restricting the simulator to the 30 trades whose A_base trigger fired before the earnings release: FA_hard = βˆ’$92, 11 better / 14 worse. Even in this allegedly safest subset, the rule is net-negative. The "noise" hypothesis isn't supported β€” pre-EPS extended-hours stops fire on real adverse moves that continue through RTH open more often than they reverse.

10.7 Halt-suspect cases caught by the simulator

Two trades flagged by the halt heuristic (>30 min gap + >3 % jump in pre/ post-market):

Under FA_hard the bot would have been forced to hold through both, with no ability to act on the price gap until RTH. These are real halt risks not modelled in the audit.

10.8 Cross-references


11. Universe View follow-up (Apr 25, 2026 β€” afternoon refresh)

A second analysis pass extended the audit to the full universe of all 900 Polymarket earnings markets in the database, not just the 109 the bot actually traded. The output is rendered live in the dashboard 🌐 Universe View section (refreshed every report run) and also mirrored in the universe_hit_rate block of scripts/strategy_analysis_report.json.

11.1 What it confirmed

11.2 What it complicated

v3 Finding C (drift filter, Queue #2) β€” verdict: WEAKENED. The original v3 C drift definition (live_entry_price βˆ’ stock_price_day_before_close) collapses to ~0 for BMO trades because the bot enters at 15:58 ET and stock_price_day_before_close is the same day's 16:00 close. Replacing it with a 5-trading-day baseline (universe view Β§ C) and re-bucketing the 253 LONG candidates:

Drift bucket n win % total $10K-notional
drift < βˆ’2% 68 60.3 % +$4,410
drift βˆ’2 to 0% 58 55.2 % +$1,964
drift 0 to 2% 53 49.1 % +$8,168
drift 2 to 5% 50 58.0 % +$3,604
drift β‰₯ 5% 24 50.0 % βˆ’$315

Skip-bucket (drift β‰₯ 2%): n=74, win 55.4 %, total +$3,289, ex-top-2 +$1,136. Skipping these would remove winners, not losers. Live trades with the original drift defn still show drift β‰₯ 2 % is bad (βˆ’$120 on n=13), but the sample is at the n β‰₯ 12 floor and the rule's effect may trace to a different underlying signal (small-caps, low-conviction overlap, etc.) than literal pre-event price drift.

Action: Queue #2 status changed from "IN-QUEUE β€” clean" to "IN-QUEUE β€” drift-baseline question must be resolved before shipping." Three options:

  1. Use a shorter baseline (e.g. 1 trading day prior) β€” closer to the v3 live-trade definition; may reproduce the live signal.
  2. Use the 5-day baseline but tighten the skip threshold (e.g. β‰₯ 5 % only, where the universe shows βˆ’$315 across 24 markets).
  3. Drop Finding C and rely on Findings B + D + E to catch the same trades via different upstream filters.

v3 Finding D (Mid-cap 80%+ skip, Queue #3) β€” verdict: CONTESTED. Universe view Β§ B splits the LONG 80-90 % bucket by market_cap_category:

mcap n win % avg $ total $
Large 36 58.3 % +$10 +$374
Mega 6 50.0 % +$61 +$364
Mid 13 46.2 % +$90 +$1,175
Unknown 40 62.5 % +$263 +$10,503

The Mid bucket is net positive at universe scale, contradicting the live-trade Mid 80-90 % LONG = βˆ’$56 / 31 trades observation. The most likely explanation per Finding #228: stops eat the winners on live Mid-cap trades (held-correct = +5 % / trade vs stopped-correct = βˆ’3.5 % / trade). The Mid bucket may NOT need a skip rule β€” it may need better stop handling (covered by Queue #7 RuleB).

Action: Queue #3 status flagged "CONTESTED" β€” re-evaluate after RuleB ships. If RuleB closes the gap on Mid-cap stops, Finding D may no longer be needed.

11.3 What it didn't change

11.4 Cross-references


12. Apr 25 evening β€” operator decisions + volume-aware Universe View

After Β§11 was written, three additional changes landed in one pass.

12.1 v3 Finding C β€” RETRACTED (option 3 chosen)

Operator decision: drop the drift filter entirely and rely on B + E + the already-IMPLEMENTED selection rules to catch high-drift bad-EV trades via different filters. Reasoning:

Re-test trigger: trade #150+ with a tighter drift definition (e.g. 1-day-prior baseline that approximates the original v3 defn).

12.2 v3 Finding D β€” ON-HOLD (pending RuleB)

New ON-HOLD status added to the Action Queue schema. v3 Finding D moved there because:

Re-evaluation criterion: after RuleB ships AND has accumulated β‰₯ 10 Mid-cap stops in production, re-measure live Mid-cap stop rate and P&L.

12.3 Universe View v3 β€” volume-aware

analyze_universe_hit_rate() extended to:

12.4 New ranked queue (post Apr 25 evening β€” first revision)

# Rule Status
1 v3 B β€” Skip downtrend LONGs IN-QUEUE β€” ship next
2 v3 E β€” Skip non-downtrend SHORTs IN-QUEUE
3 v2 #6 β€” Daily 3-stop session breaker IN-QUEUE
4 v3 F β€” BMO + VIX-bucket sizing IN-QUEUE
5 RuleB β€” Β±90 min no-stop window IN-QUEUE (unblocks v3 D re-eval)
6 Audit re-validation at trade #150+ IN-QUEUE (meta)
⏸️ v3 D β€” Mid-cap 80%+ skip ON-HOLD pending RuleB
πŸ›‘ v3 C β€” drift filter RETRACTED via option 3

Combined realistic lift: +$200 to +$350 / 100 trades.

12.5 v3 E ON-HOLD (later that evening β€” sample-size pushback)

Operator review of v3 E found the rule too aggressive given current data:

Re-evaluation criteria (strictest of): - After RuleB ships AND total SHORT count β‰₯ 30 (currently 18); OR - At trade #150+ regardless (audit re-validation gauntlet).

If at re-evaluation: - Sideways SHORT stop rate drops post-RuleB β†’ close as RESOLVED-BY-OTHER-RULE. - Sideways SHORT remains 0 % WR with high stop rate β†’ resume IN-QUEUE and ship. - Downtrend SHORT alpha is still GPRO-dominated at n β‰₯ 6 β†’ keep ON-HOLD permanently (rule has no reliable KEEP set).

12.6 Final ranked queue (post v3 E ON-HOLD)

# Rule Status
1 v3 B β€” Skip downtrend LONGs IN-QUEUE β€” ship next
2 v2 #6 β€” Daily 3-stop session breaker IN-QUEUE
3 v3 F β€” BMO + VIX-bucket sizing IN-QUEUE
4 RuleB β€” Β±90 min no-stop window IN-QUEUE (unblocks v3 D + E re-eval)
5 Audit re-validation at trade #150+ IN-QUEUE (meta)
⏸️ v3 D β€” Mid-cap 80%+ skip ON-HOLD pending RuleB
⏸️ v3 E β€” Skip non-downtrend SHORTs ON-HOLD pending RuleB + β‰₯30 SHORT trades
πŸ›‘ v3 C β€” drift filter RETRACTED via option 3

Combined realistic lift: +$160 to +$280 / 100 trades (down from +$200-350). Still flips the strategy from βˆ’$60 / 100 trades to +$100 to +$220 / 100 trades β€” driven primarily by Finding B (downtrend LONG skip) plus RuleB and behavioral guards.

Net effect of being conservative on D + E: ~$100 / 100 trades less estimated lift, but we preserve trade volume and avoid committing rules to small-sample data that may be capturing something temporary (capital constraints during a particular regime, the 3-outlier curse, or stops bleed that RuleB will fix).


13. Apr 25 evening β€” data-quality backfill (Findings #229 + #230)

After the queue updates of Β§12, two data-quality issues were uncovered and fixed in succession. They don't change any queue rule's status (the ranking from Β§12.6 is unchanged), but they materially expand the data underpinning every "is this signal valid?" check that depends on the Universe View. Future audits should rely on the post-backfill numbers.

13.1 Finding #229 β€” markets.volume_total silently 0 for resolved markets

Trigger: Universe View showed only 35 % volume coverage (111 of 314 in-band markets). User asked whether this was a real Polymarket gap or a data-pipeline gap.

Root cause: continuous_collector.sync_markets_from_gamma queried Gamma with ?active=true&archived=false. Polymarket's state machine has both active and closed flags β€” settled earnings markets sit in (active=True, closed=True, archived=False) for days/weeks before being archived. The ?active=true filter does NOT include them despite their active=true flag.

Fix: two-pass sync (active + closed-not-archived) in continuous_collector + one-off scripts/backfill_markets_volume.py. Recovered 319 markets / $5.7M of volume.

Result: markets passing the bot's $2K gate jumped 111 β†’ 546.

13.2 Finding #230 β€” earnings_markets.prob_*_before missing for 401 of 900 markets

Trigger: Even after Finding #229, the Universe View was only capturing 31 % of all earnings markets (279/900). The biggest gap was 549 rows with NULL prob_24h_before (401 past + 148 future).

Root cause: Two-layer issue. (1) Orderbook collector (continuous_collector) only starts sampling a market when its condition_id enters the active-markets list, often only a few hours before the earnings event. So the 24h / 12h / 6h-before snapshot windows fall outside the sampled timespan. (2) conviction_tracker --backfill looks for snapshots within Β±30 min of the target time β€” useless when the collector has no rows from that timeframe at all.

Fix: new scripts/backfill_conviction_from_trades.py pulls trade history from Polymarket's data-api /trades endpoint (which goes back much further), normalises each trade's price to YES-token probability (yes_prob = price if trade on YES, 1 βˆ’ price if on NO), and finds the trade closest to each target timestamp (24h/12h/6h/1h/30min before earnings_dt) within Β±60 min.

Result:

Metric Pre-backfill Post-backfill Ξ”
Universe LONG (>60 %, resolved) 253 529 +276 (+109 %)
Universe SHORT (<30 %, resolved) 26 39 +13
Total in-band universe 279 568 +289 (+104 %)
Coverage of all 900 markets 31 % 63 % +32 pp
Past markets with NULL prob_24h_before 401 54 βˆ’347
LONG ALL >60 % $10K-notional sum +$17,831 +$26,506 +$8,675
LONG 80-90 % bucket size 95 223 +128 (+135 %)

13.3 Sign-flips & re-validations

Earlier observation Pre-backfill Post-backfill Status
SHORT 20-30 % bucket n=12, βˆ’$985 ("structurally bad") n=25, +$2,870 ("positive") Sign flipped β€” small-sample artifact
LONG 95%+ bucket size n=13 ("noise") n=27, +$192/trade Emerges as strongest LONG bucket β€” defer to #150+ for action
LONG 90-95 % bucket (Mar 9 #202 skipped) n=109, +$663 (essentially flat) Skip rule still correct on average
Mid 80-90 % LONG (universe) n=13, +$1,175 n=73, +$381 (still weakest, still positive) v3 D ON-HOLD reinforced β€” 5Γ— more data confirms it's a stops issue, not selection
PM prediction accuracy LONG > 60 % 75 % (n=249) 71 % (n=412) tempered with more data
PM prediction accuracy SHORT < 30 % 88 % (n=26) 81 % (n=27) unchanged
Volume sensitivity sweet spot $2K (n=106, +$42 avg) $2K (n=479, +$57 avg) Bot's $2K gate confirmed near-optimal at 4Γ— sample

13.4 Action queue impact

Finding #230 strengthened the existing verdicts but did not change any queue ranking or status:

13.5 Forward-fix gap β†’ Finding #231

A new finding (#231) closes the data-pipeline gap so this doesn't re-accumulate:

See docs/FINDINGS.md Finding #231 for the canonical narrative.

13.6 Data hygiene rule going forward

Single source of truth. All strategy analyses must use the live production tables (earnings_markets, markets, trade_audit_log, polymarket_opportunity_log, skipped_signals, conviction_history, price_snapshots). Derived artifacts (scripts/strategy_analysis_report.json, scripts/finding_a_full.csv) are regenerated from these tables and should never be the source. When data quality issues are uncovered, the fix flow is: (a) write a backfill script to update the production table, (b) write a forward-fix to the collector / tracker so the gap doesn't re-accumulate, (c) re-run the master report so the dashboard reflects the new state, (d) document as a numbered finding in docs/FINDINGS.md.


14. Apr 25 LATE evening β€” Action Queue re-validation pass (Findings #232 + #233)

After Β§12 + Β§13 landed, the operator asked for a full re-validation of the Action Queue against the post-backfill universe data. The re-pull produced several material updates that this section documents (the canonical finding narratives are in docs/FINDINGS.md Finding #232 + Finding #233).

14.1 Headline shift (vs. handoff doc state)

Metric Handoff (Apr 25 morning) Apr 25 LATE evening Ξ”
Closed trades 109 116 +7
Net P&L +$61 +$167 +$106
Win rate 33 % 33.6 % β‰ˆ same
Universe in-band 568 604 +36 (organic + backfill timer)
Volume coverage on in-band 91 % positive 90.4 % at $2K gate comparable
actual_beat resolved past markets 524 528 +4

The +$106 net P&L improvement came from 7 new trades plus the addition of TXN +$92 (Apr 22 β€” LONG large NULL-regime, big winner) and a few smaller contributors. The strategy is now meaningfully positive on a per-trade basis (+$1.44/trade) but still well below the +$5 to +$10/trade target.

14.2 The broken stop_triggered field β€” Finding #233

The re-validation immediately surfaced a critical data-hygiene bug: the trade_audit_log.stop_triggered boolean undercounts at 1.7 % (2 of 116 closed trades flagged true) while the canonical exit_reason ILIKE '%stop%' shows the real stop frequency at 49 % (57 of 116). Cross-tab:

stop_triggered exit_reason n
true stop_loss:stp_lmt_realtime_fill 1
true stop_loss:hardware_reconciled 1
false stop_loss:hardware 40
false stop_loss:hardware_reconciled 7
false instant_exit:stop_loss_backup 5
false stop_loss_software_backup 1
false hardware_stop 1
false stop_loss_triggered 1

So 55 of 57 stop-exits set stop_triggered=false. All historical analyses depending on the boolean are wrong by orders of magnitude. This invalidates several claims in the pre-Apr-25 docs:

Forward-fix: every analysis must use exit_reason ILIKE '%stop%' until the audit-log writer is patched to set stop_triggered correctly. Canonical: docs/FINDINGS.md Finding #233.

14.3 Per-rule re-validation (4-check gauntlet on fresh data)

Rule #1 β€” v3 B (Skip downtrend LONGs) β†’ KEEP rank 1, STRONGER

Bucket n Wins Stop rate (corrected) Total $
LONG downtrend 22 5 (23 %) 68.2 % βˆ’$136
LONG uptrend 36 15 (42 %) 38.9 % +$71
LONG sideways 37 12 (32 %) 51.4 % +$1

Verdict: PASSES STRONGER than docs claim. Lift estimate revised UP from +$140 to +$140 to +$200 / 100.

Rule #2 β€” RuleB (Β±90 min stop window) β†’ PROMOTE rank 4 β†’ 2

The corrected stop classification confirms the simulator's 51-stop training population matches the live audit log's 57 stops (small difference expected from reconstruction). RuleB's +$123/100 estimate is credible. Stop bleed dominates the negative contribution to strategy P&L β†’ RuleB is the highest-impact stop-loss intervention.

Rule #3 β€” v2 #6 (3-stop session breaker) β†’ KEEP IN-QUEUE rank 3

8 historical β‰₯3-stop sessions: Feb 19 (5 stops, βˆ’$35), Mar 2 (4 stops, βˆ’$59), and six 3-stop days. Total stops on those 8 days: 27, costing βˆ’$266. Real-world fire frequency confirmed.

v3 F (BMO + VIX sizing) β†’ DROP to RETRACTED

Deep bias gauntlet:

Arm Bucket n Headline Ex-top-2 Ex-top-3 Verdict
0.6Γ— scale-DOWN BMO+VIX 18-22 30 βˆ’$54 βˆ’$135 n/a βœ… PASS
1.2Γ— scale-UP BMO+VIX <18 20 +$100 n/a βˆ’$1.58 ❌ FAIL bias #2

The +$100 winner bucket is concentrated in STT +$37 / VZ +$33 / ALLY +$32 = +$102. Drop those 3 β†’ 17 trades sum to βˆ’$1.58.

Direct multiplier-on-realized-pnl modeling: baseline BMO total +$39.72 β†’ after F (full 1.2/0.6/1.0) = +$81.24 β†’ delta +$36 / 100 trades, not the +$60 docs claim. And that's optimistic (assumes linear scaling).

Operator decision Apr 25 LATE evening: Drop both arms. Re-test the scale-DOWN arm only at trade #150+ with fresh data.

v3 D (Mid-cap LONG β‰₯80%) β†’ KEEP ON-HOLD, REASONING REINFORCED

Per the corrected stop rates:

Bucket n Stop rate Total $
Mid LONG 80-90% 31 51.6 % βˆ’$56
Mid LONG 90-95% 15 86.7 % βˆ’$55
Large LONG 80-90% 10 50 % βˆ’$3 (degraded from +$22)
Large LONG 90-95% 8 62.5 % βˆ’$18

The "stops eat winners" hypothesis is now strongly supported. RuleB likely closes the gap. Note: Large 80-90% degradation suggests the future re-frame may be "Skip ALL LONGs β‰₯ 80%" (not just Mid).

v3 E (non-downtrend SHORT) β†’ KEEP ON-HOLD, REASONING REINFORCED

Sideways SHORT 83.3 % stop rate (corrected from claimed 67 %) strongly supports the stops-eat-winners mechanism. KEEP set is 2-of-3 outliers (GPRO +$111, STUB +$58, GIS +$3.64). Total SHORT n=18 still below nβ‰₯30 commit threshold.

14.4 The 5-outlier rename (was 3-outlier)

Memory Β§K + the four-check-gauntlet docstring updated Apr 25 evening: the named outlier set extended from GPRO + DELL + SWBI (~$260 of $61 net) to GPRO + DELL + SWBI + TXN + STUB (~$430 of $167 net on 116 trades; ex-top-5 the strategy is βˆ’$263 across 111 trades).

Outlier When Bucket $
GPRO 2026-03-05 SHORT mid downtrend +$111
TXN 2026-04-22 LONG large NULL-regime +$92 (NEW)
DELL 2026-02-26 LONG mid uptrend +$80
SWBI 2026-03-05 LONG mid downtrend +$70
STUB 2026-03-04 SHORT mid downtrend +$58 (NEW)

The TXN trade also exposed a small data-hygiene gap: 5 of 116 closed trades have NULL regime_at_entry despite TXN entering on a normal trading day. Tracked in Β§15 (deferred β€” minor data-hygiene).

14.5 New observations worth tracking (not yet in queue)

14.6 Final ranked queue (post Finding #232)

# Rule Status
1 v3 B β€” Skip downtrend LONGs IN-QUEUE β€” ship next
2 RuleB β€” Β±90 min no-stop window IN-QUEUE β€” PROMOTED rank 4 β†’ 2
3 v2 #6 β€” Daily 3-stop session breaker IN-QUEUE β€” historical evidence confirmed
4 Audit re-validation at trade #150+ IN-QUEUE (meta)
⏸️ v3 D β€” Mid-cap 80%+ skip ON-HOLD β€” reasoning REINFORCED, RuleB dependency vindicated
⏸️ v3 E β€” Skip non-downtrend SHORTs ON-HOLD β€” sideways SHORT 83 % stop rate vindicates RuleB dep
πŸ›‘ v3 F β€” BMO + VIX sizing RETRACTED Apr 25 LATE (Finding #232 deep gauntlet)
πŸ›‘ v3 C β€” drift filter RETRACTED Apr 25 (operator option 3 β€” kept)
πŸ›‘ v3 A β€” block stops outside RTH RETRACTED Apr 25 (Finding #228 β€” kept)

Combined realistic lift: +$190 to +$310 / 100 trades (slightly UP from prior +$160-$280 because v3 B's bias check came out stronger and v2 #6 is no longer purely defensive; offset by v3 F retraction).

14.7 Audit-process learnings (lesson #7 β€” added Apr 25 from Finding #233)

Canonical-field-validity rule. Never trust convenience flags like stop_triggered without cross-checking against the source-of-truth field like exit_reason. Audit-log boolean fields can fall out of sync silently when writer code paths drift; the textual exit_reason is the canonical record because it's set by the order-executor on every exit. When in doubt: query both, compare counts, investigate the gap.

This joins the prior lessons: 1. (Β§1) Use the 4-check bias gauntlet on every finding. 2. (Β§6 lesson #1) Sample size: n β‰₯ 12 minimum, ideally β‰₯ 20. 3. (Β§6 lesson #2) Skip rules need full-population spillover check. 4. (Β§6 lesson #3) The named outlier set (now 5) carries the strategy. 5. (Β§6 lesson #4) Universe view + live data must agree before shipping. 6. (Β§10 lesson #6 β€” Finding #228) True-simulator rule: proxies and observational tables are NOT counterfactual evidence. 7. (Β§14.7 lesson #7 β€” Finding #233) Canonical-field-validity rule.

14.8 Cross-references


15. Deferred from Β§14 (next-pass to-do)

Update Apr 26, 2026 (Phase 1 of Strategy Analysis Revamp): the first two items below have been resolved; remaining items below the divider remain deferred.


Still deferred (next-pass to-do):


Β§15. Apr 26-27 follow-on work β€” DATA-QUALITY ONLY (no rule changes)

Verified Apr 27, 2026 LATE evening: the verified-findings table (Β§3), retracted findings (Β§4), 4-check gauntlet (Β§5), and ranked queue (docs/STRATEGY_PLAYBOOK.md Β§2) are all unchanged since Apr 25. Two days of sustained work between Apr 25 and Apr 27 produced zero new strategy rules and zero rule retractions. Everything that shipped was data infrastructure, audit completeness, or operational correctness.

For traceability, the per-day breakdown:

Net effect on the audit: none. Rule rankings unchanged. Bias gauntlet unchanged. The 5-outlier curse still defines the strategy's net P&L (GPRO + DELL + SWBI + TXN + STUB β‰ˆ $430 of $167 net on 116 trades). Re-run scripts/strategy_performance_analysis.py for current numbers; the Β§3 evidence may have refreshed slightly because regime/VIX/conviction-at-exit data is now populating forward-only on every new trade.

Re-trigger conditions for next audit pass: (a) 50 more closed trades (currently 117; trigger at 150+), or (b) Action Queue items #1 + #2 (skip downtrend LONGs + RuleB Β±90min) ship and we want to measure impact. (Update β€” (b) has shipped: Rule #1 is active (#304) and RuleB is in Phase 1 dry-run (#305); live impact measurement is signal-density-limited, which is why the Phase 3 simulator now carries the counterfactual evaluation.)


Β§16. Jul 5, 2026 β€” RuleB retraction (Finding #330): the simulator's window arm was gap-fill optimistic

Trigger: operator-directed full strategy review (Jul 5, 2026). Three independent observations converged:

  1. All four June 2026 in-window stops (LULU, ADBE, ACN, KR) were CORRECT stops β€” the price kept falling through the window. Holding them to the rearm/scheduled exit would have been worse in every case (ACN: βˆ’$26.10 actual stop vs ~βˆ’$134 at the 10:32 scheduled exit through the post-earnings crash).
  2. A gap-aware ad-hoc replay of the RuleB arm flipped its sign.
  3. Tracing simulate_window_only confirmed the bug in code: after the silent window, any bar with low <= stop returned an exit at the stop price, even when the price had crossed the stop inside the window and was still beyond it at rearm. The as-shipped #305 cancel-and-rearm mechanism re-places a STP-LMT that triggers instantly and fills near market β€” simulate_finding_a_hard already modeled this as rearm_gap_open (the arm that retracted FA_hard at βˆ’$1,545/100 in Β§10); the RuleB arm never got the same handling.

Fix + corrected numbers (prod re-run, n=129): simulate_window_only is now gap-aware (first post-window bar OPEN fill when the open is already beyond the stop; signature unchanged so the phase3_simulator reference-adapter contract holds; RuleG inherits).

Rule Pre-#330 headline / ex-top-2 (per-100) Gap-aware headline / ex-top-2 (per-100) Verdict
RuleB +$224.22 / +$122.14 βˆ’$209.41 / βˆ’$194.00 πŸ›‘ FAILED-GAUNTLET
RuleG +$137.48 / +$7.60 βˆ’$29.68 / βˆ’$11.44 πŸ›‘ FAIL (was already fragile)

Optimism bias β‰ˆ $535 across 129 trades. Ex-repeat-tickers RuleB is βˆ’$226.67/100 β€” negative on every Β§K check. Mechanism: the window's recovery saves (dip-through-then-recover, preserved in the gap-aware sim) are outweighed ~2:1 by gap-through cases where the rearm fill is far worse than the original stop fill.

Decisions: RuleB Phase-2 active flip CANCELLED (Phase-1 dry-run code remains, logging only β€” its 8,026-duplicate-line log spam also fixed); v3 D + v3 E RuleB dependencies REMOVED (Β§12.2 / Β§12.3 re-evaluate as pure selection rules at trade #150+ / SHORT nβ‰₯30); the #323.b "0.92-0.99 conditional on RuleB active" lever is dead. NO stop-loss rule variant currently survives the gauntlet.

Audit-process lesson #7: a TRUE simulator (memory Β§K) must model the REARM path of any cancel-and-rearm rule, not just the silence. The PR-1 Β§10.1 cross-check (simulator-vs-reference consistency) cannot catch a shared mechanism error β€” mechanism review against as-shipped order semantics is a separate mandatory step for any stop-manipulation rule.

Canonical narrative: docs/FINDINGS.md Finding #330.