π Jul 5, 2026 update β RuleB ALSO RETRACTED (see Β§16; Finding #330)
The Apr-25 banner below said "Only RuleB survives". That is no longer true. The Jul 5 strategy review found
simulate_window_onlyfilled window-blocked stops at the STOP PRICE after the price had gapped through β unobtainable under the as-shipped #305 cancel-and-rearm mechanism. Gap-aware (rearm fills at the first post-window bar OPEN, the samerearm_gap_openhandling that retracted FA_hard): RuleB = β$209/100 headline / β$194/100 ex-top-2 (n=129) β FAILED-GAUNTLET; RuleG also flips negative. NO stop-loss rule variant currently survives. RuleB Phase-2 active flip CANCELLED; v3 D / v3 E lose their RuleB dependency (re-evaluate as pure selection rules). Full record: Β§16 + canonical narrativedocs/FINDINGS.mdFinding #330. All RuleB-survives statements in Β§Β§4-14 below are the historical pre-#330 record.
β οΈ Apr 25, 2026 update β Finding A RETRACTED (see Β§10)
The comprehensive Finding-A simulator (
scripts/analyze_finding_a_full.py, Apr 25) showed that v3 Finding A β Block stops outside RTH is either inert (rule as written, software-only gate while hardware STP-LMT staysoutsideRth=Trueper Finding #179) or net-negative (hardware also gated: β$418 / 100 trades, opposite sign to the audit's claim). Findings D / G also fail the 4-check bias gauntlet on the simulator's apples-to-apples baseline. Only RuleB (Β±90 min window) survives, at a much smaller +$123 / 100 trades (π superseded Jul 5, 2026 β see the banner above). Full retraction + replacement queue in Β§10. Canonical narrative:docs/FINDINGS.mdFinding #228. New forward-looking source of truth (this doc + dashboard Strategy Analysis tab + simulator data combined):docs/STRATEGY_PLAYBOOK.md.
Purpose: Permanent record of the v3 audit pass (Apr 22, 2026) so future analyses can see what was tested, what survived bias-checks, and what was retracted. Mirrors the canvas at
~/.cursor/projects/Users-marius-projects-polymarket-bot/canvases/strategy-roi-audit.canvas.tsx(Cursor-local, not in git).Read order: Β§1 process learning β Β§2 v2 retractions β Β§3 v3 verified findings β Β§4 ranked recommendations β Β§5 audit trail (every hypothesis tested) β Β§6 statistical caveats β Β§10 Finding A retraction (Apr 25).
In v2 the "ATR > 5% pre-screen" rule looked like a clean +$329 winner when measured on the 49 stopped-correct trades alone. The full-sample re-check killed it: across all 109 trades the same rule would have removed +$163 of profit (mostly SHORT alpha) and turned the strategy net β$148 worse.
The diagnostic showed something deeper: 3 trades β GPRO (+$111), DELL (+$80), SWBI (+$70) β together carry ~$260 of the strategy's $61 net P&L. Multiple v2 "findings" (VIX 22+ is best, L 70-80% is the only profitable LONG band, Thursday is the alpha day, Mid-cap-SHORT bucket) all decompose into "the same 3 outliers happen to land in this slice." Without those 3 winners, the strategy is roughly β$200.
Anything that fails any check is rejected regardless of how good the isolated number looks.
Run via scripts/analyze_atr_prescreen.py against all 109 closed trades:
| ATR cutoff | Kept | Skipped | Skip % | Kept P&L | Skipped P&L | "Savings" if skipped |
|---|---|---|---|---|---|---|
| 4 % | 64 | 44 | 40 % | β$68 | +$144 | β$129 (worse) |
| 5 % | 77 | 31 | 28 % | β$87 | +$163 | β$148 (worse) |
| 6 % | 85 | 23 | 21 % | β$26 | +$102 | β$87 (worse) |
| 8 % | 104 | 4 | 4 % | β$16 | +$92 | β$77 (worse) |
The skipped trades carry the strategy's profits, not its losses. By-direction at the 5 % cutoff:
| Direction | Skipped | Skip % | skip_pnl | keep_pnl |
|---|---|---|---|---|
| LONG | 24 of 93 | 26 % | +$18 (neutral) | β$82 |
| SHORT | 7 of 16 | 44 % | +$146 (the alpha) | β$21 |
By timing at 5 %: AMC: skip 24/49 (49 %), skip_pnl = +$126 β would amputate half the AMC book including the names that drive AMC SHORT alpha.
Lesson: had we shipped rule H based on the isolated stopped-correct slice (where it showed +$329 saved), the strategy would have gone net β$148 across the full population.
| v2 # | Finding | Verdict | Action |
|---|---|---|---|
| 1 | Downtrend regime + LONG = bleed | ROBUST β 17/22 negative; ex-worst-2 still β$80; top losers are 5 distinct tickers | KEEP. Ranked #1 in v3. |
| 2 | Stops harvest noise on correct calls | ROBUST β validated by simulator: rule B saves $160 / D saves $218 | KEEP. v3 #2 refines the WHEN: extended hours. |
| 3 | Downtrend + SHORT = perfect record (3/3, +$173) | Direction robust, magnitude GPRO-driven. Ex-GPRO: 2 trades, +$62 | REPLACE with v3 #5 (skip SHORT in non-downtrend, 12/12 negative). |
| 4 | VIX 22+ is the BEST bucket (+$147) | FRAGILE β 64% of bucket P&L from GPRO + SWBI. Ex-them: +$36 across 12 trades | RETRACT. Replaced by v3 #6 (BMO + VIX < 18 sweet spot). |
| 5 | 90%+ LONG conviction destroys value | ROBUST β 22 of 26 negative; ex-best-2 still β$73 | KEEP. v3 #4 extends to β₯80% for Mid-cap. |
| 6 | L 70-80% is the only profitable LONG band (+$108) | FRAGILE β 82% of bucket P&L from SWBI. Ex-SWBI: +$15. Ex-SWBI+DAL: β$17 | RETRACT as a sizing-up signal. Keep as "don't penalize" baseline only. |
| 7 | Counterfactual cost of skipped signals | PARTIAL β small_cap_long_above_80 (+$1,558) is 9 evals of 1 ticker (MBWM) | KEEP methodology, retract that specific claim. |
| 8 | Sample-size gates (meta) | Confirmed β half of v2 sub-segments had n < 10 | PROMOTED to permanent process rule. |
π RETRACTED Apr 25, 2026 β comprehensive simulator showed the rule as written is inert (software gate; hardware STP-LMT stays
outsideRth=True) and the only effective form (outsideRth=Falseextension) is β$418 / 100 trades, not the claimed +$300β$400 / 100 trades. The audit treated Rule B as a proxy and conflated the per-session observation table (which measures where stops fired, not what would have happened if they hadn't) with counterfactual evidence. See Β§10 below for the simulator's full output anddocs/FINDINGS.mdFinding #228 for the canonical retraction narrative.
Mechanism (as originally claimed): Pre-market and after-hours liquidity is thin; STP-LMT fills at bad prices then stocks revert during RTH.
Evidence: | Session | n | Stops | Total $ | |---|---:|---:|---:| | Pre-mkt 04:00β09:30 ET | 22 | 16 (73%) | β$242 | | RTH 09:30β16:00 ET | 63 | 18 (29%) | +$428 | | After-mkt 16:00β24:00 ET | 25 | 16 (64%) | β$138 |
Bias check: No outlier dominance β 32 stops across 47 ext-hours exits are spread across distinct tickers. Mechanism is structural (liquidity) not selection. Full-population spillover is positive (RTH cell is +$428).
Implementation: Gate stop-trigger check on session-of-now in
order_executor; hardware STP-LMT order can stay placed; software gate just
refuses to act on triggers outside RTH. ~15 lines.
Estimated lift: +$300 to +$400 / 100 trades.
Full simulation results from scripts/analyze_stop_counterfactual.py
--population correct (Polygon 1m bars, exact replay vs alternative
stop-loss policies). Live total P&L on these 49 trades: β$584.
| Rule | Total $ | Ξ vs live | Stops | Skipped | Better / Worse / Same |
|---|---|---|---|---|---|
| A β Baseline (current) | n/a | β | 48/48 | β | sanity check passed |
| B β Β±90 min no-stop window around release | β$414 | +$170 | 43/49 | β | 34 / 3 / 12 |
| C β ATR(14) k=1.5 stop | β$510 | +$74 | 29/49 | β | 21 / 28 / 0 |
| D β ATR(14) k=2.0 stop | β$366 | +$218 | 19/49 | β | 26 / 23 / 0 |
| E β ATR(14) k=2.5 stop | β$444 | +$140 | 16/49 | β | 27 / 22 / 0 |
| F β No stop at all (always scheduled exit) | β$486 | +$98 | 0/49 | β | 28 / 21 / 0 |
| G β Window + ATR k=2.0 (combined B + D) | β$362 | +$221 | 18/49 | β | 26 / 23 / 0 |
| ~~H β Window + skip-if-ATR > 5%~~ | ~~β$254~~ | ~~+$329~~ | ~~26/49~~ | ~~19/49~~ | ~~42 / 2 / 5~~ |
~~H~~ retracted per Β§1 β the +$329 was a selection-bias artifact; full- population check showed β$148 net. Surviving candidates ranked by risk-adjusted lift: B is the safest (only 3 trades worse than live, no selection change, simplest implementation); D adds extra savings at the cost of 19 still-stopped trades exiting at wider losses; G is the combined optimum.
The chosen v3 Finding A is a stronger version of B β instead of just a Β±90-min window around the earnings release, it blocks ALL stops outside RTH (09:30β16:00 ET). Rationale: 80 % of stop bleed (β$380 of β$549) came from extended-hours exits regardless of how close to the release the stop fired. Per-session detail in the table at the top of this section.
Run via scripts/analyze_stop_counterfactual.py --population wrong (n=16
trades where prediction_correct=FALSE):
| Rule | Total $ | Ξ vs live |
|---|---|---|
| Live baseline | β$24 | β |
| B β Β±90 min no-stop window | +$131 | +$155 |
| D β ATR k=2.0 | +$61 | +$85 |
| F β No stop at all | +$107 | +$131 |
| G β Window + ATR | +$71 | +$95 |
Methodological caveat: the simulator does NOT model the bot's
instant_exit:earnings_miss path (the existing earnings-detector that
catches most wrong predictions β€ 60 s after release). 12 of 16 live
wrong-prediction exits used that path, not the stop. So the wrong-pop $
deltas are upper bounds. The important conclusion is the direction:
every alt-stop rule is positive on the wrong-prediction set, meaning none
of them remove the strategy's ability to catch genuine disasters. The
existing earnings_detector does the heavy lifting on wrong calls; the
stop-loss is the safety net behind it.
Per-trade detail: scripts/stop_counterfactual_correct.csv (49 rows,
35 columns) and scripts/stop_counterfactual_wrong.csv. Columns include
A_baseline_*, B_window_*, C_atr15_*, D_atr20_*, E_atr25_*,
F_nostop_*, G_combo_*, H_*, atr_daily, atr_pct_of_entry. Sort by
{rule}_pnl_d - live_pnl_dollars to inspect winners vs losers per rule.
Evidence: 22 LONG trades in downtrend: 5 wins, β$136, 14 stops (64% stop rate). Same ticker universe in uptrend: 34 trades, 15 wins, +$71, 32% stop rate. Half the win rate, double the stops.
Bias check: 17 of 22 negative; ex-worst-2 (BABA β$32, GME β$25) still β$80; top losers are 5 distinct tickers (no single name dominance).
Implementation: Skip rule on signal_generator using
MarketRegimeProvider.regime. ~10 lines.
Estimated lift: +$140 / 100 trades.
Mechanism: When a stock has already drifted into our LONG direction overnight, the EPS expectation is "priced in" β the small additional move on the announcement isn't enough to cover commission + slippage + stop risk.
Evidence: | Drift bucket | n | Wins | Total $ | |---|---:|---:|---:| | LONG drift < 2% | 75 | 31 (41%) | +$107 | | LONG drift 2β5% | 13 | 1 | β$120 | | LONG drift β₯ 5% | 2 | 0 | β$25 |
Bias check: 14 of 15 are losers; only EBAY (+$5.64) is positive. Ex-best-1 the bucket is β$150 across 14 losers. Monotonic β the more drift, the worse the outcome. No outlier rescue.
Implementation: New skip_reason='pre_event_drift_long' in
signal_generator; uses earnings_markets.stock_price_day_before_close
already populated by Finding #227 resolver (713/715 past markets covered).
~15 lines.
Estimated lift: +$145 / 100 trades.
Evidence: | mcap Γ conviction | n | Wins | Total $ | |---|---:|---:|---:| | Mid LONG 80β90% | 31 | 11 (35%) | β$56 | | Mid LONG 90β95% | 15 | 1 (7%) | β$55 | | Large LONG 80β90% | 9 | 4 (44%) | +$22 |
Bias check: Both Mid sub-buckets independently negative across n β₯ 15. Mid is 70% of our trade count, so this is a high-leverage rule. Large 80-90% is positive β keep that bucket; only Mid is the problem.
Implementation: Extension of v2 finding #5 (90%+ LONG skip across all mcaps) lowered to 80% for Mid only. ~5 lines added to existing skip logic.
Estimated lift: +$110 / 100 trades.
Evidence: SHORT in uptrend: 1/7 wins, β$2. SHORT in sideways: 0/5 wins, β$27. Combined non-downtrend: 0 wins out of 12 (excluding 1 marginal $0.47 outlier), β$63. SHORT in downtrend: 3/3 wins (GPRO-dominated +$173 but ALL 3 winners across distinct tickers).
Bias check: 12/12 negative on the skip side β perfect direction signal. The downtrend SHORT positive side is dollar-dominated by GPRO, so the estimated dollar lift on the skip rule is conservative.
Implementation: Conditional gate: skip SHORT when regime != 'downtrend'
β skip_reason='short_non_downtrend'. Replaces existing "max 2 SHORTs"
rule (which was over-cautious).
Estimated lift: +$60 / 100 trades + lower tail risk.
Evidence: | BMO Γ VIX | n | Wins | Stop rate | Total $ | |---|---:|---:|---:|---:| | BMO + VIX < 18 | 18 | 11 (61%) | 17% | +$100 | | BMO + VIX 18β22 | 30 | 11 (37%) | 53% | β$54 | | BMO + VIX 22+ | 6 | 1 | 17% | +$19 |
Bias check: BMO+VIX<18 cell ex-top-3 (STT, VZ, ALLY) still +$32 across 15 trades. Loser cell ex-top-3 (WING, BLK, GRMN) is β$160 across 27 trades β LOSERS are pervasive, wins concentrated. Both ends robust.
Implementation: risk_manager weighted multiplier on VIX bucket for BMO
trades only: VIX < 18 β 1.2Γ base; VIX 18β22 β 0.6Γ; VIX β₯ 22 β 1.0Γ. AMC
unchanged (no clean VIX signal there). ~10 lines.
Estimated lift: +$60 / 100 trades.
β οΈ Original Β§4 ranking was built on the un-validated v3 Finding A and on the pre-simulator estimates for D / G. The Apr 25 simulator killed Finding A and showed D/G fail the 4-check gauntlet. The new ranked queue below is the operative one. Original rows kept inline (struck through) for audit history.
| # | Recommendation | Source | Est. lift / 100 trades | Status |
|---|---|---|---|---|
| 1 | Skip LONGs in downtrend | v2 #1 / v3 Finding B | +$140 | NEW #1 β selection rule, robust to outlier removal, no execution dependency |
| 2 | Skip LONGs with pre-event drift β₯ 2 % | v3 Finding C | +$145 | NEW #2 β needs stock_price_day_before_close on signal-time signal_generator (already populated by Finding #227 resolver on 713/715 past markets) |
| 3 | Skip Mid-cap LONGs at conviction β₯ 80 % | v3 Finding D | +$110 | overlaps with #2 β measure post-#1+#2 before shipping |
| 4 | Skip SHORTs in non-downtrend regimes | v3 Finding E | +$60 | small dollar lift, perfect 12/12 direction signal, low-risk addition |
| 5 | Daily 3-stop session breaker | v2 #6 | tail-risk cap | LOW; behavioral cap on bad-session escalation |
| 6 | BMO + VIX-bucket sizing | v3 Finding F | +$60 | sizing rule; layer last after skip rules settle |
| 7 | Skip all LONGs at conviction β₯ 90 % | v2 #5 | (overlaps with #3) | superseded by tighter Mid-cap variant |
| 8 | RuleB (Β±90 min stop window around release) | Β§10 simulator | +$123 | only stop-loss rule that survived simulator + bias gauntlet; small lift, ship only after selection rules |
| ~~ | ~~ Block stops outside RTH | ~~v3 Finding A~~ | ~~+$300β$400~~ | π RETRACTED Apr 25 β simulator: β$418/100 (hard) or $0 (soft, inert) |
| ~~ | ~~ ATR k=2.0 / window+ATR | ~~v3 alternatives~~ | ~~+$200~~ | π FAIL bias gauntlet β outlier-driven; ex-top-2 = β$103 / β$109 |
Combined realistic lift (post-retraction): Skip rules #1+#2+#3+#4 overlap (all subset-of-LONGs); realistic combined lift is +$300β$450 / 100 trades. RuleB adds a marginal +$50β80 net of overlap. Total addressable improvement is +$350β$500 / 100 trades, not the +$500β700 the original v3 Β§4 claimed (the difference is the retracted Finding A and the gauntlet-failed D/G).
The combined queue still flips the strategy from β$60 / 100 trades (current) to roughly +$300 to +$440 / 100 trades, but via selection fixes (cut bad trades upstream) rather than execution fixes (try to rescue already-bad trades).
Implementation order (revised): Ship in this order so each rule's lift is attributable:
signal_generator, ~10 lines, uses existing MarketRegimeProvider.earnings_markets.stock_price_day_before_close, ~15 lines.Re-run scripts/strategy_performance_analysis.py and
scripts/analyze_finding_a_full.py after each deploy. Re-validate the
entire audit at trade #150+ via the 4-check gauntlet (memory Β§K).
| ID | Hypothesis | n / data | Verdict | Reason |
|---|---|---|---|---|
| H1 | Skip LONGs with drift β₯ 2% pre-event | n=15 | ACCEPT (Finding C) | 1/15 wins, no outlier rescue, full-pop check passes |
| H2 | Conviction velocity (Ξprob_30min β prob_24h) | n=216 | INFORMATIVE only | Just confirms the existing conviction signal |
| H3 | Same-day earnings clustering (n_trades_per_day) | n=109 | REJECT | Non-monotonic; cluster=2 worst (β$129) but cluster=5 +$83 β noise |
| H4 | Day-of-week effect (Thursday alpha) | n=109 | REJECT | +$209 on Thu but ex-top-3 (GPRO/DELL/SWBI) becomes β$53 |
| H5 | Flex-sizing penalty | n=109 | REJECT | Avg position % below configured (Large 19% vs 30%) β flex barely fires |
| H6 | Spread-at-entry filter | β | NO DATA | spread_at_entry NULL on 110/110 β wiring missing |
| H7 | Conviction stability across pre-event snapshots | β | DEFER | Need full 5-snapshot rows; not enough resolved markets yet |
| H8 | Conviction at exit drop signal | n=4 | NO DATA | conviction_at_exit NULL on 106/110 |
| H9 | Volume / depth quality at entry | β | DEFER | Need post-event movement check |
| H10 | Same-ticker prior-trade history | n=109 | DEFER | Only 5 tickers traded > once; sample too small |
| H11 | Mid-cap subgroup concentration | n=72 | PARTIAL β H18 | Folded into mcap Γ conviction interaction |
| H12 | Sector concentration of losses | β | DEFER | No sector field on trade_audit_log |
| H13 | AMC SHORT-only signal (+$121, n=10) | n=10 | REJECT as standalone | Same GPRO/SWBI/STUB outliers; folded into H3-extended |
| H14 | EPS surprise magnitude β stock move correlation | n=210 | DEFER | Compute surprise % then bucket β next pass |
| H15 | Quick-stop vs slow-stop dynamics | n=48 | PARTIAL β H22 | Folded into 'extended-hours stop' finding |
| H16 | Regime-conditional 90%+ LONG (does it work in uptrend?) | n=26 | REJECT | 22/26 negative across all 3 regimes |
| H17 | BMO + VIX < 18 sweet spot | n=18 + n=30 | ACCEPT (Finding F) | Both cells robust to outlier removal |
| H18 | Mid LONG β₯ 80% conviction concentration | n=47 | ACCEPT (Finding D) | Both 80-90% and 90-95% independently negative |
| H19 | Pre-event after-hours move (BMO) | β | DEFER | Need IBKR pre-mkt quote at signal time |
| H20 | Same-week clustering | n=109 | REJECT | Same as H3 |
| H21 | Slippage by execution exchange | n=109 | REJECT | DARK β IBKRATS (basically identical) |
| H22 | Hold-duration vs P&L β extended-hours stop pattern | n=109 | ACCEPT (Finding A) | Pre/after-mkt exits are 42% of trades but 80% of stop bleed |
| H23 | Earnings-detector latency cost | n=7 | DEFER | Too small. Re-test at n=20+ |
| H24 | PM post-event yes_price as instant-exit accelerator | n=12 | DEFER | Existing earnings_detector catches wrong calls at small loss |
Always run the full-population check before recommending a skip rule. The H-rule (ATR > 5% pre-screen) showed +$329 on the 49-trade subset and would have lost $148 net across 109 trades. Skip rules in particular need full-population spillover analysis because they remove WINNERS as well as losers.
The 3-outlier curse. GPRO + DELL + SWBI together dominate ~$260 of the strategy's $61 net P&L. ANY slice that includes them looks great; ANY slice that excludes them is structurally negative. Always report numbers ex-top-2 alongside the headline.
Repeated-ticker bias. The MBWM finding in v2 was 9 evaluations of 1
ticker presented as "9 wins." Every counterfactual table needs a
unique_tickers column alongside n_total.
Mechanism beats correlation. The findings that survived (extended-hours stops, drift filter, BMO + low VIX) all have a structural mechanism behind the data (liquidity gaps, price-priced-in, low pre-earnings vol). The retracted ones (VIX 22+ best, Thursday alpha) were correlation-only and traced back to the same outlier names.
The defer queue is a feature, not a failure. 11 of 24 hypotheses ended up DEFERRED. They're not bad ideas; they need either better data capture (H8 conviction_at_exit, H19 pre-event after-hours quote) or larger n (H10 same-ticker, H23 earnings-detector latency). Re-run them in 50 trades.
n=109 closed trades + n=126 calibrated markets means Β±5β8% on headline win-rate slices, Β±10β15% on regime-conditional and VIX buckets. Findings A, B, C, D are well-sampled (n β₯ 15 each) and robust to outlier removal. Finding E (skip non-downtrend SHORT) is 12/12 negative β perfect direction but small dollar impact. Findings F (BMO + VIX < 18) and the kept v2 #7 (90%+ LONG skip) are well-sampled.
Re-validate the entire audit after the next 50 trades with
scripts/strategy_performance_analysis.py per memory Β§K. The bias-check
gauntlet (4 checks per hypothesis) should be re-run every audit pass.
~/.cursor/projects/Users-marius-projects-polymarket-bot/canvases/strategy-roi-audit.canvas.tsx β mirror of this doc with charts, KPI strip, and per-finding deep-dive cards.| Script | Purpose | Outputs |
|---|---|---|
scripts/analyze_stop_counterfactual.py |
Per-trade replay against alt stop policies (B/C/D/E/F/G/H). Supports --population correct\|wrong\|all-stop. |
scripts/stop_counterfactual_correct.csv (49 rows Γ 35 cols), scripts/stop_counterfactual_wrong.csv (16 Γ 35) |
scripts/analyze_finding_a_full.py |
Comprehensive Finding-A simulator (Apr 25, 2026 follow-up). Replays the actual proposed rule (block stops outside 09:30β16:00 ET, re-arm at RTH open with gap-open slippage modeling) on the full closed-trade population. Also evaluates Rules B / D / G / F on the same bars for cross-comparison. Distinguishes FA_soft (software-only gate; hardware STP-LMT outsideRth=True still fires β INERT for P&L) vs FA_hard (hardware also gated). Models earnings_detector cap on wrong-prediction trades. Built-in 4-check gauntlet (headline / ex-top-2 / ex-repeat-tickers / by-correctness / by-action / pre-EPS-only) per rule. Halt-suspect detection. Run BEFORE shipping Finding A. | scripts/finding_a_full.csv (one row per trade Γ all rules) + stdout aggregate |
scripts/analyze_atr_prescreen.py |
ATR(14) pre-screen impact across full trade history at multiple cutoffs (4/5/6/8 %). The script that killed the H-rule. | scripts/atr_prescreen.csv (109 rows with atr_daily, atr_pct, skip_at_*pct flags) |
scripts/strategy_performance_analysis.py |
Full strategy-performance report with regime / VIX / conviction / skipped-cost sections. Memory Β§K source of truth. | scripts/strategy_analysis_report.json (consumed by dashboard Strategy Analysis tab) |
scripts/backfill_learning_loop.py |
One-shot historical backfill of the resolver across all earnings_markets / skipped_signals / opportunity_log rows. | DB writes; logs to /var/log/polymarket/backfill_learning_loop.log |
scripts/backfill_regime_for_audit_log.py |
Fills regime_at_entry + vix_at_entry on existing closed trades. |
DB writes |
scripts/compute_counterfactual_pnl.py |
Operator-only counterfactual P&L on opportunity_log rows (pre-resolver tool). | DB writes |
| Module | Role |
|---|---|
src/data_tracker/collectors/outcome_resolver.py |
Daily 06:00 ET timer; 4-pass cascade fills actual_beat, eps_actual, revenue_actual, stock_price_*, hypothetical_pnl, prediction_correct. Source priority: EPS surprise β post-event Polymarket yes_price β pre-event prob cascade. |
src/earnings/market_regime_provider.py |
Single source of truth for SPY regime (uptrend/sideways/downtrend) + VIX + SPY change. Used by signal_generator at insert time. |
src/earnings/counterfactual_pricing.py |
Polygon 1m bar fetch + simulate_minute_path (entry/exit/stop replay). Used by both the resolver and analyze_stop_counterfactual.py. |
src/data_tracker/migrations/022..024 |
Schema for regime/intended-size/counterfactual columns. |
| Doc | What's there |
|---|---|
docs/FINDINGS.md Finding #227 |
Canonical narrative for the entire learning-loop / data-infrastructure work (per memory Β§J). |
docs/DATA_ARCHITECTURE.md Β§4 |
Outcome resolver runbook + 4-tier actual_beat cascade documentation. |
docs/DATABASE_DIRECTIVES.md |
Writer-ownership table for actual_beat, vix_at_entry, etc. |
docs/OPERATIONS.md |
Polymarket-resolver service + manual backfill commands + Polygon plan guidance. |
docs/STRATEGY.md Β§6 / Β§10 |
Pointers to MarketRegimeProvider as the regime source of truth. |
.cursor/memory.md Β§M |
Cross-interface rule: any new use of regime/VIX as a gate must import from MarketRegimeProvider. |
--polygon-sleep 0.05. Documented in .env.example and docs/OPERATIONS.md.earnings_markets rows (n=715):/vX/reference/financials (Tier 3 EPS): 366 / 715 = 51 %actual_beat coverage is ~95-99.8% after the post-event snapshot + SEC EDGAR + resolver-guard fixes (#246/#247/#321). The 73% above is the Apr-25 audit-trail figure, retained for the progression history.11 hypotheses ended up DEFERRED rather than rejected β they're not bad ideas, they need either better data capture or a larger n. Re-test in the next audit pass (after another 50 trades, ~Aug 2026).
| ID | Hypothesis | What it needs to become testable | Estimated re-test trigger |
|---|---|---|---|
| H6 | Spread-at-entry filter | Wire spread_at_entry capture in signal_generator._build_trading_signal (currently NULL on 110/110). Tracks Polymarket bid-ask spread at signal time. |
After spread capture wired + 30 trades |
| H7 | Conviction stability across pre-event snapshots | Need 5-snapshot rows (prob_24h, prob_12h, prob_6h, prob_1h, prob_30min all populated). Compute stddev(prob_*) / mean(prob_*) per market. |
100+ resolved markets with full snapshot coverage |
| H8 | Conviction-at-exit drop signal | Wire conviction_at_exit capture in position_manager.persist_position_close (NULL on 106/110). Compares pre-event vs at-exit Polymarket conviction. |
After exit-time capture wired + 30 trades |
| H9 | Volume / depth quality at entry | Need pm_bid_depth_at_entry + pm_ask_depth_at_entry populated more reliably. Currently sparse. |
After 50 more trades with depth captured |
| H10 | Same-ticker prior-trade history | Only 5 tickers traded > once today. Need 20+ repeated tickers to do a meaningful "we get better at known names" analysis. | n β₯ 50 repeat-ticker trades |
| H12 | Sector concentration of losses | No sector field on trade_audit_log. Add GICS sector lookup via Yahoo / Finnhub at entry time, populate retroactively for past trades. |
After sector field wired + populated |
| H14 | EPS surprise magnitude β stock move correlation | Polygon eps_actual now populated on 210 markets. Compute (eps_actual - eps_estimate) / eps_estimate per market and bucket vs stock_return_day_close. Could be done now β just needs the analysis script. |
Could run now; deferred to keep audit scope bounded |
| H19 | Pre-event after-hours move (BMO entries) | Need IBKR pre-mkt quote at signal time (15:58 ET BMO entry would benefit from knowing if SPY is moving in after-hours). Not currently stored. | After pre-mkt SPY capture wired |
| H23 | Earnings-detector latency cost | Only 7 instant_exit:earnings_miss trades to date. Need 20+ to bucket detection_latency_seconds vs P&L. |
After 30 more wrong-prediction trades |
| H24 | Polymarket post-event yes_price as instant-exit accelerator | Sample of 12 wrong-prediction trades shows existing earnings_detector already catches them at small loss; PM signal would be marginal until detector is slow. Re-test if detector latency degrades. |
If earnings_detector 95th percentile latency exceeds 90 s |
One rejection that may want re-test: H4 (day-of-week) was rejected because Thu's +$209 was the same GPRO/DELL/SWBI outliers. Once we have 50+ more trades (and the 3-outlier dominance dilutes), re-run H4 β if Thursday is still positive, it might be a real session effect not just outlier coincidence.
Why this section exists: The original v3 Β§3 Finding A claim (block stops outside RTH = +$300β$400 / 100 trades) was supported by (a) Rule B as a proxy and (b) a per-session observational table. A follow-up review surfaced 8 unmodelled scenarios. The script
scripts/analyze_finding_a_full.pywas built to close all 8 and replay the actual proposed rule end-to-end.
scripts/analyze_finding_a_full.py (570 lines, read-only):
outsideRth=True
per Finding #179, so the broker still fills in extended hours regardless
of any software gate. By construction the price evolution is identical
to current behavior.outsideRth=False outside RTH;
stop is silent in pre/post; on first RTH bar after a non-RTH segment, if
the bar gap-throughs the stop (open β€ stop for LONG / open β₯ stop for
SHORT) we exit at the open price (gap-open slippage modeling), otherwise
the stop re-arms and intraday triggers fire as normal.exit_time is treated as a hard ceiling β the simulator can't credit the
rule for trades the live instant_exit:earnings_miss pipeline already
catches).Apples-to-apples vs the simulator's A_base baseline (cancels Polygon-bar-
vs-IBKR-fill reconstruction noise):
| Rule | n | Ξ vs A_base | per-100 trades | better | worse |
|---|---|---|---|---|---|
| FA_soft (rule as written) | 51 | $0.00 | $0.00 | 0 | 0 |
| FA_hard (hardware also gated) | 51 | β$213.23 | β$418.10 | 9 | 23 |
| RuleB (Β±90 min window only) | 51 | +$62.97 | +$123.47 | 4 | 1 |
| RuleD (ATR k=2.0) | 51 | +$99.23 | +$194.57 | 22 | 27 |
| RuleG (window + ATR k=2.0) | 51 | +$102.32 | +$200.63 | 22 | 27 |
| RuleF (no stop ever) | 51 | β$12.77 | β$25.04 | 24 | 25 |
| FA_hard action | n | Total Ξ vs A_base | Avg / trade |
|---|---|---|---|
| rearm_gap_open (stop blocked in ext-hrs, RTH opened past stop β forced fill at worse open price) | 22 | β$416.72 | β$18.94 |
| blocked_to_scheduled (stop blocked, stock recovered fully, exit at sched = profit) | 9 | +$203.61 | +$22.62 |
| earnings_detector_cap | 1 | β$0.12 | β |
| unchanged (RTH stop / no stop / re-arm intraday) | 19 | $0 | β |
The 22 worst trades include MDB (β$82), TTD (β$58), NKE (β$54), BABA (β$38), WDAY (β$26), NYT (β$23), RL (β$18), GTLB (β$15), HOOD (β$15), LOW (β$13), CSCO (β$12), VSCO (β$12). Every one of these had the stock continue past the stop level into RTH open. The audit's claim that extended-hours stops "fill at bad prices then stocks revert" assumes reversion that the data does not support β for these 22 trades the price moved the wrong direction and stayed there.
The 9 recovery wins (KSS +$66, GME +$33, BBY +$31, KR +$23, JPM +$15, TXRH +$11, WMT +$11, NET +$10, SOUN +$4) are real but outweighed 2-to-1 by the gap-open losses in both trade count and dollars.
The audit's "best" stop-loss alternative (RuleG: window + ATR k=2.0) showed +$200 / 100 headline. Robustness:
| Rule | Headline | Ex-top-2 | Ex-top-4 | Ex-top-6 |
|---|---|---|---|---|
| RuleG | +$200/100 | β$103/100 | β$163/100 | β$167/100 |
| RuleD | +$195/100 | β$109/100 | β$170/100 | β$174/100 |
Both collapse to net-negative when you drop just two outliers (the same KSS / GME / BBY recovery cluster). They fail bias-check #2 of the gauntlet β rejected as standalone rules.
Rule B (Β±90 min no-stop window around earnings release):
This is the only stop-loss intervention worth shipping, and only after the selection rules (Findings B/C/D) have been deployed and measured.
The audit claimed extended-hours stops fire "on overnight noise before any earnings." Restricting the simulator to the 30 trades whose A_base trigger fired before the earnings release: FA_hard = β$92, 11 better / 14 worse. Even in this allegedly safest subset, the rule is net-negative. The "noise" hypothesis isn't supported β pre-EPS extended-hours stops fire on real adverse moves that continue through RTH open more often than they reverse.
Two trades flagged by the halt heuristic (>30 min gap + >3 % jump in pre/ post-market):
Under FA_hard the bot would have been forced to hold through both, with no ability to act on the price gap until RTH. These are real halt risks not modelled in the audit.
scripts/analyze_finding_a_full.pyscripts/finding_a_full.csv (51 rows Γ 41 cols)docs/FINDINGS.md Finding #228docs/STRATEGY_PLAYBOOK.mdA second analysis pass extended the audit to the full universe of all 900
Polymarket earnings markets in the database, not just the 109 the bot
actually traded. The output is rendered live in the dashboard
π Universe View section (refreshed every report run) and also
mirrored in the universe_hit_rate block of scripts/strategy_analysis_report.json.
prob_24h_before > 0.60: dayClose win rate 55.3 %,
$10K-notional sum +$17,831. 26 SHORT candidates with
prob_24h_before < 0.30: dayClose win rate 61.5 %, sum +$3,124.
Polymarket's own prediction accuracy is 75 % LONG / 88 % SHORT.v3 Finding C (drift filter, Queue #2) β verdict: WEAKENED. The
original v3 C drift definition (live_entry_price β stock_price_day_before_close)
collapses to ~0 for BMO trades because the bot enters at 15:58 ET and
stock_price_day_before_close is the same day's 16:00 close. Replacing
it with a 5-trading-day baseline (universe view Β§ C) and re-bucketing the
253 LONG candidates:
| Drift bucket | n | win % | total $10K-notional |
|---|---|---|---|
| drift < β2% | 68 | 60.3 % | +$4,410 |
| drift β2 to 0% | 58 | 55.2 % | +$1,964 |
| drift 0 to 2% | 53 | 49.1 % | +$8,168 |
| drift 2 to 5% | 50 | 58.0 % | +$3,604 |
| drift β₯ 5% | 24 | 50.0 % | β$315 |
Skip-bucket (drift β₯ 2%): n=74, win 55.4 %, total +$3,289, ex-top-2 +$1,136. Skipping these would remove winners, not losers. Live trades with the original drift defn still show drift β₯ 2 % is bad (β$120 on n=13), but the sample is at the n β₯ 12 floor and the rule's effect may trace to a different underlying signal (small-caps, low-conviction overlap, etc.) than literal pre-event price drift.
Action: Queue #2 status changed from "IN-QUEUE β clean" to "IN-QUEUE β drift-baseline question must be resolved before shipping." Three options:
v3 Finding D (Mid-cap 80%+ skip, Queue #3) β verdict: CONTESTED.
Universe view Β§ B splits the LONG 80-90 % bucket by market_cap_category:
| mcap | n | win % | avg $ | total $ |
|---|---|---|---|---|
| Large | 36 | 58.3 % | +$10 | +$374 |
| Mega | 6 | 50.0 % | +$61 | +$364 |
| Mid | 13 | 46.2 % | +$90 | +$1,175 |
| Unknown | 40 | 62.5 % | +$263 | +$10,503 |
The Mid bucket is net positive at universe scale, contradicting the live-trade Mid 80-90 % LONG = β$56 / 31 trades observation. The most likely explanation per Finding #228: stops eat the winners on live Mid-cap trades (held-correct = +5 % / trade vs stopped-correct = β3.5 % / trade). The Mid bucket may NOT need a skip rule β it may need better stop handling (covered by Queue #7 RuleB).
Action: Queue #3 status flagged "CONTESTED" β re-evaluate after RuleB ships. If RuleB closes the gap on Mid-cap stops, Finding D may no longer be needed.
scripts/strategy_performance_analysis.py:analyze_universe_hit_rate()scripts/strategy_performance_analysis.py:_fetch_5d_prior_close()strategy_analysis_report.json["universe_hit_rate"]dashboard.html)STRATEGY_PLAYBOOK.md Β§2 with the new "β οΈ" markers.After Β§11 was written, three additional changes landed in one pass.
Operator decision: drop the drift filter entirely and rely on B + E + the already-IMPLEMENTED selection rules to catch high-drift bad-EV trades via different filters. Reasoning:
live entry_price β stock_price_day_before_close)
collapses to ~0 for BMO trades because both are within 2 minutes of
each other.Re-test trigger: trade #150+ with a tighter drift definition (e.g. 1-day-prior baseline that approximates the original v3 defn).
New ON-HOLD status added to the Action Queue schema. v3 Finding D moved
there because:
Re-evaluation criterion: after RuleB ships AND has accumulated β₯ 10 Mid-cap stops in production, re-measure live Mid-cap stop rate and P&L.
analyze_universe_hit_rate() extended to:
markets.volume_total for volume-coverage stats. Caveat: only
35 % of in-band markets (111 / 314) have positive volume_total β
the rest may have Gamma volume that the bot's effective gate (Finding
#224) would still pass, so this analysis is conservative and may
UNDER-count the bot's true tradeable universe.| # | Rule | Status |
|---|---|---|
| 1 | v3 B β Skip downtrend LONGs | IN-QUEUE β ship next |
| 2 | v3 E β Skip non-downtrend SHORTs | IN-QUEUE |
| 3 | v2 #6 β Daily 3-stop session breaker | IN-QUEUE |
| 4 | v3 F β BMO + VIX-bucket sizing | IN-QUEUE |
| 5 | RuleB β Β±90 min no-stop window | IN-QUEUE (unblocks v3 D re-eval) |
| 6 | Audit re-validation at trade #150+ | IN-QUEUE (meta) |
| βΈοΈ | v3 D β Mid-cap 80%+ skip | ON-HOLD pending RuleB |
| π | v3 C β drift filter | RETRACTED via option 3 |
Combined realistic lift: +$200 to +$350 / 100 trades.
Operator review of v3 E found the rule too aggressive given current data:
3-outlier rule β memory Β§K β applies). Ex-GPRO it's 2
trades / ~$60.Re-evaluation criteria (strictest of): - After RuleB ships AND total SHORT count β₯ 30 (currently 18); OR - At trade #150+ regardless (audit re-validation gauntlet).
If at re-evaluation: - Sideways SHORT stop rate drops post-RuleB β close as RESOLVED-BY-OTHER-RULE. - Sideways SHORT remains 0 % WR with high stop rate β resume IN-QUEUE and ship. - Downtrend SHORT alpha is still GPRO-dominated at n β₯ 6 β keep ON-HOLD permanently (rule has no reliable KEEP set).
| # | Rule | Status |
|---|---|---|
| 1 | v3 B β Skip downtrend LONGs | IN-QUEUE β ship next |
| 2 | v2 #6 β Daily 3-stop session breaker | IN-QUEUE |
| 3 | v3 F β BMO + VIX-bucket sizing | IN-QUEUE |
| 4 | RuleB β Β±90 min no-stop window | IN-QUEUE (unblocks v3 D + E re-eval) |
| 5 | Audit re-validation at trade #150+ | IN-QUEUE (meta) |
| βΈοΈ | v3 D β Mid-cap 80%+ skip | ON-HOLD pending RuleB |
| βΈοΈ | v3 E β Skip non-downtrend SHORTs | ON-HOLD pending RuleB + β₯30 SHORT trades |
| π | v3 C β drift filter | RETRACTED via option 3 |
Combined realistic lift: +$160 to +$280 / 100 trades (down from +$200-350). Still flips the strategy from β$60 / 100 trades to +$100 to +$220 / 100 trades β driven primarily by Finding B (downtrend LONG skip) plus RuleB and behavioral guards.
Net effect of being conservative on D + E: ~$100 / 100 trades less estimated lift, but we preserve trade volume and avoid committing rules to small-sample data that may be capturing something temporary (capital constraints during a particular regime, the 3-outlier curse, or stops bleed that RuleB will fix).
After the queue updates of Β§12, two data-quality issues were uncovered and fixed in succession. They don't change any queue rule's status (the ranking from Β§12.6 is unchanged), but they materially expand the data underpinning every "is this signal valid?" check that depends on the Universe View. Future audits should rely on the post-backfill numbers.
markets.volume_total silently 0 for resolved marketsTrigger: Universe View showed only 35 % volume coverage (111 of 314 in-band markets). User asked whether this was a real Polymarket gap or a data-pipeline gap.
Root cause: continuous_collector.sync_markets_from_gamma queried
Gamma with ?active=true&archived=false. Polymarket's state machine has
both active and closed flags β settled earnings markets sit in
(active=True, closed=True, archived=False) for days/weeks before being
archived. The ?active=true filter does NOT include them despite their
active=true flag.
Fix: two-pass sync (active + closed-not-archived) in
continuous_collector + one-off scripts/backfill_markets_volume.py.
Recovered 319 markets / $5.7M of volume.
Result: markets passing the bot's $2K gate jumped 111 β 546.
earnings_markets.prob_*_before missing for 401 of 900 marketsTrigger: Even after Finding #229, the Universe View was only
capturing 31 % of all earnings markets (279/900). The biggest gap was
549 rows with NULL prob_24h_before (401 past + 148 future).
Root cause: Two-layer issue. (1) Orderbook collector
(continuous_collector) only starts sampling a market when its
condition_id enters the active-markets list, often only a few hours
before the earnings event. So the 24h / 12h / 6h-before snapshot
windows fall outside the sampled timespan. (2) conviction_tracker
--backfill looks for snapshots within Β±30 min of the target time β
useless when the collector has no rows from that timeframe at all.
Fix: new scripts/backfill_conviction_from_trades.py pulls trade
history from Polymarket's data-api /trades endpoint (which goes back
much further), normalises each trade's price to YES-token probability
(yes_prob = price if trade on YES, 1 β price if on NO), and finds
the trade closest to each target timestamp (24h/12h/6h/1h/30min before
earnings_dt) within Β±60 min.
Result:
| Metric | Pre-backfill | Post-backfill | Ξ |
|---|---|---|---|
| Universe LONG (>60 %, resolved) | 253 | 529 | +276 (+109 %) |
| Universe SHORT (<30 %, resolved) | 26 | 39 | +13 |
| Total in-band universe | 279 | 568 | +289 (+104 %) |
| Coverage of all 900 markets | 31 % | 63 % | +32 pp |
Past markets with NULL prob_24h_before |
401 | 54 | β347 |
| LONG ALL >60 % $10K-notional sum | +$17,831 | +$26,506 | +$8,675 |
| LONG 80-90 % bucket size | 95 | 223 | +128 (+135 %) |
| Earlier observation | Pre-backfill | Post-backfill | Status |
|---|---|---|---|
| SHORT 20-30 % bucket | n=12, β$985 ("structurally bad") | n=25, +$2,870 ("positive") | Sign flipped β small-sample artifact |
| LONG 95%+ bucket size | n=13 ("noise") | n=27, +$192/trade | Emerges as strongest LONG bucket β defer to #150+ for action |
| LONG 90-95 % bucket | (Mar 9 #202 skipped) | n=109, +$663 (essentially flat) | Skip rule still correct on average |
| Mid 80-90 % LONG (universe) | n=13, +$1,175 | n=73, +$381 (still weakest, still positive) | v3 D ON-HOLD reinforced β 5Γ more data confirms it's a stops issue, not selection |
| PM prediction accuracy LONG > 60 % | 75 % (n=249) | 71 % (n=412) | tempered with more data |
| PM prediction accuracy SHORT < 30 % | 88 % (n=26) | 81 % (n=27) | unchanged |
| Volume sensitivity sweet spot | $2K (n=106, +$42 avg) | $2K (n=479, +$57 avg) | Bot's $2K gate confirmed near-optimal at 4Γ sample |
Finding #230 strengthened the existing verdicts but did not change any queue ranking or status:
A new finding (#231) closes the data-pipeline gap so this doesn't re-accumulate:
/trades fallback INTO
conviction_tracker.py:backfill_historical_convictions() so the
tracker auto-falls-back when price_snapshots is sparse.polymarket-data-backfill.timer) that runs both volume and
conviction backfills, so the gap self-heals.See docs/FINDINGS.md Finding #231 for the canonical narrative.
Single source of truth. All strategy analyses must use the live production tables (
earnings_markets,markets,trade_audit_log,polymarket_opportunity_log,skipped_signals,conviction_history,price_snapshots). Derived artifacts (scripts/strategy_analysis_report.json,scripts/finding_a_full.csv) are regenerated from these tables and should never be the source. When data quality issues are uncovered, the fix flow is: (a) write a backfill script to update the production table, (b) write a forward-fix to the collector / tracker so the gap doesn't re-accumulate, (c) re-run the master report so the dashboard reflects the new state, (d) document as a numbered finding indocs/FINDINGS.md.
After Β§12 + Β§13 landed, the operator asked for a full re-validation of the
Action Queue against the post-backfill universe data. The re-pull produced
several material updates that this section documents (the canonical
finding narratives are in docs/FINDINGS.md Finding #232 + Finding #233).
| Metric | Handoff (Apr 25 morning) | Apr 25 LATE evening | Ξ |
|---|---|---|---|
| Closed trades | 109 | 116 | +7 |
| Net P&L | +$61 | +$167 | +$106 |
| Win rate | 33 % | 33.6 % | β same |
| Universe in-band | 568 | 604 | +36 (organic + backfill timer) |
| Volume coverage on in-band | 91 % positive | 90.4 % at $2K gate | comparable |
actual_beat resolved past markets |
524 | 528 | +4 |
The +$106 net P&L improvement came from 7 new trades plus the addition of TXN +$92 (Apr 22 β LONG large NULL-regime, big winner) and a few smaller contributors. The strategy is now meaningfully positive on a per-trade basis (+$1.44/trade) but still well below the +$5 to +$10/trade target.
stop_triggered field β Finding #233The re-validation immediately surfaced a critical data-hygiene bug: the
trade_audit_log.stop_triggered boolean undercounts at 1.7 %
(2 of 116 closed trades flagged true) while the canonical
exit_reason ILIKE '%stop%' shows the real stop frequency at 49 %
(57 of 116). Cross-tab:
| stop_triggered | exit_reason | n |
|---|---|---|
| true | stop_loss:stp_lmt_realtime_fill |
1 |
| true | stop_loss:hardware_reconciled |
1 |
| false | stop_loss:hardware |
40 |
| false | stop_loss:hardware_reconciled |
7 |
| false | instant_exit:stop_loss_backup |
5 |
| false | stop_loss_software_backup |
1 |
| false | hardware_stop |
1 |
| false | stop_loss_triggered |
1 |
So 55 of 57 stop-exits set stop_triggered=false. All historical
analyses depending on the boolean are wrong by orders of magnitude.
This invalidates several claims in the pre-Apr-25 docs:
Forward-fix: every analysis must use exit_reason ILIKE '%stop%' until
the audit-log writer is patched to set stop_triggered correctly.
Canonical: docs/FINDINGS.md Finding #233.
| Bucket | n | Wins | Stop rate (corrected) | Total $ |
|---|---|---|---|---|
| LONG downtrend | 22 | 5 (23 %) | 68.2 % | β$136 |
| LONG uptrend | 36 | 15 (42 %) | 38.9 % | +$71 |
| LONG sideways | 37 | 12 (32 %) | 51.4 % | +$1 |
Verdict: PASSES STRONGER than docs claim. Lift estimate revised UP from +$140 to +$140 to +$200 / 100.
The corrected stop classification confirms the simulator's 51-stop training population matches the live audit log's 57 stops (small difference expected from reconstruction). RuleB's +$123/100 estimate is credible. Stop bleed dominates the negative contribution to strategy P&L β RuleB is the highest-impact stop-loss intervention.
8 historical β₯3-stop sessions: Feb 19 (5 stops, β$35), Mar 2 (4 stops, β$59), and six 3-stop days. Total stops on those 8 days: 27, costing β$266. Real-world fire frequency confirmed.
Deep bias gauntlet:
| Arm | Bucket | n | Headline | Ex-top-2 | Ex-top-3 | Verdict |
|---|---|---|---|---|---|---|
| 0.6Γ scale-DOWN | BMO+VIX 18-22 | 30 | β$54 | β$135 | n/a | β PASS |
| 1.2Γ scale-UP | BMO+VIX <18 | 20 | +$100 | n/a | β$1.58 | β FAIL bias #2 |
The +$100 winner bucket is concentrated in STT +$37 / VZ +$33 / ALLY +$32 = +$102. Drop those 3 β 17 trades sum to β$1.58.
Direct multiplier-on-realized-pnl modeling: baseline BMO total +$39.72 β after F (full 1.2/0.6/1.0) = +$81.24 β delta +$36 / 100 trades, not the +$60 docs claim. And that's optimistic (assumes linear scaling).
Operator decision Apr 25 LATE evening: Drop both arms. Re-test the scale-DOWN arm only at trade #150+ with fresh data.
Per the corrected stop rates:
| Bucket | n | Stop rate | Total $ |
|---|---|---|---|
| Mid LONG 80-90% | 31 | 51.6 % | β$56 |
| Mid LONG 90-95% | 15 | 86.7 % | β$55 |
| Large LONG 80-90% | 10 | 50 % | β$3 (degraded from +$22) |
| Large LONG 90-95% | 8 | 62.5 % | β$18 |
The "stops eat winners" hypothesis is now strongly supported. RuleB likely closes the gap. Note: Large 80-90% degradation suggests the future re-frame may be "Skip ALL LONGs β₯ 80%" (not just Mid).
Sideways SHORT 83.3 % stop rate (corrected from claimed 67 %) strongly supports the stops-eat-winners mechanism. KEEP set is 2-of-3 outliers (GPRO +$111, STUB +$58, GIS +$3.64). Total SHORT n=18 still below nβ₯30 commit threshold.
Memory Β§K + the four-check-gauntlet docstring updated Apr 25 evening: the named outlier set extended from GPRO + DELL + SWBI (~$260 of $61 net) to GPRO + DELL + SWBI + TXN + STUB (~$430 of $167 net on 116 trades; ex-top-5 the strategy is β$263 across 111 trades).
| Outlier | When | Bucket | $ |
|---|---|---|---|
| GPRO | 2026-03-05 | SHORT mid downtrend | +$111 |
| TXN | 2026-04-22 | LONG large NULL-regime | +$92 (NEW) |
| DELL | 2026-02-26 | LONG mid uptrend | +$80 |
| SWBI | 2026-03-05 | LONG mid downtrend | +$70 |
| STUB | 2026-03-04 | SHORT mid downtrend | +$58 (NEW) |
The TXN trade also exposed a small data-hygiene gap: 5 of 116 closed
trades have NULL regime_at_entry despite TXN entering on a normal
trading day. Tracked in Β§15 (deferred β minor data-hygiene).
| # | Rule | Status |
|---|---|---|
| 1 | v3 B β Skip downtrend LONGs | IN-QUEUE β ship next |
| 2 | RuleB β Β±90 min no-stop window | IN-QUEUE β PROMOTED rank 4 β 2 |
| 3 | v2 #6 β Daily 3-stop session breaker | IN-QUEUE β historical evidence confirmed |
| 4 | Audit re-validation at trade #150+ | IN-QUEUE (meta) |
| βΈοΈ | v3 D β Mid-cap 80%+ skip | ON-HOLD β reasoning REINFORCED, RuleB dependency vindicated |
| βΈοΈ | v3 E β Skip non-downtrend SHORTs | ON-HOLD β sideways SHORT 83 % stop rate vindicates RuleB dep |
| π | v3 F β BMO + VIX sizing | RETRACTED Apr 25 LATE (Finding #232 deep gauntlet) |
| π | v3 C β drift filter | RETRACTED Apr 25 (operator option 3 β kept) |
| π | v3 A β block stops outside RTH | RETRACTED Apr 25 (Finding #228 β kept) |
Combined realistic lift: +$190 to +$310 / 100 trades (slightly UP from prior +$160-$280 because v3 B's bias check came out stronger and v2 #6 is no longer purely defensive; offset by v3 F retraction).
Canonical-field-validity rule. Never trust convenience flags like
stop_triggeredwithout cross-checking against the source-of-truth field likeexit_reason. Audit-log boolean fields can fall out of sync silently when writer code paths drift; the textual exit_reason is the canonical record because it's set by the order-executor on every exit. When in doubt: query both, compare counts, investigate the gap.
This joins the prior lessons: 1. (Β§1) Use the 4-check bias gauntlet on every finding. 2. (Β§6 lesson #1) Sample size: n β₯ 12 minimum, ideally β₯ 20. 3. (Β§6 lesson #2) Skip rules need full-population spillover check. 4. (Β§6 lesson #3) The named outlier set (now 5) carries the strategy. 5. (Β§6 lesson #4) Universe view + live data must agree before shipping. 6. (Β§10 lesson #6 β Finding #228) True-simulator rule: proxies and observational tables are NOT counterfactual evidence. 7. (Β§14.7 lesson #7 β Finding #233) Canonical-field-validity rule.
docs/FINDINGS.md Finding #232docs/FINDINGS.md Finding #233docs/STRATEGY_PLAYBOOK.md Β§1 + Β§2 + Β§3analysis.action_queue, produced by build_action_queue())scripts/_revalidate_queue_evidence.sql,
scripts/_revalidate_f_and_ruleb.sql, scripts/_revalidate_corrected.sql
(pulled live data for this audit; safe to re-run).cursor/memory.md Β§K (3-outlier β 5-outlier)docs/AGENT_HANDOFF.md Β§11 (current state + Action Queue)Update Apr 26, 2026 (Phase 1 of Strategy Analysis Revamp): the first two items below have been resolved; remaining items below the divider remain deferred.
stop_triggered writer fix β RESOLVED as Finding #236 (Apr 26, 2026).
Both primary and fallback close paths in src/earnings/position_manager.py
now derive stop_triggered from exit_reason ILIKE '%stop%' at write time;
55 historical rows backfilled. Boolean now matches canonical 57/57.regime_at_entry β RESOLVED as Finding #234
(Apr 26, 2026). Root cause was NOT a transient SPY-fetch failure but a
divergent INSERT path: TradingOrchestrator._persist_position_open was
silently dropping the regime/VIX/SP500 columns while
PositionManager.persist_position_open was setting them. Fixed +
backfilled. regime coverage now 116/116 (100%), VIX 114/116 (98%; 2 rows
Mar-18 still NULL due to yfinance ^VIX KeyError on that specific date β
recoverable on next yfinance retry).Still deferred (next-pass to-do):
docs/STRATEGY_ANALYSIS_ARCHITECTURE.md Β§0.6. (Update β β
Phase 3 COMPLETE:
PR 1-5 shipped #306-#319, Phase 4 #320; gates met May 28, 2026.)Verified Apr 27, 2026 LATE evening: the verified-findings table (Β§3),
retracted findings (Β§4), 4-check gauntlet (Β§5), and ranked queue
(docs/STRATEGY_PLAYBOOK.md Β§2) are all unchanged since Apr 25. Two days
of sustained work between Apr 25 and Apr 27 produced zero new strategy
rules and zero rule retractions. Everything that shipped was data
infrastructure, audit completeness, or operational correctness.
For traceability, the per-day breakdown:
pm_v2 snapshot enrichment; actual_beat
98.1% via post-event price-snapshot backfill; SEC EDGAR press-release
timestamps 90.9%). No rule changes; the simulator that consumes them
is gated on the 30-day equity-bar capture window (~May 26, 2026).
(Update β that window has since CLOSED; Phase 3 PR 1-5 + Phase 4 COMPLETE
#306-#322. The gating note reflects the Apr-26 state.)hold_duration_hours writer added β
although that fix was itself broken until Apr 27 evening, see #263),
unified integrity monitor (replaces v2/v3 watchdogs), Database tab
redesign, storage hygiene + backup OOM fix, full doc refresh. No
strategy changes; closes data-recording gaps that affect Β§3 evidence
reliability going forward.:exit_time::timestamptz
pg8000 syntax error that silently broke the close path for every trade
since deploy (hold_duration_hours, conviction_at_exit,
prediction_correct, financing aggregates all NULL). Found that
Yahoo's GAAP scrape had over-ridden Polymarket's confirmed YES on VZ
earnings (Finding #264). Found that the dashboard volume display fix
(#265) was incomplete in production. Fixed all three. Then RCA'd
volume_24h_at_entry regression (Finding #272) β Polymarket's Gamma
API stopped reliably returning volume24hr field; collector wrote 0;
signal_generator treated 0 as None; 76 consecutive trades had NULL.
Backfilled 114/117 historical rows from price_snapshots. Plus 5 newly
surfaced ancillary findings (#267-#274) β most documented for operator
decision rather than auto-fixed.Net effect on the audit: none. Rule rankings unchanged. Bias gauntlet
unchanged. The 5-outlier curse still defines the strategy's net P&L
(GPRO + DELL + SWBI + TXN + STUB β $430 of $167 net on 116 trades). Re-run
scripts/strategy_performance_analysis.py for current numbers; the Β§3
evidence may have refreshed slightly because regime/VIX/conviction-at-exit
data is now populating forward-only on every new trade.
Re-trigger conditions for next audit pass: (a) 50 more closed trades (currently 117; trigger at 150+), or (b) Action Queue items #1 + #2 (skip downtrend LONGs + RuleB Β±90min) ship and we want to measure impact. (Update β (b) has shipped: Rule #1 is active (#304) and RuleB is in Phase 1 dry-run (#305); live impact measurement is signal-density-limited, which is why the Phase 3 simulator now carries the counterfactual evaluation.)
Trigger: operator-directed full strategy review (Jul 5, 2026). Three independent observations converged:
simulate_window_only confirmed the bug in code: after the
silent window, any bar with low <= stop returned an exit at the
stop price, even when the price had crossed the stop inside the
window and was still beyond it at rearm. The as-shipped #305
cancel-and-rearm mechanism re-places a STP-LMT that triggers instantly
and fills near market β simulate_finding_a_hard already modeled this
as rearm_gap_open (the arm that retracted FA_hard at β$1,545/100 in
Β§10); the RuleB arm never got the same handling.Fix + corrected numbers (prod re-run, n=129): simulate_window_only
is now gap-aware (first post-window bar OPEN fill when the open is already
beyond the stop; signature unchanged so the phase3_simulator
reference-adapter contract holds; RuleG inherits).
| Rule | Pre-#330 headline / ex-top-2 (per-100) | Gap-aware headline / ex-top-2 (per-100) | Verdict |
|---|---|---|---|
| RuleB | +$224.22 / +$122.14 | β$209.41 / β$194.00 | π FAILED-GAUNTLET |
| RuleG | +$137.48 / +$7.60 | β$29.68 / β$11.44 | π FAIL (was already fragile) |
Optimism bias β $535 across 129 trades. Ex-repeat-tickers RuleB is β$226.67/100 β negative on every Β§K check. Mechanism: the window's recovery saves (dip-through-then-recover, preserved in the gap-aware sim) are outweighed ~2:1 by gap-through cases where the rearm fill is far worse than the original stop fill.
Decisions: RuleB Phase-2 active flip CANCELLED (Phase-1 dry-run code remains, logging only β its 8,026-duplicate-line log spam also fixed); v3 D + v3 E RuleB dependencies REMOVED (Β§12.2 / Β§12.3 re-evaluate as pure selection rules at trade #150+ / SHORT nβ₯30); the #323.b "0.92-0.99 conditional on RuleB active" lever is dead. NO stop-loss rule variant currently survives the gauntlet.
Audit-process lesson #7: a TRUE simulator (memory Β§K) must model the REARM path of any cancel-and-rearm rule, not just the silence. The PR-1 Β§10.1 cross-check (simulator-vs-reference consistency) cannot catch a shared mechanism error β mechanism review against as-shipped order semantics is a separate mandatory step for any stop-manipulation rule.
Canonical narrative: docs/FINDINGS.md Finding #330.