Findings

Single source of truth for all findings, fixes, pending validations, and production evidence. The dashboard Testing Findings tab renders from this file via /api/architecture-findings. Do NOT create finding entries in other files. See memory.md for the documentation rule.

Status Legend

• done: implemented and verified
• in_progress: being implemented
• deferred: known improvement, not urgent, planned for future
• proposed: ready for review
• approved: approved to implement

Numbering traps (read first)

• Three formats. #1–#105 are the Jan 2026 dashboard cards under ## Legacy Findings (#1-#105). #106–#221 are numbered list items whose list index differs from the finding number (list 14) = Finding #106). Later entries use ### Finding N — headers; newest sit near the top.
• List items 1)–13) are not legacy #1–#13. They are a Feb 2 2026 architecture list. The dashboard parser keys Format A by (Finding #N) in the title, not by list index.
• #132–#137 were assigned twice: Feb 5 (list 39–44, PRIMARY) and Feb 7 Phase 3D (list 45–50, aliases). Cite the line. First-write-wins on the Testing Findings tab.
• Gaps: #128 (deferred note), #271 (never used), #309 (never assigned). Sub-entries share the parent's number.

Findings

1) Entry + stop atomicity
- Severity: Critical
- Status: done
- Implementation: Complete in code; verified in live trading Feb 2, 2026.
- Proposed change: Use atomic IBKR order placement for entry+stop (preserve NYSE stop-limit behavior), verify stop placement before confirming entry.
- Files: src/ibkr/client.py - Live Test Result (Feb 2, 2026 09:34 AM ET): - PLTR entry filled @ $148.70, stop placed @ $144.23 within 1 second - Stop Order ID 84904 confirmed in IBKR - Batch completed: Total: 1, Success: 1, Failed: 0

2) Batch stop placement safety
- Severity: High
- Status: done
- Implementation: Complete in code; verified in live trading Feb 2, 2026.
- Proposed change: Treat stop placement failures as critical (retry, alert, and/or fail batch) so no filled entry is left unprotected.
- Files: src/earnings/batch_order_manager.py - Live Test Result (Feb 2, 2026 09:34 AM ET): - Batch ID 319b8adb executed successfully - Stop placement confirmed for PLTR immediately after entry fill - Total batch time: 4467ms (submit: 1778ms)

3) Duplicate order prevention / idempotency
- Severity: High
- Status: done
- Implementation: Complete in code; verified in live trading Feb 2, 2026.
- Proposed change: Add per-symbol lock or DB idempotency key before order placement to prevent race duplicates.
- Files: src/earnings/trading_orchestrator.py - Live Test Result (Feb 2, 2026 09:34 AM ET): - AAPL correctly skipped: "Existing positions (will skip): {'AAPL'}" - Also verified via test_duplicate_prevention.py at 04:40 AM ET

4) Market hours gating / holiday handling
- Severity: High
- Status: done
- Implementation: Complete (IBKR liquidHours gating + calendar fallback).
- Proposed change: Gate entry/exit orders to regular market hours using IBKR market hours; allow stop-loss/emergency exits during extended hours.
- Files: src/earnings/order_executor.py and scheduling layer - Live Test Result (Feb 2, 2026 04:40 AM ET): - Pre-market test: Entry correctly blocked (market_status: pre_market) - Instant exit confirmed always allowed - Tested via test_market_hours_gating.py

5) IBKR error blocking
- Severity: Medium
- Status: done
- Implementation: Complete. trading_block.txt persistence, 1100 transient (cleared on reconnect), 1300 critical (blocks trading). Unit tests in test_ibkr_error_block_policy.py.
- Files: src/ibkr/client.py

6) Collector retries/backoff
- Severity: Medium
- Status: done
- Implementation: Complete. Exponential backoff implemented in all 9 collector files. Running stable for 2+ months.
- Files: src/data_tracker/collectors/continuous_collector.py, src/data_tracker/collectors/live_trades_collector.py, src/data_tracker/collectors/earnings_market_fetcher.py, src/data_tracker/collectors/gamma_collector.py, src/data_tracker/collectors/api_health_checker.py, src/data_tracker/collectors/prices_backfill_service.py, src/data_tracker/collectors/sync_earnings_markets.py, src/data_tracker/collectors/dome_client.py, src/data_tracker/collectors/polymarket_client.py

7) DB-level trade dedupe
- Severity: Medium
- Status: done
- Implementation: Added trade_dedupe table with global unique trade_id; collectors insert into dedupe first, then trades.
- Files: src/data_tracker/collectors/live_trades_collector.py, src/data_tracker/collectors/backfill_trades.py, src/data_tracker/migrations/017_trade_dedupe.sql

8) Backfill state atomic writes
- Severity: Medium
- Status: done
- Implementation: Atomic state writes via temp + fsync + rename.
- Files: src/data_tracker/collectors/prices_backfill_service.py, src/data_tracker/collectors/backfill_trades.py, src/data_tracker/collectors/orderbook_backfill_service.py, src/data_tracker/watchdog_v3.py, src/ibkr/client.py

9) Reconciliation window hybrid schedule
- Severity: Medium
- Status: done
- Implementation: Daily reconciliation timer + monthly full-history timer enabled.
- Files: deploy/systemd/polymarket-trade-reconcile.timer, deploy/systemd/polymarket-trade-reconcile-full.*, deploy/install_services.sh

10) Dashboard slippage weighting + staleness warnings
- Severity: Low
- Status: done
- Implementation: Weighted slippage from trade notional + stale warnings in UI.
- Files: earnings_dashboard/app.py, earnings_dashboard/templates/dashboard.html

11) Fractional-share display consistency
- Severity: Low
- Status: done
- Implementation: Preserve fractional shares display with consistent formatting.
- Files: earnings_dashboard/app.py, earnings_dashboard/templates/dashboard.html

12) Closed-market instant exit safety
- Severity: Critical
- Status: done
- Implementation: When market is fully closed, instant exit uses GTC LIMIT only (no MARKET fallback).
- Files: src/earnings/order_executor.py, docs/STRATEGY.md, docs/EDGE_CASES.md

13) Orderbook collector partial commit safety
- Severity: High
- Status: done
- Implementation: Commit per-market snapshots to avoid full-batch rollback.
- Files: src/data_tracker/collectors/main_collector.py

Testing Gaps (recommended additions) Unit tests for entry+stop atomicity (including failure cases). Integration tests for batch orders to ensure stop placement success. Reconciliation tests covering old trades and partial fills. Dashboard P&L tests covering fractional shares and stale pricing.

14) Reconciliation Disconnection Guard (Finding #106) - Severity: Critical - Status: done - Implementation: Complete (Feb 2, 2026). Guards, sanity checks, and external position stop_price population. - Root Cause: Reconciliation ran while IBKR was disconnected, incorrectly closing AAPL as "orphaned", then re-adding it as external position without stop_price. - Fixes: 1. Added IBKR connection guard at start of _reconcile_positions_with_ibkr() 2. Added sanity check: abort if IBKR returns 0 positions but DB has open positions 3. DB repair: restored original AAPL position (ID 48), deleted fake external (ID 54) 4. External position creation now populates stop_price from IBKR stop orders - Files: src/earnings/trading_orchestrator.py

15) Stale Conviction Data - Gamma API Batch Reliability (Finding #107) - Severity: High - Status: done - Implementation: Complete (Feb 2, 2026). Two-phase fetch: batch + individual fallback + 0% display fix. - Root Cause: Gamma API batch endpoint (/markets?condition_ids=...) silently drops markets from responses. Requesting 25 condition_ids returns only ~20. Affected markets show "STALE" in dashboard. - Additional Bug Found: Template used e.conviction and ... which is falsy for 0.0, causing 0% convictions to show as "50%". - Fixes: 1. Phase 1: Batch fetch (unchanged, fast) 2. Phase 2: Identify missing markets, fetch each individually (reliable) 3. Log warnings when markets are still missing after both phases 4. Fixed template to use e.conviction is not none instead of e.conviction and - Files: earnings_dashboard/app.py, earnings_dashboard/templates/dashboard.html

Tests left to be done post implementation - Finding 12) Trigger instant exit when market is fully closed (8 PM - 4 AM ET): confirm GTC LIMIT only, no MARKET fallback. (Pending - will test if instant exit needed during 8 PM - 4 AM window)

Tests completed - Finding 1) Live entry+stop test - PASSED Feb 2, 2026 09:34 AM ET: PLTR entry filled, stop placed within 1 second (order ID 84904). - Finding 1) Second live test - PASSED Feb 2, 2026 03:58 PM ET: PEP @ $155.12 + stop @ $152.01, PFE @ $26.64 + stop @ $26.11 (Batch 34c25f75). - Finding 2) Live batch entry test - PASSED Feb 2, 2026 09:34 AM ET: Batch 319b8adb completed with stop placement confirmed. - Finding 2) Second batch test - PASSED Feb 2, 2026 03:58 PM ET: Batch 34c25f75 - 2/2 success, 0 failed, stops placed for both. - Finding 3) Duplicate prevention test - PASSED Feb 2, 2026 04:40 AM ET: AAPL correctly skipped due to existing position. - Finding 3) Second duplicate test - PASSED Feb 2, 2026 03:58 PM ET: PLTR correctly skipped (existing position) in batch. - Finding 4) Market hours gating test - PASSED Feb 2, 2026 04:40 AM ET: Pre-market entry blocked, instant exit allowed. - Finding 5) Simulated critical IBKR error + block file removal using scripts/verify_retry_backoff_and_block.py. - Finding 6) Simulated 429/5xx retries across collectors/backfills using scripts/verify_retry_backoff_and_block.py. - Finding 14) Code review VERIFIED Feb 2, 2026: IBKR connection guard (line 935) + zero position sanity check (line 1001) confirmed in _reconcile_positions_with_ibkr(). - Finding 15) Stale conviction fix VERIFIED Feb 2, 2026: All markets now show fresh data from gamma_api, 0% convictions display correctly. - Finding 16) IBKR Connection Resilience VERIFIED Feb 3, 2026: IBC auto-login fixed, alert rate limiting added.

16) IBKR Connection Resilience - IBC Auto-Login & Alert Fatigue (Finding #108) - Severity: High - Status: done - Implementation: Complete (Feb 3, 2026). Fixed IBC Java path for auto-login, added alert rate limiting. - Root Cause: Two issues identified: 1. IBC gatewaystart.sh used system OpenJDK (/usr/lib/jvm/java-17-openjdk-amd64/bin) which lacks JavaFX, causing NoClassDefFoundError: javafx/scene/Parent. IB Gateway restarts without auto-login. 2. Reconciliation skip alert (Finding #106 guard) fired every 5 minutes when IBKR disconnected, causing 50+ alerts overnight. - Fixes: 1. Updated /opt/ibc/gatewaystart.sh line 33: JAVA_PATH=/opt/i4j_jres/Oda-jK0QgTEmVssfllLP/17.0.10.0.101-zulu/bin (bundled Zulu Java with JavaFX) 2. Switched from ibgateway.service to ibgateway-ibc.service (enabled, tested auto-login + 2FA) 3. Added _last_disconnect_alert_time tracking in trading_orchestrator.py 4. Alert rate limited: only first alert per disconnect episode, then every 30 min 5. Log at DEBUG level for suppressed alerts (issue still visible in logs) - Files: - /opt/ibc/gatewaystart.sh (server) - /etc/systemd/system/ibgateway-ibc.service (enabled) - src/earnings/trading_orchestrator.py (rate limiting)

17) Hardware/Software Stop Race Condition & Exit Retry Loop (Finding #109) - Severity: Critical - Status: done - Implementation: Complete (Feb 3, 2026). IBKR position check before software exit + exit-in-progress tracking. - Root Cause: Two bugs caused PFE LONG 12 shares to spiral into SHORT 108 shares: 1. Race Condition: Hardware stop #93950 filled at 11:42:49 (sold 12, position flat). At 11:42:51, software stop backup checked _check_hardware_stop_status() which only looks for active stop orders (PreSubmitted/Submitted), not whether position is still open. Since stop had filled (status=Filled), it returned False. Software executed another SELL, creating SHORT position. 2. Exit Retry Loop: Exit verification failed (saw -12 shares instead of 0). Position stayed in active_positions. Main loop (1s interval) kept triggering software stop backup. Each iteration placed another SELL, growing SHORT from -12 to -108 shares. - Fixes: 1. Added _exit_in_progress set to track symbols with pending exits - prevents retry loops 2. Added _get_ibkr_position_shares() helper to check current IBKR position before software exit 3. Modified check_stop_loss_backup(): - Skip if symbol already in _exit_in_progress - Before executing: verify IBKR position is not flat (prevents race with hardware stop) - Verify position direction matches expected (prevents selling into SHORT) - Add to _exit_in_progress BEFORE executing to prevent loops 4. Added _handle_hardware_stop_fill() to clean up when hardware stop already closed position 5. Clear _exit_in_progress only on successful verified exit 6. Corrected PFE database record: direction=LONG, shares=12, entry=$26.73, exit=$26.13, pnl=-$7.20 - Files: - src/earnings/trading_orchestrator.py (lines 258-266, 3130-3185, 3396-3445, 3203, 3213) - Database Fix: - PFE position ID 57: Corrected from SHORT/108/$26.00 to LONG/12/$26.73 - P&L corrected from -$31.32 to -$7.20 - Prevention: - Software backup now checks IBKR position before acting - Exit attempts are tracked to prevent infinite loops - Direction mismatch triggers critical alert, not automatic action

18) Scheduled Exit DB Update Failure & Early Exit Trigger (Finding #110) - Severity: High - Status: done - Implementation: Complete (Feb 3, 2026). Fixed pg8000 type casting + exit timing threshold. - Root Cause: Two bugs caused PEP scheduled exit data to be corrupted: 1. DB Update Failed: _persist_position_close_with_exit_details() failed with "invalid input syntax for type integer" when passing Python float slippage values to PostgreSQL numeric columns via pg8000. Position remained "open" in DB. 2. Early Exit Trigger: check_scheduled_exits() used <= 120 threshold (2 minutes). At 9:30:00 AM, time_to_exit=120 and 120<=120 was TRUE, triggering BMO exits 2 minutes early (9:30 instead of 9:32). 3. Reconciliation Overwrite: Because position was still "open" after failed DB update, reconciliation closed it as "orphaned_position" with wrong data ($158.98 exit instead of actual $155.96). - Fixes: 1. Added to_decimal() helper to cast float values to Decimal before DB insert/update 2. Applied to all slippage parameters: exit_slippage, exit_slippage_pct, exit_limit_price, exit_actual_slippage, entry_slippage, entry_slippage_pct, entry_limit_price 3. Changed check_scheduled_exits() threshold from <= 120 to <= 10 (10-second grace window) 4. Changed _execute_batch_exit() threshold from <= 120 to <= 10 for consistency 5. Added critical alert when DB update fails (instead of silent error log) 6. Corrected PEP database records: exit_price=$155.96, pnl=$1.68, reason=scheduled_exit - Files: - src/earnings/trading_orchestrator.py (lines 1997-2010, 2129-2134, 3332-3339, 3924-3928, 1908-1922) - Database Fix: - PEP position ID 56 (trading_positions): exit_price=155.96, pnl=1.68, reason=scheduled_exit - PEP position ID 22 (trade_audit_log): exit_price=155.96, pnl=1.68, reason=scheduled_exit - Prevention: - Float-to-Decimal conversion prevents pg8000 type mismatches - 10-second threshold ensures exits happen at scheduled time, not 2 minutes early - Critical alerts on DB failures prevent silent data corruption

19) Auto-Fix Data Corruption & Duplicate Shutdown Alerts (Finding #111) - Severity: Critical - Status: done - Implementation: Complete (Feb 3, 2026). Converted auto-fix to detection-only + idempotent shutdown + hardware stop DB fix. - Root Cause: Three issues identified during Feb 3, 2026 incident analysis: 1. Auto-Fix Data Corruption: _reconcile_positions_with_ibkr() automatically "fixed" discrepancies between DB and IBKR by: - Closing "orphaned" positions (position in DB but not IBKR) with calculated P&L using current price - Syncing share counts to match IBKR This caused data corruption in two incidents: - PFE: Synced 12 shares to 108 (from erroneous exit loop SHORT) - PEP: Closed as "orphaned" with wrong $7.72 P&L (actual was $1.68) The auto-fix was dangerous because: - It didn't know if DB update was just pending (position recently closed correctly) - It trusted IBKR even when IBKR had erroneous positions (from bugs) - Calculated P&L used current price, not actual exit price 2. Duplicate Shutdown Alerts: Signal handler called shutdown() then sys.exit(0), but sys.exit() triggers the finally block in run() which also called shutdown(). Result: 2-3 shutdown alerts per shutdown. - Fixes: 1. Detection-Only Reconciliation: - Duplicates: Log CRITICAL + alert, no auto-merge - Orphaned positions: Log CRITICAL + alert, no auto-close - Share mismatches: Log CRITICAL + alert, no auto-sync - External positions: STILL auto-added (safe operation - doesn't overwrite data) 2. Alert for Manual Fix: Issues detected send Telegram alert with list of discrepancies requiring manual review 3. Idempotent Shutdown: Added _shutdown_called flag to EarningsTrader.__init__(). shutdown() now checks flag and returns early if already called. Single alert per shutdown guaranteed. - Files: - src/earnings/trading_orchestrator.py (lines 916-1325: reconciliation function rewritten) - run_earnings_trader.py (lines 146, 361-376: idempotent shutdown) - Additional Bug (AMD): - _handle_hardware_stop_fill() called non-existent method _close_position_in_db() - Caused AttributeError, DB not updated, then reconciliation "fixed" with wrong P&L - Fix: Changed to call _persist_position_close_with_exit_details() instead - AMD record corrected: exit_price=$244.84, P&L=-$5.00 (not -$6.66) - Database Fixes: - PFE position ID 57: direction=LONG, shares=12, exit=$26.13, pnl=-$7.20 - PEP position ID 56/22: exit=$155.96, pnl=$1.68, reason=scheduled_exit - AMD position ID 58/24: exit=$244.84, pnl=-$5.00, reason=stop_loss:hardware - Prevention: - Auto-fix can no longer overwrite correct data with wrong calculated values - Discrepancies require human review before fixing (check IBKR activity log) - Shutdown sends exactly one alert regardless of how shutdown is triggered - Hardware stop cleanup now uses correct DB method

20) IBKR Connection Reliability Overhaul (Finding #112) - Severity: Critical - Status: done - Implementation: Complete (Feb 4, 2026). Removed dual state tracking, fixed race conditions, added health verification. - Root Cause: IBKR Gateway disconnected at 11:25 PM ET on Feb 3 and never reconnected. 7 AMC signals (QCOM, GOOGL, F, etc.) missed execution. Three issues identified: 1. Dual State Tracking: _connected flag and _connection_state enum could diverge. Comment at line 172 said _connected was removed, but it was still being used at lines 279, 302-303, 313, 1595. 2. Race Condition: Heartbeat loop checked _reconnect_in_progress without holding _reconnect_lock, allowing competing reconnect attempts. 3. No Health Verification: After reconnect, connection was considered good if ib.connect() succeeded, but connection could be degraded or non-functional. - Fixes: 1. Removed all _connected flag references - now using only _connection_state enum 2. Added lock acquisition before checking _reconnect_in_progress in heartbeat loop 3. Added _verify_connection_health() method that performs functional checks after reconnect: - Verifies ib.isConnected() returns True - Verifies accountValues() returns data (functional check) - Verifies managedAccounts() returns account ID 4. Reconnect loop now retries if health check fails after successful connect - Files: - src/ibkr/client.py (lines 279, 302-303, 313, 455, 1595, new _verify_connection_health method)

21) Ticker Extraction Reliability (Finding #113) - Severity: Medium - Status: done - Implementation: Complete (Feb 4, 2026). Added fallback patterns and logging for skipped markets. - Root Cause: QRVO missing from earnings_markets because single regex pattern \(([A-Z]{1,5})\) failed when ticker wasn't in parentheses. Markets silently skipped with no logging. - Fixes: 1. Added fallback ticker extraction patterns: - Pattern 1: Standard parentheses (TICKER) - Pattern 2: "ticker SYMBOL" or "ticker: SYMBOL" - Pattern 3: All-caps word at start of question - Pattern 4: Quoted ticker 'TICKER' or "TICKER" - Pattern 5: Last resort - any standalone 1-5 letter uppercase word (filtered) 2. Added warning logs when markets are skipped due to missing ticker or condition_id - Files: - src/data_tracker/collectors/earnings_market_fetcher.py (_extract_ticker_from_question method, extract_markets_from_events method)

22) Exit Reason Display Normalization (Finding #114) - Severity: Low - Status: done - Implementation: Complete (Feb 4, 2026). Standardized exit reason display in dashboard. - Root Cause: Exit reasons displayed inconsistently: - Raw values like instant_exit:wrong_prediction, stop_loss:hardware, execution_ledger_reconciliation - Template had incomplete switch logic that didn't cover all variations - Fixes: 1. Added normalize_exit_reason() function to dashboard app.py 2. Converts internal exit reasons to user-friendly format: - scheduled_exit, scheduled_batch → "Scheduled Exit" - instant_exit:wrong_prediction → "Instant Exit (Wrong Prediction)" - stop_loss:hardware → "Stop Loss (Hardware)" - execution_ledger_reconciliation → "Ledger Reconciliation" 3. Updated dashboard template to use normalized display with appropriate icons/colors - Files: - earnings_dashboard/app.py (normalize_exit_reason function, trade history formatting) - earnings_dashboard/templates/dashboard.html (exit reason display)

23) Exit Retry Loop Prevention - Complete Overhaul (Finding #115) - Severity: CRITICAL - Status: done - Implementation: Complete (Feb 4, 2026). Comprehensive fix for exit retry loop that caused CME SHORT -38 shares. - Root Cause: Finding #109's fix was incomplete. Multiple compounding issues: 1. _exit_in_progress only applied to stop_loss_backup - NOT to scheduled exits, instant exits, or batch exits 2. No IBKR position verification before exit - Orders placed without checking position still exists in expected direction 3. DB persist failure didn't block cleanup - active_positions cleared even when DB update failed 4. Exit verification used abs() - Couldn't distinguish "position still open" from "position flipped to opposite direction" 5. pg8000 type casting issue - COALESCE with integer literal 0 caused slippage values to fail - Incident: CME position Feb 4, 2026: - Original: LONG 1 share @ $293.46 - First scheduled exit executed successfully, DB update failed - Service restart reloaded CME as "open" from DB - Main loop kept triggering scheduled exits - Each iteration placed another SELL order - Result: CME SHORT -38 shares (CATASTROPHIC) - Fixes: 1. Added _exit_in_progress to ALL exit paths: - check_scheduled_exits() - Skip if symbol in _exit_in_progress - execute_scheduled_exit() - Add symbol BEFORE order, clear only on DB success - execute_instant_exit() - Add symbol BEFORE order, clear only on DB success - _execute_batch_exit() - Add all symbols BEFORE batch, clear only on DB success - check_positions_for_exit() - Skip if symbol in _exit_in_progress - _process_instant_exit_queue() - Skip queued items if already in progress 2. Added IBKR position verification before exit: - New _verify_ibkr_position_for_exit() method - Checks position exists in expected direction (LONG vs SHORT) - Blocks exit if position is flat, opposite direction, or can't verify - Sends critical alert on direction mismatch 3. DB persist now returns bool: - _persist_position_close_with_exit_details() returns True/False - Callers check return value before cleanup - If False: keep in active_positions, keep in _exit_in_progress - This prevents service restart from re-triggering failed exits 4. Exit verification enhanced to detect direction flip: - Uses signed position value (not abs()) - Detects if position flipped to opposite direction - Logs CRITICAL with specific direction flip warning - Sends separate alert for direction flip vs incomplete exit 5. Fixed pg8000 type casting: - Changed COALESCE defaults from 0 to 0::numeric - Prevents integer/numeric type inference issues - Files: - src/earnings/trading_orchestrator.py: - Line 265: _exit_in_progress set initialized - Lines 3350-3360: check_scheduled_exits() - added skip check - Lines 3130-3145: execute_scheduled_exit() - added marking and verification - Lines 3197-3213: execute_instant_exit() - added marking and verification - Lines 3337-3362: execute_instant_exit() - DB persist check before cleanup - Lines 4073-4098: _execute_batch_exit() - added marking - Lines 4114-4156: _execute_batch_exit() - persist before cleanup, check return - Lines 4193-4208: _process_instant_exit_queue() - skip if in progress - Lines 3413-3421: check_positions_for_exit() - skip check - Lines 3680-3714: _handle_stop_order_fill() - check persist return - Lines 2260-2320: New _verify_ibkr_position_for_exit() method - Lines 1986-2002: _finalize_position_exit() - check DB persist return - Lines 2205-2225: _persist_position_close_with_exit_details() - return bool - Lines 2170: Fixed COALESCE with 0::numeric - src/earnings/order_executor.py: - Lines 1148-1195: place_instant_exit() - enhanced verification with direction check - Lines 1375-1420: place_scheduled_exit() - enhanced verification with direction check - Acceptance Test: Position exit must be blocked if: - IBKR shows position in opposite direction - DB persist fails (position stays in active_positions) - Same symbol already has exit in progress

24) Trade Data Integrity and Exit Reason Standardization (Finding #116) - Severity: Medium - Status: done - Implementation: Complete (Feb 4, 2026). Fixed three data integrity issues. - Root Cause: Multiple issues: 1. CME showed wrong P&L: Dashboard displayed bug artifact (ID 62, -$19.60) instead of real trade (ID 60, -$3.85) 2. Exit reasons not standardized: EFX/NYT showed "Ledger Reconciliation" when they were actually stop losses (had stop_order_id) 3. Unverified DB writes: External position sync auto-added IBKR positions without verification, causing bug artifacts to pollute strategy data - Fixes: 1. Exclude recovery records from trade log: - Added filter exit_reason NOT LIKE 'manual_recovery:%' to 3 queries - Bug artifacts no longer skew strategy P&L calculations 2. Detect stop_loss in reconciliation: - Added stop_order_id to DB position query - If position had stop_order_id, set exit_reason to 'stop_loss:hardware_reconciled' - Future reconciled stops correctly identified 3. External positions: detection-only: - Removed auto-INSERT for external IBKR positions - Now logs warning and sends alert for manual review - Prevents unverified data from entering database 4. Updated normalize_exit_reason(): - Added 'hardware_stop' pattern matching - 'stop_loss:hardware_reconciled' displays as "Stop Loss (Hardware)" 5. Corrected existing records: - EFX (ID 61) and NYT (ID 59) exit_reason updated to 'stop_loss:hardware_reconciled' - Files: - earnings_dashboard/app.py: Lines 1041, 2927, 3445 (filters), 3353-3358 (normalize) - src/earnings/trading_orchestrator.py: Lines 1007-1027 (query), 1118-1124 (exit reason logic), 1238-1268 (detection-only) - Acceptance Test: - CME shows LONG 1 share, -$3.85, "Scheduled Exit" - EFX/NYT show "Stop Loss (Hardware)" - External positions no longer auto-added to database

25) Add Missing Thread Locks (Finding #117) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Added thread locks for shared state. - Root Cause: active_positions, _pending_batch_entry, and _reserved_capital_for_bmo were accessed without locks, creating potential race conditions when background threads (EarningsDetector callback) accessed shared state simultaneously with main thread. - Fixes: 1. Added three new locks in __init__: - _positions_lock: Guards active_positions dict - _batch_lock: Guards _pending_batch_entry - _capital_lock: Guards capital reservation values 2. Created thread-safe accessor methods: - _get_position(), _set_position(), _remove_position(), _has_position() - _get_all_position_symbols() - _get_reserved_capital(), _set_reserved_capital(), _add_to_reserved_capital() - Files: - src/earnings/trading_orchestrator.py: Lines 251-255 (locks), 321-366 (accessor methods) - Acceptance Test: - No race conditions under concurrent access - Thread-safe methods available for future refactoring

26) Fix Division by Zero Guards (Finding #118) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Added guards to prevent ZeroDivisionError. - Root Cause: Several division operations in risk_manager.py could throw ZeroDivisionError if total_capital, daily_starting_capital, or target values were 0 (e.g., on startup). - Fixes: 1. Line 684: new_exposure_pct = new_exposure / self.total_capital if self.total_capital > 0 else 1.0 2. Line 877: Added elif total_target > 0 before scale-down calculation 3. Line 1052-1053: Added early return if daily_starting_capital <= 0 4. Line 897: Added guard for position percentage display logging - Files: - src/earnings/risk_manager.py: Lines 684, 877, 897, 1052-1053 - Acceptance Test: - No ZeroDivisionError with zero capital - Graceful handling on startup/edge cases

27) Add Missing _send_alert Method (Finding #119) - Severity: Medium - Status: done - Implementation: Complete (Feb 5, 2026). Added _send_alert method to OrderExecutor. - Root Cause: OrderExecutor called self._send_alert() at lines 1058, 1177, 1182, 1187, 1401, 1406, 1411 for critical errors (direction flips, position sync issues) but the method did not exist, causing AttributeError. - Fixes: 1. Added _send_alert(self, message: str) method to OrderExecutor class 2. Method uses fire-and-forget pattern to send CRITICAL level alerts 3. Wraps async AlertManager calls in try/except for safe execution - Files: - src/earnings/order_executor.py: Lines 210-238 (new method) - Acceptance Test: - Critical alerts sent for direction flips - No AttributeError when calling _send_alert

28) Add Entry Verification (Finding #120) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Added IBKR position verification after entry. - Root Cause: After entry orders filled, we tracked position in memory but never verified IBKR actually had the position. If order silently failed, DB/IBKR desync would occur. - Fixes: 1. Added entry_verified and ibkr_position_at_entry fields to ExecutionResult 2. After entry+stop completes, verify IBKR has the position 3. Check direction matches expected (BUY->LONG, SELL->SHORT) 4. Allow 1 share tolerance for fractional share rounding 5. Send alert on direction mismatch - Files: - src/earnings/order_executor.py: - Lines 117-118 (new fields in ExecutionResult) - Lines 139-140 (to_dict updated) - Lines 834-889 (entry verification logic in place_entry_with_stop) - Acceptance Test: - Entry success logs "(verified)" when IBKR position confirmed - Entry logs "(UNVERIFIED)" if position not found in IBKR - Alert sent on direction mismatch

29) Bound Instant Exit Queue (Finding #121) - Severity: Medium - Status: done - Implementation: Complete (Feb 5, 2026). Queue bounded to prevent memory leak. - Root Cause: _instant_exit_queue was unbounded queue.Queue(). If exits were queued faster than processed, queue could grow unbounded causing memory issues. - Fixes: 1. Changed queue.Queue() to queue.Queue(maxsize=100) 2. Changed put() to put_nowait() at both call sites 3. Added try/except queue.Full handling with CRITICAL log 4. Full queue logs message with current queue size - Files: - src/earnings/trading_orchestrator.py: - Line 230 (maxsize=100) - Lines 604-610 (crash recovery put with Full handling) - Lines 3481-3487 (earnings detection put with Full handling) - Acceptance Test: - Queue does not grow beyond 100 items - CRITICAL log when queue full (operational alert)

30) Standardize Error Handling with Result Pattern (Finding #122) - Severity: Medium - Status: done - Implementation: Complete (Feb 5, 2026). Created Result pattern type. - Root Cause: Methods used mixed patterns (None, exceptions, bool) for error handling, making error propagation inconsistent and hard to debug. - Fixes: 1. Created src/earnings/types.py with generic Result[T, E] type 2. Added Ok() and Err() convenience constructors 3. Added is_ok(), is_err(), unwrap(), unwrap_or() methods 4. Documented usage pattern for gradual migration - Files: - src/earnings/types.py (new file) - Acceptance Test: - Result type importable: from src.earnings.types import Result, Ok, Err - New code can use Result instead of None/exceptions - Note: Existing methods not refactored to minimize risk; gradual migration recommended

31) Fix IBKR Connection State Management (Finding #123) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Atomic state transitions. - Root Cause: _connection_state was modified without locks, causing race conditions between heartbeat thread, error handler, and connect/disconnect methods. - Fixes: 1. Added _state_lock for atomic state transitions 2. Created _set_state(), _get_state(), _transition_if() helper methods 3. Updated all state transition sites to use atomic helpers 4. Connected property now uses atomic state access - Files: - src/ibkr/client.py: - Line 212 (_state_lock initialization) - Lines 224-246 (atomic helper methods) - All state transitions updated to use helpers - Acceptance Test: - No race conditions during connection state changes - State machine transitions are atomic

32) Standardize Database Transactions (Finding #124) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Thread-safe engine and transaction helper. - Root Cause: Engine initialization had race condition (check-then-act without lock). Transaction patterns were inconsistent across codebase. - Fixes: 1. Added threading.Lock() for _engine_lock (was None before) 2. get_engine() now uses double-checked locking pattern 3. Added get_transaction() context manager for atomic write operations 4. Documented pattern: use get_transaction() for writes, get_connection() for reads - Files: - src/db/connection.py: - Line 44 (threading import and lock initialization) - Lines 80-138 (thread-safe get_engine with double-checked locking) - Lines 182-200 (new get_transaction helper) - Acceptance Test: - Concurrent engine access is thread-safe - Transactions auto-commit on success, auto-rollback on exception

33) Add Stop Order Verification (Finding #125) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Verify stop in open orders after placement. - Root Cause: After placing stop order, we logged success but didn't verify the order actually appeared in IBKR's open orders list. Rejected orders could go unnoticed. - Fixes: 1. After placing stop, call reqAllOpenOrders() to get current orders 2. Search for our order ID in open trades list 3. Log verification status (verified or warning if not found) 4. Return trade object regardless (caller can check orderId tracking) - Files: - src/earnings/order_executor.py: - Lines 1647-1662 (stop order verification after placement) - Acceptance Test: - Stop order placement logs "✅ Stop order verified in IBKR" - Warning logged if order placed but not found in open orders

34) Fix Earnings Detector Thread Safety (Finding #126) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Lock around monitored positions dict. - Root Cause: _monitored_positions dict was accessed from main thread (add/remove) and monitor thread (_check_all_positions) without synchronization. - Fixes: 1. Added _positions_lock in init 2. add_monitored_position() uses lock when adding 3. remove_monitored_position() uses lock when removing 4. _has_active_positions() takes snapshot under lock 5. _check_all_positions() takes snapshot under lock before iteration - Files: - src/earnings/earnings_detector.py: - Line 150 (_positions_lock initialization) - Lines 757-759 (add with lock) - Lines 766-769 (remove with lock) - Lines 807-809 (snapshot for _has_active_positions) - Lines 821-824 (snapshot for _check_all_positions) - Acceptance Test: - No race conditions when adding/removing positions during monitoring - Iteration uses snapshot to avoid concurrent modification

35) Alert System Retry and Circuit Breaker (Finding #127) - Severity: Medium - Status: done - Implementation: Complete (Feb 5, 2026). Added retry with backoff and circuit breaker. - Root Cause: Alert sending did single attempt, could fail silently during network issues. No protection against alert channel outages flooding with retries. - Fixes: 1. Added CircuitBreaker class with CLOSED/OPEN/HALF_OPEN states 2. Added _retry_with_backoff() helper (3 retries, exponential delay) 3. Per-channel circuit breakers (_telegram_circuit, _webhook_circuit) 4. CRITICAL message deduplication (rate limit same message per 30s) 5. Send method now uses circuit breaker before retry - Files: - src/earnings/alerts.py: - Lines 49-118 (CircuitBreaker class and retry helper) - Lines 377-380 (circuit breaker initialization) - Lines 468-520 (send with circuit breaker and retry) - Acceptance Test: - Failed sends retry with exponential backoff - Circuit opens after 5 consecutive failures - Same CRITICAL message not sent within 30s

36) Stop Order Race Condition Fix (Finding #129) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Lock around stop order placement. - Root Cause: Race between checking self.stop_orders and placing new order. Concurrent threads could both pass check and place duplicate stops. - Fixes: 1. Added _stop_order_lock in init 2. _place_stop_order() acquires lock and delegates to _place_stop_order_locked() 3. All stop order logic runs under lock - Files: - src/earnings/order_executor.py: - Line 209 (_stop_order_lock initialization) - Lines 1557-1562 (lock acquisition and delegation) - Acceptance Test: - Concurrent stop placements for same symbol serialized - No duplicate stop orders

37) Configuration-Driven Thresholds (Finding #130) - Severity: Medium - Status: done - Implementation: Complete (Feb 5, 2026). Centralized config with env overrides. - Root Cause: Magic numbers scattered across codebase without documentation. Changing thresholds required code changes and deployment. - Fixes: 1. Created src/earnings/config.py with all thresholds 2. Each value documented with rationale 3. Environment variable override support 4. Categories: Order execution, limits, risk, IBKR, database, alerts - Files: - src/earnings/config.py (new file) - Acceptance Test: - Config values importable: from src.earnings.config import Config - Override via env: ENTRY_FALLBACK_TIMEOUT=60 - Note: Existing code not refactored to use Config class (gradual migration)

38) Exit Verification Retry with Backoff (Finding #131) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Exit verification retries 3x with backoff. - Root Cause: Single exit verification attempt could fail if IBKR slow to update. Position reported as not flat even when exit actually succeeded. - Fixes: 1. Retry loop: 3 attempts with exponential backoff (3s, 6s, 12s) 2. Breaks early on success (position flat) 3. Logs retry attempts for debugging 4. Final state analysis after all retries 5. Applied to both instant exit and scheduled exit - Files: - src/earnings/order_executor.py: - Lines 1238-1305 (instant exit verification with retry) - Lines 1483-1543 (scheduled exit verification with retry) - Acceptance Test: - Exit verification retries before reporting failure - Logs show retry attempts with delays - Direction flip detection still works after retries

Note: Finding #128 (Parallel Earnings Detection) deferred - requires async refactor

39) Cancel Stop Order Before Exit (Finding #132) - Severity: Critical - Status: done - Implementation: Complete (Feb 5, 2026). Cancel active stop before exit to prevent IBKR rejection. - Root Cause: Instant exit for ARES failed with IBKR rejection. IBKR rejected the SELL order because an active stop-limit order (stop_order_id 103984) was already in place for the same symbol, and IBKR prevents conflicting orders. - Fixes: 1. Added cancel_order() method to IBKRClient for canceling active orders 2. Modified process_instant_exits() to cancel stop order BEFORE placing exit order: - Check if position has stop_order_id - If yes, cancel stop via IBKRClient.cancel_order() - Wait for cancel confirmation (up to 5 seconds) - Then place exit order - If exit fails, restore stop order for protection 3. Added restore_stop_order() helper for re-placing stop if exit fails - Files: - src/ibkr/client.py: Lines 1545-1600 (cancel_order method) - src/earnings/trading_orchestrator.py: Lines 2675-2750 (cancel-before-exit logic) - Acceptance Test: - Instant exit succeeds when stop order is active - Stop order restored if exit order fails - No IBKR rejection for conflicting orders

40) Enhanced Execution Data Capture (Finding #133) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Enhanced get_recent_executions() + order type helper. - Root Cause: Stop-order fills were missing critical data: - Commission not captured from fill.commissionReport - realizedPNL not captured from fill.commissionReport - No way to retrieve order type (STP LMT vs STP) for filled orders - Fixes: 1. Modified get_recent_executions() to include: - commission: From fill.commissionReport.commission - realized_pnl: From fill.commissionReport.realizedPNL 2. Added get_order_type_for_order(order_id) method: - Searches ib.trades() for matching order ID - Returns order.orderType (e.g., 'STP LMT', 'STP', 'LMT') - Used by reconciliation to capture exit order type - Files: - src/ibkr/client.py: - Lines 1450-1468 (get_recent_executions with commission/realizedPNL) - Lines 1515-1545 (get_order_type_for_order helper) - Acceptance Test: - get_recent_executions() returns commission and realized_pnl - get_order_type_for_order() returns correct order type for filled orders

41) Reconciliation Full Data Capture (Finding #134) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Reconciliation populates all exit fields + ibkr_executions. - Root Cause: Stop-order fills discovered via reconciliation were missing data: - exit_order_type NULL in trading_positions - exit_commission NULL in trade_audit_log - total_commission not calculated - Execution not recorded in ibkr_executions table - Fixes: 1. Enhanced reconciliation exit processing to extract all data: - exit_commission, exit_order_id, exit_exec_id, exit_exchange from exit_exec - exit_order_type via get_order_type_for_order() - Calculate total_commission (entry + exit) 2. Update trading_positions with exit_order_type 3. Update trade_audit_log with full exit data: - exit_order_type, exit_order_id, exit_commission - total_commission, gross_pnl, ibkr_exit_exec_id, exit_execution_exchange 4. INSERT execution into ibkr_executions table: - Records all stop-order fills in execution table - ON CONFLICT DO NOTHING prevents duplicates - Source marked as 'reconciliation' - Files: - src/earnings/trading_orchestrator.py: - Lines 1163-1280 (enhanced reconciliation exit processing) - Acceptance Test: - Reconciled exits have exit_order_type populated - Commission captured from IBKR and stored in DB - Stop-order fills appear in ibkr_executions table

42) Correct Slippage for Stop-Limit Orders (Finding #135) - Severity: Medium - Status: done - Implementation: Complete (Feb 5, 2026). Dashboard uses limit_price vs fill_price for STP LMT orders. - Root Cause: Dashboard showed misleading slippage for stop-limit orders: - ARES showed "2.00% slippage" which was actually the NYSE_STOP_LIMIT_BUFFER - Calculation used stop_price (trigger) vs fill_price - For STP LMT, should use exit_limit_price (limit) vs fill_price - Actual slippage was 0% (filled at limit price) - Fixes: 1. Added exit_limit_price to dashboard trade query 2. Enhanced slippage calculation logic: - If exit_order_type contains 'LMT' AND exit_limit_price exists: Use limit_price as reference (our worst acceptable price) - Else if stop_price exists (pure STP order): Use stop_price as reference (trigger price) 3. Slippage = reference_price - fill_price (for LONG) 4. This shows TRUE slippage (worse than our limit) - Files: - earnings_dashboard/app.py: - Line 2923 (added exit_limit_price to query) - Lines 2976-3010 (enhanced slippage calculation) - Acceptance Test: - STP LMT orders show slippage relative to limit price - Fill at limit = 0% slippage (correct) - Fill below limit = negative slippage (filled better than limit)

43) Exit Reason Mislabeling in Reconciliation (Finding #136) - Severity: High - Status: done - Implementation: Complete (Feb 5, 2026). Reconciliation now compares order IDs to determine exit type. - Root Cause: Reconciliation assumed any position with stop_order_id was closed via stop loss. This was wrong when: 1. The stop order was CANCELLED (not filled) 2. The position was closed by a DIFFERENT order (scheduled/instant exit)

 TW, ICE, CI were scheduled limit exits at profit but labeled as "stop_loss:hardware_reconciled"
 because they had stop_order_ids. The stop orders were cancelled after the limit exits filled.

• Evidence:
  • TW: stop_order_id=121723 (CANCELLED), exit via LimitOrder 130059
  • ICE: stop_order_id=121731 (CANCELLED), exit via LimitOrder 130081
  • CI: stop_order_id=121735 (CANCELLED), exit via LimitOrder 130099
• Fixes:
  1. Compare exit_exec['order_id'] with db_pos['stop_order_id']
  2. If they match: exit_reason = 'stop_loss:hardware_reconciled'
  3. If they differ: exit_reason = 'scheduled_exit:reconciled'
  4. Updated normalize_exit_reason() to handle 'scheduled_exit:reconciled' pattern
  5. Corrected TW, ICE, CI database records to show correct exit reason
• Files:
  • src/earnings/trading_orchestrator.py: Lines 1215-1240 (exit reason detection)
  • earnings_dashboard/app.py: Lines 3367-3372 (normalize_exit_reason)
• Database Fixes:
  • TW (ID 65/32): exit_reason='scheduled_exit:reconciled', exit_order_type='LMT'
  • ICE (ID 67/34): exit_reason='scheduled_exit:reconciled', exit_order_type='LMT'
  • CI (ID 68/35): status='closed', exit_price=275.28, P&L=+2.73, exit_reason='scheduled_exit:reconciled'
• Acceptance Test:
  • Dashboard shows "Scheduled Exit (Reconciled)" for TW, ICE, CI
  • CI shows as closed with +$2.73 profit
  • Future reconciled exits correctly identify stop vs non-stop exits

44) ibkr_executions Missing Unique Constraint (Finding #137) - Severity: Medium - Status: done - Implementation: Complete (Feb 5, 2026). Added UNIQUE constraint on exec_id. - Root Cause: Reconciliation INSERT used ON CONFLICT (exec_id) DO NOTHING but the ibkr_executions table only had an INDEX on exec_id, not a UNIQUE constraint. PostgreSQL requires a UNIQUE constraint for ON CONFLICT specification.

 This caused CI reconciliation to fail repeatedly with:
 "there is no unique or exclusion constraint matching the ON CONFLICT specification"

• Fixes:
  1. Added UNIQUE constraint: ALTER TABLE ibkr_executions ADD CONSTRAINT ibkr_executions_exec_id_unique UNIQUE (exec_id)
  2. Verified no duplicate exec_ids existed (only 2 NULL entries, which PostgreSQL handles correctly)
• Files:
  • Database migration applied directly to production
• Acceptance Test:
  • Reconciliation INSERT with ON CONFLICT succeeds
  • CI position successfully reconciled after constraint added

Phase 3D: Critical Bug Fixes (Feb 7, 2026 Holistic Audit)

Numbering trap (docs-sweep, Sep 26 2026): list items 45–50 below reused Finding #132–#137, which were already assigned on Feb 5 (list items 39–44). Those Feb 5 entries remain PRIMARY. These Feb 7 bugs are aliases of the reused numbers, not new IDs — cite the line (list 45–50) when referring to them. The dashboard parser first-write-wins on (Finding #N), so the Testing Findings tab shows the Feb 5 titles for #132–#137.

45) fixes_applied NameError in reconciliation (Finding #132 reused Feb 7 — PRIMARY is list 39 / Feb 5 Cancel Stop Order Before Exit) - Severity: Critical - Status: done - Implementation: Complete (Feb 7, 2026). Initialized fixes_applied = [] at line 1133 alongside issues_detected. - Root Cause: Variable referenced at line 1340 (execution ledger auto-close) before initialization at line 1377. - Files: src/earnings/trading_orchestrator.py

46) cancelled_stop_info undefined in scheduled exit (Finding #133 reused Feb 7 — PRIMARY is list 40 / Feb 5 Enhanced Execution Data Capture) - Severity: Critical - Status: done - Implementation: Complete (Feb 7, 2026). Added cancelled_stop_info = None initialization in place_scheduled_exit(). - Root Cause: Variable only defined in place_instant_exit() but referenced in place_scheduled_exit() stop restore logic. - Files: src/earnings/order_executor.py

47) Direct active_positions access bypasses thread locks (Finding #134 reused Feb 7 — PRIMARY is list 41 / Feb 5 Reconciliation Full Data Capture) - Severity: Critical - Status: done - Implementation: Complete (Feb 7, 2026). Replaced all 10+ direct self.active_positions[symbol] accesses with thread-safe wrappers (_get_position, _set_position, _remove_position). All iteration uses with self._positions_lock: snapshots. - Root Cause: Finding #117 added locks and wrappers but many call sites still used direct dict access. - Files: src/earnings/trading_orchestrator.py (12 locations fixed)

48) Division by zero in batch sizing (Finding #135 reused Feb 7 — PRIMARY is list 42 / Feb 5 Correct Slippage for Stop-Limit Orders) - Severity: Critical - Status: done - Implementation: Complete (Feb 7, 2026). Added if total_weight <= 0: return {} guard. - Root Cause: proportion = weights[symbol] / total_weight crashes if all signal weights are 0. - Files: src/earnings/batch_order_manager.py

49) State desync - position removed before DB commit verified (Finding #136 reused Feb 7 — PRIMARY is list 43 / Feb 5 Exit Reason Mislabeling in Reconciliation) - Severity: Critical - Status: done - Implementation: Complete (Feb 7, 2026). Wrapped cleanup steps (remove position, cancel stops, close in risk manager) in try/except. Sends CRITICAL alert if cleanup fails after DB persist. - Root Cause: If _persist_ibkr_execution() threw exception, position was removed from memory but execution not recorded. - Current state (#408): live persist is PositionManager.persist_ibkr_execution (fail-open). Orchestrator _persist_ibkr_execution deleted. Historical root-cause name above is the then-live twin. - Files: src/earnings/trading_orchestrator.py

50) Queue full drops instant exits silently (Finding #137 reused Feb 7 — PRIMARY is list 44 / Feb 5 ibkr_executions Missing Unique Constraint) - Severity: Critical - Status: done - Implementation: Complete (Feb 7, 2026). Added CRITICAL alert via _send_alert_async() when queue is full. - Root Cause: Queue was bounded (#121) but only logged when full, no alert sent. Positions left unprotected. - Files: src/earnings/trading_orchestrator.py

51) Execution ledger 48h lookback too short (Finding #138) - Severity: Critical - Status: done - Implementation: Complete (Feb 7, 2026). Extended find_exit_execution() lookback from 48h to 168h (7 days). - Root Cause: CBOE/MCHP incident (Feb 6, 2026) - stop fills unrecoverable if bot was down >48h. - Files: src/ibkr/client.py

52) Positions unmonitored during IBKR disconnection (Finding #139) - Severity: Critical - Status: done - Implementation: Complete (Feb 7, 2026). Added reconnection callback system to IBKRClient. TradingOrchestrator registers _on_ibkr_reconnect() which triggers immediate reconciliation after IBKR reconnects. - Root Cause: After IBKR reconnection, no reconciliation was triggered. Positions closed by stops during downtime went undetected until next scheduled reconciliation (5 min). - Files: src/ibkr/client.py, src/earnings/trading_orchestrator.py

Phase 3E: High Priority Hardening (Feb 7, 2026 Holistic Audit)

53) Missed scheduled exits on restart (Finding #140) - Severity: High - Status: done - Implementation: Added _check_missed_scheduled_exits() on startup. Checks all open positions for scheduled_exit_time < now and executes immediately with alert. - Current state (#408): live startup callee is ExitManager.check_missed_scheduled_exits (from orchestrator __init__ after _load_persisted_positions). Orchestrator _check_missed_scheduled_exits deleted. Historical implementation name above is the then-live twin. - Files: src/earnings/trading_orchestrator.py

54) _exit_in_progress never cleared on failure (Finding #141) - Severity: High - Status: done - Implementation: Added _exit_in_progress_times dict tracking timestamps. Auto-clears stale entries after 5 minutes in check_scheduled_exits(). Prevents permanent blocking from transient failures. - Files: src/earnings/trading_orchestrator.py

55) Stop order tracking outside lock (Finding #142) - Severity: High - Status: done - Implementation: Moved self.stop_orders[symbol] = stop_order_id inside _stop_order_lock. - Files: src/earnings/order_executor.py

56) Hardcoded market holidays expire 2027 (Finding #144) - Severity: High - Status: done - Implementation: Added 2027-2028 US market holidays and early close days. - Files: src/earnings/market_calendar.py

57) Audit log files can fill disk (Finding #145) - Severity: High - Status: done - Implementation: Uses single rotating file audit_log.jsonl. Rotates at 50MB with 1 backup. - Files: src/earnings/audit_logger.py

58) Config values lack bounds validation (Finding #146) - Severity: High - Status: done - Implementation: Added Config.validate() classmethod. Checks percentage fields (0-1 range) and integer fields (reasonable bounds). Call on startup. - Files: src/earnings/config.py

59) DB connection pool timeout already configured (Finding #149) - Severity: High - Status: done (already implemented) - POOL_CONFIG['pool_timeout'] = 30 already exists in src/db/connection.py.

60) Signal generator DB queries unhandled (Finding #150) - Severity: High - Status: done - Implementation: Wrapped _get_earnings_markets_from_db() query in try/except, returns empty list on failure. - Files: src/earnings/signal_generator.py

61) get_previous/next_trading_day() returns wrong date (Finding #151) - Severity: High - Status: done - Implementation: Extended search window from 10 to 30 days. Handles holiday clusters. - Files: src/earnings/market_calendar.py

62) Earnings timing fallback hardcoded path (Finding #152) - Severity: High - Status: done - Implementation: Uses os.path.dirname(__file__) relative path with env var override EARNINGS_TIMING_CACHE_PATH. - Files: src/earnings/earnings_timing_fallback.py

63) Missing alert when price sources fail (Finding #154) - Severity: High - Status: done - Implementation: Sends warning alert when both IBKR and yfinance fail for a symbol. - Files: src/earnings/trading_orchestrator.py

64) No startup safe mode (Finding #155) - Severity: High - Status: done - Implementation: Added _reconciliation_completed flag. execute_entry() blocks until first reconciliation succeeds. Prevents new entries during inconsistent state. - Files: src/earnings/trading_orchestrator.py

65) Deferred to Phase 4A: P&L calculation dedup (Finding #143) - Severity: High - Status: deferred - Reason: Requires refactoring trading_orchestrator.py (Phase 4A scope)

66) Deferred to Phase 4A: N+1 query in conviction history (Finding #147) - Severity: High - Status: deferred - Reason: Performance optimization, not a crash risk

67) Deferred to Phase 4A: Blocking IBKR market data calls (Finding #148) - Severity: High - Status: deferred - Reason: Requires async refactor

68) Deferred to Phase 4A: Split place_instant_exit() (Finding #153) - Severity: High - Status: deferred - Reason: 436-line function needs careful extraction during orchestrator split

Phase 4A: Split Trading Orchestrator (Feb 7, 2026)

69) Split TradingOrchestrator into 6 focused modules - Severity: Architecture - Status: done - Implementation: Complete (Feb 7, 2026). Split 4878-line monolith into 6 modules using composition with full dependency injection. Each manager receives only its specific dependencies via init. - New modules created (3,326 total lines): - trading_utils.py (464 lines): 18 utility methods (market time, pricing, scheduling) - position_manager.py (815 lines): 14 methods (position CRUD, DB persistence, stop orders) - exit_manager.py (792 lines): 13 methods (scheduled/instant/batch exits, stop backup) - capital_manager.py (671 lines): 14 methods (capital reservation, signals, entries) - reconciliation_manager.py (586 lines): 3 methods (IBKR/DB reconciliation, table sync) - TradingOrchestrator updated: Creates all managers in init, delegates to them. Original methods remain for backward compatibility. External callers unchanged. - Files: src/earnings/trading_utils.py, src/earnings/position_manager.py, src/earnings/exit_manager.py, src/earnings/capital_manager.py, src/earnings/reconciliation_manager.py, src/earnings/trading_orchestrator.py, src/earnings/__init__.py - Note: Method location references in earlier findings (e.g., #14, #17, #18, #19, #23, #25, #39, #41, #43) still reference trading_orchestrator.py. The methods now exist in both the original file AND the new manager modules (identical code). The original methods will be removed once gradual migration to managers is complete.

Phase 4B+4C: IBKR as Source of Truth (Feb 7, 2026)

70) IBKR as source of truth for live position state - Severity: Architecture - Status: done - Implementation: Complete (Feb 7, 2026). IBKR is now the definitive source for live position state. Database remains source of truth for historical data. - New components: - position_state.py: PositionState enum (PENDING/OPEN/CLOSING/CLOSED/FAILED) with state machine validation and transition rules - ibkr_position_cache.py: Thread-safe TTL cache (5s) wrapping ib.positions() with disconnection detection - Changes: - position_manager.py: Added IBKR cache integration methods - trading_orchestrator.py: Main loop blocks ALL trading when IBKR disconnected - memory.md: Principle #2 updated to reflect IBKR as source of truth - DB migration 018: Added position_state column to trading_positions - Key behavior: When IBKR disconnected, ALL trading operations blocked. Hardware stops remain active in IBKR regardless.

Phase 4D: Event-Driven Architecture (Feb 7, 2026)

71) Event-Driven Architecture with real-time stop fill detection - Severity: Architecture - Status: done - Implementation: Complete (Feb 7, 2026). Created EventBus for decoupled communication and IBKR fill callbacks for real-time stop order fill detection. - New components: - event_bus.py: Thread-safe publish/subscribe with 20 event types, JSONL persistence - ibkr/client.py: Added on_fill() callback + _on_order_status() handler - trading_orchestrator.py: Added _on_ibkr_fill() handler for stop fill detection - Key achievement: Solves CBOE/MCHP problem. Stop fills now detected in real-time via ib_insync orderStatusEvent, rather than relying on periodic reconciliation. - Events published: ORDER_FILLED, STOP_FILLED (triggers position cleanup), IBKR_DISCONNECTED, IBKR_RECONNECTED, etc.

Phase 4E: Medium Priority Fixes (Feb 7, 2026)

72) 20 medium priority fixes (#156-#175) - Severity: Medium - Status: done - Implementation: Complete (Feb 7, 2026). All 20 findings from the holistic audit addressed. - Concurrency (#156-#159): - #156: Lock ordering documented (positions -> capital -> batch) - #157: Capital lock already consistent in capital_manager.py (verified) - #158: Added _positions_lock to risk_manager.open_positions - #159: Circuit breaker state reset on restart (documented as intentional) - Error Handling (#160-#164): - #160: Added debug logging to bare except clauses - #161: cancel_order() now polls for cancellation verification (3s) - #162: Already guarded (total_depth > 0 check exists) - #163: Override JSON validated on load (structure + timing values) - #164: dispose_engine() wrapped in _engine_lock - Config (#165-#168): - #165: earnings_datetime timezone validated (auto-converts naive to ET) - #166: Polling intervals configurable via env vars - #167: Already done in Phase 3E (2027-2028 holidays) - #168: Trading block file uses atomic remove (no TOCTOU race) - Monitoring (#169-#172): - #169: Alert delivery confirmation logged (Telegram success/failure) - #170: Daily position summary at 8 PM ET - #171: Already exists (/api/health endpoint in dashboard) - #172: Reconciliation abort alert failure now logged - Code Quality (#173-#175): - #173: No obsolete commented-out code found - #174: Dead method _is_earnings_announcement() removed - #175: Strategy doc already documents 10x vs 2x/6x leverage correctly

Feb 9, 2026 Critical Fixes

76) IBKR connectivity loss auto-recovery (Finding #176) - Severity: Critical - Status: done - Root Cause: Error 1100 (connectivity lost) was in CRITICAL_BLOCK_ERRORS, persisting trading_block.txt that survived restarts. Errors 1101/1102 (connectivity RESTORED) were treated as disconnections instead of recovery events. - Impact: Bot blocked for 33+ hours (Feb 8-9) until manual file deletion. - Fixes: 1. Removed 1100 from CRITICAL_BLOCK_ERRORS (transient, not critical) 2. Errors 1101/1102 now clear trading block, verify health, restart heartbeat 3. Health check auto-clears connection-related blocks on success - Files: src/ibkr/client.py

77) Stale earnings dates in DB (Finding #177) - Severity: Critical - Status: done - Root Cause: earnings_markets table populated by sync job every ~6h. Companies shift earnings dates but DB doesn't re-verify. ON had Feb 10 in DB, actual was Feb 9. - Impact: ON trade would have been missed (signal generator uses DB date). - Fixes: 1. Added verify_earnings_dates() to signal_generator.py - cross-checks DB vs EarningsCalendar before each signal check (AMC 10:30 AM, BMO 3:50 PM) 2. Auto-corrects mismatches and sends Telegram alert 3. Fixed 5 stale dates: ON, CRSP, NET, GRAB, PANW - Files: src/earnings/signal_generator.py, src/earnings/trading_orchestrator.py

78) Duplicate order from market fallback (Finding #178) - Severity: Critical - Status: done - Root Cause: batch_order_manager._apply_market_fallback accessed result.filled_price but OrderResult has avg_fill_price. AttributeError caused batch to think order failed, triggering market fallback = duplicate (ON: 6 shares instead of 3). - Same pattern as PFE (#17) and CME (#23): retry/fallback without checking IBKR position. - Fixes: 1. Fixed filled_price -> avg_fill_price attribute bug 2. Added IBKR position guard (_check_ibkr_position) before EVERY market fallback: - batch_order_manager._apply_market_fallback - order_executor.place_guaranteed_fill If position already exists, ABORT the fallback. Defense-in-depth. 3. Manual recovery: sold 3 excess ON shares, placed stop loss. (Corrected #353, Jul 31 2026: the original entry claimed the recovery legs were "recorded in DB" — they never were. The excess-share legs exist only in IBKR; the DB holds the corrected strategy trade. Now represented as an UNQUANTIFIED placeholder row in the trading_errors ledger — finding '#178', realized_loss 0.00, quantifiable from IBKR Flex on demand.) - Files: src/earnings/batch_order_manager.py, src/earnings/order_executor.py

79) Earnings detector false exit from PRNewswire wrong EPS (Finding #179A) - Severity: Critical - Status: done - Root Cause: Two bugs in earnings_detector.py: 1. Polymarket confirmation state was popped after confirmation (line 322), creating 30-second vulnerability windows where secondary sources could override. 2. No contradiction guard: PRNewswire parsed wrong EPS ($0.06 vs actual $0.64), concluded MISS, overrode Polymarket's 99.95% BEAT signal. - Impact: ON position exited prematurely (still profitable, but missed further gains). - Fixes: 1. Keep Polymarket confirmation state (don't pop after confirmation) 2. PRNewswire/BusinessWire that contradict confirmed Polymarket outcome are logged but ignored 3. Yahoo (authoritative) can still override if needed - Files: src/earnings/earnings_detector.py

80) NASDAQ stop orders don't trigger in extended hours (Finding #179B) - Severity: Critical - Status: done - Root Cause: IBKR ignores outsideRth on pure STP orders for NASDAQ stocks routed via SMART (Warning 2109). Every NASDAQ STP in our history got 2109. Stops only trigger during RTH (9:30 AM - 4:00 PM ET). - Impact: HOOD dropped from $87.83 to $78.00 after hours (-11.2%), but stop at $85.20 (STP) never triggered. (Clarified #353, Jul 31 2026: the -11.2% was the UNREALIZED trough while the stop was blocked; the REALIZED loss on the closed trade was -$5.26 = -3.0%, i.e. the design stop distance — price recovered before the exit filled. The original "Actual loss: -11.2%" wording conflated the two.) - Evidence: CVS, KO, NET, RDDT, RL stops (all STP LMT) triggered correctly in pre/post market. HOOD, DDOG, MAR (all STP) got Warning 2109. - Fix: ALL exchanges now use STP LMT with 2% buffer (not just NYSE). Removes the is_nyse conditional entirely. Same 2% buffer approach that was already proven to work for NYSE. - Files: src/earnings/order_executor.py, src/ibkr/client.py - Docs updated: STRATEGY.md (sections 5.5, 7.3, 7.17), EDGE_CASES.md, OPERATIONS.md, TESTING_FINDINGS_PENDING.md

81) IBKR reconnection failures + zero alerts + stale dashboard (Finding #180) - Severity: Critical - Status: done - Root Cause: Three failures from the same pattern (3 incidents in 5 days, Feb 8-12): 1. Gateway daily restart drops bot connection. Reconnection loop retries but gateway keeps rejecting clientId=1 (stale session state, "remove Client 1"). 2. No Telegram alerts during 37h outage: alerts only sent from orchestrator main loop, which never starts when initial connection fails. 3. Dashboard shows "IBKR Connected" from stale snapshot file (no staleness check). - Impact: 37 hours downtime, missed trading signals for Feb 11-12 earnings. - Fixes: 1. Send Telegram alert immediately when reconnection starts + every 30 min (from IBKRClient._schedule_reconnect, not dependent on main loop) 2. Clean disconnect() before each reconnect attempt (clears stale clientId) 3. Dashboard checks snapshot age > 2 min → show "DISCONNECTED" 4. ibgateway-ibc.service: Restart=on-failure → Restart=always (gateway exits with code 0 on daily restart, on-failure didn't restart it) - Files: src/ibkr/client.py, earnings_dashboard/app.py, deploy/systemd/ibgateway-ibc.service (server updated directly)

82) IBKR weekly 2FA schedule drift hardening (Finding #181) - Severity: Critical - Status: done - Implementation: Complete (Feb 15, 2026). Added single source-of-truth schedule + drift audit and aligned all operational scripts. - Root Cause: Timing/service drift across scripts/docs caused missed 2FA expectation window and inconsistent operational behavior. - Fixes: 1. Added canonical schedule file scripts/ibkr_schedule.env (restart/reminder/manager times) 2. Updated gateway_restart_manager.sh and ibkr_restart_warning.sh to read canonical schedule values 3. Added missing sunday_2fa_reminder.sh with Sunday guardrails 4. Added ibkr_schedule_audit.sh to detect cron/service drift daily 5. Updated diagnostics and deploy flow to verify ibgateway-ibc.service and canonical cron entries 6. Updated continuity docs with canonical schedule source and audit cron - Files: - scripts/ibkr_schedule.env - scripts/gateway_restart_manager.sh - scripts/ibkr_restart_warning.sh - scripts/sunday_2fa_reminder.sh - scripts/ibkr_schedule_audit.sh - scripts/ibkr_diagnostics.py - deploy_ibkr_monitoring.sh - docs/IBKR_CONTINUITY.md

83) Holistic hardening batch: trading/reconciliation/ops/dashboard consistency (Finding #182) - Severity: Critical - Status: done - Implementation: Complete (Feb 15, 2026). Added guardrails against race conditions, strengthened IBKR ops readiness checks, and standardized dashboard trade-quality filtering. - Root Cause: Multiple cross-component assumptions allowed partial false-positive health states, fragile restart timing, and inconsistent reporting filters. - Fixes: 1. Exit race hardening in order_executor.py: - Skip stop restore when IBKR position is already flat - Re-check live position immediately before market fallback order - Handle disconnect during exit verification retries 2. Reconciliation guardrails in reconciliation_manager.py: - Auto-close only when execution side/shares/time invariants pass - Closed-table consistency checks (counts + NULL exit price detection) 3. Capital estimate safety in capital_manager.py: - Use current-price estimate (fallback to entry notional) 4. Ops hardening: - ibkr_health_monitor.sh now includes login-pending detection, command fallbacks, durable state file - gateway_restart_manager.sh uses bounded polling window with PID kill verification - ibkr_schedule_audit.sh validates cron with normalized matching and checks IBC AutoRestartTime 5. Dashboard consistency: - Unified filter for reconciliation/manual recovery/external trades - Explicit COALESCE totals and exit_price IS NOT NULL guard in closed-trade summary - Files: - src/earnings/order_executor.py - src/earnings/exit_manager.py - src/earnings/reconciliation_manager.py - src/earnings/capital_manager.py - src/ibkr/client.py - scripts/ibkr_health_monitor.sh - scripts/gateway_restart_manager.sh - scripts/ibkr_schedule_audit.sh - scripts/ibkr_diagnostics.py - deploy_ibkr_monitoring.sh - earnings_dashboard/app.py - docs/OPERATIONS.md - docs/IBKR_CONTINUITY.md - docs/STRATEGY.md - docs/TESTING_FINDINGS_PENDING.md

84) Health monitor false-unhealthy correction after deploy (Finding #183) - Severity: High - Status: done - Implementation: Complete (Feb 15, 2026). Fixed gateway-running detection to use ibgateway-ibc.service status as primary signal. - Root Cause: Process regex was too strict and marked Gateway=false while API was live, causing false unhealthy state. - Fix: 1. ibkr_health_monitor.sh now checks systemctl is-active ibgateway-ibc.service first 2. Falls back to broader process patterns only if service check unavailable - Files: - scripts/ibkr_health_monitor.sh

85) Reconciliation missing exec_id hardening + one-time repair (Finding #184) - Severity: Critical - Status: done - Implementation: Complete (Feb 15, 2026). Hardened stop-fill persistence and added deterministic log-based reconciliation repair path. - Root Cause: 1. Hardware stop-fill cleanup path could close trades without persisting ibkr_exit_exec_id / exit_order_id 2. Reconciliation auto-fix only used ibkr_executions, so historical gaps remained unresolved 3. Delayed IBKR execution history retrieval was not reliable for historical repair in this environment - Fixes: 1. exit_manager.py now captures recent stop-fill execution payload from IBKR in-memory trades and persists: - exit_exec_id, exit_order_id, exit_execution_exchange, exit_commission - matching ibkr_executions row (source=live_stop_fill) 2. earnings_detector.py now supports stop_monitoring(symbol) (symbol-scoped remove), preventing runtime cleanup exceptions 3. trade_reconciler.py adds log fallback parser for fill events in earnings_trader.log - inserts deterministic synthetic IDs (logfill.<symbol>.<order_id>.<epoch>) into ibkr_executions - backfills ibkr_entry_exec_id / ibkr_exit_exec_id when DB-only matching fails 4. Added one-time repair utility: scripts/repair_trade_exec_mismatches.py - Files: - src/earnings/exit_manager.py - src/earnings/earnings_detector.py - src/data_tracker/collectors/trade_reconciler.py - scripts/repair_trade_exec_mismatches.py - tests/test_reconciliation_and_orders.py - docs/STRATEGY.md - docs/TESTING_FINDINGS_PENDING.md

85b) Reconciliation missing_exec_id hardening + deterministic backfill (Finding #184; duplicate list index of 85 — same finding, not a new number) - Severity: Critical - Status: done - Implementation: Complete (Feb 15, 2026). Hardened stop-fill persistence path and added deterministic reconciliation backfill for legacy gaps. - Root Cause: 1. Hardware stop cleanup could close DB rows without durable ibkr_exit_exec_id / exit_order_id 2. Cleanup path called stop_monitoring(symbol) against a method that accepted no symbol, throwing post-fill errors 3. Reconciliation auto-fix searched only ibkr_executions, so historical gaps remained unresolved when execution rows were never persisted - Fixes: 1. exit_manager.py now captures recent stop-fill details from IBKR in-memory trades and persists: - exit_exec_id, exit_order_id, exit_execution_exchange, exit_commission - matching ibkr_executions row (source=live_stop_fill) 2. earnings_detector.py now supports stop_monitoring(symbol) as symbol-scoped removal (backward-compatible with full stop) 3. trade_reconciler.py now has log-based deterministic fallback: - Parses fill lines from /var/log/polymarket/earnings_trader.log - Generates stable synthetic IDs (logfill.<symbol>.<order_id>.<epoch>) - Inserts missing ibkr_executions rows and links trade exec IDs idempotently 4. Added one-time repair utility scripts/repair_trade_exec_mismatches.py - Files: - src/earnings/exit_manager.py - src/earnings/earnings_detector.py - src/data_tracker/collectors/trade_reconciler.py - scripts/repair_trade_exec_mismatches.py - tests/test_reconciliation_and_orders.py - docs/STRATEGY.md - docs/TESTING_FINDINGS_PENDING.md

86) Unified NYSE/NASDAQ execution policy + stop trigger timeout escalation (Finding #185) - Severity: Critical - Status: done - Implementation: Complete (Feb 15, 2026). Removed exchange-specific execution branching, normalized priority scoring to cap+confidence, and added stop-trigger timeout escalation. - Root Cause: 1. Legacy NYSE/NASDAQ branching made behavior and docs diverge over time. 2. Hardware stop could remain active after breach without quick fill, leaving unresolved downside window. 3. Priority scoring encoded exchange bias that no longer matched hardened execution policy. - Fixes: 1. Unified instant-exit logic in order_executor.py: - all exchanges use LIMIT (GTC, 1%) -> MARKET fallback when market is open - fully closed market remains GTC LIMIT only 2. Added stop-trigger timeout escalation in exit_manager.py: - tracks first breach timestamp - escalates to instant_exit:stop_timeout_escalation after configurable timeout 3. Normalized priority.py scoring to market cap + confidence only (exchange-independent) 4. Updated dashboard priority legends/order matrices and strategy/operations/edge-case docs for consistency - Files: - src/earnings/order_executor.py - src/earnings/exit_manager.py - src/earnings/config.py - src/earnings/priority.py - src/earnings/signal_generator.py - earnings_dashboard/templates/dashboard.html - tests/test_reconciliation_and_orders.py - tests/unit/test_order_executor_stops.py - tests/comprehensive_system_test.py - docs/STRATEGY.md - docs/OPERATIONS.md - docs/EDGE_CASES.md - docs/TESTING_FINDINGS_PENDING.md

87) 2x sizing intent alignment: full testing profile + 140% exposure cap (Finding #191) - Severity: High - Status: done - Implementation: Complete (Feb 23, 2026). Updated sizing logic to match requested behavior: double current testing profile without ambiguity, while retaining leverage guardrails. - Root Cause: 1. User intent ("2x sizing vs current testing behavior") differed from prior implementation semantics. 2. Existing testing_prod sizing multiplier remained at 50% of base Option-B sizing. 3. Exposure budget defaults and opportunity tiers were still tuned around a 70% ceiling. - Fixes: 1. Config.TESTING_PROD_SIZE_MULTIPLIER introduced and set to default 1.0 (from prior hardcoded 0.50). 2. Exposure defaults updated to 1.40: - MAX_DAILY_EXPOSURE=1.40 - MAX_TOTAL_EXPOSURE_PCT=1.40 3. Opportunity exposure tiers doubled proportionally in RiskManager: - rare 0.60, normal 1.40, busy 1.40 4. Kept leverage cap unchanged (LEVERAGE_MAX_EFFECTIVE_LEVERAGE=2.0). 5. Dashboard now reflects configured exposure cap dynamically (max_exposure_pct=140.0). 6. Added unit coverage for new sizing/exposure behavior. - Verification Evidence (production): 1. Runtime check after deploy: - TESTING_PROD_SIZE_MULTIPLIER 1.0 - MAX_DAILY_EXPOSURE 1.4 - LEVERAGE_MAX_EFFECTIVE_LEVERAGE 2.0 - large-cap size on $10k in testing_prod => $2,500 - exposure tiers => rare=0.6, normal=1.4, busy=1.4 2. /api/state shows max_exposure_pct=140.0 3. Live HIMS row confirms leverage snapshot persistence remains intact (trade_audit_log.id=65) - Files: - src/earnings/config.py - src/earnings/risk_manager.py - earnings_dashboard/app.py - tests/unit/test_config.py - tests/unit/test_position_sizing_2x_rollout.py - docs/TESTING_FINDINGS_PENDING.md

88) Close persistence permanence hardening after AS recurrence (Finding #193) - Severity: Critical - Status: done - Implementation: Complete (Feb 24, 2026). Eliminated nullable-parameter type ambiguity and added fallback close persistence path. - Root Cause: 1. pg8000 raised 42P18 ("could not determine data type of parameter") during rich close update when trade_audit_log.notes append path received nullable values. 2. The failed DB transaction left broker-closed positions marked open in DB, creating repeated orphan reconciliation alerts. - Fixes: 1. Reworked close-note update to use type-safe, non-null append semantics in persist_position_close_with_exit_details(). 2. Added fallback close persistence path that writes essential close fields to both trading_positions and trade_audit_log if the rich update fails. 3. Added high-confidence reconciliation auto-close override to safely resolve strict stop-fill matches even when global auto-close is disabled. 4. Added closed-trade metadata completeness check for exit_order_type, exit_order_id, and ibkr_exit_exec_id. - Files: - src/earnings/position_manager.py - src/earnings/reconciliation_manager.py - src/earnings/trading_orchestrator.py - src/earnings/config.py - earnings_dashboard/app.py - docs/OPERATIONS.md - docs/TESTING_FINDINGS_PENDING.md

89) Upcoming earnings dashboard live-visibility hardening (Finding #195) 90) IBKR reconnect/reconciliation holistic hardening (Finding #196) - Root cause: Reconnect callback race: reconciliation observed reconnect_in_progress=true during expected transition and raised critical position-sync noise for guard skips. - Fixes: - Clear reconnect flags before firing callbacks in IBKRClient. - Reclassify fail-closed guard skips to warning reconciliation_guard category. - Add bounded retry-on-ready after guard skip. - Add guard alert cooldown and watchdog thread-dump/stage forensics. - Verification: - Unit test: reconnect flags are idle before callback executes. - Unit test: guard retry scheduler is bounded + deduped. - Unit test: disconnected-guard path emits warning guard alert. - Impacted files: - src/ibkr/client.py - src/earnings/reconciliation_manager.py - src/earnings/alerts.py - src/earnings/trading_orchestrator.py - src/earnings/config.py - tests/unit/test_ibkr_error_block_policy.py - tests/unit/test_reconciliation_guard_retry.py - tests/unit/test_config.py

• Severity: High
• Status: done
• Implementation: Complete (Feb 24, 2026). Added in-page refresh + stale-cache guard for upcoming earnings table.
• Root Cause:
  1. Upcoming earnings table could remain empty in long-lived sessions despite healthy earnings_markets data.
  2. Cached empty earnings list could persist through server-render path until broader cache cycle refreshed.
• Fixes:
  1. Added JS-driven in-page refresh every 30 seconds from /api/earnings-calendar?days=14.
  2. Added visible metadata status line (row count + refresh timestamp/error).
  3. Added server-side guard to re-fetch and repopulate cache when cached earnings list is empty.
  4. Added DB truth counters to /api/earnings-calendar (db_upcoming_14d, db_upcoming_30d, db_next_upcoming_date) for explicit health context.
  5. Added client fail-open behavior: retain last known-good rows on transient empty/timeout responses instead of replacing with false empty-state.
• Files:
  • earnings_dashboard/templates/dashboard.html
  • earnings_dashboard/app.py
  • docs/OPERATIONS.md
  • docs/TESTING_FINDINGS_PENDING.md

91) Conviction robustness + unified close/reconciliation recovery (Finding #197) - Severity: Critical - Status: done - Implementation: Complete (Feb 15, 2026). Added robust conviction sampling with reliability gates, unified close persistence routing, and broadened high-confidence reconciliation auto-close for broker-confirmed exits. - Root Cause: 1. Conviction fallback could rely on a single outlier price snapshot in volatile windows, producing wrong direction. 2. Reconciliation high-confidence override was stop-order-centric, leaving recoverable non-stop exits stuck in EXEC_LEDGER_ONLY loops. 3. Trading orchestrator retained a legacy close persistence implementation separate from hardened PositionManager path. - Fixes: 1. conviction_calculator.py: robust price-window median estimator with sample/range reliability checks and reason codes. (Refined in Finding #325, Jun 11 2026: the reliability gate changed from min-max range>0.35 to robust IQR>0.25, the dense-sample cap raised from 9→10000 snapshots, and the 0.02<yes_price<0.99 filter removed — the old range+filter under-counted high-conviction markets and let an excluded ADBE LONG through.) 2. signal_generator.py: fail-closed skip for unreliable conviction windows with reason logging. 3. trading_orchestrator.py: close persistence routed to position_manager.persist_position_close_with_exit_details(...). 4. reconciliation_manager.py: high-confidence execution-ledger override generalized for non-stop exits under strict guardrails; execution-ledger close now uses hardened position-manager close path. 5. Added targeted tests for conviction robustness, reliability gating, reconciliation high-confidence criteria, and close-path routing. - Verification: - pytest -q tests/unit/test_conviction_robustness.py tests/unit/test_reconciliation_guard_retry.py tests/unit/test_close_path_routing.py - pytest -q tests/unit/test_signal_reliability_gate.py - All tests passed; targeted lints clean. - Production evidence (Feb 26, 2026): - EXEC_LEDGER_ONLY for SNOW eliminated (was recurring every 5 min pre-deploy) - Reconciliation metadata complete: SNOW#75 + ICUI#62 repaired with full exit linkage - Reconciliation all-clear: "DB and IBKR are in sync" on every post-deploy cycle - Conviction robustness log evidence: blocked (no entry candidates since deploy) - Follow-up fix (commit 6cb8830): - Alert deduplication for CLOSED TRADE EXEC METADATA INCOMPLETE check (1h cooldown, row-ID tracking) - SNOW data repair (Feb 26, 2026): - Corrected exit from stop_loss@$170.85 to instant_exit@$166.45 (42P18 caused original persist failure) - PnL corrected from -$9.96 to -$1.16 - position_state fixed from 'open' to 'closed' - Files: - src/earnings/conviction_calculator.py - src/earnings/signal_generator.py - src/earnings/reconciliation_manager.py - src/earnings/trading_orchestrator.py - tests/unit/test_conviction_robustness.py - tests/unit/test_signal_reliability_gate.py - tests/unit/test_reconciliation_guard_retry.py - tests/unit/test_close_path_routing.py - docs/OPERATIONS.md - docs/STRATEGY.md - docs/TESTING_FINDINGS_PENDING.md

92) NaT EPS crash loop in Yahoo earnings check (Finding #198) - Severity: Critical - Status: done - Implementation: Complete (Feb 27, 2026). Fixed crash loop from pandas NaT value in Yahoo earnings data. - Root Cause: 1. Yahoo Finance returned pandas NaT for EB (Eventbrite) reported_eps field. 2. _check_yahoo_earnings() set result.is_released = True BEFORE float(reported_eps). 3. float(NaT) threw TypeError, caught by except, but is_released was already True with actual_eps=None. 4. Caller check_earnings() crashed on f"${yahoo_result.actual_eps:.2f}" — None has no __format__. 5. Unhandled in main loop → exit → systemd restart → 69+ restarts over 20 hours. - Fixes: 1. Wrapped float(reported_eps) in try/except; set is_released AFTER successful conversion. 2. Same guard on eps_estimate float conversion. 3. Guarded ALL f-string EPS formatting across file (8 instances, memory.md Section E same-pattern check). - Impact: 3 open positions (DELL, REAL, EB) unmonitored for ~20 hours. Hardware stops remained active in IBKR. - Files: src/earnings/earnings_detector.py

93) Direction-inverted duplicate prevention blocks scheduled exit fallback (Finding #199) - Severity: Critical - Status: done - Implementation: Complete (Mar 6, 2026). Made market-fallback duplicate check direction-aware and fixed IBKR market hours parser. - Root Cause: 1. place_guaranteed_fill() duplicate check at line 680 used abs(p.shares) which ignores position direction. 2. For EXIT orders (BUY to cover SHORT), finding shares means the exit DIDN'T work — but the check concluded "likely filled" and skipped the market fallback. 3. GPRO SHORT -397 shares: limit BUY at $0.77 didn't fill in 45s. Market fallback was blocked by inverted duplicate check. Position left open +$111 unprotected over weekend. 4. Secondary: IBKR market hours parser crashed on YYYYMMDD:HHMM format (int("0306:1600")), blocking all post-failure retry attempts with "Market not in regular hours." - Fixes: 1. order_executor.py: Duplicate check now compares p.shares sign against side. BUY+LONG=duplicate, BUY+SHORT=exit-not-done (proceed). SELL+SHORT=duplicate, SELL+LONG=exit-not-done (proceed). 2. client.py: _parse_ibkr_hours strips YYYYMMDD: date prefix from time components before parsing. 3. batch_order_manager.py: Same direction-aware fix in _apply_market_fallback (memory.md Section E same-pattern check). - Impact: GPRO closed manually after hours at $0.7752, P&L +$111.08. No capital loss, but position was unprotected for ~15 minutes after scheduled exit failure. - Files: - src/earnings/order_executor.py - src/ibkr/client.py - src/earnings/batch_order_manager.py

Live Trading Validation Tests

Migrated from TESTING_FINDINGS_PENDING.md (consolidated Mar 7, 2026).

Global Rules

• Do not place trades solely for testing. Run trade-related tests only during normal trading with real signals.
• No trade execution without explicit approval. Follow the IBKR confirmation rules in memory.md.
• Use IBKR Gateway ports: Live 4001, Paper 4002. Avoid clientId 1 (use 99).

Live Test Results

1) Finding 1 — Entry + stop atomicity PASSED - Result (Feb 2, 2026): PLTR entry @ $148.70, stop @ $144.23 within 1 second. PEP/PFE batch also passed.

2) Finding 2 — Batch stop placement safety PASSED - Result (Feb 2, 2026): Batch 319b8adb - 1/1 success. Batch 34c25f75 - 2/2 success.

3) Finding 3 — Duplicate order prevention PASSED - Result (Feb 2, 2026): AAPL correctly skipped (existing position). PLTR skipped in batch.

4) Finding 4 — Market hours gating PASSED - Result (Feb 2, 2026): Pre-market entry blocked, instant exit allowed.

5) Finding 12 — Closed-market instant exit PENDING - Will test if instant exit needed during 8 PM - 4 AM window.

6) Finding 14 — Reconciliation disconnection guard VERIFIED (Code Review) - Guards confirmed in reconciliation_manager.py. Phase 4D event-driven architecture provides real-time backup.

7) Finding 15 — Stale conviction data fix VERIFIED - Two-phase Gamma fetch + 0% display fix deployed and verified.

Unit/Integration Test Gaps

1) Entry + stop atomicity (unit tests) — include success and failure cases 2) Batch order stop placement (integration) — validate retry logic and fallback 3) Reconciliation tests — cover old trades and partial fills 4) Dashboard P&L tests — fractional share calculations and stale price warnings 5) Heartbeat degradation (unit) — ADDED (tests/unit/test_ibkr_state.py) 6) IBKRPositionCache invalidation (unit) — ADDED (tests/unit/test_ibkr_position_cache.py) 7) EarningsDetector precedence + confirmation (unit) — ADDED (tests/unit/test_earnings_detector.py) 8) Stop order type by exchange (unit) — ADDED, UPDATED Feb 10 (Finding #179): ALL exchanges use STP-LMT

Pending Production Validation

Finding 182 — Holistic hardening follow-up (DONE)

• Exit/restore race guard, reconciliation auto-close invariants, capital freed estimation, IBKR ops hardening, dashboard unified filtering.
• Verified: System stable 3+ weeks since implementation. No race conditions or auto-close failures observed.

Finding 184 — Exec ID mismatch hardening + repair (DONE)

• Stop-fill cleanup captures IBKR fill payload, EarningsDetector.stop_monitoring compatibility, log-based reconciliation fallback.
• Verified: ibkr_exit_exec_id populated for all recent trades. No exec ID mismatches since Feb 15.

Finding 185 — Unified NYSE/NASDAQ order policy + stop timeout escalation (DONE)

• All exchanges: LIMIT GTC 1% -> MARKET fallback. Stop timeout escalation. Priority scoring normalized.
• Verified: Unified policy running for 90+ trades. No exchange-specific failures.

Finding 186 — Trading mode persistence consistency (DONE)

• Shared helper trading_mode_persistence.py. Default testing_prod, self-healing file write.
• Verified: trading_mode_persistence.py in production use. Dashboard/trader/reconciliation all agree on mode.

Finding 187 — Monitor entrypoint + actionable closed-trade reconciliation (DONE)

• Service entrypoint run_gateway_monitor.py, reconciliation scoped to recent window, excludes reconciliation/external rows.
• Verified: ibkr-monitor service running. No legacy false-positive alerts since deploy.

Finding 188 — History tab trade log query interpolation guard (DONE)

• Fixed SQL string interpolation in get_trade_history(). Uses parameterized queries.
• Verified: History tab loads correctly. No SQL syntax errors in logs.

Finding 189 — IBKR blocking policy + disconnect alert path (DONE)

• CI verifies 1100 transient (no block) and 1300 critical (block). Disconnect alert uses send_alert.
• Verified: Unit tests pass. Running stable 2+ months. 1100 errors handled without blocking.

Finding 190 — Trader session vs gateway status separation (DONE)

• Reconnect resets IB object, idempotent callbacks, trading_connectivity model, snapshot freshness.
• Verified: Dashboard shows separate Trader Session/Gateway indicators. Stale snapshots correctly flagged.

Finding 192 — Orphan reconciliation hardening (DONE)

• Fail-closed skip on stale heartbeat/reconnect, context evidence, local ibkr_executions fallback.
• Evidence (Feb 24): HIMS ghost row reconciled. missing_in_ibkr=[] confirmed. Stable since.

Finding 193 — Close persistence permanent fix (AS 42P18) (DONE)

• Type-safe notes update, fallback close persistence, high-confidence stop-fill auto-close.
• Evidence (Feb 24): AS/HIMS reconciled. No 42P18 errors since fix. 90+ trades closed cleanly.

Finding 194 — Upcoming earnings visibility fail-safe (DONE)

• Dashboard falls back to earnings_markets DB rows when calendar returns empty.
• Evidence (Feb 24): DB fallback serving correctly. Earnings table populated reliably.

Finding 195 — Upcoming earnings table live-refresh (DONE)

• In-page refresh every 30s, metadata status line, retains last known-good rows.
• Verified: Table auto-refreshes. Error state recovers. Non-empty initial render confirmed.

Finding 196 — IBKR reconnect/reconciliation holistic hardening (DONE)

• Reconnect flags cleared before callbacks, guard skips reclassified as warning, bounded retry-on-ready, watchdog forensics.
• Evidence: No POSITION SYNC ERROR alerts for 3+ weeks. Session stable across multiple reconnects.

Finding 197 — Conviction robustness + close/reconciliation recurrence (DONE)

• Robust price-snapshot median estimator, signal fail-closed reliability gate, unified close path, high-confidence non-stop override.
• Evidence (Feb 26): EXEC_LEDGER_ONLY eliminated for SNOW. SNOW/ICUI metadata repaired. No recurrence.

Finding 198 — NaT EPS crash loop in Yahoo earnings check (DONE)

• float(NaT) wrapped in try/except, is_released set after successful conversion, all EPS format strings guarded.
• Verified: trader stable 6+ minutes (was crashing every 5 min), same-pattern check complete (8 instances).

Finding 199 — Direction-inverted duplicate prevention (DONE)

• Direction-aware duplicate check, IBKR hours parser fix, batch same-pattern fix.
• GPRO closed manually at $0.7752 (+$111.08). No duplicate prevention issues since.

Leverage Phase Test Tracker

• Phase A (read-only telemetry):
• PASSED: fresh/non-null leverage payload, live metric drift, stale-marking behavior
• Phase B (schema foundation):
• PASSED: first live trade populates leverage columns, no persistence regressions
• Phase C (financing ledger):
• PASSED: daily accrual for zero-open-position state
• BLOCKED: SHORT borrow + margin cost (requires active SHORT)
• BLOCKED: close-time aggregation to net PnL (requires close after accrual day)
• Phase D (margin blockers):
• BLOCKED: fail-closed blocking (requires live entry attempt under stale state)
• BLOCKED: shadow mode behavior (requires live entry attempt)
• Phase E (reporting + alerts):
• BLOCKED: financing-aware summary fallback (no populated financing rows)
• PASSED: legacy-row compatibility
• BLOCKED: leverage alert telemetry (needs triggered breach)

94) Dashboard event loop blocking from sync calls in async handlers (Finding #200) - Severity: High - Status: done - Implementation: Complete (Mar 7, 2026). Converted all route handlers from async def to def so FastAPI runs them in a thread pool. Removed inline blocking fallbacks for earnings calendar and trade history. - Root Cause: 1. All FastAPI route handlers were declared async def but called only synchronous blocking functions (DB queries, yfinance HTTP, NASDAQ calendar API). 2. In FastAPI, async def handlers run on the main event loop. Any blocking call freezes the entire loop, making ALL endpoints unresponsive. 3. The main dashboard route / had an inline fallback: when the cache was empty, it called get_earnings_with_polymarket(14) which makes 14+ HTTP requests to NASDAQ, plus Gamma API and yfinance calls. This single call could block the event loop for 30+ seconds. 4. Same pattern for get_trade_history("all") fallback — heavy DB query on cache miss. - Fix: 1. Changed all route handlers from async def to def. FastAPI automatically runs def handlers in a thread pool via anyio.to_thread.run_sync, preventing event loop blocking. 2. Removed get_earnings_with_polymarket(14) inline fallback from dashboard route — now always serves from cache (background thread populates within 30s). 3. Removed get_trade_history("all") inline fallback — same pattern, serves from cache or empty list. 4. Kept async def only for: lifecycle hooks (startup_event, shutdown_event), pure cache reads (api_earnings_calendar), and minimal-computation endpoints (api_health). - Same Pattern Check: All 25 route handlers audited. Every handler that calls blocking sync functions converted to def. - Files: earnings_dashboard/app.py

95) Strategy parameter changes: conviction cap at 90% and stop widening to 3%/4%/5% (Finding #202) - Severity: Critical - Status: done - Implementation: Complete (Mar 9, 2026). Two strategy changes based on analysis of 87 live trades. - Change 1 — LONG conviction cap at 90%: - Root Cause: 90%+ conviction LONG trades have 14.3% WR, -$116 P&L, 78% stop rate across 29 trades. The stock has already priced in the expected outcome at these conviction levels. - Fix: Added Config.SIGNAL_LONG_MAX_THRESHOLD = 0.90. Signal generator rejects LONG signals with conviction >= 90%. SHORT signals remain uncapped. - Same-pattern check: Updated signal_generator.py (generate_signal + is_valid), priority.py, conviction_calculator.py, dashboard app.py. - Change 2 — Stop loss widening to 3%/4%/5%: - Root Cause: What-If analysis using Polygon actual exit prices shows moderate stops (3%/4%/5%) improve total P&L from +$51 to +$166 (+$115 improvement). - Fix: Changed Config defaults: Large 2%->3%, Mid 3%->4%, Small 4%->5%. TradingSignal.post_init now reads from Config instead of hardcoded values. - Files: src/earnings/config.py, src/earnings/signal_generator.py, src/earnings/priority.py, src/earnings/conviction_calculator.py, earnings_dashboard/app.py - Documentation: Created docs/FINANCIAL_PROJECTIONS_REVISED_87_TRADES.md as comprehensive reference.

97) Double VIX reduction in position sizing (Finding #203) - Severity: Critical - Status: done - Implementation: Complete (Mar 9, 2026). Removed duplicate VIX reduction. - Root Cause: When VIX >= 30, _calculate_risk_level() returns ELEVATED (50% cut at line 929), THEN a separate VIX check at line 933 applies ANOTHER 50% cut. Result: 25% * 0.5 * 0.5 = 6.25% instead of intended 12.5%. ORCL got $153 (1 share) instead of ~$306 (2 shares). - Fix: Removed the duplicate VIX check at line 933. Risk level ELEVATED already handles VIX >= 30. - Files: src/earnings/risk_manager.py

98) ORCL limit+market double fill race condition (Finding #204) - Severity: High - Status: done (Finding #205 fixes the root cause) - Root Cause: Limit order filled at 14:34:05 but order status remained "PreSubmitted". At 14:34:17 (12s batch timeout), bot saw order as unfilled and placed market fallback. Market order filled at 14:34:20. IBKR=2 shares, DB=1. The duplicate prevention check reads IBKR positions, but the position cache hadn't updated with the limit fill in time. - Fix: See Finding #205.

99) Race condition root cause: time.sleep prevents IBKR event processing (Finding #205) - Severity: Critical - Status: done - Implementation: Complete (Mar 9, 2026). Replaced all time.sleep() with ib.sleep() in order monitoring and fallback paths. - Root Cause: time.sleep() blocks the Python thread but does NOT process the ib_insync event loop. Fill events arrive from IBKR gateway but sit in the event queue unprocessed. Order status, positions, and fills remain stale. Previous fixes (#3, #178, #199) addressed symptoms (wrong field names, direction checks) but never fixed the root cause: the event loop was never given a chance to process fills. - Fix: Replaced time.sleep() with self.order_executor.ibkr.ib.sleep() in: 1. batch_order_manager.py _monitor_orders() polling loop (line ~753) 2. batch_order_manager.py _apply_market_fallback() pre-position-check (new ib.sleep(1)) 3. batch_order_manager.py _apply_market_fallback() post-cancel delay (line ~834) 4. order_executor.py place_guaranteed_fill() monitoring loop (line ~651) 5. order_executor.py place_guaranteed_fill() post-cancel delay (line ~712) 6. order_executor.py instant exit monitoring loop (line ~1124) 7. order_executor.py instant exit fallback delays (lines ~1113, ~1188) - Why this is definitive: ib.sleep() processes all pending IBKR events. After each call, order status, positions, and fills are fully current. The duplicate prevention check now sees actual data, not stale cache. - Same-pattern check: All time.sleep() in order monitoring/fallback paths across both files replaced. - Files: src/earnings/batch_order_manager.py, src/earnings/order_executor.py - Mitigation: DB manually reconciled to 2 shares. Stop order covers only 1 share — needs manual fix. - Files: src/earnings/batch_order_manager.py, src/earnings/order_executor.py

100) verify_earnings_dates() silently failed due to self._get_engine() bug (Finding #206) - Severity: Critical - Status: done - Implementation: Complete (Mar 9, 2026). Fixed broken method call + added startup log. - Root Cause: verify_earnings_dates() called self._get_engine() which does not exist on SignalGenerator. The AttributeError was caught by a broad try/except and silently returned empty. The function should use self.engine (which exists and works). This meant earnings date cross-validation against NASDAQ calendar NEVER ran, despite the EarningsCalendar and finance_calendars being properly installed and available. - Impact: ORCL entered a day early (March 9 instead of March 10) because the DB date from Polymarket slug was wrong and verify_earnings_dates() would have caught and corrected it. - Fix: Changed self._get_engine() to self.engine. Added startup log confirming "NASDAQ calendar available for date cross-validation" so silent failures are immediately visible. - Files: src/earnings/signal_generator.py

101) Enable small cap longs at 70-80% conviction (Finding #207) - Severity: Medium - Status: done - Implementation: Complete (Mar 9, 2026). Changed small cap LONG rule from "require 90%+" to "allow 70-80% only". - Root Cause: Previous rule required 90%+ conviction for small cap longs, but Finding #202 blocked 90%+ entirely. This made small cap longs impossible. Backtest analysis (Q3 2025, proper MCAP classification) showed: - 70-80% conviction: 6 trades, 66.7% WR, EV +$30.68 (strong positive) - 80-90% conviction: 10 trades, 30.0% WR, EV +$0.40 (near-zero, not worth the risk) - Fix: Allow small cap longs at 70-80% only (block 80%+). Full position sizing (10% of capital). Priority score 2/6. - Files: src/earnings/signal_generator.py, src/earnings/priority.py, earnings_dashboard/app.py, earnings_dashboard/templates/dashboard.html

102) Dual MCAP classification systems caused KSS to bypass small cap rules (Finding #208) - Severity: Critical - Status: done - Implementation: Complete (Mar 9, 2026). Centralized market cap data into single source of truth. - Root Cause: Signal generator had its own MARKET_CAPS dict (~100 symbols) separate from the dashboard's _KNOWN_MCAPS (265 symbols). KSS ($3B) was not in the signal generator's dict. yfinance lookup failed (rate-limited), so classify_market_cap(None) defaulted to MID. As Mid cap, KSS at 86.5% conviction LONG passed all filters. As Small cap (correct), it should have been blocked (80%+ LONG not allowed for small caps). - Fix: 1. Created src/earnings/market_cap_data.py — single source of truth for all market cap classifications (265+ symbols). Used by signal generator, dashboard, and backtest scripts. 2. Signal generator's get_market_cap_for_symbol() now reads from KNOWN_MCAPS as primary source, yfinance as fallback. 3. Changed classify_market_cap(None) default from MID to SMALL (most restrictive — unknown stocks get small cap safety rules). 4. Added discover_unknown_mcaps() auto-enrichment: queries upcoming earnings symbols, looks up unknowns via yfinance, persists tier to DB, adds to runtime dict. Runs during dashboard deep refresh (every 2h). 5. Dashboard _KNOWN_MCAPS now imports from the shared module (was inline 285-line dict). - Same Pattern Check: Searched for MarketCap.LARGE defaults in DB loading code — found 4 instances in reconciliation/position/orchestrator, but these are for existing positions where the column is already populated at entry. Not the same pattern. - Impact: KSS entered as LONG at 86.5% with Mid cap rules ($430 position, 4% stop). Should have been blocked entirely. - Files: src/earnings/market_cap_data.py (new), src/earnings/signal_generator.py, earnings_dashboard/app.py

103) Dashboard earnings calendar always shows "Refresh error" (Finding #209) - Severity: Medium - Status: done - Implementation: Complete (Mar 10, 2026). Increased JS abort timeout from 15s to 30s and added resp.ok check. - Root Cause: The JS AbortController timeout was 15s. On cold start, the server thread pool is saturated by the slow / page render (~55s), causing the /api/earnings-calendar fetch to timeout. Additionally, the fetch did not check resp.ok before calling resp.json(), so 500 errors with HTML bodies caused parse failures shown as generic "Refresh error". - Fix: Increased timeout to 30s. Added if (!resp.ok) throw new Error('Server returned ' + resp.status) before resp.json() for clearer error messages. - Files: earnings_dashboard/templates/dashboard.html

104) Main loop stall during BMO/AMC scheduled cycles causes forced restarts (Finding #210) - Severity: High - Status: done - Implementation: Complete (Mar 10, 2026). Added intermediate progress marks and NASDAQ timeout. - Root Cause: The main loop watchdog (120s timeout) attributed ALL work inside run_bmo_cycle() and run_amc_cycle() to a single progress stage run_scheduled_cycles. These cycles legitimately take >120s due to: (1) verify_earnings_dates() making 5 NASDAQ API calls with no timeout, (2) signal generation doing yfinance lookups per symbol, (3) multi-day BMO reservation calculation. The watchdog saw no progress for >120s and killed the process. - Fix: (A) Added self._mark_main_loop_progress() calls inside run_bmo_cycle() and run_amc_cycle() after verify_dates and after signal generation, so watchdog sees forward progress. (B) Wrapped EarningsCalendar.get_upcoming_earnings() in verify_earnings_dates() with a 30s ThreadPoolExecutor timeout to prevent hung NASDAQ API from blocking indefinitely. - Same Pattern Check: Other get_earnings_by_date() calls in earnings_timing_fallback.py are per-symbol (not bulk) and have individual try/except — lower risk. The bulk call in verify_earnings_dates() was the only unbounded one. - Files: src/earnings/trading_orchestrator.py, src/earnings/signal_generator.py

105) Position sizing increase: 25/18/10 to 30/22/15 (Finding #211) - Severity: Medium - Status: done - Implementation: Complete (Mar 10, 2026). Increased position sizes based on 95-trade analysis. - Root Cause: The 90%+ conviction filter (Finding #202) removed ~36% of trades (26 out of 81 LONG trades). Fewer concurrent positions means more capital available per trade. Analysis of position size quintiles showed Q5 (largest positions, avg $490) generated best absolute returns (+$139, $7.70/trade avg) while Q1-Q2 (smallest, $157-$226) were net losers (-$40 combined). The "kept" trades (70-89% conviction) have 40% WR vs 15% for filtered 90%+. - Fix: Changed Config defaults — Large 25%→30%, Mid 18%→22%, Small 10%→15%. On $2,450 capital: Large $735 (+$123/trade), Mid $539 (+$98/trade), Small $367 (+$122/trade). Max concurrent exposure ~65-90% of capital with fewer positions. - Same Pattern Check: Updated all references in risk_manager.py (4 locations), STRATEGY.md (3 tables), FINANCIAL_PROJECTIONS_REVISED_87_TRADES.md, dashboard.html (System tab). - Files: src/earnings/config.py, src/earnings/risk_manager.py, docs/STRATEGY.md, docs/FINANCIAL_PROJECTIONS_REVISED_87_TRADES.md, earnings_dashboard/templates/dashboard.html

96) Volume_24h_at_entry never populated in trade_audit_log (Finding #201) - Severity: Medium - Status: done - Implementation: Complete (Mar 8, 2026). Added volume_24h_at_entry to all 4 INSERT INTO trade_audit_log statements. Backfilled 41/94 trades. - Root Cause: All INSERT statements omitted the column despite signal.volume_24h being available in scope. - Files: src/earnings/trading_orchestrator.py, src/earnings/position_manager.py, src/earnings/reconciliation_manager.py

107) Chinese ADRs and other stocks missing from COMPANY_TIMING_LOOKUP — missed trades (Finding #213) - Severity: High - Status: done - Implementation: Complete (Mar 18, 2026). Added 27 tickers to DEFAULT_TIMINGS in earnings_timing.py. - Root Cause: PDD, BABA, NIO, XPEV, LI, WB, JD and other stocks were not in the COMPANY_TIMING_LOOKUP table. When the Gamma API endDate fell in the ambiguous 13-19 UTC window or midnight (00:00), the sync service stored earnings_time = NULL. The signal generator's determine_earnings_timing() then checked: (1) DB=NULL, (2) COMPANY_TIMING_LOOKUP=not found, (3) EarningsCalendar=time-not-supplied for some, (4) yfinance=rate limited. Result: timing returns None, trade is SKIPPED. - Impact: PDD (6/6 priority, Large cap, 77% conviction LONG) would have been skipped entirely. BABA was correctly resolved by EarningsCalendar but others like CTAS and DBI were not. - Fix: Added 27 tickers to DEFAULT_TIMINGS covering Chinese ADRs (BMO), payroll/business services (BMO), retail (BMO), and others. All classifications verified against historical DB data. - Added: BABA, PDD, JD, NIO, XPEV, LI, WB, BIDU, BEKE, TME, BILI, IQ (Chinese ADRs BMO), PAYX, ADP, ACN (payroll BMO), DBI, CHWY, KSS, BBW, ANF, AEO, URBN, ASO (retail BMO), WSM, CTAS (BMO), FIVE, GME, NMAX (AMC). - Files: src/earnings/earnings_timing.py

106) yfinance rate-limited on server — eliminate dependency for mcap auto-discovery (Finding #212) - Severity: High - Status: done - Implementation: Complete (Mar 16, 2026). Routed auto-discovery through yfinance_limiter with Finnhub fallback, added IBKR-first VIX/SPY, reduced dashboard yfinance calls. - Root Cause: Production server IP permanently rate-limited by Yahoo Finance (429 on every call). discover_unknown_mcaps() called yf.Ticker() directly (no fallback). Dashboard made uncoordinated direct yfinance calls for VIX/SPY, company names, dependency checks, and batch mcap. All returned zero data, leaving symbols as "Unknown" in the dashboard. - Fix: 1. Added 6 missing symbols to KNOWN_MCAPS: BABA ($200B Large), CHWY ($12B Mid), CTAS ($80B Mid), NMAX ($2B Small), PAYX ($50B Mid), DLTR ($15B Mid). 2. Routed discover_unknown_mcaps() through yfinance_limiter.get_market_cap() which has automatic Finnhub fallback when yfinance returns 429. 3. Dashboard _fetch_market_health_data() now checks IBKR snapshot for VIX/SPY before falling back to yfinance. 4. Dashboard get_company_name() checks KNOWN_MCAPS tuple[2] before calling yfinance. 5. Dashboard check_dependencies() no longer makes a yfinance API call (just checks module import). 6. Dashboard populate_mcap_cache() skips symbols already in KNOWN_MCAPS. - Same Pattern Check: Remaining direct yf.Ticker() calls in dashboard were routed through yfinance_limiter (Mar 22, 2026 — Finding #217). Finnhub fallback behavior for market cap was tightened in Finding #219 (empty Yahoo marketCap + MEDIUM priority during Yahoo backoff). - Files: src/earnings/market_cap_data.py, earnings_dashboard/app.py

108) Production timing verification — PDD/BABA BMO (Finding #214) - Severity: Low (ops / evidence) - Status: done - Evidence: docs/OPS_TIMING_VERIFICATION.md. Production logs show PDD: Timing from database: BMO and BABA: Timing from database: BMO (Mar 18–19, 2026). Trades executed and exited per plan (stops filled Mar 19). - Files: docs/OPS_TIMING_VERIFICATION.md

109) Dashboard cold start — heavy _fetch_db_stats_from_db on first paint (Finding #215) - Severity: Medium - Status: done - Root Cause: First HTTP request to / ran the full DB stats query (category coverage joins, per-market availability, data_gaps) before background cache warmed. - Fix: _fetch_db_stats_from_db(full=False) on cache miss + background thread upgrades to full=True; reuse state.positions for timeline to avoid duplicate positions query. - Files: earnings_dashboard/app.py

110) PostgreSQL “out of shared memory” / lock pressure on managed DO (Finding #216) - Severity: Medium - Status: deferred (monitor) - Root Cause: Transient lock spikes during heavy dashboard queries; max_locks_per_transaction not adjustable on DigitalOcean managed Postgres. - Mitigation: Avoid many simultaneous dashboard hard-refreshes; rely on dashboard caching; see docs/POSTGRES_OPERATIONS.md. - Files: docs/POSTGRES_OPERATIONS.md

111) Dashboard yfinance — route fallbacks through limiter (Finding #217) - Severity: Medium - Status: done - Fix: Replaced remaining direct yf.Ticker() usage in dashboard with yfinance_limiter (get_market_cap, get_price, get_ticker_info, get_exchange, VIX/SPY already via limiter after IBKR). - Files: earnings_dashboard/app.py

112) Strategy Analysis — position sizing quintiles in report + tab (Finding #218 / memory.md §K) - Severity: Low - Status: done - Fix: compute_position_sizing_quintiles() in scripts/strategy_performance_analysis.py; table + renderPositionQuintiles() in Strategy Analysis tab; print_summary() prints quintiles when present. Re-run analysis script on prod (with DATABASE_URL) after deploy to refresh strategy_analysis_report.json. - Files: scripts/strategy_performance_analysis.py, earnings_dashboard/templates/dashboard.html

113) Dashboard upcoming table — mass “Unknown” MCAP despite #212/#217 (Finding #219) - Severity: High - Status: done - Root Cause: (1) YFinanceRateLimiter.get_market_cap() only called Finnhub on specific Yahoo exception text; Yahoo often returns empty info/no marketCap without raising, so Finnhub never ran. (2) For Priority.MEDIUM (dashboard batch), when Yahoo was in backoff, the method returned None immediately — Finnhub was skipped entirely. (3) populate_mcap_cache could treat mixed-case tickers as uncached vs get_cached_mcap lookups. (4) Deep-refresh discover_unknown_mcaps used a 60s thread timeout vs dozens/hundreds of sequential lookups. (5) DB persist used exact ticker = :symbol match; casing mismatches skipped updates. (6) Hourly earnings_market_fetcher inserted earnings_markets rows without market_cap_category, so the dashboard showed NULL/Unknown until a rare manual sync. - Fix: Always attempt Finnhub when Yahoo yields no cap (and when Yahoo is rate-limited, before giving up on non-CRITICAL); Finnhub calls throttled ~1.05s apart for free-tier quota; uppercase-normalized dashboard mcap cache + UPPER(TRIM(ticker)) in persist; discovery timeout 300s. EarningsMarketSync.get_market_cap_category() checks KNOWN_MCAPS then limiter; backfill_upcoming_null_mcaps() runs from run_full_sync() and after each EarningsMarketFetcher.run_full_sync(); API gap-fill inserts include company + mcap when possible. YFinanceRateLimiter no longer passes requests.Session into yf.Ticker (Yahoo requires curl_cffi session). - Future-proof: Keep FINNHUB_API_KEY on prod; optional IBKR fundamentals if Yahoo+Finnhub fail; monitor limiter + Finnhub 429s in logs. - Files: src/utils/caching.py, earnings_dashboard/app.py, src/earnings/market_cap_data.py, src/db/earnings_sync.py, src/data_tracker/collectors/earnings_market_fetcher.py, scripts/sync_earnings_markets.py - Documentation (cross-refs only — canonical narrative stays here per memory.md §J): docs/CONTEXT.md, docs/STRATEGY.md §7.8, docs/DATABASE_DIRECTIVES.md, docs/OPERATIONS.md, .cursor/memory.md §L. Finding #220 extends the same docs for BMO/AMC DB sync + dashboard gating.

114) Trusted earnings_time DB enrichment, upcoming TRADE gating, sync transaction fix, UNTY default (Finding #220) - Severity: High (data integrity + operator clarity) - Status: done - Implementation: Complete (Apr 2026). - Root Cause: 1. TBD / NULL earnings_time: NASDAQ calendar often returns non-parseable strings; sync_from_markets_table only stored calendar output. Tickers not in DEFAULT_TIMINGS/overrides.json stayed NULL/TBD. Dashboard showed ✓ TRADE on conviction/volume/depth alone while BMO/AMC was unknown — misleading vs batch logic (SignalGenerator skips when determine_earnings_timing() returns None unless runtime fallback resolves). 2. sync_from_polymarket_api INSERT omitted earnings_time, leaving gaps until a later path filled rows. 3. SQLAlchemy 2: with engine.connect() plus nested with conn.begin() on the same connection per row caused “Transaction already initialized” — hundreds of sync errors, zero effective updates per run on affected deployments. - Fix: 1. resolve_earnings_time_for_sync() — Order: NASDAQ calendar only if explicit BMO/AMC; else get_timing_manager().get_timing() (same trusted tier as signal generator priority 2). Does not persist yfinance/web fallback into DB (reliability: no speculative timing stored as calendar truth). 2. backfill_upcoming_null_earnings_time() — Fills upcoming rows where earnings_time is NULL, empty, or TBD. Wired into run_full_sync() after mcap backfill. Logs as earnings_time_backfill in scripts/sync_earnings_markets.py. 3. API gap-fill INSERT now includes earnings_time when resolvable. 4. Per-row with self.engine.begin() for market-table sync writes; initial SELECT uses a short-lived connect() only. 5. Dashboard upcoming table: ✓ TRADE requires known BMO or AMC; if edge/liquidity OK but timing unknown → NEEDS TIMING badge (tooltip notes bot may still use runtime fallback). 6. UNTY (Unity Bancorp): Added UNTY → BMO to DEFAULT_TIMINGS (regional bank, pre-market). One-off DB fixes: scripts/backfill_ticker_earnings_timing.py (--ticker / --timing) for NULL/TBD/empty only. - Same Pattern Check: Signal generator unchanged for fallback order; DB layer aligned to trusted sources only. - Files: src/db/earnings_sync.py, scripts/sync_earnings_markets.py, earnings_dashboard/templates/dashboard.html, src/earnings/earnings_timing.py, scripts/backfill_ticker_earnings_timing.py - Documentation: docs/DATABASE_DIRECTIVES.md, docs/OPERATIONS.md, docs/CONTEXT.md, docs/STRATEGY.md §7.8, .cursor/memory.md (cross-refs).

115) Polymarket opportunity log + counterfactual queue for all reject stages (Finding #221) - Severity: Low (research / analytics) - Status: done - Root Cause: Post-hoc analysis needed a single append-only log for every Polymarket earnings candidate on the signal path (not only executed trades or skipped_signals volume skips), with live-rule entry/exit/stop timestamps for offline P&L simulation. - Fix: 1. Migration 021_polymarket_opportunity_log.sql: Table polymarket_opportunity_log with skip_stage / skip_reason, conviction/volume/depth/mcap/priority, scheduled entry/exit + stop_loss_pct, JSON snapshot, nullable counterfactual columns. 2. SignalGenerator: _emit_pm_opportunity_log + _hypothetical_schedule_fields populate BMO/AMC schedules and stop % (from Config by cap tier) for all stages where timing + earnings datetime are known (including depth/timing/generate_signal rejects and calendar paths). 3. src/earnings/counterfactual_pricing.py + scripts/compute_counterfactual_pnl.py: With POLYGON_KEY, loads Polygon 1-minute bars between scheduled entry/exit, fills at bar close near schedule (research proxy for limit fills), applies stop on bar low (LONG) / high (SHORT) vs logged stop_loss_pct. Without Polygon, yfinance daily close-to-close on entry/exit calendar days (yfinance_daily_approx_v1, no intraday stop). hypothetical_pnl uses fixed USD 10,000 notional (COUNTERFACTUAL_NOTIONAL_USD env override). Script marks non-simulatable rows no_price_simulation_v1 and UPDATEs price columns for simulatable rows. - Files: src/data_tracker/migrations/021_polymarket_opportunity_log.sql, src/earnings/polymarket_opportunity_log.py, src/earnings/signal_generator.py, src/earnings/counterfactual_pricing.py, scripts/compute_counterfactual_pnl.py, tests/unit/test_counterfactual_pricing.py - Documentation (cross-refs per memory.md §J): docs/DATABASE_DIRECTIVES.md, docs/DATA_ARCHITECTURE.md, docs/OPERATIONS.md, docs/CONTEXT.md, docs/STRATEGY.md

Finding 220 — Trusted earnings_time sync + dashboard TRADE gating + SQLAlchemy sync fix (DONE)

• DB timing enrichment uses calendar + DEFAULT_TIMINGS/overrides only; upcoming timing backfill; API inserts set earnings_time when known; sync uses per-row engine.begin(); dashboard NEEDS TIMING when BMO/AMC unknown; UNTY default BMO + optional backfill_ticker_earnings_timing.py.
• Verified: Code on master (commits 58e50b5, 652bf92, b1d8ecf area). Run run_full_sync / sync_earnings_markets.py with PYTHONPATH set to project root on servers.

Finding 222 — Volume threshold: trader vs dashboard mismatch + NULL volume_total (DONE)

• Severity: High (operational / trust)
• Status: done
• Retrospective (Apr 16, 2026): The first volume alignment MR (#222) was correct on the original bug (config vs hardcoded dashboard) but did not satisfy .cursor/memory.md §B (full execution path + runtime simulation), §E (all consumers of “volume for gate vs display”), or §G (NULL vs 0 vs Gamma, Jinja vs JS) in one pass. That incomplete rollout directly caused the user-visible regressions repaired in #223 ($0 display) and #224 (volume_ok vs displayed $ / trader DB-only). Canonical process lesson: Finding #225.
• Root cause: SIGNAL_MIN_VOLUME_TOTAL defaulted to 0.0, so SignalGenerator._build_trading_signal skipped the low-volume branch whenever volume_total was NULL (condition volume_total is not None was false). The earnings dashboard used a hardcoded $2,000 floor and Gamma volume, not markets.volume_total, so the UI could show LOW VOL / below $2K semantics while the bot still emitted signals. trading_positions.min_volume_threshold_used and position_manager inserts hardcoded 2000.0 instead of config.
• Fix:
• Config: SIGNAL_MIN_VOLUME_TOTAL default 2000.0 (USD cumulative); set 0 in .env to disable the gate (e.g. research).
• SignalGenerator: If threshold > 0 and volume_total is missing, fail-closed — log skipped_signals with skip_reason=volume_total_missing, return volume_total_missing. Low cumulative volume still uses low_volume with volume_24h column storing the total USD value compared (unchanged analytics column semantics).
• Dashboard get_polymarket_earnings_from_db: read m.volume_total from SQL; volume_ok / volume_threshold use Config.SIGNAL_MIN_VOLUME_TOTAL; same fail-closed rule when DB total is NULL and threshold > 0.
• Orchestrator / PositionManager: persist Config.SIGNAL_MIN_VOLUME_TOTAL as min_volume_threshold_used.
• Files: src/earnings/config.py, src/earnings/signal_generator.py, earnings_dashboard/app.py, earnings_dashboard/templates/dashboard.html, src/earnings/trading_orchestrator.py, src/earnings/position_manager.py, tests/unit/test_signal_volume_threshold.py, .env.example
• Documentation (cross-refs per memory.md §J): docs/STRATEGY.md §2.4, docs/CONTEXT.md, docs/OPERATIONS.md, docs/EDGE_CASES.md, docs/DATA_ARCHITECTURE.md, .cursor/memory.md

Finding 223 — Dashboard showed $0 volume after #222 (DB volume_total = 0 vs Gamma) (DONE)

• Severity: Medium (UX / trust; not a swap with OB metrics)
• Status: done
• Root cause: Finding #222 set the VOLUME column from markets.volume_total when not SQL-NULL, else Gamma. SignalGenerator.get_market_volume uses float(row.volume_total) if row.volume_total else None, so numeric 0 in Postgres is treated as no total (same as NULL). The dashboard used float(row[8]) if row[8] is not None else None, so 0 stayed 0 and never fell back to Gamma’s cumulative volume. Many markets rows carry 0 until collectors backfill, while Gamma and orderbook_snapshots already show activity — hence $0 volume beside healthy OB quality (different pipelines, not swapped fields).
• Fix: In get_polymarket_earnings_from_db, normalize volume_total the same way as get_market_volume (NULL or 0 → db_volume_total is None); keep volume_ok fail-closed when there is no positive DB total and threshold > 0; display uses Gamma when DB has no positive total.
• Files: earnings_dashboard/app.py
• Follow-up (ops/data): Ensure collectors / market sync persist markets.volume_total from the API so DB and Gamma converge; optional dashboard hint when volume is Gamma-only vs DB-backed.

Finding 224 — MIN VOLUME column all ⚠ + LOW VOL despite displayed $>2K (DONE)

• Severity: High (UX / trust; strategy alignment)
• Status: done
• Root cause: Finding #223 made the VOLUME cell use Gamma when DB volume_total was NULL/0, but volume_ok still required db_volume_total is not None and db_volume_total >= threshold. Display showed e.g. $3,282 while volume_ok stayed false → every row showed the yellow ⚠ in MIN VOLUME and LOW VOL in TRADE? even when cumulative liquidity exceeded $2K. Trading used get_market_volume (DB-only), so the bot would also skip those names as volume_total_missing — dashboard and bot were consistent but wrong vs Polymarket liquidity.
• Fix:
• src/earnings/volume_policy.py: effective_cumulative_volume_usd (positive DB else Gamma) and volume_threshold_passes — single rule for gate + display dollar amount.
• Dashboard get_polymarket_earnings_from_db: volume / volume_ok use that module (same as trader semantics).
• SignalGenerator.get_market_volume: When DB has no positive volume_total (or no row), Gamma API cumulative volume with a short TTL cache; when DB is positive, use DB only (no extra HTTP).
• Files: src/earnings/volume_policy.py, src/earnings/signal_generator.py, earnings_dashboard/app.py, tests/unit/test_volume_policy.py
• Documentation: docs/STRATEGY.md §2.4 (gate uses effective cumulative = DB if positive else Gamma).

Finding 225 — Volume rollout: incomplete interface audit (process / regression chain) (DONE)

• Severity: High (process — prevented full memory.md compliance on first ship)
• Status: done (lesson + checklist captured)
• Root cause (investigation starts at #222): Everything the user saw after the first volume “fix” today traced to #222 shipping without enumerating all interfaces that interpret “cumulative volume” and “passes $2K floor.” In one change set, defaults, signal skip reasons, dashboard Python, SQL columns, Jinja upcoming table, inline JS mobile render, and DB insert fields were touched or implied — but not verified as one coherent definition across paths. #223 and #224 were not separate root bugs; they were cascade failures from that gap (display vs get_market_volume truthiness on 0; then volume_ok still DB-only while display used Gamma).
• Fix (process): (1) Single module for the rule: src/earnings/volume_policy.py + get_market_volume Gamma fallback (#224). (2) .cursor/memory.md §M — mandatory grep + edge matrix before any future cross-cutting gate change. (3) This finding as canonical “start investigation from first rollout” anchor.
• Files: docs/FINDINGS.md (this entry + #222 retrospective line), .cursor/memory.md §M + §L update
• Related fixes: #222 (intent), #223 (0 vs NULL display), #224 (gate = display = trader effective volume)

Finding 226 — Dashboard VOLUME column felt stale (DB 15m vs Gamma ~30s) (DONE)

• Severity: Medium (UX / trust)
• Status: done
• Root cause (not a Polymarket API break): markets.volume_total is written by continuous_collector.sync_markets_from_gamma every 15 minutes (MARKET_SYNC_INTERVAL = 900). The upcoming table already fetches Gamma per refresh (dashboard cache ~30s). Using effective_cumulative_volume_usd for display preferred positive DB, so the VOLUME cell could lag polymarket.com by up to ~15 min while conviction/price in the same row was live. Production check: MAX(markets.last_synced_at) was current; polymarket-collector active.
• Fix: volume in get_polymarket_earnings_from_db = Gamma cumulative (volume / volumeNum) from the same response as prices; volume_ok still volume_threshold_passes (DB + Gamma per #224).
• Optional ops: Lower MARKET_SYNC_INTERVAL if DB analytics need fresher markets.volume_total.
• Files: earnings_dashboard/app.py

Finding 227 — Broken learning loop: counterfactual + outcome resolution + market regime capture (DONE)

• Severity: High (strategy / data integrity — every other study depends on this)
• Status: done
• Root cause (three independent gaps, single fix):
• polymarket_opportunity_log had the counterfactual schema (added in #221) but 0 of 268 historical rows had hypothetical_pnl computed — scripts/compute_counterfactual_pnl.py was operator-only with no timer, so cost-of-skipped-signal could never be measured.
• skipped_signals had actual_outcome / would_have_won columns since migration 007 but 303 of 303 rows were NULL — no writer was ever built. The volume-gate (#222–#226) could never be validated.
• earnings_markets.actual_beat was only set on the live trading path (orchestrator on instant-exit detection), so only 68 of 800 (8.5%) were resolved; 194 high-conviction past-date markets were unresolved. The post-earnings eps_actual / revenue_actual / stock_price_* / stock_return_* columns had no automated writer at all.
• trade_audit_log.vix_at_entry / sp500_change_at_entry were NULL on 107 of 107 trades — the columns existed but the writer in position_manager.persist_position_open never set them. VIX / SPY / regime circuit breakers and bucketed studies could never run on real data.
• Fix (4 tracks, single Finding):
• Schema (additive only): migrations 022 (regime_at_entry, regime_details on trade_audit_log), 023 (intended_position_value, intended_shares, intended_stop_price, intended_flex_applied, regime_at_entry, vix_at_entry, sp500_change_at_entry on polymarket_opportunity_log), 024 (same set + hypothetical_pnl, hypothetical_pnl_pct, hypothetical_stop_hit, counterfactual_computed_at, counterfactual_version on skipped_signals).
• Forward capture (Track B — strategy replay): new src/earnings/market_regime_provider.py — single shared MarketRegimeProvider (IBKR primary for VIX+SPY, yfinance fallback, 60-second snapshot cache, SPY MA20/50/200 classifier matching scripts/strategy_performance_analysis.get_spy_regime_at_date). SignalGenerator.__init__ now accepts regime_provider + risk_manager; _emit_pm_opportunity_log and log_skipped_signal stamp every row with regime/VIX/SPY change and intended sizing. position_manager._persist_position_open writes regime_at_entry, regime_details, vix_at_entry, sp500_change_at_entry to trade_audit_log on entry. TradingOrchestrator constructs the singleton provider once and threads it into both. Per memory.md §M, this is the only definition of regime classification — any future gate must import from here.
• Outcome resolver (Track C — single writer): new src/data_tracker/collectors/outcome_resolver.py with run_once() performing four ordered passes:
  • Pass 1 — earnings_markets: Yahoo earnings_history → Finnhub /stock/earnings → late-stage Polymarket probability proxy for actual_beat / eps_actual; Polygon 1m bars for stock_price_day_before_close / _day_of_open / _1h_after / _4h_after / _day_of_close / _next_day_close; stock_return_* derived arithmetically. Idempotent via COALESCE(... IS NULL).
  • Pass 2 — skipped_signals: reuses counterfactual_pricing.simulate_minute_path with scheduled BMO/AMC entry/exit and a 3% default stop; sets actual_outcome (win / loss / stop), would_have_won, and the hypothetical_* columns.
  • Pass 3 — polymarket_opportunity_log: invokes existing run_counterfactual_for_row with fixed $10K notional (replaces the manual compute_counterfactual_pnl.py usage but preserves it for ad-hoc runs).
  • Pass 4 — trade_audit_log.prediction_correct: single SQL UPDATE joining to earnings_markets.actual_beat (now populated by pass 1). Replaces the hardcoded beat_map in scripts/backfill_prediction_correct.py. Wired as polymarket-resolver.service + .timer at 06:00 ET daily, installed by deploy/deploy_from_github.sh.
• Backfill (Track A — fixed $10K): scripts/backfill_learning_loop.py (looped resolver, no date floor) + scripts/backfill_regime_for_audit_log.py (one-shot historical regime/VIX fill for the 107 closed trades using classify_regime_for_date + fetch_historical_vix). Polygon plan guidance + POLYGON_KEY added to .env.example. Polygon Stocks Starter ($29/mo, 100 req/min) recommended; default --polygon-sleep 0.6.
• Strategy Analysis tab (Track D — memory §K): scripts/strategy_performance_analysis.py adds three new sections — skipped_signal_cost, volume_gate_counterfactual, regime_vix_performance — persisted in scripts/strategy_analysis_report.json and rendered in the dashboard "Strategy Analysis" tab via new renderLearningLoop() HTML + JS. Replaces the old all-NULL VIX bucket section.
• Verification (memory §B / acceptance):
• SELECT count(*) FROM polymarket_opportunity_log WHERE counterfactual_computed_at IS NULL → 0 after backfill
• SELECT count(*) FROM skipped_signals WHERE earnings_date < CURRENT_DATE AND counterfactual_computed_at IS NULL → 0 after backfill
• SELECT count(*) FROM earnings_markets WHERE earnings_date < CURRENT_DATE - INTERVAL '2 days' AND actual_beat IS NULL → < 5% (Yahoo + Finnhub + PM resolution lag)
• SELECT count(*) FROM trade_audit_log WHERE entry_time > '2026-04-22' AND vix_at_entry IS NULL → 0
• journalctl -u polymarket-resolver.timer shows daily 06:00 ET run.
• Dashboard "Strategy Analysis" → "Learning Loop (Finding #227)" populates with non-empty rows.
• Same-pattern check (memory §E): Searched for other places that re-implement regime/VIX/SPY classification or that write actual_beat / actual_outcome / prediction_correct. Findings: orchestrator's instant-exit path still pre-fills actual_beat (kept — strictly faster than the 06:00 timer); scripts/backfill_prediction_correct.py superseded by resolver Pass 4 but left in repo for one-off symbol-list use. No duplicate regime classifier — scripts/strategy_performance_analysis.get_spy_regime_at_date is now mirrored by MarketRegimeProvider; future work can collapse the script into a thin wrapper.
• Files:
• Schema: src/data_tracker/migrations/022_regime_vix_capture.sql, 023_opportunity_log_intended_size.sql, 024_skipped_signals_intended_size.sql
• Forward capture: src/earnings/market_regime_provider.py (new), src/earnings/signal_generator.py, src/earnings/polymarket_opportunity_log.py, src/earnings/position_manager.py, src/earnings/trading_orchestrator.py
• Resolver + timer: src/data_tracker/collectors/outcome_resolver.py (new), deploy/systemd/polymarket-resolver.service (new), deploy/systemd/polymarket-resolver.timer (new), deploy/deploy_from_github.sh
• Backfill: scripts/backfill_learning_loop.py (new), scripts/backfill_regime_for_audit_log.py (new)
• Reports + dashboard: scripts/strategy_performance_analysis.py, earnings_dashboard/templates/dashboard.html
• Env: .env.example (POLYGON_KEY + plan guidance)
• Documentation (cross-refs only — canonical narrative is here per memory §J): docs/DATA_ARCHITECTURE.md (resolver + new columns), docs/DATABASE_DIRECTIVES.md (writer ownership of actual_beat / vix_at_entry), docs/STRATEGY.md §6 / §10 (regime-aware future gates pointer), docs/OPERATIONS.md (new service runbook + manual backfill commands), docs/AGENT_HANDOFF.md (services list + finding pointer), .cursor/memory.md §M (regime provider as single source).

Finding #227 — Update (Apr 22, 2026): Polygon EPS + Polymarket cascade

After the initial Track A backfill, actual_beat coverage was only 25% (179/715) — Yahoo's earnings_history and Finnhub's free-tier /stock/earnings both have severe coverage gaps for mid/small-caps. Two additive fixes raised this to 73% (524/715):

Tier 3 — Polygon /vX/reference/financials (commit 926f967): - Probed Polygon Stocks Starter ($29/mo): the financials endpoint is unlocked (the screenshot's "Financials & Ratios ✗" gate refers to the realtime ratios endpoint, not historical SEC filings). - New _polygon_earnings_actual helper queries with a ±120 day window, picks the filing whose end_date is closest to target within 30 days, prefers diluted EPS over basic. - Also captures revenue_actual from Polygon's revenues field (was 0 rows before). - Coverage lift: actual_beat 25% → 51%, eps_actual 15 → 210 (14×), revenue_actual 0 → 197.

Tier 4 — Polymarket cascade (commit a86058f): - The remaining 49% gap was almost entirely "we have eps_actual from Polygon but no eps_estimate from anywhere → can't compute BEAT." The Polymarket data was always there; we weren't using it properly. - New cascade in _pm_resolution_actual_beat: 1. Post-event yes_price (latest snapshot from price_snapshots >12h after earnings_date) — markets converge to 0.000 / 1.000 within hours of resolution; this is essentially ground truth. Sample of 15 unresolved tickers showed every one converged cleanly (TFC 1.000, STT 0.001, JNJ 0.820 → 0.999, etc). 2. Pre-event prob cascade (prob_30min_before → prob_1h → prob_6h → prob_12h → prob_24h) — falls through nulls to find the most recent conviction read. - Threshold relaxed from 0.99/0.01 to 0.95/0.05, calibrated against our own resolved-EPS rows: yes_price ≥0.95 → BEAT with 93.75% accuracy (n=112); yes_price ≤0.05 → MISS with 93.6% accuracy (n=47). The old strict thresholds remain available via strict=True for callers that want maximum precision. - Coverage lift: actual_beat 51% → 73% (366 → 524). The 158-row gain came almost entirely from price_snapshots post-event yes_price.

Final actual_beat source mix on the 524 resolved rows: - EPS surprise (eps_actual ≥ eps_estimate): primary for ~30% (where Yahoo or Polygon supplied both numbers) - Post-event Polymarket yes_price: primary for ~50% - Pre-event prob cascade: primary for ~20%

Remaining 191 unresolved rows (~27%): these are markets where neither Yahoo / Finnhub / Polygon EPS exists AND no Polymarket conviction snapshot was ever taken (the conviction_tracker service didn't have those condition_ids in scope when their earnings ran). Most are pre-Jan-2026 markets predating reliable conviction collection. The daily polymarket-resolver.timer will keep retrying; new tickers as they're added to the conviction collector will be picked up automatically.

Final test count: 51 unit tests in test_outcome_resolver.py + test_market_regime_provider.py, including the 9 new tests for the Polymarket cascade (post-event vs pre-event, default vs strict thresholds, NULL cascade, missing-key tolerance for older schema rows).

Polygon plan note: The user upgraded to Stocks Starter ($29/mo). Verified via burst test (50 req in 5s with zero 429s — Starter is unlimited API calls, only restriction is 15-min-delayed real-time). The default --polygon-sleep 0.05 in scripts/backfill_learning_loop.py and the polymarket-resolver.service ExecStart was tuned accordingly. Earlier docs that cited "100 req/min" for Starter were corrected in commit 9324af5.

Finding 228 — v3 audit Finding A retracted via comprehensive simulator (DONE — no code change)

• Severity: High (strategy analysis integrity — a $300/100-trade headline finding was the #1 ranked recommendation and was about to be shipped)
• Status: done — Apr 25, 2026
• Trigger: A pre-implementation review of v3 Finding A ("Block all stop-loss execution outside RTH 09:30–16:00 ET", est. lift +$300–$400 / 100 trades) surfaced 8 unmodelled scenarios that the audit had not addressed:
• The audit's lift estimate was extrapolated from Rule B (±90 min window), not from a true simulation of Finding A.
• The audit's per-session evidence table was observational (where stops fired) not counterfactual (what would have happened if they hadn't).
• Wrong-prediction extended-hours stops fired BEFORE the EPS release — the earnings_detector "saves us" defense only applies after publication.
• Halt risk was not modelled.
• Gap-open slippage at RTH re-arm was not modelled.
• Hardware-order ambiguity: the bot's STP-LMT carries outsideRth=True per Finding #179, so a software-only gate is functionally inert.
• Long-weekend BMO trades extend the no-stop window across multiple nights.
• The 4-check bias gauntlet had not been applied to Rules D / G.
• Investigation: Built scripts/analyze_finding_a_full.py (570 lines, read-only). The simulator:
• Replays every closed trade (default population: 51 stopped trades) against Polygon 1m bars.
• Distinguishes FA_soft (software-only gate; hardware STP-LMT stays outsideRth=True; price evolution identical to A_base — INERT for P&L) from FA_hard (hardware also flipped to outsideRth=False; stop is silent in pre/post; on first RTH bar after a non-RTH segment, gap-throughs the stop are exited at the open price, otherwise stop re-arms intraday).
• Models earnings_detector cap on wrong-prediction trades (recorded exit_time is treated as a hard ceiling so the simulator can't fake-credit the rule for trades the live instant_exit:earnings_miss path catches).
• Halt detection (>30-min bar gap + >3 % price jump in extended hours).
• Cross-evaluates Rules B / D / G / F on the same fetched bars.
• Built-in 4-check bias gauntlet per rule: headline / ex-top-2 outliers / ex-repeat-tickers / by prediction-correctness / by FA_hard action / pre-EPS-trigger only.
• Result (vs simulator's A_base baseline; cancels Polygon-bar-vs-IBKR-fill reconstruction noise; n=51 stopped trades):

• Mechanism (FA_hard breakdown by action):
• rearm_gap_open — n=22 — −$416.72 (avg −$18.94 / trade): stop blocked in extended hours, RTH opened past stop → forced fill at the worse opening price. Worst trades: MDB (−$82), TTD (−$58), NKE (−$54), BABA (−$38), WDAY (−$26), NYT (−$23), RL (−$18), GTLB (−$15), HOOD (−$15), LOW (−$13), CSCO (−$12), VSCO (−$12).
• blocked_to_scheduled — n=9 — +$203.61 (avg +$22.62 / trade): stop blocked, stock recovered fully, exited at scheduled = profit. Best trades: KSS (+$66), GME (+$33), BBY (+$31), KR (+$23), JPM (+$15).
• Gap-opens outweigh recoveries 2-to-1 in both trade count and dollars.
• The audit's claim that ext-hours stops "fill at bad prices then stocks revert" assumes reversion the data does not support — for the 22 worst trades the stock continued past the stop into RTH open.
• Pre-EPS-trigger only (the audit's strongest claimed scenario): on the 30 trades where the would-be stop fired BEFORE the earnings release, FA_hard = −$92 (11 better / 14 worse). Even in this allegedly-safest subset the rule is net-negative.
• Halt-suspect cases the simulator flagged: ARES (134-min pre-mkt gap + 3.71 % jump on Feb 5) and NYT (41-min gap + 3.92 % jump on Feb 4). FA_hard would have forced the bot to hold through both halts.
• What survives the gauntlet: Only RuleB (±90 min window) survives. +$123 / 100 trades, 4 better / 1 worse / 46 unchanged → 96 % no-effect. The mechanism is sound (disable stop only inside the legitimate earnings-volatility spike window). Risk profile is very low. Recommended for ship-after-selection-rules. All other stop-loss interventions are rejected.
• Process learning (audit-method discipline — added to STRATEGY_AUDIT_v3.md §6 as lesson #6): Every counterfactual rule needs a TRUE simulator that replays the proposed rule end-to-end on the actual data. Proxies ("Rule B is close enough to Finding A") and observational tables ("stops fire in pre-market AND pre-market trades have negative P&L → therefore blocking pre-market stops would have helped") are NOT counterfactual evidence — they conflate selection with causation. The +$300–$400 audit estimate compounded both errors plus +$117 of Polygon-bar-vs-IBKR-fill simulator noise that was attributed to rule "lift."
• Decision: Drop Finding A from the queue. Drop Findings D / G as standalone stop-loss interventions. Keep RuleB but ship after the selection rules (B / C / D / E from v3) settle. Pivot the implementation queue: v3 Finding B (skip LONGs in downtrend) is the new #1.
• Files (no production code modified — pure analysis + docs):
• scripts/analyze_finding_a_full.py (new, 570 lines)
• scripts/finding_a_full.csv (output, 51 rows × 41 cols)
• docs/STRATEGY_AUDIT_v3.md — §10 retraction added, §4 ranked recommendations revised
• docs/AGENT_HANDOFF.md §11 — outstanding-work table re-ranked
• docs/STRATEGY_PLAYBOOK.md (new) — forward-looking consolidated source of truth
• scripts/strategy_performance_analysis.py — new finding_a_simulator + audit_v3_status sections
• earnings_dashboard/templates/dashboard.html — new dashboard tab sections rendering the simulator results
• Acceptance test: Re-run scripts/analyze_finding_a_full.py on prod; FA_soft Δ vs A_base should be $0.00 (sanity check that the rule-as-written is inert), FA_hard Δ vs A_base should be −$213.23 (within Polygon-fetch variance). Re-run after each new ~10 closed trades to track whether the conclusion holds; the conclusion will be re-validated as part of the trade-#150+ audit-gauntlet pass (memory §K).

Finding 286 — Dashboard cold-start race: _empty_summary missing 13 fields added by Finding #284 → / returned HTTP 503 with UndefinedError: 'roi_tc_total_pct' for ~2min after every restart; deploy_from_github.sh smoke test exited 1 (RESOLVED May 9, 2026)

• Severity: Medium (operator-facing — every dashboard restart 503'd for a ~2-minute cache-warmup window since May 6; deploy script aborted with exit 1 blocking final daemon-reload step on every routine deploy). User impact mostly invisible because dashboard self-recovered as soon as _bootstrap_cache_from_db() thread finished, but every subsequent housekeeping deploy (#274/#273/#269) would have tripped the same 503 and confused the operator.
• Status: RESOLVED May 9, 2026 LATE evening — caught during the §G post-deploy probe of Finding #270.
• Trigger: During Finding #270 deploy (May 9 ~19:25 UTC), deploy/deploy_from_github.sh's built-in dashboard health probe (sleep 3 && curl / after restart) returned HTTP 503 and exited 1. journalctl trace showed: jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'roi_tc_total_pct' File "earnings_dashboard/templates/dashboard.html", line 2391, in top-level template code <div class="value {% if trade_history.summary.roi_tc_total_pct >= 0 %}positive{% else %}negative{% endif %}" ... Dashboard returned to HTTP 200 within ~2 minutes (cache bootstrap thread completed), but the deploy was already marked failed by then.
• Root cause: earnings_dashboard/app.py:4359-4371 defines _empty_summary (used as the cold-start fallback at line 4372 when _dashboard_cache.get('trade_history') returns None). Pre-fix, this dict had 24 fields — the original pre-#284 set. Finding #284 (May 6, 2026) added 13 new TC-ROI fields to the populated summary.update({...}) block at app.py:3584-3600 (trading_capital_usd, trading_capital_source, trading_capital_net_liq_observed, period_days, alltime_days, alltime_total_pnl, alltime_n_closed, roi_tc_total_pct, roi_tc_monthly_avg_pct, roi_tc_annualized_pct, roi_tc_alltime_total_pct, roi_tc_alltime_monthly_avg_pct, roi_tc_alltime_annualized_pct) but did not mirror them into _empty_summary. Result: in the ~2-minute window after every dashboard restart (cache cold, bootstrap thread still running its DB queries), the / endpoint hit the empty fallback path and Jinja's {% if trade_history.summary.roi_tc_total_pct >= 0 %} (line 2391) blew up — the displayed value at 2392 had |default(0), but the conditional check did NOT. Same lesson as Finding #265.v2 + Finding #284 v1→v2 (the missing t. alias caught only by §G probe): every fix MUST end with a probe of the changed output — not just code-level review of the populated-path. §E same-pattern check on data-shape contracts was missed during the #284 ship.
• Memory §B 5-step verification:
• Step 1 trace: dashboard / → templates.TemplateResponse('dashboard.html', context) → Jinja env renders → trade_history resolves to {"trades": [], "summary": _empty_summary} (cache miss) → template at line 2391 reads trade_history.summary.roi_tc_total_pct → KeyError (Jinja raises UndefinedError for missing dict keys with StrictUndefined semantics) → Starlette returns 503.
• Step 2 simulate: replay May 9 19:25 UTC. systemctl restart polymarket-dashboard → FastAPI starts at T+0 → @app.on_event("startup") schedules init_cache thread (daemon=True) → bootstrap thread runs get_trade_history(period="all") (DB queries take ~1-3s) → meanwhile deploy script's smoke test fires curl / at T+3s → endpoint hits empty fallback → 503.
• Step 3 lines: app.py:4359-4400 (extended _empty_summary to mirror all 13 new fields with safe defaults — keeps the dict shape contract synchronized between cold-start and populated paths). dashboard.html:2391, 2397, 2403, 2417, 2423, 2429 (added (... | default(0)) wrapper to the 6 numeric {% if %} conditionals in the new TC ROI panel for defense-in-depth — pattern enforcement so future field additions don't break this exact way).
• Step 4 failure modes: (a) _empty_summary field name typo — verified each new key against app.py:3585-3599 letter-for-letter. (b) trading_capital_source set to string "fallback" — matches the == comparison at dashboard.html:2378 (renders fallback branch correctly). (c) trading_capital_net_liq_observed=None (sentinel "no IBKR snapshot yet") — Jinja renders as None/empty in |default(0) displays; consistent with populated-path behavior when net_liq is 0. (d) Jinja (expr | default(0)) >= 0 — verified default() filter applies to the value before the >= comparison; works because default(0) returns 0 for undefined, and 0 >= 0 is True (positive class) which matches the original behavior on the empty fallback path (was previously crashing instead of rendering positive — net behavior change: 0% in green, not 503). (e) Pre-#284 fields like total_pnl / win_rate / long_win_rate etc. — already in _empty_summary since Day 0; {% if %} checks on them never broke; intentionally not adding |default(0) to those conditionals per §C surgical-changes (don't refactor things that aren't broken).
• Step 5 acceptance: post-deploy systemctl restart polymarket-dashboard && curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:8080/ (no sleep) returns 200, not 503. journalctl -u polymarket-dashboard --since "1 minute ago" | grep -c UndefinedError returns 0. deploy_from_github.sh final smoke test exits 0 cleanly.
• Memory §E same-pattern check (data-shape contracts, May 9 LATE evening):
• Other dict shape contracts in the dashboard? _dashboard_cache.get('X') is called for: trade_history, earnings_calendar, db_stats, high_conviction, recent_trades, dependencies, positions — only trade_history has the empty-fallback dict shape problem because only it has a multi-key summary sub-dict consumed by Jinja {% if %}. The others are lists (empty list → no key access) or simpler shapes; no analogous bug.
• Other Jinja {% if trade_history.summary.X %} peers? Grep returned 14 references (lines 2331, 2339, 2346, 2356, 2378, 2391, 2397, 2403, 2417, 2423, 2429, 2494, 2502, 2518, 2526). 6 are the new TC ROI panel ones (FIXED). The remaining 8 reference pre-#284 fields (total_pnl, total_pnl_pct, win_rate, trading_capital_source — wait, this last one IS new! See note below; long_win_rate, short_win_rate, bmo_win_rate, amc_win_rate) — all of which ARE in _empty_summary post-fix, so they cannot trigger the same UndefinedError. Intentionally not adding |default(0) to them per §C.
• trading_capital_source (string == comparison at line 2378): populated value is "ibkr_net_liq_minus_pdt" or "fallback". Empty-fallback now sets "fallback", matching the existing else-branch. No defensive filter needed (Jinja handles missing keys in == differently than in >= — would still UndefinedError; but with the dict fix it's no longer missing).
• Memory §H deploy:
• Standard deploy: commit + push + bash deploy/deploy_from_github.sh. Dashboard restart triggers smoke test, which now succeeds because _empty_summary has all expected keys.
• Forward-looking observation: the bootstrap thread takes ~1-3s on warm prod (DB queries against trade_audit_log for the get_trade_history(period="all") summary). The deploy script's sleep 3 && curl / smoke test threads the needle — sometimes bootstrap finishes before the curl, sometimes after. Pre-#286, the latter case 503'd; post-#286, both cases return 200. Could optionally extend the smoke test's sleep to 10-15s so it only ever sees the warm path, but that's belt-and-suspenders given the dict fix is the structurally correct one. Not done in this commit per §C.
• Lesson logged for the agent (reinforcement of #265.v2 + #284 v1→v2):
• §E "data-shape contracts" rule (NEW, May 9 LATE evening): when ADDING fields to a dict that has multiple producer paths (populated path + empty/fallback path + cached path), enumerate every producer and update them in lockstep. This is structurally identical to the cross-interface §M rule for volume gates (Finding #225) — same lesson, different domain. The "single shared definition" should ideally be a @dataclass or a _make_empty_summary() helper that's called from both paths so the shape contract is enforced by the typechecker, not by hand. Not refactoring to that today (§C surgical), but flagged for a future cleanup pass.
• §G "probe the changed output" rule keeps catching this exact failure mode: Finding #265 → #265.v2 (incomplete fix on except-fallback caught by re-audit), Finding #284 → #284 v2 (missing t. alias caught by re-probe), Finding #286 (today — caught by deploy-script smoke test, which IS a §G probe baked into infrastructure). The deploy-script smoke test introduced as part of Finding #262 is doing exactly what it was designed to do; the dict-shape bug would otherwise have been silent for who-knows-how-long.
• Files:
• earnings_dashboard/app.py (extend _empty_summary with 13 new fields)
• earnings_dashboard/templates/dashboard.html (add |default(0) to 6 TC ROI {% if %} conditionals — defense-in-depth)
• docs/FINDINGS.md (this entry)
• docs/AGENT_HANDOFF.md §11 (May 9 LATE evening housekeeping block extended)
• docs/CONTEXT.md (What changed this week — May 9 LATE evening line)

Finding 285 — Rare-day exposure cap (60% on days with 1-2 opportunities) silently rejected BOTH valid signals when one would fit; rejection wasn't logged anywhere (Phase A: greedy-fill + opportunity_log emission RESOLVED May 9, 2026; Phase B rule re-evaluation DEFERRED to Phase 3 strategy-revamp May 26)

• Severity: CRITICAL (real trade-rejection bug — silent, unaudited; rule applies to majority of trading days; today missed 2 valid signals)
• Status: Phase A (greedy-fill + logging) done — May 9, 2026 ~22:30 UTC. Phase B (rule re-evaluation) DEFERRED-POST-PHASE-3 alongside Findings #278 / #279 / #283 (sizing + priority + depth-gate v2 review).
• Trigger: Operator pushback May 7, 2026 — "why we didn't open positions just now on mchp and TTD?" Investigation showed both passed all selection gates (conviction, volume, depth, priority), generated valid signals, then got rejected at the orchestrator's exposure-cap check at 14:30:14 UTC with ❌ TTD: Would exceed 60% exposure limit (63.0%) + ❌ MCHP: Would exceed 60% exposure limit (63.0%). Operator: "this is wrong, we are missing opportunities for no reason! the logic seems to be flawed when we have less opportunities we don't take trades!"
• Investigation surfaced THREE distinct problems:
• Implementation bug (greedy-fill missing): the rule says "max 60% exposure", but the loop pre-aggregated intended exposure and rejected ALL signals whose combined value exceeded the cap. APP carryover (18%) + TTD (22%) = 40% < 60% ✓ — TTD alone fit. Pre-fix logic rejected both → ZERO entries. Greedy-fill fix lets TTD enter (top priority), then rejects MCHP (40% + 22% = 62% > 60%). Net: ONE trade instead of zero.
• Visibility bug: searched polymarket_opportunity_log for any skip with exposure / limit / capital in skip_reason — 0 rows ever recorded. The rule has been firing silently for the lifetime of the bot. Phase A fix wires can_open_position rejections through _emit_pm_opportunity_log with skip_stage='exposure_cap'.
• Rule design question: in the last 90 trading days, 22 of 34 days were "rare" (1-2 entries) — i.e. the rule's stricter cap applies to the MAJORITY of trading days. Operator's framing: "most days become rare days... our bottleneck is not no of markets is no of markets with volume above 2k usd that qualify as per our rules". The rule's premise (lower diversification → higher variance → cap exposure) doesn't hold when the small opportunity count is structural (most earnings markets fail our $2k volume gate), not a function of low overall earnings activity. Phase B: re-evaluate whether the rare-day cap should exist at all — bundled with the priority/sizing v2 deferral (#279) for the May 26 Phase 3 strategy-analysis revamp.
• Phase A fix shipped (UNAMBIGUOUS — no rule change, just correctness + visibility):
• src/earnings/risk_manager.py:can_open_position — new optional kwarg extra_pending_exposure: float = 0.0. Default preserves all existing call-site semantics (capital_manager, paper_trade_test, fixture validation, etc — n=8 callers, all backward-compatible). When >0, the exposure check uses current_exposure + extra_pending_exposure + new_position so batch loops can run greedy-fill.
• src/earnings/trading_orchestrator.py:check_bmo_signals + check_amc_signals — both loops now maintain a pending_exposure = 0.0 running total. Each iteration calls can_open_position(signal, extra_pending_exposure=pending_exposure); on accept, pending_exposure += position_size so the NEXT iteration sees the running total. Top-priority signal goes first → next signals see growing pending exposure.
• src/earnings/trading_orchestrator.py:_log_exposure_skip — new helper that calls the signal_generator's existing _emit_pm_opportunity_log with skip_stage='exposure_cap'. Reused for BMO loop, AMC loop, AND BMO-reservation rejections (capital constraint = same class of skip). Fail-soft (try/except → debug log, never raises) so exposure gating works even if log path is broken.
• Phase B deferred to post-Phase-3 (May 26 strategy-revamp Phase 3 task):
• Bundled with sizing-priority-v2 (Finding #279) since the exposure-cap rule and the priority-tier rule are interconnected (both gate which valid-signal trades actually enter).
• Operator's reasoning to encode in the Phase 3 work: "most days become rare days as our bottleneck is not no of markets is no of markets with volume above 2k usd that qualify as per our rules". The rule's premise needs revisiting against Polymarket's actual liquidity distribution.
• With Phase A logging in place, by May 26 we should have ~1-3 weeks of skip_stage='exposure_cap' rows in polymarket_opportunity_log — providing the n ≥ 12 evidence base memory.md §K's bias gauntlet requires for any rule change.
• Specific Phase B questions to answer: (a) Is the rare-day cap helpful at all, or pure opportunity cost? (b) Should "carryover" positions scheduled to exit within the next ~6 hours (e.g., today's APP exiting at 15:57 ET) be excluded from the exposure calc? (c) Should the rare-day cap scale with carryover? (d) Should the cap depend on conviction/priority of the queued signals?
• §B 5-step verification (Phase A):
• Step 1 trace: AMC entry tick → check_amc_signals() → for each signal in priority order → can_open_position(signal, extra_pending_exposure=pending_exposure) → if accept, pending_exposure += position_size; if reject, _log_exposure_skip(signal, reason) → signal_generator._emit_pm_opportunity_log(...) writes to DB. Same flow for BMO loop.
• Step 2 simulate: replay today's TTD + MCHP. Pre-fix: TTD checked vs APP-only state ($460 + $552 = $1012 = 40.4% < 60% ✓ — but log shows both at 63%, suggesting current_exposure pre-aggregation we didn't fully trace). Post-fix: explicit greedy-fill loop. TTD first (pending=0, sees APP only) → 18% + 22% = 40% < 60% ✓ → enters → pending=$552. MCHP next (pending=$552, sees APP + TTD-reserved) → 18% + 22% + 22% = 62% > 60% ✗ → rejects → log to opportunity_log.
• Step 3 lines: risk_manager.py lines 749-815 (signature + check); trading_orchestrator.py lines 3045-3097 (_log_exposure_skip), 3257-3286 (BMO loop), 3360-3445 (AMC loop, all 3 accept paths + reject paths).
• Step 4 failure modes: (a) extra_pending_exposure < 0 (defensive: clamped to 0 via max(..., 0.0)); (b) _emit_pm_opportunity_log not present on a mock signal_generator (helper checks hasattr first); (c) DB write fails (try/except → debug log, gating continues); (d) signal missing condition_id (defaults to empty string — opportunity_log accepts NULL); (e) signal.market_cap is enum None (defensive getattr + mcap_enum is not None check). All handled.
• Step 5 acceptance: post-deploy first BMO/AMC tick should: (a) emit at most ONE rejection per "competing signal" (top priority enters, lower priority rejected) instead of ALL rejected; (b) every rejection generates a polymarket_opportunity_log row with skip_stage='exposure_cap' and skip_reason=.... Cannot test on TTD/MCHP today (window passed); next test fires on next BMO/AMC tick with rare-day classification.
• §E same-pattern check (re-audited May 9 evening before declaring done): rg "can_open_position\(" returned 8 callers across 4 files. Compliance audit broke them down by semantic:
• Batch-loop callers (greedy-fill needed) — 4 sites:
  • trading_orchestrator.py check_bmo_signals (line ~3309) ✓ patched
  • trading_orchestrator.py check_amc_signals (line ~3433) ✓ patched
  • capital_manager.py check_bmo_signals (line ~289) ✓ patched (same pattern; module is currently dead-code per Phase 4A extraction but kept in lockstep so it can never be a re-activation footgun)
  • capital_manager.py check_amc_signals (line ~360) ✓ patched
• Independent single-signal validators (greedy-fill not needed) — 4 sites:
  • trading_orchestrator.py _calculate_multi_day_bmo_reservation (line ~3147) — by design, evaluates each signal independently against current state to compute MAX possible future-BMO capacity; greedy-fill would systematically under-estimate. Default extra_pending_exposure=0.0 is the correct semantic. ✓ unchanged
  • trading_orchestrator.py execute_entry final risk check (line ~3677) — single-signal, last-look gate at order time. ✓ unchanged
  • capital_manager.py two equivalents (lines ~198, 462) — same single-signal semantic. ✓ unchanged
  • Tests + archived paper_trade_test — single-signal validators. ✓ unchanged
• Unit tests added in tests/unit/test_risk_manager.py:TestPhase285GreedyFillExposureMath — 9 new tests covering: top-priority sees only registered exposure ✓; second signal sees first's pending ✓; today's TTD/MCHP scenario recovery ✓; defensive None / negative pending → clamped to 0 ✓; zero capital returns max-pct ✓; normal-day 140% cap allows 4 signals ✓; rare-day 60% cap blocks 3rd ✓; today's exact APP-carryover scenario enters 1 of 2 ✓. All 41 unit tests passing.
• §F project-specific: does NOT touch trading_positions / trade_audit_log writes. Does NOT change order-execution path. Does NOT change the rare-day cap rule itself (Phase B). Pure correctness + visibility shipment.
• §M cross-interface: the same _emit_pm_opportunity_log helper is used by the trader's signal generation path (signal_generator) AND the orchestrator's exposure path (new _log_exposure_skip). One serialization, one schema, one DB destination. The new skip_stage='exposure_cap' value is documented in docs/DATA_ARCHITECTURE.md under polymarket_opportunity_log.
• §G post-fix verification: live probe + monitoring on next BMO/AMC tick. (TTD/MCHP today specifically: window already passed when the operator pushed back ~3 hours after entry slot; cannot recover.)
• Files (Phase A ship-now):
• src/earnings/risk_manager.py (can_open_position signature + computation).
• src/earnings/trading_orchestrator.py (_log_exposure_skip helper + BMO loop + AMC loop greedy-fill + 4 reject-paths now log to opportunity_log).
• src/earnings/capital_manager.py (Phase 4A extracted module — same greedy-fill patches in check_bmo_signals + check_amc_signals to keep it in lockstep with the orchestrator's canonical implementation per memory §E).
• tests/unit/test_risk_manager.py (9 new tests covering greedy-fill math across the failure-mode matrix; replays today's TTD/MCHP exact scenario).
• docs/FINDINGS.md (this entry — canonical per memory §J).
• docs/STRATEGY_PLAYBOOK.md §7 (Phase B deferred-post-Phase-3 row, bundled with #279).
• docs/AGENT_HANDOFF.md §11 (DEFER queue bullet referencing #285 Phase B).
• scripts/strategy_performance_analysis.py:build_action_queue() (sizing-priority-v2 row updated to call out exposure-cap re-evaluation).
• Acceptance test (any time after next BMO/AMC tick fires): psql ... -c "SELECT symbol, evaluated_at::date, skip_reason FROM polymarket_opportunity_log WHERE skip_stage = 'exposure_cap' ORDER BY evaluated_at DESC LIMIT 10" should return rows on rare-day cap fires. journalctl -u earnings-trader | grep "Would exceed" should show greedy-fill behavior — at most one ❌ per overflow batch, with the top-priority signal showing ✅.

Finding 284 — P&L Summary needed tooltips + a Trading Capital ROI panel (period + all-time, total / monthly / annualized) so the operator can see annualized-on-actual-trading-capital, not just per-dollar-deployed (RESOLVED May 6, 2026)

• Severity: Low (operator UX — dashboard didn't expose a clear "what's my actual trading capital making per year" view)
• Status: done — May 6, 2026 ~23:30 UTC.
• Trigger: Operator deep-dive Q on the P&L Summary (May 6 ~21:30 UTC): "what is the yearly ROI? is it the ROI on trading capital extrapolated for the rest of the year based on current gain? what is the profit factor?" — followed by a request to add tooltips on each existing tile AND a new "Trading Capital ROI" panel showing past monthly + annualized projection.
• Operator design choices (3-question form May 6 23:27 UTC):
• Trading capital value: option (c) — computed dynamically as net_liquidation − $25,000 PDT reserve, falling back to $2,500 when IBKR snapshot is unavailable. Auto-tracks account growth.
• Period scope: option (c) — show BOTH the time-filter-aware period view AND a separate stable all-time view.
• Annualized formula: option (b) — compound CAGR (1 + return)^(365/days) − 1, not simple linear monthly × 12.
• Implementation (smallest correct surface):
• Backend (earnings_dashboard/app.py:get_trade_history): added trading-capital read from get_cached_ibkr_snapshot() with PDT_RESERVE_USD=25_000, TRADING_CAPITAL_FALLBACK=2_500, TRADING_CAPITAL_FLOOR=100. New helper _compute_roi_triplet(pnl, tc, days) -> (total_pct, monthly_avg_pct, annualized_pct) reused for both period + all-time. All-time computed via a separate small MIN(entry_time) + COUNT + SUM query (respects TRADE_METRICS_FILTER_SQL_T so corrupt rows are excluded the same way the rest of the metrics do). 12 new fields surfaced on summary: trading_capital_usd, trading_capital_source ("ibkr_net_liq_minus_pdt" or "fallback"), trading_capital_net_liq_observed, period_days, alltime_days, alltime_total_pnl, alltime_n_closed, roi_tc_total_pct, roi_tc_monthly_avg_pct, roi_tc_annualized_pct, roi_tc_alltime_total_pct, roi_tc_alltime_monthly_avg_pct, roi_tc_alltime_annualized_pct. Defensive: annualization base floored at 1e-9 to avoid negative-base fractional-exponent errors on extreme drawdowns.
• Frontend (earnings_dashboard/templates/dashboard.html): tooltips added to ALL six existing P&L Summary tiles (Total P&L, Total ROI, Total Trades, Win Rate, Profit Factor, Avg Position) explaining formula + caveat (Win Rate tooltip cross-refs memory.md §K 5-outlier rule; Profit Factor tooltip has the industry-standard buckets 1.0/1.2/1.5/2.0/3.0). New "🎯 Trading Capital ROI" panel below the 6 tiles with two side-by-side columns (PERIOD + ALL-TIME), each showing 3 numbers (Total / /mo avg / annualized) on bg-secondary cards. Capital source badge displayed inline ("dynamic" green or "fallback" yellow). Footer line explains the compound annualization formula, caveats past-performance-doesn't-guarantee-future-results, references the 5-outlier dependency from STRATEGY_PLAYBOOK.md §1, and links to /docs/STRATEGY_PLAYBOOK#1-current-verdict.
• JS update (filterTradeHistory): new helper _setPct(id, pct) and _setText(id, text) consolidate the 6+3 ID writes; the time-filter buttons (All Time / Year / Month / Week / Today) now also refresh the period-column TC ROI numbers. All-time numbers recompute server-side and re-flow through the same JSON.
• §B 5-step verification:
• Step 1 trace: page-load → Jinja renders all 6 tiles + new TC ROI panel directly from trade_history.summary. Time-filter click → filterTradeHistory(period) → fetch /api/trade-history?period=X → api_trade_history → get_trade_history(period) → builds summary dict (now with 12 extra TC fields) → JS read + _setPct/_setText rewrite both columns + capital-source badge.
• Step 2 simulate: $158.74 P&L over a 365-day Year filter, trading_capital = max(27484-25000, 100) = $2,484. Period total = 158.74/2484 × 100 = +6.39%. Period months = 365/30.4375 = 11.99 → monthly avg = 6.39/11.99 = +0.53%. Period compound annualized = (1 + 0.0639)^(365/365) − 1 = +6.39% (matches total when period_days exactly 365). All-time may differ if first trade < 365 days ago — earlier trades, denser P&L over fewer days → higher monthly + much higher annualized.
• Step 3 lines: app.py:get_trade_history ~lines 3492-3585 (new _compute_roi_triplet + 12 summary fields); dashboard.html Jinja ~lines 2336-2440 (tooltips + new panel); dashboard.html JS ~lines 4395-4435 (_setPct/_setText + 9 ID writes).
• Step 4 failure modes: (a) IBKR snapshot unavailable / net_liq missing → falls back to $2,500 with tc_source="fallback" — UI shows yellow "fallback" badge; (b) period_days = 0 (e.g., today before any trade) → guarded max(... , 1); (c) total_pnl/tc <= -1 (account wipeout, near-impossible) → annualization base floored at 1e-9 so we render a finite (very negative) number instead of crashing; (d) all-time query fails → keeps alltime_* at zeros + logs warning, doesn't crash the response; (e) operator picks "Today" filter → period_days=1 → compound annualized = (1+r)^365 — extremely volatile on tiny windows. Mitigated by tooltip text explicitly warning "very volatile on short windows — read it as 'noisy' under those filters."
• Step 5 acceptance: post-deploy curl /api/trade-history?period=year | jq '.data.summary | {tc: .trading_capital_usd, src: .trading_capital_source, net_liq: .trading_capital_net_liq_observed, period: .roi_tc_total_pct, period_mo: .roi_tc_monthly_avg_pct, period_ann: .roi_tc_annualized_pct, all_total: .roi_tc_alltime_total_pct, all_mo: .roi_tc_alltime_monthly_avg_pct, all_ann: .roi_tc_alltime_annualized_pct}' returns all 9 numbers populated and consistent (e.g., period_ann ≈ period_total when period_days = 365); HTML render shows the new panel with both columns and the dynamic capital badge.
• §E same-pattern check: searched for other consumers of the summary dict — only api_trade_history returns it as {"data": history, ...}, only the trade-history Jinja block + filterTradeHistory JS render it. No other surface needs updating.
• §F project-specific: does NOT touch trading_positions / trade_audit_log writes. Does NOT change order-execution path or any gate. Pure dashboard-side enrichment + UX.
• §M cross-interface: Jinja initial render and JS refresh produce the SAME field-set (12 new + 6 existing) via _setPct/_setText helpers and field IDs. Single source of truth (get_trade_history); single render mapping.
• §G post-fix verification (lesson from Finding #265.v2): live probe documented in chat — curl /api/trade-history returns all 12 new TC fields; visual on dashboard shows the new panel rendered with PERIOD + ALL-TIME columns + dynamic badge.
• Files:
• earnings_dashboard/app.py (helper + 12 summary fields + all-time query).
• earnings_dashboard/templates/dashboard.html (tooltips on 6 existing tiles + new TC ROI panel + JS refresh helper).
• docs/FINDINGS.md (this entry).
• Acceptance test (any time): curl -s 'http://127.0.0.1:8080/api/trade-history?period=year' | jq '.data.summary | with_entries(select(.key | test("trading_capital|roi_tc|period_days|alltime")))' returns 12 fields, all numeric (with trading_capital_source as a string).

Finding 283 — Polymarket "depth quality" gate (badge "LOW OB") is functionally a SPREAD gate; 37/38 historical kills were spread-dominated; UX rename + post-Phase-3 deferral (RESOLVED May 31, 2026 via Finding #319 — KEEP gate)

• Severity: Medium UX (badge name lied) + opportunity-cost question (n=3 real-signal kills, 1 known +$158 missed counterfactual)
• Status: RESOLVED May 31, 2026 — KEEP gate (see Finding #319). UX rename done Apr 30. PR 5 widened reconstruction (n=22 binding kills vs the n=3-4 originally available, replaying the real conviction code over the full historical universe) showed the gate is a roughly break-even low-information-signal filter (blocked 13 losers / −$142.56, killed 9 winners / +$165.09, net removal +$22.54 over ~7 months = noise) — and is mostly a snapshot-reliability filter (18/22 kills: hollow book → sparse/unstable price-snapshot conviction), not the pure spread gate this entry's title implied. Removing/carving-out adds variance for ~$0 EV → operator-confirmed KEEP. Original deferral context (UX rename + n=3 below gauntlet floor) preserved below.
• Trigger: Operator screenshot of MRNA #2026-05-01 BMO with LOW OB badge despite the dashboard's effective volume showing $4,210 (post Finding #282 fix). Operator: "now MRNA is showing as low OB... are we trading that based on current rules or not? what is low OB? is it important?"
• Investigation (deep dive in chat): ran 4-part probe on prod —
• MRNA right now: latest orderbook snapshot has best_bid=$0.0100 / best_ask=$0.9900 (98¢ wide spread!), bid_depth=$1,163, ask_depth=$3,759 (depth is fine — 47× the $100 floor). The _calculate_depth_quality score collapses to 0.30 because of the spread penalty (0.20/0.98 = 0.30), NOT because of any actual depth issue. Trader's is_reliable flag is False → hard skip at signal_generator.py:1568. Conviction (18.5% SHORT) comes from last_trade_price / price_snapshots — independent of the bid/ask, so the signal IS real even though the visible orderbook is hollow.
• Sub-cause analysis on last 50 production depth-skips: 37 spread-dominant, 1 depth-dominant, 0 asymmetry. The gate's name is a misnomer — it's effectively a spread gate.
• Conviction profile of the 38 historical depth-skips: 35 had conviction = 0.500 (the calculator's default when no real signal exists — these would have been skipped at the conviction gate anyway, so the depth gate is redundant for 92% of its kills); only 3 had a tradeable conviction (0.70-0.90 LONG): ICE 2026-04-28 conv=89.9% (counterfactual P&L = +$158.22 — real missed winner), MPC 2026-04-29 conv=74.5% twice (counterfactual $0.00 = wash).
• Statistical significance: n=3 real-signal kills is well below memory.md §K's 4-check bias-gauntlet floor (n≥12, ideally ≥20). Cannot ship a gate-tuning rule change based on this evidence alone.
• Fix shipped now (UX rename, no behaviour change):
• earnings_dashboard/templates/dashboard.html Jinja badge (line ~1284) + JS render (line ~3855): LOW OB → WIDE SPREAD. Both surfaces now also have a title tooltip explaining the gate logic + cross-reference to this finding.
• src/earnings/signal_generator.py log messages (lines ~1570 + ~2008, BMO + AMC paths): "Skipping - poor orderbook quality" → "Skipping - PM orderbook unreliable" with reason field defaulting to "wide_spread_or_low_depth" (was "unspecified"). Comment block above each cite Finding #283.
• Gate logic itself is UNCHANGED. is_reliable = depth_quality >= 0.5 still skips the same 38 historical and any future markets with the same profile. No trader behavior change.
• Deferred to post-Phase-3 (Finding #283 deferral — mirrors Findings #278 / #279 deferral pattern):
• Decompose the gate into named sub-gates so the badge is honest about what fired:
  • min_depth_gate (real depth check, n=1 historical kill)
  • spread_gate (the actual functional gate, n=37 historical kills)
  • Each tunable independently.
• Re-evaluate threshold (0.50). Lower → catches more real signals (e.g. ICE Apr 28 +$158) but adds variance. Higher → stricter. Need counterfactual P&L on a wider universe than the 2 currently computed.
• Skip the gate entirely when conviction comes from last_trade_price or price_snapshots (signals from real trades — bid/ask spread quality is irrelevant to signal quality). Would unblock MRNA-style markets with active trade history but hollow inside quotes.
• §B 5-step verification (UX rename only — gate-tuning follows the deferred path):
• Step 1 trace: Jinja path → templates/dashboard.html line ~1284 emits badge in the upcoming-earnings table; JS path → renderUpcomingEarningsRows at line ~3855 emits the same badge during the 30s polling refresh. Both produce identical text per memory §M.
• Step 2 simulate: MRNA today — (conv_short_ok && is_reliable && !depth_ok) → True → renders WIDE SPREAD badge with tooltip. NO change to tradeBadge selection logic — same conditions, same skip.
• Step 3 lines: dashboard.html ~1284 (Jinja), ~3855 (JS); signal_generator.py ~1570 (BMO log), ~2008 (AMC log).
• Step 4 failure modes: (a) future depth-dominant kill (1 historical case) would still render WIDE SPREAD — Phase-3 decomposition addresses; (b) tooltip text was added — no breakage if title is unrendered (older browsers) — operator just doesn't see the explanation; (c) both Jinja + JS render the same text per memory §M cross-interface.
• Step 5 acceptance: post-deploy curl /api/earnings-calendar | jq '.upcoming[] | select(.symbol=="MRNA")' row should still surface skip indicators; visual on dashboard shows WIDE SPREAD not LOW OB. Hovering shows the explanation tooltip. journalctl -u earnings-trader | grep "PM orderbook unreliable" shows the new log text on the next signal evaluation pass. Action Queue includes new depth-gate-v2 row with DEFERRED-POST-PHASE-3 status.
• §E same-pattern check: rg "LOW OB|poor orderbook" returns ONLY the 4 lines fixed (2 dashboard + 2 trader). No other surface uses these strings.
• §F project-specific: does NOT touch trading_positions / trade_audit_log writes. Does NOT change order-execution path. Does NOT change the is_reliable semantics or the 0.50 threshold — pure UX rename + log-text consistency.
• §M cross-interface (mandatory for gates/displayed metrics): ✓ grep done; ✓ both Jinja + JS render the same text; ✓ tooltip explains the gate logic uniformly; ✓ canonical narrative this entry; ✓ trader log message + dashboard badge both reference Finding #283.
• §G post-fix verification (lesson from Finding #265.v2): live probe documented in chat — verified MRNA badge changed + log-message change validated on next signal pass.
• Files (UX rename ship-now):
• earnings_dashboard/templates/dashboard.html (Jinja badge + JS badge — both LOW OB → WIDE SPREAD with explanatory title tooltip).
• src/earnings/signal_generator.py (BMO + AMC log messages updated for accuracy — was "poor orderbook quality", now "PM orderbook unreliable" with wide_spread_or_low_depth default reason).
• docs/FINDINGS.md (this entry — canonical per memory §J).
• docs/STRATEGY_PLAYBOOK.md §7 (deferred-post-Phase-3 entry mirroring Findings #278 + #279 pattern).
• docs/AGENT_HANDOFF.md §11 (DEFER queue bullet).
• scripts/strategy_performance_analysis.py:build_action_queue() (new depth-gate-v2 row with status=DEFERRED-POST-PHASE-3).
• Acceptance test (any time): curl -s 'http://127.0.0.1:8080/' | grep -c "LOW OB" should return 0; curl -s 'http://127.0.0.1:8080/' | grep -c "WIDE SPREAD" should return ≥2 (Jinja initial + JS renderer fallback markup). Strategy Action Queue API: curl -s http://127.0.0.1:8080/api/strategy-analysis | jq '.analysis.action_queue.summary.deferred_post_phase_3' should return ≥3 (was 2 — adds depth-gate-v2 alongside regime-detection-v2 + sizing-priority-v2).

Finding 282.v2 — Same-day post-deploy live probe caught dashboard ≠ trader misalignment ($2,224 vs $4,210 for MRNA) because Gamma batch chronically drops some markets and the except-fallback used cached_volume_24h (rolling, not cumulative); now both surfaces share the SAME single-shot Gamma fetcher (RESOLVED Apr 30, 2026)

• Severity: CRITICAL (operator's explicit requirement: "if dashboard shows volume >2000 usd trader uses the same info for decision")
• Status: done — Apr 30, 2026 ~19:30 UTC.
• Trigger: Per memory §G post-deploy live probe of #282 fix — dashboard MRNA showed $2,224.94 while trader's get_market_volume('0xc0a568...') returned 4210.34. All 8 May 1 BMO markets misaligned by 7%-50%. The MRNA $2k gate WAS now passing on the trader (✓ — primary #282 fix worked) but dashboard still showed LOW VOL because the dashboard's number was wrong.
• Root cause (compound — both observed in one probe):
• Dashboard's batch Gamma fetch chronically drops these 8 markets. Logs: WARNING:earnings_dashboard.app:Gamma API: 8/111 markets still missing after individual fetch. My direct probe to Gamma succeeded for all 8 in a single batch call, but the dashboard's fetch_gamma_markets_batch (Phase 1 chunk-of-25 + Phase 2 individual retry) consistently fails for them. Likely a transient Polymarket Gamma-side issue (Finding #15 batch-drop pattern). Result: all 8 fall into the except Exception fallback path.
• Except-fallback used cached_volume_24h as the gamma arg, which is the rolling 24h field from price_snapshots, NOT cumulative. So effective_cumulative_volume_usd(db_total, cached_volume_24h) returned a max between two-different-semantic numbers — even with the #282 max() fix, the dashboard's number was a 24h figure while the trader's was true cumulative.
• Fix (memory §M consolidation — single shared fetcher across both surfaces):
• New shared helper volume_policy.fetch_gamma_cumulative_single(condition_id, *, timeout_sec=5.0) -> Optional[float] performs a single-shot GET /markets?condition_ids=X and extracts the volume (cumulative USD) field. Returns None on any error / missing field / zero volume so callers can fall back. ONE implementation, identical results for both surfaces.
• Trader path — signal_generator._fetch_gamma_cumulative_volume now delegates to the new shared helper (was inline duplicate of the same logic). Trader's existing _gamma_cumulative_cached TTL wrapper is unchanged.
• Dashboard except-fallback — before falling back to cached snapshots, ALWAYS tries the same fetch_gamma_cumulative_single. If it returns a positive value, uses it as the "gamma" arg to effective_cumulative_volume_usd AND flips is_reliable=True + data_source="gamma_singleshot" so the operator can see this row was rescued from the cached fallback. Only when the single-shot ALSO fails does it fall through to the historical cached-volume_24h behavior (preserved as a defensive lower bound, never inflates above true cumulative since 24h ≤ cumulative).
• §B 5-step verification (this v2 — the v1 entry below covers the original):
• Step 1 trace: dashboard refresh → get_polymarket_earnings_from_db → batch Gamma fetch fails for cid → except runs → NEW: fetch_gamma_cumulative_single(cid) → if returns gamma_volume, use that with effective_cumulative_volume_usd(db_total, gamma_volume) (max). Trader path unchanged externally — get_market_volume calls _gamma_cumulative_cached which calls _fetch_gamma_cumulative_volume which now delegates to the SAME fetch_gamma_cumulative_single. Same call, same result.
• Step 2 simulate: MRNA cid passed to fetch_gamma_cumulative_single → returns 4210.34 (current live cumulative). Dashboard except-fallback: effective_cumulative_volume_usd(6.28, 4210.34) = 4210.34 → volume=4210.34, volume_ok=True. Trader: same helper, same args, same result. Aligned to the cent.
• Step 3 lines: volume_policy.py lines 95-135 (new fetch_gamma_cumulative_single); signal_generator.py lines 346-353 (delegate); app.py lines 35-40 (import) + 2188-2245 (rewritten except-fallback to call helper before falling to cached_volume_24h).
• Step 4 failure modes: (a) fetch_gamma_cumulative_single raises any exception → returns None (never propagates); (b) Gamma returns non-200 → None; (c) Gamma returns empty list → None; (d) Gamma returns market without volume field → None; (e) Gamma returns zero volume → None (treated as "no live data"); (f) all of the above → except-fallback uses cached_volume_24h (preserved historical behavior, is_reliable=False, data_source="price_snapshot").
• Step 5 acceptance: re-run alignment probe → ALL 8 May 1 BMO markets show dash.volume == trader.vt to the cent; MRNA volume_ok=True.
• §E same-pattern check: searched for other Gamma-cumulative call sites — only the trader's _fetch_gamma_cumulative_volume (now delegated) and the dashboard's fetch_gamma_markets_batch (kept — different semantic: bulk fetch of many at once, fast happy path). No other consumers.
• §G post-fix verification: alignment probe ran post-deploy — see chat for table showing all 8 markets identical between dashboard and trader.
• Tests: 6 new tests in tests/unit/test_volume_policy.py covering the single-shot helper: network-failure, non-200, empty response, success, zero-volume → None, missing field. All 20 tests passing.
• Files (v2 follow-up — additive to v1 below):
• src/earnings/volume_policy.py (new fetch_gamma_cumulative_single helper).
• src/earnings/signal_generator.py (_fetch_gamma_cumulative_volume now delegates).
• earnings_dashboard/app.py (import + except-fallback now tries single-shot before cached snapshots).
• tests/unit/test_volume_policy.py (6 new tests for the helper).

Finding 282 — effective_cumulative_volume_usd returned stale DB value when live Gamma cumulative was much higher; trader silently rejected MRNA #2026-05-01 at $6.28 < $2k while live Gamma was $3,734 (RESOLVED Apr 30, 2026)

• Severity: CRITICAL (real trade-rejection bug — both trader gate AND dashboard display silently dropped tradeable opportunities; not just a UI staleness issue)
• Status: done — Apr 30, 2026 ~19:00 UTC.
• Trigger: Operator screenshot of May 1 BMO Upcoming Earnings table showed all 8 markets with implausibly low volumes ($6-$25). Investigation hit Polymarket Gamma API directly for each condition_id and found a 10×-500× gap between displayed and live cumulative volume: Ticker Dashboard DB.volume_total Gamma.volume Gamma.volume24hr ARES $207 $0 $207.42 $206.08 ← only one matching (DB=0 → Gamma fallback fired) CBOE $18 $18.12 $123.77 $16.22 CL $25 $24.73 $78.15 $6.02 CVX $8 $7.63 $244.49 $188.21 EL $6 $6.32 $153.27 $146.96 MRNA $6 $6.28 $3,734.76 $2,234.94 ← gated out at 6.28 < $2k despite 3.7k live cumulative PIPR $10 $10.00 $261.62 $168.88 XOM $8 $7.53 $186.53 $18.60
• Root cause: the Apr 16, 2026 design of effective_cumulative_volume_usd(db, gamma) (Finding #224) was "if DB has any positive value, use it; otherwise fall back to Gamma" — based on the assumption that "DB is authoritative when synced". In practice, continuous_collector snaps markets.volume_total only every 15 minutes while Polymarket trades land continuously, so as soon as ANY positive value made it into the DB, every subsequent trade was invisible to both the dashboard display and the trader's gate until the next sync. Cumulative volume is monotonically non-decreasing so max(db, gamma) is the strictly correct semantic. Compounding: the trader had its OWN short-circuit at signal_generator.get_market_volume:441 (if vt_db > 0: return vt_db) that bypassed the helper entirely — so even fixing the helper alone wouldn't have helped the trader path. Both paths arrived at the same wrong number for different code reasons.
• Trader-correctness impact: for MRNA #2026-05-01 SHORT signal (conviction 18.5%, Mid-cap, BMO), pre-fix the trader would have skipped the trade because volume_threshold_passes(2000, 6.28, ...) returned False. Post-fix it returns True (max(6.28, 3734.76) = 3734.76 ≥ 2000) — the SHORT signal becomes tradeable assuming its other gates (depth quality, conviction, regime) pass. Same dynamic for any earnings market whose DB row was synced before recent Polymarket activity.
• Fix shipped (smallest correct surface):
• src/earnings/volume_policy.py:effective_cumulative_volume_usd — change semantics from "DB if positive else Gamma" to max(db, gamma) with defensive _safe_nonneg_float coercion (handles None, NaN, Inf, negative, string garbage — all → 0). volume_threshold_passes is unchanged (calls the fixed helper).
• src/earnings/signal_generator.py:get_market_volume — REMOVED the short-circuit if vt_db > 0: return vt_db. Now ALWAYS goes through the shared effective_cumulative_volume_usd helper. This is the §M consolidation: trader and dashboard share one definition with zero wrapper logic divergence.
• tests/unit/test_volume_policy.py — flipped the broken test_effective_prefers_positive_db_over_gamma assertion (which encoded the bug) to the new max() semantic, plus added 8 new defensive tests covering the regression case (stale-DB-low + Gamma-high → max), both-zero, both-None, negative inputs, NaN/Inf inputs, string garbage. All 14 tests passing locally pre-deploy.
• §B 5-step verification:
• Step 1 trace: trader path → signal_generator._build_trading_signal → get_market_volume(condition_id) → effective_cumulative_volume_usd(vt_db_raw, gamma_v) → returns max. Dashboard path → _refresh_cache() → get_polymarket_earnings_from_db() → effective_cumulative_volume_usd(db_volume_total, gamma_volume) → returns max. Same call, same args, same result. Single shared module.
• Step 2 simulate: MRNA condition_id passed through get_market_volume. Pre-fix: row.volume_total=6.28 → if vt_db > 0: return 6.28 → trader sees vt=6.28 → signal.volume_total=6.28 → _build_trading_signal rejects below 2000 threshold. Post-fix: vt_db_raw=6.28, gamma_v=3734.76, effective_cumulative_volume_usd(6.28, 3734.76) = 3734.76 → trader sees vt=3734.76 → 3734.76 ≥ 2000 → signal qualifies (subject to other gates).
• Step 3 lines: volume_policy.py lines 30-83 (rewritten); signal_generator.py lines 441-455 (rewritten — short-circuit removed, now always calls helper); test_volume_policy.py rewritten with 14 tests.
• Step 4 failure modes: (a) Gamma returns NaN/Inf → _safe_nonneg_float coerces to 0 → never poisons gate; (b) DB pg8000 returns negative → coerced to 0; (c) both sources return 0 → returns 0 → fails any positive threshold (correct); (d) _gamma_cumulative_cached returns None (Gamma API down) → coerced to 0 → falls back to DB value via max; (e) DB connection fails inside get_market_volume → existing except Exception returns (None, None) (unchanged); (f) test that encoded broken behavior was identified + flipped + replaced with regression test.
• Step 5 acceptance: post-deploy, curl /api/earnings-calendar?days=14 for May 1 markets — MRNA volume should be ≈ $3,735 (was $6); the same call to trader's get_market_volume('0xc0a568...mrna...') from a Python REPL on prod must return the same (v24, vt) tuple with vt ≈ 3735. volume_threshold_passes(2000, db, gamma) returns True for MRNA. Verified live in §G probe below.
• §E same-pattern check: ALL callers of the helper grepped:
• earnings_dashboard/app.py:2087 (Gamma success branch) — now returns max ✓
• earnings_dashboard/app.py:2219 (except-fallback branch) — now returns max ✓ (with the existing semantic that the "gamma" arg is cached_volume_24h — see Finding #265.v2 design note)
• src/earnings/signal_generator.py:444 (trader path, no-DB-row branch) — was already calling helper with (None, gamma_v) ✓
• src/earnings/signal_generator.py:441-450 (trader path, with-DB-row branch) — previously short-circuited; now consolidated to call helper ✓
• No other callers in the repo. Both surfaces share one definition (memory §M).
• §F project-specific: does NOT touch trading_positions / trade_audit_log writes. Does NOT change order-execution path (only the pre-trade gate). Does NOT change earnings_markets.earnings_time semantics (Finding #220 unaffected). Cross-interface check (memory §M, mandatory for volume changes): grep ✓; edge matrix covered by tests; one definition ✓; canonical narrative this entry.
• §G post-fix verification (lesson from Finding #265.v2 — never trust code-review alone): post-deploy live probe documented in chat — MRNA on /api/earnings-calendar shows $3,735; trader-side probe via SignalGenerator.get_market_volume from a prod REPL returns (volume_24h, 3734.76). Trader and dashboard now agree to the cent.
• Files:
• src/earnings/volume_policy.py (helper rewritten + new _safe_nonneg_float defensive helper).
• src/earnings/signal_generator.py (trader path consolidated to use the shared helper — removed the bypass short-circuit).
• tests/unit/test_volume_policy.py (existing test flipped + 8 new defensive tests).
• docs/FINDINGS.md (this entry).
• .cursor/memory.md §L (line that referenced "positive DB else Gamma" updated to reflect max() semantic).
• Derived docs (cross-references to "DB else Gamma" updated): docs/STRATEGY.md, docs/STRATEGY_PLAYBOOK.md, docs/STRATEGY_ANALYSIS_ARCHITECTURE.md, docs/EDGE_CASES.md, docs/API_REFERENCE.md, docs/AGENT_HANDOFF.md, docs/CONTEXT.md, README.md, scripts/strategy_performance_analysis.py Action Queue note.
• Acceptance test (any time): for cid in <list>; do compare = python3 -c "from src.earnings.volume_policy import effective_cumulative_volume_usd; ..."; done should report dashboard_volume == trader_get_market_volume() for every May 1 BMO market. Visual: dashboard now shows MRNA at ~$3,735 instead of $6, with ✓ TRADE badge instead of LOW VOL (assuming other gates pass).

Finding 281.v3 — Resolution selector (auto / 1m / 15m / 1h / 1d) on trade-chart modal; pre-Apr-26 trades no longer locked to daily-only (RESOLVED Apr 29, 2026)

• Severity: Medium (operator-facing UX; daily-only fallback for pre-Apr-26 trades was too coarse to be useful)
• Status: done — Apr 29, 2026 ~22:50 UTC.
• Trigger: Operator screenshot of TXN #115 (Apr 22 entry, Apr 23 exit) showed the chart rendered as polygon_daily with 16 1d bars. Quote: "some of the historical charts have only hourly candles, while other only 1 minute candles, what is the logic here? the daily candles dont help at all, as a minimum we should have the posibility to switch all candles to 1h (option to do 1h, 15 min, 1 min ) should be there". Documented routing logic for context: live OR closed-within-24h → fresh Polygon 1m; older closed (post-Apr-26) → equity_bars_1m cache; older closed (pre-Apr-26 cache start) → polygon_daily fallback. TXN entered the third bucket because the cache started Apr 26.
• Fix shipped (operator-controlled override + always-available alternative resolutions):
• Endpoint accepts resolution query param: auto (default, current routing) or 1m / 15m / 1h / 1d. Explicit resolutions bypass the auto routing and fetch directly from Polygon at the requested interval, with a resolution-appropriate window pad (1m: ±12h, 15m: ±5d, 1h: ±14d, 1d: ±30d) so wider intervals show more context bars.
• New generic helper _fetch_polygon_aggs_for_chart(symbol, multiplier, timespan, start, end) wraps /v2/aggs/ticker/{S}/range/{N}/{minute|hour|day}/{from}/{to} with proper pagination via next_url. Returns the chart shape directly. The legacy _fetch_polygon_minute_bars_for_chart and _fetch_polygon_daily_bars are unchanged (memory §C surgical — auto-mode is verified working, do not touch).
• Frontend resolution selector — five buttons (Auto / 1m / 15m / 1h / 1d) above the chart canvas. Click handler setTradeChartResolution(res) is no-op if same as current; otherwise re-fetches with the new resolution and re-renders. State variable _tradeChartResolution persists within a single modal session, resets to auto on each open. The backend echoes meta.resolution_requested so the UI button stays in sync with what was actually returned.
• Active-button styling: selected button has accent-blue background + white text + bold; others are transparent border-only. Helper _setTradeChartResolutionUI(res) is the single source of truth for button state.
• Resolution → window-pad map (_TRADE_CHART_RESOLUTION_MAP in app.py): 1m → ±12h (~ 1k bars max for 2 RTH days) 15m → ±5d (~ 350 bars across 12d window) 1h → ±14d (~ 230 bars across 30d window) 1d → ±30d (~ 60 bars across 60d window) Pads scale with the resolution so wider intervals show enough context for trend analysis. Polygon Stocks Starter ($29/mo) supports all four resolutions with 5y of 1m history — well within our use case.
• §B 5-step verification:
• Step 1 trace: operator clicks 15m button → setTradeChartResolution('15m') updates state + UI → calls _fetchAndRenderTradeChart(...) with new resolution → fetch URL has &resolution=15m → backend dispatches to _fetch_polygon_aggs_for_chart(SYMBOL, 15, "minute", entry-5d, exit+5d) → returns 15m bars → JS renders candles + markers + anchor + price lines.
• Step 2 simulate: TXN #115 (entry Apr 22 14:34 UTC, exit Apr 23 19:56 UTC). With resolution=15m: window = Apr 17 to Apr 28 (10 trading days × 26 RTH bars/day ≈ 260 bars). With resolution=1h: window = Apr 8 to May 7 (~160 bars). With resolution=1m: tight window Apr 22 02:34 to Apr 24 07:56 UTC (~780 bars). With resolution=1d: ±30d window (~40 bars).
• Step 3 lines: app.py constants ~lines 4634-4650 (_TRADE_CHART_RESOLUTION_MAP); _fetch_polygon_aggs_for_chart ~4740-4800; endpoint signature with resolution param ~4870; bar-selection branch on resolution != "auto" ~4920-4940; meta.resolution_requested echo ~4990. dashboard.html resolution-selector buttons ~6320; _tradeChartResolution state + setTradeChartResolution + _setTradeChartResolutionUI ~6370-6420; URL threading ~6480; UI sync from echoed m.resolution_requested ~6510.
• Step 4 failure modes: (a) bad resolution value → endpoint returns {ok: false, error: "resolution must be one of [...]"}; (b) Polygon returns 0 bars at requested resolution → modal shows "No bar data available"; (c) operator clicks resolution mid-fetch → setTradeChartResolution no-ops if same value; otherwise the next render is the latest click (last-write-wins on the chart instance); (d) auto-refresh interval re-fires with old resolution → uses _tradeChartResolution (current value) so the auto-refresh respects the latest selection.
• Step 5 acceptance: probe /api/trade-chart-data?source=audit&id=115&resolution=1h should return interval=1h, data_source=polygon_1h, ~230 bars. Same for 15m / 1m / 1d. Modal visual: clicking each button changes the chart and bar count.
• §E same-pattern check: the new _fetch_polygon_aggs_for_chart is generic enough to replace the existing minute + daily helpers. NOT done in this commit (memory §C surgical) — auto-mode is verified working, refactor would be cosmetic. Future work: collapse the three helpers into one when the next chart-related change lands.
• §G post-fix verification: live probe documented in chat — TXN #115 returns 1m / 15m / 1h / 1d at the requested resolution.
• Files:
• earnings_dashboard/app.py — new _TRADE_CHART_RESOLUTION_MAP + _fetch_polygon_aggs_for_chart helper + resolution query param + bar-selection branch + meta.resolution_requested echo.
• earnings_dashboard/templates/dashboard.html — five resolution-selector buttons in modal header; _tradeChartResolution state + setTradeChartResolution + _setTradeChartResolutionUI JS; URL threading + UI echo sync.
• docs/FINDINGS.md — this entry.
• Acceptance test (any time): for r in 1m 15m 1h 1d; do curl -s "http://127.0.0.1:8080/api/trade-chart-data?source=audit&id=115&resolution=$r" | jq "{r: .meta.resolution_requested, src: .meta.data_source, n: .meta.bar_count}"; done should print four objects, one per resolution, with data_source=polygon_<r> and non-zero bar_count for each.

Finding 281.v2 — Same-day operator-flagged bugs on the first ship: live chart "stops at entry"; SOFI exit marker positioned at $18.67 instead of the $17.56 fill price (RESOLVED Apr 29, 2026)

• Severity: Medium (operator-facing; both bugs surfaced on the first chart-modal screenshot — chart misled rather than informed)
• Status: done — Apr 29, 2026 ~22:00 UTC.
• Trigger: Operator screenshots Apr 29 ~21:46 UTC. (1) MSFT #157 LIVE chart showed 99 bars but the last bar was at 09:45 UTC while the entry happened at 14:34 UTC — chart appeared to "stop at entry" with no live updates. (2) SOFI #123 CLOSED chart showed exit marker text "Exit $17.56 ↓" but the marker arrow was positioned at the bar at $18.67 (the y-axis only spanned 18.30-19.10), making the exit visually look like it filled at $18.67.
• Root cause (single shared cause): the equity_bars_1m collector (Finding #240) runs once daily at 10:00 UTC. Any bar after that fire is missing from the cache. Per-trade fallout:
• MSFT live: entry at 14:34 UTC, modal opened at 16:30 UTC; cache last bar was 09:53 UTC (~7h before entry, 7h before modal open). Chart showed only pre-entry bars. The operator's "auto-refresh every 10 min" request was the right intervention but doesn't help if the underlying source is once-daily-stale.
• SOFI exit: exit at 11:01 UTC, but cache had ZERO bars between 10:00-14:00 UTC for SOFI. Marker time=11:01:16 had no corresponding bar; Lightweight Charts anchored the marker to the nearest in-range bar (~$18.67) and the operator saw a phantom exit at $18.67. Compounding visual bug: the $17.56 price line we add via createPriceLine does NOT trigger price-scale auto-fit in Lightweight Charts, so the chart auto-scaled to the in-range bars (18.30-19.10) and the actual exit price line fell off-screen.
• Fix shipped (4 changes, 1 commit):
• Live + recent-close: prefer fresh Polygon minute bars over cache. New helper _fetch_polygon_minute_bars_for_chart reuses src.earnings.counterfactual_pricing.fetch_polygon_minute_bars (single shared Polygon-paginate impl per memory §M) and converts to chart shape. Bar-selection logic in api_trade_chart_data now: live → fresh fetch; closed within last 24h → fresh fetch; older closed → cache (always sufficient, bars don't change post-exit); fall back to daily Polygon for pre-Apr-26-cache trades.
• Hidden anchor line series so price scale includes entry/exit/stop. New invisible (color: rgba(0,0,0,0)) line series with two points {time: bars[0].time, value: min(allPrices)} and {time: bars[-1].time, value: max(allPrices)} where allPrices includes bar highs/lows + entry/exit/stop prices. Price-scale auto-fit picks up the line series so the chart's Y range now ALWAYS includes the markers. SOFI exit at $17.56 now visible as a horizontal red dotted line.
• 10-min auto-refresh for live charts. Backend adds meta.live_refresh_seconds=600 (only for live trades). JS sets a setInterval(..., live_refresh_seconds*1000) that re-fetches and re-renders when the modal is open on a live trade. Cleared on modal close + on opening a different trade. Defensive guard verifies modal still open AND same trade before firing.
• Wider historical context. Pad bumped from entry-12h / exit+12h to entry-2d / exit+2d for closed trades; daily-fallback pad bumped from 7d to 14d. Matches operator's "graph should move further and should not be limited to the days when the trade was open" request from the original Finding #281 spec.
• §B 5-step verification:
• Step 1 trace: open modal → _fetchAndRenderTradeChart(source, id, false) → /api/trade-chart-data → backend selects bars (fresh Polygon for live/recent, cache for older, daily fallback for very old) → JSON has bars + markers + meta.live_refresh_seconds → JS renders candles + anchor + markers + price lines + arms setInterval if live.
• Step 2 simulate: MSFT #157 live → fresh Polygon fetch returns ~390 bars including post-entry → chart extends past entry; auto-refresh interval fires at 10 min, re-renders with newer bars; "last update" timestamp in meta line updates each tick. SOFI #123 (exit 6.5h ago) → recent_close=True → fresh Polygon fetch includes the 11:01 UTC exit-time bar; anchor series adds 17.56 to price range; exit price line visible; exit marker still anchored to its time-bar but chart extends down to show the gap.
• Step 3 lines: app.py constants ~4602-4631; _fetch_polygon_minute_bars_for_chart ~4720; selection logic ~4840-4880; meta live_refresh_seconds ~4925. dashboard.html closeTradeChartModal updated ~6363; openTradeChartModal split into open + _fetchAndRenderTradeChart ~6395; anchor line series ~6470-6500; setInterval wiring ~6610-6630.
• Step 4 failure modes: (a) Polygon fetch fails → falls through to cache, then daily; (b) POLYGON_KEY missing → cache + daily fallbacks still work; (c) auto-refresh fires after modal closed → defensive modal.style.display !== 'flex' check clears the timer; (d) auto-refresh fires for a different trade than originally clicked → _tradeChartCurrent guard skips; (e) allPrices.length < 2 → anchor not added (degenerate case); (f) anchor times within bar range → never extends time-scale, only price-scale; (g) auto-refresh fetch errors mid-modal → isAutoRefresh=true flag suppresses error overlay so a transient hiccup doesn't blank the working chart.
• Step 5 acceptance: probe /api/trade-chart-data?source=position&id=157 → bars[-1].time should be > markers.entry.time AND data_source="polygon_realtime"; probe SOFI #123 → bars should include the 11:01 UTC range; probe NKE #106 → unchanged daily fallback. Visual: open MSFT live chart → bars extend past entry; meta line shows "auto-refresh every 10 min · last update HH:MM:SS"; reopen 10 min later → new last-update timestamp + newer last bar. Open SOFI history chart → exit price line visible at $17.56 with the chart Y range extending down to include it.
• §E same-pattern check: searched for other paths reading equity_bars_1m for live use cases — none exist on the dashboard side. The collector consumers (analyze_finding_a_full.py, compute_counterfactual_pnl.py) only run on closed/historical trades where the once-daily cadence is fine. No further surfaces to update.
• §G post-fix verification (lesson from Finding #265.v2): verified live on prod by reopening the SOFI #123 chart — exit price line at 17.56 now visible; chart auto-scales correctly. Verified MSFT #157 (or whichever live trade is active) shows bars extending past entry and the "auto-refresh every 10 min · last update" suffix in the meta line.
• Files:
• earnings_dashboard/app.py — new constants + new _fetch_polygon_minute_bars_for_chart helper + reworked selection logic + live_refresh_seconds in meta.
• earnings_dashboard/templates/dashboard.html — split openTradeChartModal into open + reusable _fetchAndRenderTradeChart; added hidden anchor line series; setInterval for live auto-refresh; defensive close + cross-trade guards; meta line shows "last update HH:MM:SS" on live charts.
• docs/FINDINGS.md — this entry.

Finding 281 — Active Trades + History trade log had no entry/exit-marked chart link; new inline chart modal with Lightweight Charts (RESOLVED Apr 29, 2026)

• Severity: Medium (operator UX — slows post-trade review and live trade monitoring; no trader correctness impact)
• Status: done — Apr 29, 2026 ~21:30 UTC.
• Trigger: Operator request Apr 29 evening: "for all live trades in active trades tab i would like a link to a graph showing the entry point and exit point for the trade ... it can be a link to any existing website where we can actually mark entry on a chart, similar to this is the following point: ... in the trade log table on history tab, i would like a link to a chart showing both entry and exit point for each trade".
• Investigation finding: surveyed major free public charting sites (TradingView free, Yahoo Finance, Google Finance, Stockanalysis, Investing.com, Finviz, Webull). None of them support URL-based entry/exit markers. TradingView requires a saved layout (paid plan, per-user, not URL-shareable to other users). The only realistic options are (a) link to symbol-only chart and rely on the operator to find the entry timestamp, or (b) render an inline chart in the dashboard with markers using the data we already collect. Option (b) chosen — equity_bars_1m cache (Finding #240) covers Apr 26+ trades at full 1-minute resolution; older trades fall back to Polygon /v2/aggs/.../range/1/day.
• Fix shipped:
• New endpoint GET /api/trade-chart-data?source={position|audit}&id=N returning {ok, meta, bars[], markers}. Source modes: position reads trading_positions (Active Trades), audit reads trade_audit_log (History). Single endpoint, single response shape — frontend modal is symmetric.
• Bar selection: tries equity_bars_1m first for [entry-12h, exit+12h] (or [entry-12h, now+12h] for live). Fall-back to Polygon daily for [entry-7d, exit+7d] only when equity_bars_1m has < 10 bars in window (older trades pre-Apr-26 cache start). Heuristic threshold = _TRADE_CHART_MIN_1M_BARS = 10.
• Markers: entry (color-coded green for LONG / red for SHORT, arrow-up/down setMarkers), exit (red/green inverse), horizontal stop-price line (orange dashed), vertical earnings-release marker (purple 📢 circle) when actual_earnings_time is known via the LEFT JOIN trading_positions lookup.
• Frontend modal: TradingView Lightweight Charts (MIT-licensed, ~40KB) loaded from unpkg.com/lightweight-charts@4.1.7. Single shared modal HTML + openTradeChartModal(source, id, title) JS function used by both tables (memory §M cross-interface — one shared definition, not parallel renderers). Chart icon 📈 button added as a new column on both Active Trades (JS-rendered) and History trade log (Jinja-rendered).
• Graceful degradation: (a) CDN blocked → typeof LightweightCharts === 'undefined' check → friendly message but bars still in API JSON for inspection. (b) Polygon key missing → daily fallback returns [] → modal shows "No bar data available". (c) Position/audit row not found → {ok: false, error: "..."} → modal shows error. (d) Orphan IBKR-only position with no DB id → button replaced with "-" placeholder.
• §B 5-step verification:
• Step 1 trace: user clicks 📈 → openTradeChartModal(source, id) → fetch('/api/trade-chart-data?source=...&id=...') → api_trade_chart_data() → _load_position_for_chart or _load_audit_for_chart → _fetch_chart_bars_1m (or _fetch_polygon_daily_bars fallback) → JSON response → JS renders candlestick + markers + price-lines.
• Step 2 simulate: SPGI audit_id=121 (Apr 27-28 closed) → ~1232 bars in cache → 1m chart with entry $437.24 / exit $449.46 / stop $424.12 markers. NKE audit_id=106 (March, no cache) → daily Polygon fallback → ~14 daily bars + entry/exit markers. MSFT position_id=157 (live, just opened today) → may be 1m or daily depending on cache state at click time.
• Step 3 lines: app.py endpoint + 4 helpers ~lines 4602-4842; app.py trading_positions id field at SELECT line ~2473 + dict line ~2575; dashboard.html Active Trades <th>Chart</th> at line ~537; JS chart-cell render at line ~5180 + colspan bumps 13→14; History <th>Chart</th> at line ~2470 + Jinja per-row chart-cell at line ~2557 + colspan 17→18; modal HTML + Lightweight Charts script + openTradeChartModal/closeTradeChartModal at dashboard.html ~lines 6310-6510.
• Step 4 failure modes: enumerated and handled — (a) library load failure, (b) Polygon key missing, (c) row not found, (d) orphan position with no id, (e) entry_time NULL, (f) modal opened twice (chart.remove() before re-render), (g) ResizeObserver not supported (graceful — chart just doesn't auto-resize), (h) timezone mismatch (datetime fields converted to unix seconds).
• Step 5 acceptance: curl /api/trade-chart-data?source=audit&id=121 returns ok=true, interval=1m, ≥1000 bars, markers with entry+exit+stop. curl /api/trade-chart-data?source=position&id=<live> returns ok=true. curl /api/trade-chart-data?source=audit&id=999999 returns ok=false, error="audit #999999 not found". Dashboard visual: chart icon on Active Trades + History rows opens modal with rendered chart and markers.
• §E same-pattern check: searched for other consumers of trade-chart data — none exist (this is a new endpoint). The chart icon is added in two places (Active Trades JS render + History Jinja per-row) but both call the SAME openTradeChartModal() function with different source arg. No duplicate render logic.
• §F project-specific: does NOT touch trade_audit_log writes; does NOT change trader behavior; does NOT modify any gate/sizing; pure read-only enrichment. Adding id to the trading_positions SELECT + dict is the minimum surface needed for the chart link.
• §C surgical surface review: every changed line traces to "operator can see entry/exit on a chart". No adjacent improvements. No re-formatting of unrelated code.
• Files:
• earnings_dashboard/app.py — new helpers (_load_position_for_chart, _load_audit_for_chart, _fetch_chart_bars_1m, _fetch_polygon_daily_bars); new endpoint /api/trade-chart-data; id added to active-positions SQL + dict.
• earnings_dashboard/templates/dashboard.html — chart-modal HTML + Lightweight Charts CDN script + openTradeChartModal/closeTradeChartModal JS; Active Trades + History trade-log gain a Chart column with the 📈 button; empty-state colspan bumps (13→14 Active, 17→18 History).
• docs/FINDINGS.md — this entry.
• Acceptance test (any time): curl -s 'http://127.0.0.1:8080/api/trade-chart-data?source=audit&id=121' | jq '.ok, .meta.bar_count, .markers' should return true, >= 100, and a markers dict with entry/exit/stop populated. Visual: open /, click 📈 on any History row → candlestick chart with markers renders within ~500ms.

Finding 280 — Upcoming Earnings widget had no post-event status; today's BMO rows rendered as "SKIP / -- / 0% / NO EDGE" hours after release (RESOLVED Apr 29, 2026)

• Severity: Medium (operator-facing; no trader correctness impact — gates were pre-event-only by design, but the UI made settled rows visually indistinguishable from pre-event rows)
• Status: done — Apr 29, 2026 ~20:30 UTC.
• Trigger: Operator screenshot Apr 29 16:00 UTC: GRMN, YUM, HUM, TW, SOFI, PAG, VIRT, WING (all today's BMO) all rendered as SKIP / -- / 0% ✗ / LONG / NO EDGE even though they had released hours earlier. Screenshot also showed today's AMC markets (CMG 4/6 / 72.0% ✓ / LOW VOL, EBAY SKIP / 91.5% / 100% / NO EDGE) which are still pre-event and correctly rendered.
• Root cause (3 contributing factors, none of them bugs in isolation):
• polymarket-resolver.timer runs daily at 06:00 ET and processes YESTERDAY's earnings (data needs to settle) — so actual_beat is NULL for today's earnings until tomorrow morning.
• continuous_collector stops sampling resolved markets (Polymarket drops them from ?active=true); recent price_snapshots rows stop arriving for them. Live PM yes_price evidence exists for some markets (e.g. HUM @ 0.9995, CAT @ 0.815) but not uniformly.
• The pre-event gating columns (Conviction --, Min Required 0% ✗, Direction LONG, Trade? NO EDGE) are exactly what they should be for a settled market — there's no Trade? action available because the trade window has closed. But the row visually looks identical to a low-quality opportunity that we're skipping.
• Fix shipped: new OUTCOME column between Trade? and Hold with five states computed from data we already collect:
• released_beat_eps / released_miss_eps — from earnings_markets.actual_beat. Gold-standard, overrides PM. Badge: ✅ BEAT · EPS / ❌ MISS · EPS.
• released_beat_pm / released_miss_pm — latest price_snapshots.yes_price >= 0.99 or <= 0.01 within last 12h. Same threshold as Finding #264's _live_polymarket_extreme_outcome so trader and dashboard agree on "PM at extreme = released". Badge: ✅ BEAT · PM / ❌ MISS · PM.
• released_pending — past typical BMO/AMC release window in ET (BMO ≥ 9:30 AM, AMC ≥ 4:30 PM), no clear outcome signal yet. Badge: ⏳ PENDING. Resolver will pick this up on the next cycle.
• pre_event — default; no badge; current behaviour.
• Settled rows (any released_*) get opacity:0.55 so the operator's eye skips past them.
• Single source of truth (memory §M): new helper _compute_outcome_status(actual_beat, latest_yes_price, latest_yes_price_age_sec, earnings_date_str, earnings_time, now_et) in earnings_dashboard/app.py. Called once per row in get_earnings_with_polymarket. JS render and Jinja initial render BOTH read the same 5-state outcome_status field; no duplicate logic in HTML/JS.
• §B 5-step verification:
• Step 1 trace: browser → _refresh_cache() (initial Jinja render) AND /api/earnings-calendar (30s JS poll) → get_earnings_with_polymarket() → _compute_outcome_status() → row dict gets outcome_status etc. → renderer paints badge + dim.
• Step 2 simulate: HUM today BMO, actual_beat=NULL, latest yes_price=0.9995 (~3h old) → helper returns released_beat_pm. CAT today BMO, latest yes=0.815 → no PM extreme → past 9:30 AM ET → released_pending. GRMN today BMO, no recent PM snapshot → past 9:30 AM ET → released_pending. AMZN today AMC, no PM, before 4:30 PM ET → pre_event.
• Step 3 lines: helper at app.py lines ~1740-1875; SQL extended at app.py line ~1791; row-loop fields at app.py line ~1908; both polymarket_data[ticker] dicts (success + except-fallback) at app.py lines ~2030 and ~2114; entry dict at app.py line ~2330; Jinja header at dashboard.html line ~1062; Jinja per-row cell at dashboard.html line ~1293; Jinja TR dim at dashboard.html line ~1071; JS render outcome cell at dashboard.html line ~3920; JS render TR dim at dashboard.html line ~3865.
• Step 4 failure modes: (a) row with no pm_data → helper called with all None → returns pre_event. (b) timezone mismatch → helper uses dt.now(ET) (ZoneInfo("America/New_York")); intra-day comparison in ET. (c) cache stale → entry dict has fields; backend serves cache → JS still renders. (d) actual_earnings_time is a datetime kept INTERNAL to polymarket_data; entry dict only carries scalars (no JSON serialization risk).
• Step 5 acceptance: curl /api/earnings-calendar returns rows with outcome_status / outcome_label / outcome_source / outcome_settled; today's HUM = released_beat_pm, CAT = released_pending, GRMN/YUM/TW/SOFI/PAG/VIRT/WING = released_pending, all today's AMC + future-dated rows = pre_event. Visual: settled rows dimmed with badges; pre-event rows unchanged.
• §E same-pattern check: searched for other readers of earnings_with_polymarket rows. Only _refresh_cache() populates the cache, only /api/earnings-calendar + the Jinja initial render consume it, and both use the new field. No other dashboard surface needs the field.
• §F project-specific checks: does NOT touch trading_positions or trade_audit_log; does NOT change trader behavior or earnings_markets.earnings_time semantics (Finding #220 unaffected); does NOT change any gate (memory §M cross-interface for gates not triggered, but the cross-interface principle of "one shared definition, both surfaces" was followed anyway via the helper).
• Files:
• earnings_dashboard/app.py (helpers + SQL + 2 dict additions + entry-dict outcome plumbing).
• earnings_dashboard/templates/dashboard.html (header + Jinja cell + Jinja TR dim + JS render outcome cell + JS TR dim + 2 colspan bumps 14→15).
• docs/FINDINGS.md (this entry).
• Acceptance test (run any time): curl -s http://127.0.0.1:8080/api/earnings-calendar | jq '.[] | select(.date == "<today>") | {symbol, time, outcome_status, outcome_label, outcome_source, outcome_yes_price}'. Today's BMO rows post-release should be released_beat_pm / released_miss_pm / released_pending / released_beat_eps / released_miss_eps; today's pre-AMC + future rows should be pre_event.

Finding 278 — Market regime detection is structurally lagging (median 31 trading days from SPY peak); evaluate faster regime alternatives post-Phase-3 (RESOLVED May 31, 2026 via Finding #318 — KEEP compound_v1)

• Severity: Low → Medium for opportunity cost (the signal works on the trades it does catch; the cost is downtrend-LONG entries we miss in the lag window). Not a correctness bug.
• Status: RESOLVED May 31, 2026 — KEEP compound_v1 (see Finding #318). PR 4 of the Phase 3 simulator built the 6-classifier sweep (scripts/phase3_simulator/regime_classifiers.py + replay_regime_v2.py) and ran it on prod: faster detectors add NO robust ROI. The fastest (below_ema20) gains +$70 on the live-traded set but the gain evaporates to −$17 (outlier-driven) on the capacity-aware universe at the live 1.40 cap, and it whipsaws ~40% of LONGs (vs 17% for compound). Operator-confirmed KEEP. The faster detectors' only durable benefit is a better worst-week tail at ~breakeven P&L — a downside trade-off, not a money-maker. Historical context below preserved verbatim. (Originally documented Apr 28, 2026 — deferred to post-Phase-3; the deferral conditions were met and the analysis is now done.)
• Trigger: Operator asked Apr 28 evening: "our system catches regime changes fast enough? by the time we show downtrend there are already a number of days of markets going down when we do not apply the downtrend rules — is there any consensus way to improve this?"
• Background — what we measured: read-only empirical analysis (scripts/analyze_regime_detection_lag.py, committed Apr 28) replayed MarketRegimeProvider._classify over SPY 2018-2026 daily history and measured trading days from each significant peak (≥8% drop within 60 days, n=8 peaks) to first regime == "downtrend" flip, vs faster alternative detectors: ``` Peak date Peak $ Trough $ DD% compound below_50 below_200 below_ema20 dd_3pct dd_5pct dd_10pct 2018-08-27 257.23 210.18 -18.3% 37d 31d 40d 27d 31d 32d 76d 2019-12-12 288.40 204.94 -28.9% 54d 48d 51d 29d 33d 49d 51d 2020-02-11 306.38 204.94 -33.1% 14d 8d 11d 8d 8d 9d 11d 2020-09-01 325.55 299.08 -8.1% never 12d never 3d 2d 4d never 2021-12-10 442.51 392.67 -11.3% 30d 6d 28d 2d 6d 26d 49d 2024-07-16 553.22 506.72 -8.4% 18d 7d never 3d 6d 13d never 2025-01-22 597.77 490.85 -17.9% 31d 22d 32d 8d 23d 28d 35d 2026-01-06 689.93 631.97 -8.4% 37d 9d 51d 9d 41d 50d never

Detector medians: compound (current) 31d [22 historical downtrend-LONG trades sized against this] below_50 12d faster, classic SMA breach below_ema20 8d fastest, EMA smoothing dd_3pct 23d drawdown filter dd_5pct 28d "correction" textbook def below_200 40d slower dd_10pct 49d slowest `` Two takeaways: (1) our compound 4-AND uptrend / 3-AND downtrend rule is the **second-slowest** detector in the family — onlyclose < MA200is slower; (2) our rule **missed an entire drawdown** (Sep 2020 −8.1%) because the move wasn't deep enough to invert MA20<MA50. - **Why this is the right design today (do NOT change before Phase 3):** - Action Queue rule #1 (Skip LONGs in downtrend) was sized on the **22 historical downtrend-LONG trades** identified by exactly this classifier. Switching to a faster detector retroactively re-classifies those trades and may shrink the bucket below the n≥12 gauntlet floor (memory §K). The +$140-$200 / 100-trades estimate is *causally tied to the current classifier's lag*. - Per memory §K + §N, every regime-rule change requires a true-simulator end-to-end re-run on the actual data (not just lag analysis). That re-run is a meaningful piece of work and should be done as a deliberate "regime v2" project after rule #1 has accumulated live evidence. - The lag is a **selection problem (we miss some good downtrend-skips)**, not a **damage problem (the rule itself works on the trades it does catch)** — opportunity cost, not loss. - **Industry-consensus options (ranked by cost-of-integration, surveyed Apr 28):** 1. **SMA → EMA on the same windows** — median 8d vs our 31d; drop-in replacement; no new data; trade-off is more sideways↔uptrend whipsaw. 2. **Drawdown circuit-breaker as OR clause** —downtrend = (compound) OR (close ≤ 95% of 60-day high); 28d median; catches the Sep 2020 case our rule missed; textbook "correction" definition. 3. **VIX term structure (VIX vs VIX3M)** — backwardation = stress; the single most-watched leading vol-regime indicator on institutional vol desks; typically flips 5-15 days BEFORE the price-MA breakdown. We already capture VIX — VIX3M is one extra yfinance call (^VIX3M). 4. **Breadth deterioration** — % of S&P 500 stocks below 50-DMA; leads index by ~10 days; needs universe data we don't currently have. 5. **Credit spreads** — HYG/LQD ratio or HY OAS; leads equity by 5-20 days; free from FRED. 6. **Composite financial-conditions index (FCI)** — Goldman / Bloomberg / Chicago Fed NFCI; full institutional standard. Heavier integration cost. 7. **Hidden Markov Models / Markov-switching** — academic-grade; probably overkill for our purpose. - **Asymmetry to flag in any redesign:** the current downtrend rule does NOT requireclose < MA200. So a sustained bull-market dip that briefly breaks MA50 + inverts MA20<MA50 can flip us to "downtrend" while SPY is still above MA200 — a slight false-positive risk. Symmetric construction would also requireclose < MA200. Worth raising during the post-Phase-3 redesign even if all other choices are unchanged. - **Re-test triggers (when to revisit):** 1. Phase 3 unified simulator has shipped (~May 26, 2026 — equity-bars 30-day capture window completes, see Finding #240). 2. Action Queue rule #1 has been live for ≥30 closed trades AND its measured lift is within ±$30 of the +$140-$200 / 100 estimate (i.e. the current-classifier baseline holds). 3. THEN: buildMarketRegimeProvider v2candidate (likely option B above — compound rule ORdd_5pctORVIX > VIX3M for ≥3 days), re-run the full Action Queue analysis with v2 labels, apply the 4-check gauntlet (memory §K) to every rule that uses regime, ship only if v2 shows ≥+30% improvement and survives the gauntlet. - **Saved artifacts (per operator request "ensure everything is clearly saved"):** -scripts/analyze_regime_detection_lag.py— read-only, re-runnable, committed Apr 28. Re-run with:[hidden]/venv/bin/python3 scripts/analyze_regime_detection_lag.py. Output is the table above + aggregate medians. - This Finding (canonical narrative per memory §J). -docs/STRATEGY_PLAYBOOK.md§7 (forward-looking deferral entry per memory §N). -docs/AGENT_HANDOFF.md§11 DEFER queue. - Action Queue entryregime-detection-v2with statusDEFERRED-POST-PHASE-3(mirrored to dashboard viabuild_action_queue()per memory §K). - **Files (current implementation, no changes):** -src/earnings/market_regime_provider.py— single source of truth for regime classification (memory §M). -scripts/strategy_performance_analysis.py:get_spy_regime_at_date— historical analysis mirror (must stay in sync with_classifyif the rule ever changes). - **Acceptance test (when this finding is eventually closed):** post-Phase-3, after building v2, re-runscripts/analyze_regime_detection_lag.pyANDscripts/strategy_performance_analysis.py` AND the 4-check gauntlet. Lag table should show v2 median ≤ 15d. Action Queue rule #1 evidence under v2 must still pass the gauntlet. Then update this finding to RESOLVED and the Playbook §7 entry.

Finding 279 — Position-sizing percentages and 0-6 priority-tier system have not been re-validated against the current strategy data; defer comprehensive re-evaluation to post-Phase-3 (DOCUMENTED — deferred to post-Phase-3; May 28, 2026 amendment: Phase B scope expanded to include the $2k SIGNAL_MIN_VOLUME_TOTAL gate retune after PM earnings-market volume collapse was quantified)

• Severity: Medium (real money in production at sizing chosen Mar 9 against an earlier, smaller dataset and a smaller capital base; risk is mis-allocated capital across mcap × conviction tiers, not a correctness bug). Severity unchanged by May-28 amendment — the volume-collapse evidence is structural-context, not a new live-trading defect.
• Status: RESOLVED May 30-31, 2026 — core sub-items SHIPPED across Phase 3 PR 3. Volume gate $2k→$1k (#311), priority tier system 0-6→0-8 LIVE IN PRODUCTION (#313; priority.py MAX_PRIORITY_SCORE=8, base 0-6 + volume bump), sizing 30/22/15 EVALUATED + KEPT (#314.c), exposure cap KEEP 1.40 + daily tiering SIMPLIFIED to uniform 140% (#315/#316/#317), 4 SKIP cells evaluated (#314.e, analysis-only). Residual deferred to after PR 4 (now un-gated, #318 shipped): SHORT ≤10%-conviction bucket validation + regime-conditioned sizing + position×regime interaction. (Originally documented Apr 28, 2026 deferred-post-Phase-3 alongside #278; May-28 amendment folded the volume-collapse evidence into Phase B scope. The deferral conditions were met and the core re-evaluation is now done; historical body preserved below.)
• Trigger: Operator request Apr 28 evening: "another point that we need to reevaluate post phase 3 strategy analysis completion is the position sizing and priority system". Parked alongside the regime-detection finding (#278) so both flow back into the same post-Phase-3 audit pass.
• Current state being deferred (do not change before Phase 3):
• Position-sizing percentages (src/earnings/config.py lines 141-143):
  • POSITION_SIZE_PCT_LARGE = 0.30
  • POSITION_SIZE_PCT_MID = 0.22
  • POSITION_SIZE_PCT_SMALL = 0.15
  • TESTING_PROD_SIZE_MULTIPLIER = 1.00 (line 151) — currently 100% of standard sizing; will downshift to live sizing after the trade #150+ transition.
  • POSITION_SIZE_FLEX_MAX (Finding #74) — fractional-share flex allowing up to +57% over base size when rounding to 1 share.
• Priority tier system (src/earnings/priority.py, full file):
  • 0-6 scale: 6=Large 70-80%, 5=Large 80-90%, 4=Mid 70-80%, 3=Mid 80-90%, 2=Small 70-80%, 1=(reserved), 0=SKIP.
  • Hard-coded SKIP rules: Large/Mid LONGs at conviction ≥90% (Finding #202); Small LONGs at conviction ≥80% (Finding #207); ALL Small SHORTs (squeeze risk); ALL signals in conviction dead-zone 30-70%.
  • Tiebreaker on equal scores: descending conviction.
• Why these MUST be re-evaluated (evidence summary):
• Sizing percentages were set Mar 9, 2026 when the strategy had ~50 closed trades and ~$25k capital. We now have 116 closed trades and the strategy verdict has shifted materially: stops are the biggest leak (−$676 across 57 stops); strategy is structurally negative ex-top-5 outliers (−$263 across 111 trades); Mid-cap stop rate at 80-90% conviction is 52%, at 90-95% is 87%. The Mid 22% sizing was chosen when Mid was a profitable bucket; today's data shows Mid LONGs at the 80%+ conviction tier are the worst losers. Position-size-by-mcap-tier is currently treating different EV regimes as if they were the same.
• Priority tiers conflate "expected ROI" with "execution priority": tier 6 (Large 70-80%) was rated "best ROI" Mar 9 but the most recent live data shows the highest live wins are coming from elsewhere (LONG 95%+ universe-bucket: +$192 / trade across 27 markets — currently SKIPPED by Finding #202 — see docs/AGENT_HANDOFF.md §11 follow-on observation). The tier ordering predates Findings #228 (stops eat winners), #232 (5-outlier curse rename), #233 (corrected stop classification), and the pending Action Queue rules.
• Position-sizing quintile analysis already exists (Finding #218, in scripts/strategy_performance_analysis.py) but has not been used to inform a sizing-policy update. The quintile-by-position_value data is rendered on the dashboard Strategy Analysis tab but consumed by no rule.
• Capital base has grown from ~$25k (PDT-reserve era) to ~$27.5k net liq with potential for further growth post-trade-#150 transition. Sizing-as-percent-of-equity scales naturally, but the absolute $ at risk per trade changes — and so does the ratio of fixed costs (commission + slippage + spread) to position value. A 30% Large position at $25k = $7,500 has different fee economics than at $50k = $15,000.
• The 0-6 tier system has dead values: tier 1 is "reserved" / unused (priority.py PRIORITY_TABLE has no row for it); tier 2 covers Small 70-80% LONGs only (very rare in production, n<5 historical). The 0-6 range is not actually 7 distinct tiers in production.
• Why deferred (do NOT change before Phase 3):
• The Action Queue's selection rules (#1 Skip downtrend LONGs, #2 RuleB no-stop window) directly change which trades get sized and stopped. Their live impact must be measured at the CURRENT sizing first, otherwise we conflate "rule effect" with "sizing effect".
• Phase 3's unified simulator (~May 26, 2026 dependency) will be able to replay sizing-policy variants on the same trade history with realistic execution costs (Polygon 1m bars, equity_bars_1m table from Finding #240). Without it, any sizing change is a guess against today's lag-prone analyses.
• Finding #218 quintile analysis showed sizing effects ARE present in the data but did not isolate them from outlier dominance (5-outlier curse, memory §K). A proper re-evaluation needs the full bias gauntlet.
• Memory §K explicitly requires the 4-check gauntlet for every new tier/sizing change. That's a meaningful piece of work and the right time to do it is once.
• Re-test triggers (when to revisit):
• Phase 3 unified simulator has shipped.
• Action Queue rule #1 + RuleB have been live for ≥30 closed trades each AND their measured lift is within ±$30 of estimates.
• Trade #150+ audit gauntlet pass has run (Action Queue rank #4).
• THEN: comprehensive sizing/priority re-evaluation in one focused project — replay history with the unified simulator under (a) current sizing/tiers, (b) flat-equal sizing across mcaps as null hypothesis, (c) 4+ candidate redistributions (e.g. "shift Mid 22% → 18%; Large 30% → 33%" if Mid stays negative; "merge tiers 5+6 with same sizing" if conviction band has no measurable EV difference). Apply 4-check gauntlet to each variant. Ship only the candidate that wins on the gauntlet AND has positive ex-top-5-outlier P&L AND robust full-population spillover.
• Scope of the eventual re-evaluation (so Phase 3 planning includes it):
• Re-derive sizing percentages from per-bucket live + simulator EV, not Mar 9 priors.
• Collapse the 0-6 tier system to whatever number of tiers is actually informative (likely 3-4: HIGH / MED / LOW / SKIP). Drop dead tier values.
• Tighten or relax the SKIP rules from Findings #202 / #207 against current data:
  • LONG ≥ 90% (Finding #202): LONG 95%+ universe-bucket = +$192/trade across 27 markets — currently SKIPPED. Live n at 95%+ is just 2 trades — far below gauntlet floor. Re-test at trade #150+ if signal persists. (Per docs/AGENT_HANDOFF.md §11 follow-on observation.)
  • SHORT ≤ 10% — symmetric application of #202 to SHORTs (added May 6, 2026). The implementation pattern (omit ("size", "90+") from PRIORITY_TABLE so confidence ≥ 90% returns SKIP) applies symmetrically to LONG-90% AND SHORT-10% paths, but the rule was only data-validated on LONGs. SHORT-side: total SHORT history n=18; zero historical SHORTs at conviction ≤ 10% to compare against. Operator caught this on COIN #2026-05-07 (yes_price=0.0755 → confidence 92.45% → mid 90+ bucket → SKIP). The skip might be over-strict for SHORTs because miss-direction has different EV characteristics than beat-direction at the conviction extreme. Phase-3 evaluation must explicitly bucket SHORTs at confidence 90-95% / 95-99% / 99%+ separately and decide per bucket. Sample candidate when COIN-style markets accumulate: ("mid", "90+SHORT") = 3 (same priority as ("mid", "80-90") = 3 if data supports it).
  • Small LONG ≥ 80% (Finding #207) — currently SKIP for small-cap squeeze risk. Re-evaluate against fresh data when small-cap n grows.
• Re-evaluate POSITION_SIZE_FLEX_MAX (currently +57%, Finding #74) against the now-larger capital base — flex was sized when 1-share-rounding was a frequent constraint at $25k; at $50k it is less common.
• Add explicit position-sizing × regime interaction (the sized_by_regime hypothesis) — currently size is independent of regime, which the data now suggests is sub-optimal for downtrend LONGs.
• Saved artifacts (per operator request "ensure everything is clearly saved"):
• This Finding (canonical narrative per memory §J).
• docs/STRATEGY_PLAYBOOK.md §7 (forward-looking deferral entry per memory §N).
• docs/AGENT_HANDOFF.md §11 DEFER queue.
• Action Queue entry sizing-priority-v2 with status DEFERRED-POST-PHASE-3 (mirrored to dashboard via build_action_queue() per memory §K).
• Files (current implementation, no changes):
• src/earnings/config.py lines 141-151 (sizing percentages + TESTING_PROD multiplier).
• src/earnings/priority.py (entire file, 0-6 tier system).
• src/earnings/risk_manager.py:calculate_position_size (line 881 + flex logic ~953-974).
• scripts/strategy_performance_analysis.py (Finding #218 quintile analysis already plumbed; will be the foundation for the v2 evaluation).
• Acceptance test (when this finding is eventually closed): post-Phase-3, after running the comprehensive re-evaluation, the Action Queue should show either (a) IMPLEMENTED — sizing-priority v2 (Finding #279) with concrete new percentages + tier definitions in the Playbook, OR (b) KEPT — sizing-priority v1 verified at trade #150+ if the gauntlet showed the current values are still optimal. Either outcome closes this finding.

Phase B framing premise — PM earnings-market volume collapse (May 28, 2026 amendment)

This subsection was added May 28, 2026 (was a chat-only observation from May 19, escalated to canonical narrative here per operator's AskQuestion "option B — fold #304 evidence into #279 Phase B" decision — i.e. the PM volume-collapse evidence got folded INTO this #279 subsection rather than creating a separate Finding entry for the collapse itself). Note for chronology: the Finding #304 number itself WAS subsequently used the same afternoon — assigned to the Queue Rule #1 ship (Skip LONGs in downtrend regime) per docs/FINDINGS.md Finding #304. The two events are independent: the volume-collapse evidence remains folded in here under #279; the next-numbered ship (Rule #1) took #304 in the natural sequence. This subsection captures the volume-environment shift that any Phase B sizing/priority/gating decision must now account for.

• Why this is here, not as a standalone Finding: the volume-collapse evidence is INPUT to the sizing+priority+gating decision that #279 Phase B already owns. Splitting it into two Findings would fragment the narrative; folding it in preserves canonical single-source per memory §J. The chat-dump from May 19 framed it as "Finding #298 candidate" — that label is hereby retired in favor of this subsection.
• Evidence (SQL on earnings_markets LEFT JOIN markets USING (condition_id)`, May 28, 2026 ~12:30 UTC):

• Headline orders-of-magnitude (Jan → May): average cumulative volume per market dropped ~109× ($33,475 → $306); median dropped ~1,847× ($24,730 → $13.39); the $2k-gate pass rate dropped ~52× (100.0 % → 1.9 %).
• May 2026 by market-cap tier (no recovery in any tier):
• Mid: 74 markets, 0 crossed $2k; avg $252, median $17, max $1,557.
• Small: 59 markets, 1 crossed $2k (AMPL); avg $270, median $10, max $2,441.
• Large: 21 markets, 2 crossed $2k (NVDA + LOW); avg $597, median $15, max $9,358.
• The 3 May 2026 gate-passing markets and why only 1 produced an entry:
• AMPL (Small, AMC 2026-05-06, vol $2,441) — prob_24h_before = 0.1205 (12 %). Bot's SHORT path skipped because Finding #207 symmetric application (added May 6, 2026, see sub-finding in this entry above) treats SHORT conviction ≤ 10 % as SKIP. AMPL at 12 % was borderline; market resolved MISS (actual_beat = False) — would have been a winning SHORT had the rule allowed it. (Already on the Phase B re-evaluation scope as the SHORT-≤-10 % sub-rule.)
• NVDA (Large, AMC 2026-05-20, vol $9,358) — prob_24h_before = 0.9720 (97 %). Bot's LONG path skipped via Finding #202 (Skip all LONGs at conviction ≥ 90 %). LONG 95 %+ universe-bucket = +$192/trade across 27 markets (currently skipped); Phase B's LONG ≥ 90 % re-evaluation already lists NVDA-class markets as the population this might re-admit.
• LOW (Large, BMO 2026-05-20, vol $2,253) — prob_24h_before = 0.8900 (89 %), Large LONG. Below the 90 % skip cliff. The Mid-LONG-≥80 % skip (queue rule #3 ON-HOLD pending RuleB) does NOT apply because LOW is Large, not Mid. The reason LOW produced no entry is not on the current investigation surface; flagged as a sub-question for Phase B planning (likely either a entry_volume_24h < 1k bot-side check, or a depth/spread gate kill via Finding #283 / WIDE SPREAD).
• Live consequence: 21-day entry drought. From trading_positions: last pre-drought entry was id = 162 MCD on 2026-05-06 19:58 UTC; next entry was id = 163 HPQ on 2026-05-27 14:34 UTC — 20 days, 18 hours, 36 min with zero new entries. HPQ itself only crossed the gate via the Finding #282 Gamma-cumulative fallback path — the DB-recorded markets.volume_total = $23.84 is far below the $2k threshold, but volume_threshold_met = True was recorded in trading_positions at entry time, so the effective max(DB, Gamma) cumulative WAS ≥ $2k at the gate-decision instant (the DB value subsequently drifted or was overwritten by a stale sync — separate data-capture observation; see "Data-capture gap" bullet below).
• Why the cliff happened in March: outside the scope of this Finding (PM market-maker activity / Polymarket platform-side dynamics / shift in retail interest in earnings markets — all hypotheses, none verified here per INFERENCE DISCIPLINE). What matters for Phase B: the signal is directionally persistent across 3 consecutive months (Mar/Apr/May) with severity now an order of magnitude worse than at first cliff. Treat as the NEW operating regime, not as a transient.
• Implication for Phase B scope (NEW — to be addressed in the Phase 3 re-evaluation):
• The $2k SIGNAL_MIN_VOLUME_TOTAL gate must be re-evaluated against fresh data. The historical bias-gauntlet population that justified $2k was built on the Dec-2025 / Jan-2026 / Feb-2026 regime when 95-100 % of markets passed; in the May regime the gate eliminates 98.1 % of potential signal sources. A retune may be appropriate, BUT (per memory §K + §N) requires a TRUE simulator pass first (counterfactual P&L on a wider universe of currently-skipped markets across a range of candidate thresholds, e.g. $1k / $500 / $250 / $0). Lowering the gate without simulator evidence is exactly the shortcut NO SHORTCUTS rule #9 forbids.
• Sizing percentages must be re-evaluated against the new market-cap mix. May 2026 saw 0 / 74 Mid-cap markets cross the gate but 2 / 21 Large-cap markets — i.e., if any signal flow continues in the current regime, it's structurally Large-cap-dominated. The Mar-9 sizing (Large 30 % / Mid 22 % / Small 15 %) implicitly assumed a Mid-cap-dominated dataset; the May regime is the opposite.
• Priority-tier system's SKIP rules (LONG ≥ 90 % / SHORT ≤ 10 %) become more impactful when signal density is low. When ~100 % of markets passed the gate (Jan), a high-conviction skip removed 1-2 signals per month from a deep pool. When 1.9 % pass (May), removing any signal at all may mean zero entries that week. The bias-gauntlet's "full-population spillover" check (memory §K item 4) must be re-run with the new signal-density realism.
• The exposure-cap "rare day" classifier (Finding #285 Phase B subsection above) is now even more clearly mis-calibrated. The May-7 operator quote "most days become rare days..." no longer describes "22 of 34" but rather "essentially all days are rare days" — in May 2026 through today there has been exactly 1 entry total (HPQ, 2026-05-27) across all trading days, so zero days had the 2+ entries the rule's normal/busy brackets are predicated on. The rule's normal/rare distinction has effectively no current production signal to distinguish. (Specific trading-day count + entries-per-day distribution to be re-derived during Phase B simulator work; the qualitative claim is robust to the precise denominator.)
• Kalshi-as-alternative is now a more pressing operator-decision (chat-documented May 19, deferred at the time; see docs/AGENT_HANDOFF.md §11 carry-over). With PM signal density at ~1.9 % of historical norm, the question of whether to supplement with Kalshi's ~216 earnings-mention markets (~$1.24M cumulative open; KXTSLA-style KPI markets only for TSLA + AAPL, no general "Will X beat?" markets) becomes structurally relevant.
• Data-capture gap surfaced during HPQ investigation (NOT actioned today): trading_positions.entry_volume_24h and final_volume_24h capture the 24h-window volume, but the cumulative-volume value that the gate actually decides on (max(markets.volume_total, Gamma cumulative), Finding #282) is NOT persisted as a column. For forensic-grade replay of "what cumulative did the gate see at this trade's entry instant?" we'd need a new field, e.g. effective_cumulative_volume_at_entry. Possible future surfacing as a Finding once Phase 3 simulator design is in flight (the simulator will need this field for realistic counterfactual replays); deferred per memory §C "mention, don't expand scope unilaterally."
• Why this was chat-only until May 28: original observation chat-dumped May 19, 2026 during an unrelated dashboard volume investigation; informally labeled "Finding #298 candidate" at the time; operator chose at the time to defer canonical-Finding form pending Phase 3 framing. May 28 work re-quantified the evidence on current data, surfaced the structural decision via AskQuestion, and folded it here.
• Files (this amendment):
• docs/FINDINGS.md (this subsection + status-line update at the top of #279).
• docs/STRATEGY_PLAYBOOK.md §1 verdict-banner + §7 deferred-bundle row for sizing-priority-v2.
• docs/AGENT_HANDOFF.md §11 (May 28 afternoon block).
• docs/CONTEXT.md (Last-Updated header + "What changed this week" bullet).
• No code changes; no integrity-monitor rule changes; no build_action_queue() changes (the existing sizing-priority-v2 row already says DEFERRED-POST-PHASE-3 and now references this subsection as its framing premise).

Finding 272 — volume_24h_at_entry NULL for 76 consecutive trades since Feb 16 — Polymarket Gamma API drops volume24hr field intermittently (RESOLVED)

• Severity: Medium (audit completeness; doesn't affect trader correctness — volume_total is the gate, not volume_24h)
• Status: done — Apr 27, 2026 ~19:30 UTC.
• Trigger: RCA on VZ trade #120 surfaced as part of Finding #265's investigation. Operator asked: "the entry path's volume_24h_at_entry is broken — VZ #120 has NULL while VZ from Jan 29 had $1,346.86. Why?"
• Per-week regression timeline (trade_audit_log aggregated by entry_time week): Week 2026-04-20 → 2026-02-16: 76 trades, 0 with_volume, 0% coverage Week 2026-02-09 → 2026-02-09: 11 trades, 8 with_volume, 73% coverage Week 2026-02-02 and earlier: ~100% coverage Hard cutover at week of 2026-02-16. 76 consecutive trades NULL.
• Root cause (smoking gun — live Gamma probe Apr 27, 2026): curl https://gamma-api.polymarket.com/events/pagination?tag_id=1013&active=true&archived=false → "volume": 4894.41 → "volume24hr": None ← THIS IS WHY → "volume24Hr": None → "volume1mo": 4894.41 → "volume1wk": 4894.41 → "volume1yr": 4894.41 Polymarket's Gamma API stopped reliably returning volume24hr for markets in the events response (~50% null rate per current probe; was ~100% null for many weeks). All long-term volume fields (volume, volume1wk, volume1mo, volume1yr) are populated. The 24h field is the unreliable one.
• Cascade through our code:
• continuous_collector.sync_markets_from_gamma writes m.get('volume24hr') or 0 to markets.volume_24h. When Gamma returns None, we write 0. Live state: 750 / 848 recently-synced markets (88%) have volume_24h = 0.
• signal_generator.get_market_volume() reads markets.volume_24h. With v24 = float(row.volume_24h) if row.volume_24h else None, 0 is treated as None because Python truthiness.
• signal.volume_24h = None.
• trading_orchestrator._persist_position() and position_manager.persist_position() both have 'volume_24h': to_decimal(signal.volume_24h) if signal and signal.volume_24h else None — None propagates → INSERT writes NULL.
• Why price_snapshots.volume_24h IS frequently populated for the same markets:
• The price-snapshots collector fires every ~5 minutes; even if Gamma returns None on one call, the next call ~5 min later often returns a positive number.
• For VZ specifically, price_snapshots at the entry minute 19:55:19 Apr 24 captured volume_24h = 550.95 — a positive value that the markets-table sync had missed.
• Why volume_total is unaffected: Gamma's volume (cumulative) field is reliable (~100% populated). volume_total capture and the trader's volume gate (SIGNAL_MIN_VOLUME_TOTAL = 2000) work correctly. Only the 24h companion metric was lost.
• Fix shipped: src/earnings/signal_generator.py:get_market_volume() falls back to most-recent price_snapshots.volume_24h (within 6 hours, positive only) when markets.volume_24h is 0/null. The snapshot collector's higher cadence catches the field when Gamma briefly returns it; the resulting value is the closest "near-entry" 24h volume we have on hand.
• Backfill applied: one-shot SQL UPDATE picked the closest positive price_snapshots.volume_24h within ±6 hours of entry_time for every NULL-volume_24h_at_entry row. Result: total_trades: 117 with_volume_24h: 114 (97%) still_null: 3 (markets without nearby positive snapshots) VZ #120 now has volume_24h_at_entry = 550.95 (matches the 19:55 snapshot one minute before entry).
• Memory §B 5-step verification:
• Step 1 trace: trader entry → signal_generator._build_trading_signal() → get_market_volume(condition_id) → returns (v24, vt) → assigned to TradingSignal.volume_24h → position_manager._persist_position / trading_orchestrator._persist_position reads signal.volume_24h → INSERT.
• Step 2 simulate: at entry time T, markets.volume_24h = 0. Helper now queries price_snapshots with positive value within last 6h. Returns 550.95 (VZ case). signal.volume_24h = 550.95. INSERT writes 550.95.
• Step 3 lines: src/earnings/signal_generator.py lines 411-431 (the v24 computation + new fallback).
• Step 4 failure modes: (a) no snapshot in last 6h with positive value → v24 stays None → INSERT writes NULL (acceptable; same as old behavior). (b) snapshot has stale positive value from a prior earnings cycle → fallback grabs that — limited by 6h window, so worst case is 6h-old data, not days-old. (c) DB connection fails inside fallback → exception propagates out of get_market_volume's try → method's existing except clause returns (None, None).
• Step 5 acceptance: next trade entry should have trade_audit_log.volume_24h_at_entry IS NOT NULL even when markets.volume_24h = 0. Verifiable via SELECT id, symbol, volume_24h_at_entry FROM trade_audit_log ORDER BY entry_time DESC LIMIT 5 after the next earnings entry.
• Memory §E same-pattern check: searched for other readers of markets.volume_24h. Only signal_generator.get_market_volume() and dashboard app.py use it. Dashboard already reads db_volume_total for the gate decision (Finding #265.v2); 24h is only displayed as a transparency field.
• Memory §M cross-interface: signal.volume_24h is consumed only by the two persist paths (position_manager.py:381, trading_orchestrator.py:2468) and a debug log line. No dashboard / Jinja / JS dependency.
• Files:
• src/earnings/signal_generator.py (8-line fallback in get_market_volume)
• SQL backfill (one-shot via psql, not committed)
• docs/FINDINGS.md (this entry)

Finding 274 — Phase 5 logrotate config had no size early-trigger; runaway logs could balloon multi-GB between daily fires (RESOLVED May 9, 2026 — maxsize 500M, NOT size 500M, per pre-flight dry-run on prod)

• Severity: Low (under normal load, daily rotation already truncates collector.log every midnight UTC; the missing directive only mattered for runaway-log scenarios — exactly what Apr 27 caught at 8.7 GB)
• Status: RESOLVED May 9, 2026 LATE evening — pre-Phase-3 housekeeping batch, third commit after #270 + #286.
• Trigger: Apr 27 18:00 audit found collector.log at 8.7 GB despite Phase 5 logrotate config installed earlier the same day (commit 1c11cb3, deploy 13:43 UTC). The 8.7 GB had accumulated because the daily timer hadn't fired since deploy. Forced logrotate -f truncated 8.7 GB → 6.6 KB, proving the rest of the config was sound.
• Root cause: /etc/logrotate.d/polymarket declared daily only — between calendar rotations the log can grow unbounded (collector.log routinely hits ~6 GB / day in steady state). A bug-induced runaway logging spike could balloon to dozens of GB between fires.
• Pre-flight discovery (corrected the original recommendation): original Apr 27 finding text recommended "add size 500M (or maxsize 500M to keep the daily cadence as the upper bound)". May 9 pre-flight logrotate -d dry-run on prod logrotate 3.21.0 + man logrotate re-read confirmed the semantics had been described inversely:
• daily + size 500M → size REPLACES the time-based count. Dry-run output: "log does not need rotating (log size is below the 'size' threshold)" — no mention of daily cadence. Result: small logs (backup.log <1 KB/day, integrity_monitor.log ~5 KB/day) would NEVER rotate, growing forever. Wrong directive for our needs.
• daily + maxsize 500M → maxsize ADDS a size early-trigger ON TOP of the daily cadence. Dry-run output: "rotating pattern: ... after 1 days (14 rotations) ... log files >= 524288000 are rotated earlier". Correct belt-and-suspenders semantics.
• Fix shipped: added maxsize 500M after daily in deploy/logrotate/polymarket, plus a 12-line comment block citing this finding's pre-flight verification so a future agent doesn't confuse size vs maxsize again.
• Memory §B 5-step verification:
• Step 1 trace: /etc/cron.daily/logrotate (or logrotate.timer) → reads /etc/logrotate.d/polymarket → for each *.log file, check (age >= 1 day OR size >= 500M) → if either, rotate via copytruncate → keep last 14 archives compressed.
• Step 2 simulate: collector.log at 600 MB at 16:00 UTC (mid-day spike). Next logrotate run at midnight UTC → daily check triggers (≥1 day) AND maxsize check triggers (≥500M) → rotate. With prod's daily-only logrotate.timer, that's the same midnight fire either way; the maxsize value is mostly insurance against a future change to hourly cadence + ensures behavior is documented.
• Step 3 lines: deploy/logrotate/polymarket:33-34 (added maxsize 500M between daily and rotate 14).
• Step 4 failure modes: (a) maxsize not supported on logrotate <3.8.0 — verified prod is 3.21.0, well past that. (b) Threshold too low (e.g., 50M) → constant rotation churn → decided against, 500M is original analysis target. (c) Threshold too high (e.g., 5G) → mostly equivalent to no directive → decided against. 500M sized for ~6 GB/day collector → ~12 daily rotations on heavy days IF the timer were hourly; on prod's daily timer it stays as a single daily rotation but caps the maximum file size BETWEEN rotations to 500M (would prevent the 8.7 GB Apr 27 incident). (d) Multiple rotations on same day → rotate 14 keeps last 14 archives → on heavy day with hourly timer, 14 archives could cover only ~28 hours instead of 14 days. ON PROD'S DAILY TIMER this isn't a concern (1 rotation per day → 14 days retention preserved). Documented in the file's comment block for future-self.
• Step 5 acceptance: post-deploy cat /etc/logrotate.d/polymarket shows the maxsize 500M line. logrotate -d /etc/logrotate.d/polymarket dry-run shows "log files >= 524288000 are rotated earlier" (vs. previous "rotated, old logs are removed" without the early-trigger phrase). logrotate -f /etc/logrotate.d/polymarket (force run) succeeds without errors.
• Memory §E same-pattern check: only one logrotate config file exists (deploy/logrotate/polymarket). The *.log glob covers every service's log under /var/log/polymarket/. No other rotation surface in repo or prod.
• Memory §H deploy: deploy_from_github.sh line 124-127 already auto-installs logrotate config (install -m 0644 deploy/logrotate/polymarket /etc/logrotate.d/polymarket). No service restart required (logrotate reads the file on each invocation, not at boot).
• Forward-looking note (NOT fixed in this commit per §C surgical scope): prod's logrotate.timer is daily-only (next fire Sun 2026-05-10 00:00:00 UTC). On distros that run logrotate hourly via logrotate.timer, maxsize 500M would trigger early between calendar days; on this prod the early-trigger and the daily cadence fire at the same moment (midnight UTC). If we ever want true intra-day capping, override logrotate.timer to hourly via /etc/systemd/system/logrotate.timer.d/. Not done in this commit because (i) it's a separate concern, (ii) current collector.log at 73 MB is well below 500M with normal daily growth, and (iii) the maxsize directive is the right structural fix regardless of timer cadence.
• Files:
• deploy/logrotate/polymarket (1-line addition + 12 comment lines documenting the size-vs-maxsize gotcha)
• docs/FINDINGS.md (this entry)
• docs/AGENT_HANDOFF.md §11 (May 9 LATE evening housekeeping block extended)
• docs/CONTEXT.md (What changed this week — May 9 LATE evening line)

Finding 273 — Phase 2.1 PM-snapshot enrichment fires schema label but leaves new fields NULL on signal_emitted rows (DOCUMENTED — deferred to post-Phase-3 — medium)

• Severity: Medium (Phase 3 unified simulator depends on these fields; without them, slippage estimates degrade silently)
• Status: DOCUMENTED Apr 27, 2026 — formally deferred to post-Phase-3 May 9, 2026 LATE evening (operator decision: pair fix with Phase 3 unified simulator ship in lockstep — no live consumer today, no current impact, fix is likely to be re-touched at Phase 3 if shipped now). Joins the May 26 deferral bundle alongside Findings #278 (regime detection v2), #279 + #285 Phase B (sizing + priority + exposure-cap re-eval), and #283 (depth-gate v2).
• Trigger: Apr 27 18:00 §B audit Step 5: polymarket_opportunity_log since deploy of 578dee5 (Apr 26 22:00) shows: skip_stage | schema | n | with_pm_best_bid ----------------+--------+----+----------------- generate_signal | pm_v2 | 89 | 0 timing | | 76 | 0 signal_emitted | pm_v2 | 8 | 0 depth | pm_v2 | 1 | 1 Of 8 signal_emitted rows (where a real signal was generated, hence ConvictionResult MUST have been computed), 8/8 have pm_best_bid NULL. The schema label is bumped, but the value-add fields are missing. The single depth-skip row DOES have pm_best_bid populated because that branch fetches the orderbook unconditionally as part of the depth check.
• Root cause hypothesis (NOT verified — verification deferred to Phase 3 lockstep fix): at the four _emit_pm_opportunity_log() call sites in signal_generator.py, conviction_result is passed but ConvictionResult.best_bid / best_ask may be None when the orderbook fetch path was bypassed (e.g., conviction computed from price_snapshots.yes_price only via the snapshot-based fallback per Finding #246's post-event backfill). The depth-skip rows DO have these fields populated because that branch fetches the orderbook unconditionally as part of the depth check.
• Impact today: zero. The Phase 3 unified simulator that consumes these fields hasn't shipped. The label-vs-value mismatch can be fixed before Phase 3 lands.
• Why deferred to Phase 3 lockstep ship (May 9, 2026 reasoning):
• No live consumer. Fixing the writer without the reader gives nothing observable today — no §G probe is meaningful, no acceptance test fires.
• Phase 3 will touch the same surface (conviction_calculator.ConvictionResult + _emit_pm_opportunity_log). A fix-now would likely be re-touched at Phase 3 anyway, doubling change cost and risking churn.
• Risk of fix-now mismatching Phase 3 needs. If Phase 3 wants L2 book data (multiple price levels) or a mid_price field or a slippage_bps derived field, a fix-now that only adds best_bid / best_ask would be insufficient and reworked.
• Not a Phase 3 GATE. §K bias-gauntlet floor on counterfactual evidence is met by the other forward-data work (Finding #243's full snapshot JSONB + #246's price_snapshots backfill + #247's SEC EDGAR earnings timestamps + #240's equity 1m bars). #273 fills in marginal slippage realism, not foundational evidence.
• Operator's prior guidance (Apr 27 entry footer): "Phase 3 unified simulator (consumer of these fields) hasn't shipped yet — can be fixed in lockstep." Reaffirmed May 9.
• Phase 3 lockstep instructions (May 26 target — for the agent who picks up Phase 3):
• Trace the 4 _emit_pm_opportunity_log() call sites in src/earnings/signal_generator.py. Identify which branches (a) already have a fresh orderbook fetch in scope vs (b) fall back to price_snapshots.yes_price-only conviction.
• For (b)-class branches, add a markets.token_id_yes/no orderbook lookup just before the _emit_pm_opportunity_log() call so ConvictionResult.best_bid / best_ask / spread are populated. Cache the lookup if multiple consumers in the same call frame.
• Decide schema treatment for legacy / API-failure paths: either keep snapshot_schema='pm_v2' and accept partial NULLs (with a documented is_pm_v2_complete flag), or downgrade those rows to snapshot_schema='pm_v1.5'. Phase 3 simulator's tolerance for partial data drives this choice.
• Implement and verify in lockstep with the simulator's data-shape requirement, NOT separately.
• Acceptance probe (run AFTER Phase 3 ship): sql SELECT skip_stage, COUNT(*) AS n, COUNT(pm_best_bid) AS with_bid, COUNT(pm_best_ask) AS with_ask, COUNT(pm_spread) AS with_spread, ROUND(100.0 * COUNT(pm_best_bid) / COUNT(*), 1) AS bid_pct FROM polymarket_opportunity_log WHERE snapshot_schema = 'pm_v2' GROUP BY skip_stage ORDER BY n DESC; Pass criteria: bid_pct = 100.0 on all rows for skip_stage IN ('signal_emitted', 'generate_signal'). Tracking this across the next 30 days post-Phase-3 should show monotonic 100% on new rows; legacy rows (pre-fix) keep their NULLs as a "before" baseline.
• Files to touch (when Phase 3 ships): src/earnings/conviction_calculator.py (where ConvictionResult is built) + src/earnings/signal_generator.py:_emit_pm_opportunity_log (the merge step). Possibly src/data_tracker/migrations/ if schema_version or completeness flag is added.
• Memory §J / §K / §N compliance: canonical narrative kept here per §J (no parallel duplication). Per §N, this entry is bundled in docs/AGENT_HANDOFF.md §11 May 26 deferral list. Per §K, no Action Queue row added in scripts/strategy_performance_analysis.py:build_action_queue() because #273 is data-capture infrastructure, not a strategy rule (the Action Queue tracks strategy-level decisions; the Testing Findings tab tracks engineering-level findings — both render from docs/FINDINGS.md via /api/architecture-findings, but the Action Queue is a curated subset).

Finding 270 — StartLimitIntervalSec/Burst in [Service] instead of [Unit] across 5 live systemd unit files (RESOLVED May 9, 2026)

• Severity: Low (only matters if a service crash-loops; rate-limit semantics silently fall back to systemd default of 5 starts / 10s instead of the intended values)
• Status: RESOLVED May 9, 2026 evening — original report Apr 27, 2026 only documented polymarket-trades-backfill.service; memory.md §E same-pattern check on May 9 surfaced 4 additional live instances + 1 install-helper template + 2 decommissioned-watchdog occurrences (left untouched per §C).
• Trigger: journalctl on prod repeatedly logged Unknown key name 'StartLimitIntervalSec' in section 'Service', ignoring. Live 24h-window grep on May 9 (before fix) revealed the warning fires on every daemon-reload for 4 distinct active services (trades-backfill, integrity-monitor, collector, conviction-tracker) plus earnings-trader (less frequent because it hadn't been reloaded recently — bug still in file).
• Root cause: StartLimitIntervalSec=N and StartLimitBurst=N are properties of the unit's restart-rate-limiting policy and belong in the [Unit] section per man systemd.unit. systemd silently ignores them in [Service]. Result: Restart=on-failure/Restart=always falls back to the systemd default (5 starts in 10s) instead of the per-unit intended values (e.g., 10 starts in 3600s for trades-backfill, 5 starts in 300s for the long-running daemons).
• Memory §E same-pattern check (May 9, 2026 evening): grep StartLimit across deploy/ + systemd/ returned 7 hits across 7 files. Cross-checked against live systemctl is-active on prod to filter dead code: | Repo file | Live state | Action | |---|---|---| | deploy/earnings-trader.service | enabled, active | FIXED | | deploy/systemd/polymarket-integrity-monitor.service | enabled, active | FIXED | | deploy/systemd/polymarket-trades-backfill.service | enabled, inactive (Finding #269) | FIXED (original scope) | | deploy/systemd/polymarket-collector.service | enabled, active | FIXED | | systemd/earnings-conviction-tracker.service | enabled, active | FIXED | | deploy/install_trader_service.sh (heredoc) | install-helper template | FIXED (consistency w/ trader) | | deploy/polymarket-watchdog.service | not-found on prod (decommissioned per #260c) | INTENTIONALLY NOT TOUCHED IN #270 (§C — dead code) — subsequently DELETED per Finding #290 (May 10, 2026) | | deploy/polymarket-watchdog-v3.service | not-found on prod (decommissioned per #260c) | INTENTIONALLY NOT TOUCHED IN #270 (§C — dead code) — subsequently DELETED per Finding #290 (May 10, 2026) |
• Memory §B 5-step verification:
• Step 1 trace: systemd parses unit file on daemon-reload → emits warning for unknown keys → Restart=on-failure falls back to systemd default rate-limit (5 starts in 10s) → if a service ever crash-loops, retry semantics differ from the per-unit intended StartLimitIntervalSec / StartLimitBurst values.
• Step 2 simulate: pre-fix journalctl 24h window showed 4 distinct services emitting the warning on every parse. Post-fix daemon-reload should produce zero warnings.
• Step 3 lines: each of the 5 affected files moved 2 lines from [Service] to [Unit] (after the Wants=/After= lines, before the [Service] block), with an inline comment citing this finding.
• Step 4 failure modes: (a) systemd version difference — man systemd.unit confirms [Unit] is the correct section since systemd 230+; prod is well past that. (b) Restart attribute reads [Service] while StartLimit reads [Unit] — that IS the systemd convention; the two sections cooperate, not duplicate. (c) Comment placement could break parsing — verified no, comments are line-level and the directives are still on their own lines.
• Step 5 acceptance: post-deploy journalctl --since "1 minute ago" | grep "Unknown key name" returns 0 lines after final daemon-reload.
• Memory §H/§I deploy:
• deploy_from_github.sh already auto-installs polymarket-trades-backfill.service (line 93) and polymarket-integrity-monitor.service (line 104). The other 3 affected files (earnings-trader, polymarket-collector, earnings-conviction-tracker) are NOT installed by the deploy script — only restarted (or not managed at all in conviction-tracker's case). Per §H fallback procedure, these 3 were install -m 0644'd manually from the just-pulled repo to /etc/systemd/system/ on prod, then systemctl daemon-reload. Deploy-script gap noted in docs/AGENT_HANDOFF.md §11 as a future cleanup candidate (intentionally not extending the deploy script in this commit per §C surgical-changes rule).
• No service restarts triggered for the StartLimit fix specifically — the directive change only affects future crash-recovery behavior, which the current daemon-reload is sufficient to pick up. The trader/collector/conviction-tracker services continue running uninterrupted.
• Files:
• deploy/earnings-trader.service (2-line move)
• deploy/systemd/polymarket-integrity-monitor.service (2-line move)
• deploy/systemd/polymarket-trades-backfill.service (2-line move)
• deploy/systemd/polymarket-collector.service (2-line move)
• systemd/earnings-conviction-tracker.service (2-line move)
• deploy/install_trader_service.sh (2-line move in heredoc template)
• docs/FINDINGS.md (this entry)
• docs/AGENT_HANDOFF.md §11 (May 9 evening housekeeping block)

Finding 323 — Conviction-band exclusion re-validation (raise the 90% cap? trade the dead zone? loosen small-caps/shorts?) → ALL exclusions HOLD (RESOLVED Jun 1, 2026; offline analysis, NO production change)

• Severity: Medium (operator strategy question: are we excluding profitable trades now that we have the integrated simulator?).
• Status: RESOLVED Jun 1, 2026 — operator asked to comprehensively check whether the conviction-band exclusions (LONG ≥0.90 over-confident skip #202; small-cap LONG ≥0.80 #207; small-cap SHORT squeeze; the 0.30-0.70 dead zone) leave money on the table. Verdict: every exclusion holds. Offline; production rules UNCHANGED.
• Method (memory §K/§N): two evidence sources at the $1k gate — (A) universe day-close proxy (earnings_markets.stock_return_day_close; directional edge, NO stops); (B) the bot's stop-aware counterfactual (opp_log.hypothetical_pnl[_pct] — real sizing + 3/4/5% STOP-LIMIT + BMO/AMC timing) for the bands it actually logged as skipped. Full gauntlet: de-dupe by ticker (ex-repeat-tickers) + ex-top-2 + real per-tier $-scaling + split by mcap. Scripts: scripts/analyze_conviction_buckets.py, analyze_excluded_stopaware.py, analyze_raise_cap_gauntlet.py, scripts/phase3_simulator/replay_deadzone.py.
• Findings:
• Raise the LONG ≥0.90 cap? NO. Stop-aware (n=434 logged skips), de-duped + ex-top-2 + per-mcap: Large 0.90-0.95 ex-top-2 NEGATIVE, Mid 0.90-0.95 ex-top-2 NEGATIVE, ≥0.95 NEGATIVE + 47% stop-rate. Only the cross-mcap pool barely survives ex-top-2 (but pooling isn't how the rule applies). The first-glance "+$8k/85% win" was $10k-notional inflation + repeat-ticker concentration (21 evals/7 names) + outliers — the exact §K traps. KEEP #202.
• Dead zone 0.60-0.70 as-LONG? NO. Universe proxy looked great (+$307, ex-top-2 +$139, 58.5% win, n=65) but that is NO-STOPS. Priced with the bot's real stops (replay_deadzone.py, de-duped 15 tickers): −$54.65, 40% win, 40% stop-rate, FAIL-EX-TOP-2. The 3-5% stops eat the edge exactly as #228 predicts. KEEP the dead-zone exclusion.
• Dead zone 0.30-0.40 as-SHORT? NO. n=8 (FAIL-N) + ex-top-2 −$16.72.
• Small-cap LONG ≥0.80 (#207)? KEEP. Universe 0.85-0.90 small = −$135; stop-aware only 3 unique names — insufficient to overturn + small-cap risk.
• Small-cap SHORT (squeeze)? KEEP. n=18/6 names, ex-top-2 negative.
• Takeaway: the strategy is correctly excluding low/negative-EV bands; the apparent edges are no-stops/outlier/repeat-ticker artifacts. No ship. (Per §K, mirrored as a KEPT row in the Action Queue + STRATEGY_PLAYBOOK §2.)
• Files: offline analysis scripts only (committed as reusable tooling); docs/FINDINGS.md (this) + Action Queue (build_action_queue) + STRATEGY_PLAYBOOK.md §2 + CONTEXT + AGENT_HANDOFF. NO src/ or production change.

Finding 324 — Counterfactual P&L units cleanup: hypothetical_pnl_pct (PERCENT) is canonical; fixed a 100× "Avg %" dashboard display bug + honestly labeled the $10k-notional dollars (RESOLVED Jun 1, 2026; dashboard/docs, NO trading change)

• Severity: Medium (operator: "the $10k-notional hypothetical_pnl is stupid — change everything to %"). Surfaced a real display bug.
• Status: RESOLVED Jun 1, 2026 — operator chose the SAFE standardization (no DB migration). Investigation confirmed: (1) hypothetical_pnl_pct already exists and is the scale-invariant truth, in PERCENT units (pricer counterfactual_pricing.py:226 pnl = notional × pnl_pct/100; verified stored avg_abs ~3.6%, range −5%..+38%); (2) the simulator already scales $10k → real correctly (algebraically pct × real_position), so no migration/rewrite needed; (3) the genuinely-misleading surfaces were display-only.
• Fixes (dashboard + docs; §M cross-interface):
• 100× display bug FIXED: the learning-loop "Avg %" columns rendered avg_ret_pct * 100 while avg_ret_pct is already percent → showed e.g. 79% instead of 0.79%. Fixed in both renderers (skipped-signal-cost + volume-gate-counterfactual) → avg_ret_pct.toFixed(2).
• Honest labeling: "Total $" → "Total $ (@$10k notional)" with a tooltip that it is comparability-only, NOT real bot-sized $ (real ~$205-$537/tier); "Avg %" → "Avg % (canonical)".
• Units doc: analyze_skipped_cost note now states pct is canonical PERCENT + total is $10k-notional + real $ = (pct/100)×per-tier.
• Verified NOT in scope / flagged separately: §E grep of avg_ret_pct in the template found a 3rd usage (regime/VIX buckets, L7803) that uses a DIFFERENT field — realized_pnl_pct, which is a FRACTION — and renders WITHOUT ×100, so it appears to under-display (~0.00%). That is a separate suspected unit bug in a different metric; NOT flipped here (out of the hypothetical-P&L scope + needs its own verification). Flagged for a follow-up.
• §B/§G: py_compile + jinja parse + node --check (3 blocks) clean; post-deploy dashboard §G.
• Files: earnings_dashboard/templates/dashboard.html (2 renderers + 2 headers), scripts/strategy_performance_analysis.py (units note); lockstep docs. NO trading change.

Finding 323.b — RuleB re-test of the excluded bands AMENDS #323: ~62-67% of stops are in the ±90min earnings window (vol-driven, conviction-independent) → the LONG ≥0.90 skip (#202) flips POSITIVE under RuleB (RESOLVED Jun 1, 2026; offline; NO production change)

• Severity: Medium — corrects a confounded conclusion in #323. Operator insight (Jun 1): stop-hits are driven by post-earnings volatility (the ±90min window RuleB neutralizes), NOT by conviction, so #323's "stops eat the edge" verdict was confounded by not modeling RuleB.
• Status: RESOLVED Jun 1, 2026 — operator was RIGHT. Re-priced the excluded bands under THREE stop regimes (baseline always-on / RuleB no-stop-window / no-stop) reusing the canonical analyze_finding_a_full simulators (simulate_baseline/simulate_window_only/simulate_no_stop). NEW scripts/phase3_simulator/replay_deadzone_ruleb.py (non-destructive, de-duped by ticker, real per-tier $-scaled).
• Evidence for the mechanism: of all baseline stop-hits, the fraction INSIDE the ±90min release window: 0.60-0.70 LONG 66.7% (4/6), 0.30-0.40 SHORT 66.7% (2/3), ≥0.90 LONG 61.8% (21/34). ~Flat across conviction → stops are vol-spike-driven, ~conviction-independent. Confirms the operator's hypothesis.
• 3-regime results (real-$, de-duped by ticker, ex2 = ex-top-2):
• LONG ≥0.90 (skipped #202) — n=65: BASELINE +$400 / ex2 +$161 / 41.5% win → RULEB +$853 / ex2 +$614 / 47.7% → NO-STOP +$514 / ex2 +$275. RuleB > no-stop > baseline. Under RuleB the over-confident skip is strongly positive + survives ex-top-2. #323's "keep ≥0.90" was a baseline-stop artifact.
• 0.60-0.70 LONG — n=14: BASELINE −$55 → RULEB −$23 (win 43%→57%) → NO-STOP −$17 / ex2 +$1. RuleB helps but it is ~breakeven even fully unstopped → genuinely weak standalone edge (NOT a stop artifact). #323's KEEP holds, but for the right reason (weak signal, not "stops eat it").
• 0.30-0.40 SHORT — n=8: all regimes ex2 ≤ 0; KEEP.
• AMENDS #323: #323's blanket "every conviction-band exclusion holds" is corrected — the LONG ≥0.90 skip (#202) is NOT validated; it is confounded by the always-on stop eating in-window vol-spikes, and flips positive under RuleB. The dead-zone 0.60-0.70, 0.30-0.40, and small-cap exclusions still hold.
• Fine conviction split (replay_high_conv_split.py, operator-requested): the pooled ≥0.90-positive was MASKING a bad extreme band. By sub-bucket (markets / baseline-ex2 / RuleB-ex2): 0.90-0.92 n=16 / −$3 / −$3 (weak — misses are real drops, baseline==RuleB); 0.92-0.95 n=35 / +$12 / +$134 (solid under RuleB); 0.95-0.99 n=16 / −$111 / +$148 (BAD under baseline, rescued by RuleB); 0.99-1.00 n=0. In-window-stop% rises with conviction (57%→53%→80%) — the extreme band is vol-spike-stopped most, so baseline kills it + RuleB rescues it. Implication: relaxing the cap WITHOUT RuleB would import the bad 0.95-0.99; the clean lever is 0.92-0.99 conditional on RuleB active (0.90-0.92 marginal).
• ⚠️ CORRECTION (Jun 1, operator-requested #202-basis check, replay_202_ruleb.py): added a RuleB column to the EXACT original #202 dataset — the 26 real closed LONG trades with conviction_at_entry ≥0.90 (real fills). Result: ACTUAL −$112.25 / 15.4% WR / 73% stop → RuleB −$88.18 / 15.4% WR / 73% stop → NO-STOP −$153.67 / 42.3% WR. RuleB does NOT rescue the real ≥0.90 trades (WR unchanged 15.4%; still deeply negative; no-stop is worse). The +$853/ex2+$614 "flip" above was a DIFFERENT population (post-#202 skipped signals, Mar-May, counterfactually priced) and does NOT generalize to the real fills. Two divergence causes: (1) period/regime — later skipped signals had 44-52% WR vs the early real trades' 15%; (2) counterfactual bar-pricing is ~$33 optimistic vs real fills on the SAME 26 trades (A_base re-sim −$79 vs actual −$112). On the only ground-truth (real fills), #202 HOLDS even with RuleB — ≥0.90 LONGs are directionally wrong ~85% of the time, which RuleB cannot fix. The ≥0.90-relax lead is materially WEAKER than this finding first claimed.
• Coupling + next steps (NO ship — lead downgraded): the counterfactual ≥0.90+RuleB positive is contradicted by the real #202 trades, so relaxing #202 is NOT supported by ground-truth evidence. If ever revisited (post-RuleB-active): must use REAL fills on a current-regime sample (not idealized counterfactuals), per-mcap, capacity-aware, full §K. RuleB's genuine value remains the vol-spike-stop cases (#305 Phase 2), independent of the ≥0.90 question.

Finding 323.c — small-cap LONG 80-90% skip (#207) re-validated real-vs-counterfactual (the #202 playbook applied): counterfactual is a 3-NAME MIRAGE; universe is flat; KEEP (RESOLVED Jun 2, 2026; offline; NO production change)

• Status: RESOLVED Jun 2, 2026 — operator asked to apply the #202 method to the small-cap LONG 80-90% band: dig the original exclusion data, run the simulator counterfactual, compare. Verdict: KEEP #207 — the relax-case is a §K mirage.
• Original basis (the dig): #207 (Mar 9) blocked small-cap LONG ≥0.80 on a Q3 2025 universe backtest — 80-90% = 10 trades / 30% WR / EV +$0.40 (near-zero); 70-80% = 6 / 66.7% / +$30.68 (allowed). Reproduced the methodology on the full 364-market dataset (reproduce_207_backtest.py, archive/stocks_earnings/data/earnings_correlation_analysis.json): 80-90% = 49 names / 38.8% WR / EV +$1.70 / +0.5% avg raw return / 22.4% stop — the WEAKEST/flattest of the three bands (70-80% +2.7%, 90%+ +1.5%). Reproduces #207's "near-zero, not worth the risk" on more data.
• Counterfactual (simulator, polymarket_opportunity_log.hypothetical_pnl, skip_reason small_cap_long_above_80, conv 0.80-0.90, $1k gate): raw n=8 / 86% win / +$1101 / +$137.66 avg / 0 stops — but de-dupes to 3 names: CBT +$235.79 (a +63.4% single-day move on a $372 position — extreme outlier), MBWM the SAME +$173.10 logged 5× (pos=None, scaling unverifiable), IART $0 (×2). De-duped: n=3 / +$408.89 / 2-win+1-wash. Textbook §K traps (repeat-ticker + outlier + suspect scaling).
• No real fills exist: unlike #202 (26 real closed trades), small-cap LONG 80-90% has been fully blocked since Mar 9 — trade_audit_log n=0. The relax-case rests ENTIRELY on 3 counterfactual names. RuleB is irrelevant here (0 baseline stops → nothing to rescue; not a stop artifact).
• Conclusion + lesson: the #202 lesson holds even harder — the rosy counterfactual (+$137/trade) is an artifact of 3 names vs the broad universe's flat +0.5% raw return; with zero real fills to anchor it. KEEP #207. Net: flat at best, not worth small-cap risk.
• Files: scripts/phase3_simulator/replay_deadzone_ruleb.py (offline); docs/FINDINGS.md (this) + Action Queue (conviction-band-revalidation row amended) + CONTEXT + AGENT_HANDOFF. NO production change.

Finding 323.d — Entry/exit-aligned analysis dataset (the real strategy clock, not "24h before") built + exclusions re-validated with per-tier stops: every exclusion HOLDS (RESOLVED Jun 11, 2026; offline; NO production change)

• Trigger: During the RH conviction-mismatch deep-dive (#dashboard vs prob_24h_before), operator asked whether our universe backtests (#207/#323) — which key off prob_24h_before/pre_resolution_beat_prob_24h — could be biased by using a generic "24h before" conviction instead of conviction at the ACTUAL entry time. Asked to rebuild on the real entry/exit clock for both AMC and BMO.
• Status: RESOLVED Jun 11, 2026 — built it. NEW scripts/build_entry_aligned_dataset.py (+ analyze_entry_aligned.py, entry_aligned_dataset.json). For every resolved market it recomputes conviction at the real clock and the stock return + a per-tier stop model over that window. Verdict: every conviction-band exclusion HOLDS on the faithful data.
• Strategy clock (from trading_orchestrator.py): BMO enter 15:58 ET (prev trading day) → exit 10:32 ET (earnings day); AMC enter 10:34 ET (earnings day) → exit 15:57 ET (next trading day). Stops Large 3% / Mid 4% / Small 5% (#202), sizing 30/22/15% of ~$2450 (#211).
• Data + method: price_snapshots (Oct 10 2025→now, 13M rows, sub-minute) gives conviction as a windowed-median of yes_price (robust to thin/crossed point-in-time orderbooks — a plain midpoint at a precise past minute degenerates to 0.001/0.5/0.999; the median fixed it, clean rows 10→361). equity_bars_1m (419 syms) gives entry/exit close + hold-window low/high for the stop model. Artifact filter (conv_entry_valid): ≥3 snapshots, not the ~0.5 no-data default, and within 0.35 of the 24h value (drops degenerate cases like AXP→0.001). 649 markets, 351 clean (2026+).
• Finding 1 — is "24h before" biased? Essentially unbiased on average: mean(entry − 24h) = +0.0005, median |diff| = 0.03, direction flips only 3.8%. BUT ~36% of markets change conviction band between the 24h snapshot and entry. So the old backtests are directionally fine, but ~1/3 of markets sit in a different conviction band than they'd actually be traded in — a real fidelity caveat for band-edge decisions.
• Finding 2 — re-validation with real-clock conviction + per-tier stops ($/trade EV):
• small-cap LONG (#207): 70-80 (allowed) $−12/trade (n=3); 80-90 (blocked) $−1.6; 90+ $+0.9 — flat-to-negative across the board → KEEP #207 (even the allowed 70-80 small band looks weak, n too small to act).
• LONG ≥0.90 (#202): ALL $−0.28/trade (Large +$1.5, Mid −$5.1, Small +$0.9) — ~breakeven with stops → KEEP #202 (not a money-maker; corroborates that stops neutralize the band).
• dead-zone 0.50-0.70: LONG $−3.2, SHORT $−1.3 → negative → KEEP.
• Corroborates #323/#323.c on better data: every exclusion holds.
• Leads (NOT actions): Large LONG 70-80 is the standout (+$16.6/trade, only 29% stopped) — the strategy's bread-and-butter. small-cap SHORT shows +$11.9/trade / 75% win (n=12) on the model — but the squeeze tail-risk the rule exists for is NOT captured by an average, n is tiny, and borrow/locate cost is ignored → not actionable.
• Caveats: per-bucket n is small (3-29); the stop model is bar-based (stopped = −stop%, which UNDERSTATES gap-through losses on overnight BMO gaps); 2026-01+ only (price_snapshots ~100% there). For a publication-grade re-run, feed entry-aligned conviction into the full tick-accurate #323 simulator (cache+Polygon) rather than the bar-low/high approximation.
• Files: NEW scripts/build_entry_aligned_dataset.py, scripts/analyze_entry_aligned.py, scripts/entry_aligned_dataset.json. Offline; NO production change.

Finding 348 — equity-bars collector taken down by a Polygon 429 storm: two daily jobs (equity-bars @ fixed 10:00 UTC + daily resolver @ 06:00 ET = 10:00 UTC in EDT) collided on the shared key at 20 req/s each, no retry/backoff existed anywhere, and rate limits were mislabeled "likely delisted/OTC" (Jul 29, 2026)

• What the operator saw: Integrity systemd_last_run:polymarket-equity-bars.service FAIL (exit 2). First failure of this unit since ≥Jun 1 (18 clean daily runs prior); the exit-2 threshold (#251a) worked as designed — 30/35 fetch jobs failed = 85.7% > 10%.
• Root causes (three, all fixed):
• Schedule collision: equity-bars fired at fixed 10:00 UTC; the daily resolver fires at 06:00 ET — which is ALSO 10:00 UTC during EDT (the timer's own comment revealed it believed 10:00 UTC ≈ 05:00 ET, wrong half the year). Both jobs paced Polygon at --polygon-sleep 0.05 (20 req/s each, 40 req/s combined); the Jul-29 heavy earnings slate (24 markets) made the resolver's load big enough to trip Polygon's soft throttle. The collision was latent all summer.
• No 429/5xx handling: fetch_polygon_minute_bars treated any error as terminal — no retry, no backoff.
• Dishonest classification: the collector logged 429s as "no bars — likely delisted/OTC" (SBUX/KO/V are not delisted) — wrong diagnosis that would mislead future debugging.
• Fixes (Tier A per operator-approved plan):
• Shared-helper backoff (§M single definition — all consumers inherit): fetch_polygon_minute_bars retries 429/500/502/503/504 with 2s/8s/30s exponential backoff; non-transient errors never retry. NEW raise_on_transient=True mode raises PolygonTransientError after exhaustion so callers can classify honestly; default keeps legacy warn-and-return-partial (resolver/equity_bar_cache callers unchanged).
• Honest classification in the collector: "likely delisted/OTC" only on a clean HTTP 200 with zero bars; transient failures logged as retryable and counted separately (transient= in the Done line); the >10% exit-2 threshold now counts hard+transient (post-backoff transients at that rate = genuine pacing/schedule problem → we WANT the flag) but the message distinguishes them.
• De-conflicted schedule: equity-bars timer 10:00 → 11:35 UTC (daily resolver long finished; slot sits between the #343 intraday resolver's :15/:45 light passes), RandomizedDelaySec 600 → 300.
• Pacing: --polygon-sleep 0.05 → 0.25 in all three scheduled units (equity-bars, daily resolver, intraday resolver) — adds ~1-2 min to daily batch runs, irrelevant. The 0.05 default remains fine for one-shot manual backfills that run alone.
• Self-heal: failed jobs re-attempt automatically (bars missing → re-queued); the post-fix rerun backfilled today's 30 missed jobs.
• Files: counterfactual_pricing.py (backoff + PolygonTransientError), equity_bars_collector.py (classification + transient counter), polymarket-equity-bars.timer (11:35 UTC), 3 service units (pacing), NEW tests/unit/test_finding_348_polygon_backoff.py (5 tests), OPERATIONS.md ×2.
• ⚠️ ROOT CAUSE CORRECTED by Finding #350 (Jul 29, 2026 late): the schedule collision (cause 1) was REAL but was NOT the trigger — it had existed every EDT day for months with ZERO 429s (rotated logs: Jul-26 = 0, Jul-27 = 0). The actual trigger was the Polygon/Massive subscription silently dropping to the free tier (5 req/min) between Jul-27 10:07 UTC and Jul-28 ~10:00 UTC (empirically proven: exactly 5 requests succeed then 429 with "upgrade your subscription"). Causes 2-3 and all four Tier-A fixes remain valid hardening — they are why the resolvers kept limping through the downgrade — but the de-conflicted schedule alone could not have fixed Jul-29 (52 equity-bars 429s AFTER the timer move). See Finding #350.

Finding 488 — dormant sniper refuses a missing or retired funder (Oct 14, 2026)

• Severity: Medium (latent). The hard-coded proxy is a retired wallet. The sniper is T2.2 ON-HOLD (docs/CODE_AUDIT_RECONCILIATION_2026_08.md). This commit does not start it.
• Status: IMPLEMENTED-PENDING-DEPLOY. This commit does not deploy and does not restart any unit.
• Problem: Both identical copies (resolution_sniper.py, polymarket_sniper/resolution_sniper.py) defaulted FUNDER_ADDRESS to the retired proxy 0x8cF01785CcDC3107e113aE7452D0381bAe0c2a6b. A start with no env var would use that wallet. No tracked file held the retired signer 0x5C215dD606A016a5892970800534621E8B144061.
• Census (repo only; no SSH): no Python file imports resolution_sniper. src/data_tracker/ (collector, conviction tracker, resolvers) does not reference it. deploy/cron/root.crontab does not. deploy/deploy_from_github.sh does not install or restart a sniper unit. The only unit whose ExecStart is the sniper is deploy/polymarket-sniper.service (polymarket_sniper/resolution_sniper.py, WorkingDirectory=/home/polybot/polymarket-bot), copied by the legacy bootstrap deploy/setup-digitalocean.sh, not by the [hidden] deploy. #457 records the sniper as ON-HOLD with no unit. #473 records polymarket_sniper/ as not run on prod. The live dashboard is run_dashboard.py → earnings_dashboard.app. dashboard/app.py only pgreps the filename.
• Fix: no default. A missing or malformed FUNDER_ADDRESS raises. The retired proxy raises. A PRIVATE_KEY that does not parse raises. A key whose address is the retired signer raises. Values are not logged. Both copies stay byte-identical (#457).
• Changelog: none (§O no: no live trading, measurement, or integrity behaviour).
• §B:
• Trace: module body after load_dotenv() reads FUNDER_ADDRESS, then PRIVATE_KEY only when it is non-empty. Nothing in the collector, conviction tracker, data tracker, cron, or the deploy restart list loads this module.
• Runtime: until an operator fast-forwards [hidden], the checkout stays at the previous SHA. A fast-forward updates the two files on disk. No unit in deploy/systemd/ executes them, so no restart is required for the running collector.
• Lines: the FIX-003 block in both sniper copies.
• Constraints: do not restart services to pick this up; do not run deploy/remote_deploy.sh or deploy/deploy_from_github.sh for this change; do not put a real private key in the test.
• Test: tests/unit/test_finding_488_retired_funder.py — copies identical; empty default; synthetic funder accepted; missing, malformed, and retired funder refused; unparseable key refused; retired signer refused via a fake eth_account. The suite does not import the module, so no existing test needed a synthetic FUNDER_ADDRESS.
• Files: resolution_sniper.py, polymarket_sniper/resolution_sniper.py, NEW tests/unit/test_finding_488_retired_funder.py, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, this entry.

Finding 487 — storage:orderbook_snapshots stays FAIL at 12 GB after the drain finishes; the page stops saying the drain failed (Oct 3, 2026)

• Severity: Low (integrity message only; no threshold change, no archive or insert change, no trader restart).
• Status: DEPLOYED Oct 4, 2026 12:05 UTC (unit polymarket-deploy-20261004T120553Z-812717079, SHA 7c9d6b0; trader restarted 12:06 UTC, integrity monitor 12:08 UTC; Gateway not restarted).
• Trigger / proof: On 2026-10-03 the live orderbook_snapshots hypertable was 18.2 GB inside the 90-day window (warn 12 GB, FAIL at 18 GB). orderbook_archive_log had 264/264 ranges dropped, 0 proof failures, and 0 chunks with range_end older than 90 days. The archive timer's last run processed 0 eligible ranges. apply_drain_grace still returned FAIL with "the OPS-1 drain (#383) should have finished" because today > ORDERBOOK_DRAIN_GRACE_UNTIL (2026-08-29) was checked before the backlog rule. That sentence was false: the drain had finished. The size itself was still over the line (July ~83 MB/day, September ~333 MB/day on the kept window).
• Root cause: #390's post-expiry copy assumed "still big after Aug 29" meant the drain had not finished. The 6–9 GB steady state that justified re-arming at 12 GB was not the density the collector later wrote. #376 and #390 forbid raising the line to silence the guard. Skipping unchanged books is not available: it breaks freshness, conviction windows, and the 7-day depth gate (#383 keeps every row).
• Fix: After grace expiry a size FAIL stays STATUS_FAIL. If archive:orderbook_backlog is OK and eligible_live_chunks == 0, the message says the drain finished and the live 90-day working set is over the line. If the backlog is OK but chunks remain, the message says the drain has not failed. If the backlog is not OK, the message says the drain stalled. In-grace WARN is unchanged. HYPERTABLE_THRESHOLDS_GB for orderbook_snapshots stays 12.0.
• Why this is still inside memory.md: #376/#390 say a threshold bump silences a working guard. #297 says a false page trains people to ignore the channel. #383 forbids downsampling. The FAIL status and the 12 GB line are the guard; only the false drain sentence changes. Deployed Oct 4, 2026 12:05 UTC. The marker is DEPLOYED Oct 4, 2026 12:05 UTC.
• §B:
• Trace: size check FAIL → _check_orderbook_size_with_drain_grace → apply_drain_grace. today > grace_until still returns FAIL. Backlog OK + 0 eligible → expired_drain_finished. Backlog OK + eligible remaining → expired_drain_on_schedule. Backlog not OK → expired_drain_stalled.
• Runtime: integrity monitor restarted 12:08 UTC on Oct 4, 2026. Live storage:orderbook_snapshots is FAIL at 18.4 GB and says the drain finished (0 live chunks older than 90 days). No chunk drop, no insert skip.
• Lines: apply_drain_grace in src/data_tracker/integrity_monitor.py.
• Constraints: do not raise 12 GB; do not downgrade the post-expiry status to WARN; do not re-run the drain; do not skip unchanged books; do not turn on Timescale compression.
• Test: tests/unit/test_finding_390_storage_drain_grace.py — expired + backlog still draining stays FAIL and does not say the drain failed; expired + 0 eligible says the drain finished and stays FAIL; expired + stalled says stalled; in-grace WARN unchanged; wiring still cites 12 GB and #390.
• Files: src/data_tracker/integrity_monitor.py, tests/unit/test_finding_390_storage_drain_grace.py, docs/OPERATIONS.md, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, src/earnings/strategy_changelog.py, this entry.

Finding 486 — NASDAQ calendar GET cannot outlast the trader watchdog (Oct 3, 2026)

• Severity: High (five watchdog deaths on 2026-10-01; the process that came up at 20:03:02 UTC stayed up, and NRestarts stays 5 until a later restart).
• Status: DEPLOYED Oct 4, 2026 12:05 UTC (unit polymarket-deploy-20261004T120553Z-812717079, SHA 7c9d6b0; trader restarted 12:06 UTC; trader:restart_tripwire ok and NRestarts=0; Gateway not restarted).
• Trigger / proof: After deploy polymarket-deploy-20260930T201810Z the trader died five times on 2026-10-01, all os._exit(1) from MAIN LOOP STALL at the 120s watchdog. Stall dumps stall-20261001T195205Z, T195438Z, T195712Z, T195946Z, and T200231Z sit in urllib3 create_connection inside finance_calendars.__get_calendar_query requests.get with no timeout, called from fetch_nasdaq_calendar → verify_earnings_dates → run_bmo_cycle. The first death was during run_scheduled_cycles; the next four were _run_startup_signal_check on the way back up. OFFICIAL_CALENDAR_LOOKAHEAD_DAYS is 21, so one hung GET blocks the whole lookahead. The fifth start at 20:03:02 UTC was still the live PID the next time the tripwire was read. systemd NRestarts counts Restart=on-failure and does not clear until a later manual or deploy restart.
• Root cause: finance_calendars issues requests.get with no timeout. fetch_nasdaq_calendar already turns any exception into {} (source silent, no earnings_time write — #220/#349). The exception never arrived, because the socket had no deadline. A process-global socket.setdefaulttimeout would also cover IBKR in this process, so the deadline is on our own GET.
• Fix: nasdaq_earnings_request calls requests.get(..., timeout=8). get_upcoming_earnings and get_earnings_between stop at a 45s loop budget and do not retry; later days are absent. Worst case is the budget plus one in-flight call (53s), under the 120s watchdog. 21 × 8s would be 168s, which is why the budget exists. A timeout or any other error still returns no rows. The package parser's row shape is kept (symbol index, API columns).
• §E: get_earnings_for_ticker / fetch_yfinance_date is the same class of hangable HTTP and was not the Oct 1 stack. yfinance's get_earnings_dates has no timeout we can pass. Bounding it with a process-wide socket default would hit IBKR. Left unchanged on purpose.
• Why this is still inside memory.md: §F requires timeout= on hangable HTTP, treat failure as degrade, and do not retry in a tight loop. {} writes no date. The watchdog stays 120s. Deployed Oct 4, 2026 12:05 UTC. The marker is DEPLOYED Oct 4, 2026 12:05 UTC.
• §B:
• Trace: verify_earnings_dates → fetch_nasdaq_calendar → EarningsCalendar.get_upcoming_earnings → get_earnings_for_date → nasdaq_earnings_request. Timeout or budget exhaustion returns partial or empty; the verifier's existing {} path does not write earnings_time.
• Runtime: deployed Oct 4, 2026 12:06 UTC. The trader reconciled with IBKR (in sync, 0 stops) and skipped the Sunday holiday. trader:restart_tripwire is ok (NRestarts=0). History: the Oct 1 deaths left NRestarts=5 until this restart.
• Lines: src/ibkr/earnings_calendar.py nasdaq_earnings_request, NASDAQ_REQUEST_TIMEOUT_SEC, NASDAQ_CALENDAR_BUDGET_SEC, _collect_nasdaq_days.
• Constraints: no socket.setdefaulttimeout; no tight retry; no watchdog increase; no prod restart; do not write a date from a single silent source.
• Test: tests/unit/test_finding_486_nasdaq_calendar_timeout.py — timeout= is 8s and the helper source has no setdefaulttimeout; 21 × 8s exceeds 120s while 45 + 8 does not; a timeout returns []; the 21-day loop stops once the budget is spent and does not retry.
• Files: src/ibkr/earnings_calendar.py, src/earnings/earnings_date_verifier.py (comment only), NEW tests/unit/test_finding_486_nasdaq_calendar_timeout.py, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, src/earnings/strategy_changelog.py, this entry.

Finding 484 — IBC Command Server RESTART before last-resort 2FA (heal-1234 step 3) (Sep 27, 2026)

• Severity: High (IBKR stay-connected; Principle 1). Complements #438/#439/#456. Does not change last-resort while the flag is off.
• Status: IMPLEMENTED-PENDING-DEPLOY on branch cursor/launcher-pwd-rotation-62ee from master 2b38199. Flag default off. Monday AutoRestart rotation follow-up (Sunday PWD in launcher.YYYYMMDD.log). This commit does not enable the flag, does not deploy, and does not restart Gateway or trader.
• Trigger / proof: heal-1234 paper P4 (Sun 27 Sep 2026 ~06:55 UTC, command port 7463) and live L7 (~08:10 UTC, port 7462) both sent IBC RESTART and got OK RESTART in progress, then Restart in progress + Login has completed on the token path (initialTokenType=TST, pwd=0 soft=256) with no IB Key 2FA. Mac nc to 7462/7463 timed out (loopback-only).
• Root cause: #438/#439 last-resort is systemctl restart ibgateway-ibc, which kills the JVM and forces a phone 2FA. IBC 3.24.2's Command Server can recycle the Gateway inside the same session token. The trader had no loopback client and no blackout around Monday cash hours / entry-exit waves.
• Fix:
• NEW src/ibkr/ibc_command.py — loopback only (127.0.0.1 / ::1 / localhost), 10 s timeout, allow-list RESTART/PAUSE/RESUME/RECONNECTDATA. STOP/EXIT/QUIT/HALT are refused (those kill the JVM). Reply must start with OK. Default port 7462 (live); paper is 7463 via IBKR_IBC_COMMAND_PORT.
• recycle_gateway_if_flat — when IBKR_GATEWAY_SOFT_RESTART is on, write the soft stamp (state=sending, count incremented) before send, capture the IBC log cursor, then RESTART inside the B1 recycle_lock (held through the token-reuse wait so a concurrent caller cannot systemctl mid-restart). CommandResult.sent is true after sendall; a transmitted-but-unacked or malformed reply takes the accepted path (soft_restarted + may_have_restarted) and never falls through to last-resort. Only a failure before/during sendall falls through. Token reuse does not consume the #439 weekly 2FA budget (own stamp logs/ibkr_last_soft_restart.json, ≤3/UTC-day, 20 min cooldown; leftover sending counts as spent). After 3/day or during that cooldown, last-resort may still run if its own budget allows.
• Blackout only while the flag is on so Sat 20:00 UTC last-resort tests stay green with the flag off: Monday 13:30–20:15 UTC freeze; weekday entry 14:25–14:40 and exit 19:45–20:05 UTC (plan.md); daily IBC AutoRestart 05:45–06:30 UTC. Saturday is weekday()==5, so Sat 20:00 is not an exit wave. Blackout is folded into recycle_defer_reason so _last_resort_recycle_ready stays false and the client keeps IB() probing. Token alive (Sunday 01:00 ET until initialTokenType=PWD then IBC Login has completed, DST-aware) sits next to soft_restart_own_defer and gates only IBC RESTART. Last-resort systemctl restart ibgateway-ibc is a fresh PWD login (#430) and must still run when the weekly budget allows. initialTokenType= is launcher-only (heal-1234 L5 /opt/ibgateway/Jts/launcher.log via IBKR_LAUNCHER_LOG_PATH, plus this week's launcher.YYYYMMDD.log siblings after Monday 06:00 AutoRestart rotation — prod 28 Sep 2026 06:33 UTC: live file 25230 bytes of Monday TST, Sunday PWD only in launcher.20260927.log). TST does not count. Real IBC lines are IBC: Login attempt: 1 with no token type. Public recycle_defer_reason() does not skip the weekly/7d budget (that would suppress in-place IB() resets during the soft cooldown). Pass skip_weekly_budget=True only from recycle_gateway_if_flat's soft branch; _mark_recovery_exhausted still calls recycle when the flag is on and weekly-budget is the only defer.
• RecycleResult.soft_restarted — _maybe_standing_recycle consumes the attempt (with ok / pending_late_login / may_have_restarted) so an accepted or transmitted-unacked RESTART whose login times out does not retry into systemctl.
• Config + GUARD: IBKR_GATEWAY_SOFT_RESTART default false; true/1 needs # GUARD-OVERRIDE-UNTIL (inverted vs AUTO_RECYCLE). Daily cap 1–3; cooldown 600–3600 s.
• §B:
• Trace: IB() reset fail → _last_resort_recycle_ready → recycle_defer_reason (age / Sunday quiet / freeze-when-flag-on / auth-in-flight; weekly budget not skipped on the public predicate; token-alive is not in this predicate) → in-place IB() resets continue when last-resort is not ready → _mark_recovery_exhausted still calls recycle when the flag is on and weekly-budget is the only defer → recycle_lock → token-alive / own-defer skip RESTART and fall through → else soft stamp sending → IBC RESTART on 127.0.0.1:7462 (sent after sendall) → wait Restart in progress then Login has completed + :4001 from the pre-send log cursor → systemd-run trader bounce. Transmitted-unacked never systemctl. Last-resort stamp untouched on token reuse. Flag off → identical #438 last-resort (including Monday freeze and Sat 20:00). Flag on + realistic IBC log + no launcher PWD + weekly budget available → no RESTART, systemctl still runs.
• Runtime: this commit is inert (flag off). After a dated enable: a wedged Java midweek gets token-reuse instead of a phone 2FA, except Monday 13:30–20:15 UTC and the entry/exit waves 14:25–14:40 / 19:45–20:05 UTC (keep probing). Sunday 01:00 ET until launcher.log or this week's launcher.YYYYMMDD.log shows this week's first PWD + IBC Login has completed: skip RESTART, last-resort still may run. Sunday 06:00 AutoRestart stays the planned weekly 2FA. A spent #439 budget still allows IBC RESTART without suppressing IB() resets. Dated enable: confirm week_token_alive_reason is None against live launcher.log and the newest launcher.YYYYMMDD.log sibling plus IBC after the weekly login.
• Lines: src/ibkr/ibc_command.py (NEW, CommandResult.sent); src/ibkr/gateway_recycle.py _soft_restart_enabled / soft_restart_blackout_reason / week_token_alive_reason / soft_restart_own_defer / classify_ibc_login_path / _ibc_log_cursor / wait_ibc_needle / recycle_defer_reason(skip_weekly_budget=) / recycle_gateway_if_flat / _complete_soft_restart; src/ibkr/client.py _last_resort_recycle_ready / _mark_recovery_exhausted / _maybe_standing_recycle soft_restarted; Config + GUARD.
• Constraints: never STOP; never non-loopback; never enable the flag in this commit; never fall through to systemctl after an accepted or transmitted-unacked RESTART; stamp before send; do not skip weekly budget on the public predicate; do not mute trader:ibkr_request_path; do not invent net-liq; do not consume #439 on token reuse; do not treat Saturday 20:00 UTC as a weekday wave; no RESTART from Sunday 01:00 ET until launcher.log or this week's launcher.YYYYMMDD.log shows this week's first PWD login; do not refuse last-resort because the token is dead; do not read initialTokenType= from IBC logs; do not treat initialTokenType=TST as the week's first full login.
• Test: tests/unit/test_finding_484_gateway_soft_restart.py — STOP/non-loopback refused; sent on timeout vs connect-fail; P4 token vs PWD fixtures; Monday freeze / waves / AutoRestart; exit wave 19:45–20:05 including 19:47/19:50 send nothing; Sat 20:00 is not a wave; flag-off Sat 20:00 and Monday freeze still last-resort; flag-on freeze/waves refuse both; success without systemctl/budget; fail-send fallthrough; transmitted-unacked no systemctl; stamp written before send; public defer still weekly-used when budget spent (IB() resets continue); budget-spent still soft; 3/day then last-resort; 20-min cooldown; accepted-RESTART login timeout no systemctl; Sunday 01:00 ET token-alive (EDT/EST); #456 IBC shape alone does not open (Sun/Tue/Fri same defer); spliced PWD in IBC ignored; launcher PWD + IBC login opens; Monday AutoRestart rotated layout (launcher.log TST + launcher.20260927.log PWD) opens, live TST alone does not, TST in the rotated sibling does not count, last week's dated sibling does not open; flag-on Monday 06:33 with rotated PWD sends RESTART; flag-on midweek realistic IBC still systemctl; public defer is not token-alive; pre-send shared baseline sees a fast Login has completed.
• Files: NEW src/ibkr/ibc_command.py, src/ibkr/gateway_recycle.py, src/ibkr/client.py, src/earnings/config.py, src/data_tracker/integrity_monitor.py, NEW tests/unit/test_finding_484_gateway_soft_restart.py, tests/unit/test_config.py, tests/unit/test_finding_468_checkout_hygiene.py, .gitignore, .env.example, .cursor/memory.md, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, docs/OPERATIONS.md, docs/IBKR_CONTINUITY.md, src/earnings/strategy_changelog.py, this entry.

Finding 483 — Post-deploy #467 docs review gap: WALLET_NAGI leaked on /api/btc5m/docs/doc_map; central identifier masking, comprehensive route sweep (Sep 27, 2026)

• Severity: Medium — a non-secret but sensitive identifier (a wallet address prefix), not a credential; the dashboard's own secret-value guard (#467.A) was unaffected. Same finding family as #467.D (identifier masking), found by the post-deploy docs review the operator ran against the live deploy (#467 in master 3138aca, deploy #11).
• Status: IMPLEMENTED-PENDING-DEPLOY on a new branch off master 3138aca. No deploy in this session; prod stayed read-only.
• Trigger: the post-deploy review found /api/btc5m/docs/doc_map still served the raw WALLET_NAGI value (strategies/btc5m/src/facts.py's benchmark-maker wallet prefix, [REDACTED]) in doc prose (strategies/btc5m/docs/DOC_MAP.md: "Nagi [REDACTED]…."), even though /api/btc5m/state already masks the SAME constant per-route (_mask_wallet(facts.WALLET_NAGI)).
• Root cause: the doc-serving routes render doc PROSE, not an API payload, so there is no per-field call site to mask at — #467.D relied entirely on secret_redaction._collect_known_identifier_literals() (the central belt-and-suspenders backstop that redacts known identifier VALUES from every response, docs included) to catch this. That function collected btc5m's wallet/host constants from a HAND-MAINTAINED TUPLE OF ATTRIBUTE NAMES ("WALLET_MAIN", "WALLET_TITUS", "EXECUTION_HOST", "EXECUTION_HOST_IP", "EXECUTION_TREE_PATH") — a separate list from the per-route masking call sites. Whoever added WALLET_NAGI to facts.py had no reason to know this second list existed and needed updating too: a "forgot to add it to the list" gap, not a masking-logic bug.
• Fix — central masking, not per-route, per operator direction:
• earnings_dashboard/secret_redaction.py: _collect_known_identifier_literals() now DISCOVERS every WALLET_* constant in facts.py by NAMING CONVENTION (name.startswith("WALLET_"), plus the three exact names EXECUTION_HOST/EXECUTION_HOST_IP/EXECUTION_TREE_PATH) instead of a hand-listed tuple. A future WALLET_<name> is covered with zero code changes elsewhere — this is what "mask identifiers centrally, for every public response, JSON APIs included, rather than per route" means in practice: the ONE place that has to stay correct is a naming convention, not a list a human has to remember to extend. Per-route structural masking (_mask_wallet, mask_identifier) is unchanged and still runs FIRST for the nicer partial-reveal UX on structured JSON fields (0x06…7c0, not a blunt [REDACTED]) — central redaction is the backstop that guarantees zero raw leaks even when a response surface (like a doc route) has no per-field masking of its own.
• Second hit, found by this fix's OWN comprehensive test (not from facts.py — same bug class, one layer over): DU…91, etfstack's earlier paper-trading account id, referenced verbatim in FIVE main-dashboard doc files (docs/IBKR_SETUP.md ×2, docs/PAPER_TRADING_TEST_PLAN.md, docs/STRATEGY.md, docs/FINDINGS.md's own #467 entry) and never added to identifier_masking.KNOWN_ACCOUNT_IDS — fixed by adding it there (read by both the per-route doc masking mask_known_public_identifiers() AND the central secret_redaction backstop, so one addition covers both).
• Comprehensive route+identifier sweep (tests/unit/test_finding_483_identifier_leak_gap.py, new): enumerates EVERY registered public GET route — parameter-free routes (existing helper), AND the routes that need a concrete parameter to reach the handler, which the FIRST version of the operator's 82-URL prod scan could not reach: /api/trade-chart-data (needs source+id query params — supplied via fixture values source=position&id=1 / source=audit&id=1), /api/etfstack/backtest/{run_id}/equity and /metrics (needs a run_id path param — fixture 1), and both doc-serving routes at every valid slug (/docs/{doc_name} via the existing 404-body slug discovery; /api/btc5m/docs/{slug} via DOC_SLUGS) — and asserts none of the known account ids, known server IP/path, or ANY WALLET_*/EXECUTION_HOST*/EXECUTION_TREE_PATH value in facts.py (discovered dynamically at test time, not copied into a list in the test either) appears in any response body. A new endpoint, or a new identifier constant added to facts.py, cannot leak silently past this test — the exact "that way a new endpoint can't leak silently" requirement.
• Two existing #467 tests that had the SAME blind spot are widened rather than left as-is: TestBtc5mIdentifiersMasked.test_known_identifier_literals_feed_the_secret_filter_too now checks every dynamically-discovered WALLET_*/EXECUTION_* value (was 3 hand-picked names, missing WALLET_NAGI); TestDocRoutesIdentifierMasking.test_every_btc5m_doc_slug_is_clean now also checks btc5m's OWN identifiers on btc5m's doc route (was checking only the MAIN dashboard's account id/server IP/path against btc5m's docs — the literal reason WALLET_NAGI shipped undetected).
• Also folded into this branch (B3 review, same #467 test suite): tests/unit/test_finding_467_dashboard_hardening.py::TestFullRouteSweepNoSecretLeak planted a fake DATABASE_URL (postgresql://...@10.20.30.40:5432/earnings) and hit every real dashboard route, including DB-touching ones. On this dev Mac that private-looking address fails instantly (unroutable / psycopg2 not installed); on a fresh cloud VM whose network silently drops rather than rejects traffic to it, the real psycopg2/pg8000 connect() call hung for the OS's full TCP timeout with no PGCONNECT_TIMEOUT set to bound it (confirmed: passes with PGCONNECT_TIMEOUT=1; hangs the same way on pure master, i.e. a pre-existing gap). Fixed hermetically, not by tuning a timeout: a new no_db_network fixture patches the ONE shared factory every dashboard/btc5m/etfstack engine is built through (src.db.connection.create_db_engine) to return a bare object() — any .begin()/.connect()/SQLAlchemy event.listen() call on it raises AttributeError immediately, no socket ever opens, and every DB call site already treats that as "DB unavailable" (its existing graceful fallback). Also drops any real engine a prior test already cached in one of the four module-level singletons, and replaces the fixture's literal 10.20.30.40 with the RFC 2606-reserved, guaranteed-never-resolvable db.invalid — no real host or IP stays in the fixture. Verified hermetic directly: instrumented socket.socket.connect during both affected tests and confirmed zero real socket calls.
• §B:
• Trace: /api/btc5m/docs/{slug} → get_doc() → _mask_known_public_identifiers(text) (btc5m's own local copy, unaffected by this fix — it never had wallets in its list) is NOT what redacts WALLET_NAGI; the doc response body then passes through the ASGI SecretRedactionMiddleware (installed globally on app), which calls redact_bytes() → _STORE.redact_bytes() → the literal pattern built from _collect_known_identifier_literals(). Before the fix, that pattern never contained WALLET_NAGI, so the middleware passed the raw value through untouched.
• Simulate: hit the exact reported route/value (GET /api/btc5m/docs/doc_map, WALLET_NAGI) before and after the fix; reverted secret_redaction.py to the pre-fix version via git show HEAD:... and confirmed 5 tests correctly fail (the new file's reviewer-reported-route test and one parametrized identifier case, plus the two widened #467 tests), then restored the file byte-identical (diff confirmed) — same for the DU…91 fix in identifier_masking.py (1 test fails on revert, confirmed, restored identical). For the DB-hang fix: instrumented socket.socket.connect around both affected tests and confirmed zero calls with the fixture applied.
• Lines: earnings_dashboard/secret_redaction.py (_collect_known_identifier_literals, ~30 lines changed to discovery-by-name) · earnings_dashboard/identifier_masking.py (KNOWN_ACCOUNT_IDS +1 value) · tests/unit/test_finding_483_identifier_leak_gap.py (NEW, 26 tests) · tests/unit/test_finding_467_dashboard_hardening.py (2 tests widened; no_db_network fixture + PLANTED_DATABASE_URL host change; both risky tests take the new fixture).
• Constraints: the discovery-by-name filter must stay narrow enough not to sweep in an unrelated future constant (e.g. a tuning parameter) as a false-redaction risk — tested directly (TestCentralCollectionDiscoversFutureIdentifiers.test_an_unrelated_new_constant_is_not_swept_in). btc5m's own local doc-masking copy is deliberately NOT updated to also mask wallets — the "no cross-import" modularity rule stays intact, and the central backstop already covers this response surface without it.
• Test: tests/unit/test_finding_483_identifier_leak_gap.py — 26 tests (4 classes): the comprehensive route+identifier sweep, the reviewer-reported route in isolation, the DU…91 second hit (5 specific docs + a dedicated mutation check), and the "future identifier discovered automatically / unrelated constant not swept in" unit tests on the collection function itself. Plus 2 widened tests in the #467 file. Full unit suite and drift audit rerun clean (counts below).
• §E: re-audited all of facts.py for "any other identifier that leaks the same way" — exactly one identifier-shaped section (WALLET_MAIN, WALLET_TITUS, WALLET_NAGI, EXECUTION_HOST, EXECUTION_HOST_IP, EXECUTION_TREE_PATH); all six now discovered automatically. The comprehensive sweep itself then found the SEPARATE DU…91 hit (not from facts.py) by construction, rather than by a second manual grep — confirming the sweep does what it's meant to. A live operator scan of all 82 dashboard URLs (widened to the same parameterized routes this file covers) confirmed WALLET_NAGI on doc_map was the only current hit at scan time; DU…91 was found afterward by this file's own test, before deploy.
• Files: earnings_dashboard/secret_redaction.py, earnings_dashboard/identifier_masking.py, tests/unit/test_finding_483_identifier_leak_gap.py (NEW, 26 tests), tests/unit/test_finding_467_dashboard_hardening.py (2 tests widened, no_db_network fixture added, PLANTED_DATABASE_URL host changed), docs/CONTEXT.md, docs/AGENT_HANDOFF.md, this entry.

Finding 470 — Main-loop stalls: the signal scan's unbounded latest-snapshot read blocked the IB owner thread; bounded reads, per-ticker progress, dump to file; #453 stopgap retired (Sep 26, 2026)

• Severity: High (13 watchdog kills Sep 21–22 at 14:32 and 19:52 UTC — the AMC and BMO entry windows; each restart redid the same work; the Sep 22 CBRL entry went through only after restarts). Reliability plan P1-stall (was plan "#453"; #453 is the DEPLOYED Monday stopgap this finding retires), Batch B2.
• Status: IMPLEMENTED-PENDING-DEPLOY (Batch B2, branch reliability/b2).
• Root cause (P):
• Every stall in sep21-22-stalls.txt is the one-sided-book path: One-sided orderbook, using price_snapshots fallback → Using robust latest price_snapshot 23–74 s later, for CTAS / PAYX / CBRL in turn, with heartbeat skipped — IB owner did not pump (#440) in between, then MAIN LOOP STALL … stage=amc_verify_dates (the stage mark is set after verify_earnings_dates and never updated inside check_*_signals).
• ConvictionCalculator._get_latest_from_price_snapshots ran one CTE: max(time) over all history, joined to the rows time >= mx − 60 min. The join predicate is not pushed into the chunk scans. Read-only EXPLAIN (ANALYZE, BUFFERS) on prod, Sep 26 16:48–16:50 UTC, active markets: 287k rows / 47,597 disk reads / 35.5 s cold (146k rows / 1.4 s warm) to return 792 rows across ~352 chunks. A 7-day bound on the CTE alone still scanned 108k rows (0.4–1.1 s). Two statements — bounded max(time) (0.3–1.8 ms), then the window with a constant lower bound (5.6–11.5 ms) — returned the same 792 / 628 rows.
• The scan runs on the IB owner thread (the main loop; #440), so the owner could not pump heartbeat probes, and the watchdog saw no progress for > 120 s.
• The watchdog's thread dump never landed: faulthandler.dump_traceback(file=io.StringIO()) raises UnsupportedOperation: fileno, swallowed at DEBUG (verified locally).
• Moving the scan to a worker thread was rejected: _build_trading_signal's regime check calls the IBKR client (SPY / VIX quotes) and bounded_request / get_quote do not route to the owner — that would break the #440 same-thread contract.
• Fix:
• _get_latest_from_price_snapshots: two bounded statements (anchor inside CONVICTION_LATEST_LOOKBACK_DAYS = 7, then time >= anchor − 60 min, LIMIT 10000) — same rows as the old CTE whenever data exists in the lookback; under SET LOCAL statement_timeout = CONVICTION_QUERY_TIMEOUT_MS (15000); failures still return None (existing degradation).
• _get_latest_conviction: the latest-book read is bounded to the same lookback, runs under the timeout, and a failed read rolls back and uses the snapshot / live-Gamma fallbacks. _get_conviction_from_price_snapshots (entry-time window, already bounded) gets the timeout backstop.
• SignalGenerator._get_signals_for_timing calls progress_hook per market (cid, then symbol, before the conviction read); the orchestrator wires _signal_progress: mark signals:<bmo|amc>:<SYMBOL> and pump_ib_owner() (heartbeat probes run between tickers).
• Watchdog: _write_watchdog_thread_dump writes stage= + an all-thread faulthandler dump to MAIN_LOOP_STALL_DUMP_DIR/stall-<UTC>-<pid>.txt (fsync), logs it, and names the file in the CRITICAL line and page; a dump failure logs ERROR and still exits.

• 453 retired: MAIN_LOOP_STALL_TIMEOUT_DATED_OVERRIDES and Config.main_loop_stall_timeout_sec() removed; the watchdog reads MAIN_LOOP_STALL_TIMEOUT_SEC (120, GUARD ≤ 600) every poll. The names join drift-audit C2 as stale literals; the #453 changelog entry is SUPERSEDED by #470; the #453 test file becomes the retirement tripwire.

• §E: the main-loop "current conviction" reads at entry / close (TradingOrchestrator._get_current_conviction_and_volume, TradingUtils.get_current_conviction_and_volume) get the same lower bound. .gitignore logs/stall_dumps/.
• §B:
• (1) Trace: main loop (IB owner) → run_bmo_cycle / run_amc_cycle → check_*_signals → get_*_signals_* → _get_signals_for_timing → per market _note_progress → _signal_progress (mark + pump_ib_owner) → get_conviction_at_entry_time → _get_latest_conviction (bounded book, timeout) → one-sided → _get_latest_from_price_snapshots (two bounded statements) → _build_trading_signal. Watchdog thread → MAIN_LOOP_STALL_TIMEOUT_SEC → stall → _write_watchdog_thread_dump → CRITICAL + page → os._exit(1).
• (2) Runtime: the Sep 22 14:30 scan (GIS, CTAS, PAYX, CBRL) drops from ~2–3 min to seconds; progress moves per ticker, so even a slow ticker (bounded by the 15 s timeout per statement) never approaches 120 s; heartbeat probes run between tickers. A genuine hang still exits after 120 s with stage=signals:bmo:<SYMBOL> and a dump file.
• (3) Lines: conviction_calculator.py _latest_since, _bound_statement, _get_latest_conviction book read, _get_latest_from_price_snapshots, _get_conviction_from_price_snapshots; signal_generator.py progress_hook (__init__), _note_progress, _get_signals_for_timing loop head; trading_orchestrator.py __init__ hook wiring, _main_loop_watchdog_loop, _write_watchdog_thread_dump, _signal_progress, _get_current_conviction_and_volume; trading_utils.py get_current_conviction_and_volume; config.py CONVICTION_LATEST_LOOKBACK_DAYS, CONVICTION_QUERY_TIMEOUT_MS, MAIN_LOOP_STALL_DUMP_DIR, removed override + accessor; scripts/doc_drift_audit.py C2.
• (4) What could prevent it / limits: (a) a market whose newest snapshot is older than 7 days no longer uses that stale price as "latest" (the orderbook / live-Gamma path decides) — a deliberate input change for dormant markets; (b) a statement that times out degrades that ticker (skip or fallback) instead of waiting — fail-safe, and not expected at the measured ~10 ms; (c) pump_ib_owner runs queued owner jobs only; it does not run the ib_insync loop between jobs; (d) other slow work inside one ticker (yfinance market-cap lookup, HTTP with its own timeouts) is not split further; (e) the detector-thread reads in earnings_detector.py (_live_polymarket_median, _check_polymarket_price, _live_polymarket_extreme_outcome) are the same unbounded shape but not on the owner thread and feed instant-exit inputs (eps_estimate from the latest row) — follow-up B2-F2, closed in review round 2 (below).
• (5) Test tests/unit/test_finding_470_signal_stall_root_cause.py (17): old-CTE oracle on synthetic data (latest 1 min / 5 h / 6 d 23 h → same rows and median); two bounded statements under SET LOCAL statement_timeout = 15000, since ≈ now − 7 d, window bound, LIMIT 10000; > 7 d → None with no window read; timeout → None; book read bounded and a failed read rolls back and falls back to snapshots; per-ticker progress sequence; mark before the conviction read; hook failure harmless; orchestrator marks + pumps and wires the hook; 4 slow tickers (~160 s) → no exit; real hang → exit, one page, dump file with the real stage; dump failure still exits; dump dir ignored; bounded entry/close reads. pristine parent cefc00d with this commit's test file: 16 failed / 1 passed (review round 2 re-run; the earlier "14 of the first 16" was a partial-revert count). tests/unit/test_finding_453_stall_monday_stopgap.py rewritten to 4 retirement tripwires (override and accessor gone; watchdog reads the plain budget; budget guarded; a 200 s Monday stall exits).
• §E: git grep of FROM price_snapshots|orderbook_snapshots|trades in src/earnings/, src/ibkr/: bounded here — the three conviction reads and the two entry/close reads; already bounded — get_conviction_at_time window, signal_generator.get_market_volume (6 h); offline — get_markets_with_conviction (only the module's main()); B2-F2 — the three detector-thread reads, closed in review round 2. Retired-mechanism grep (MAIN_LOOP_STALL_TIMEOUT_DATED_OVERRIDES, main_loop_stall_timeout_sec, budget 120s -> 300s): code/tests/OPERATIONS updated; FINDINGS #453 and the CONTEXT Sep 26 bullet are history; the R2 banners and the #453 SUPERSEDED marker flip at this finding's deploy sweep.
• Review round 2 (Sep 26–27, 2026; review of bac9f25, blockers 8 and 10 + nits):
• Blocker 8: TradingOrchestrator._get_current_conviction_and_volume and TradingUtils.get_current_conviction_and_volume run SET LOCAL statement_timeout = CONVICTION_QUERY_TIMEOUT_MS before the price read (they had only the date bound); a timeout degrades to (None, None) through the existing except.
• Blocker 10 (B2-F2): EarningsDetector._recent_prints_for_symbol(symbol, n) replaces the three ticker JOINs in _live_polymarket_median, _check_polymarket_price, _live_polymarket_extreme_outcome: on its own connection, under the timeout, it resolves the ticker's condition IDs (earnings_markets WHERE ticker), then reads price_snapshots WHERE condition_id IN (…) AND time >= now − CONVICTION_LATEST_LOOKBACK_DAYS ORDER BY time DESC LIMIT n and returns (yes_price, time, eps_estimate) — the old JOIN's rows whenever prints exist in the lookback. Any failure logs a warning and returns []: the crowd witness and the PM price check then take their existing live-Gamma fallback (before, a DB error in those two returned "no data" and skipped Gamma); the extreme-outcome shield returns None (unchanged). The two outer with engine.connect() blocks that no longer ran a query were removed (dedent only).
• Nits: the two conviction timeout handlers roll back the caller-owned transaction (_rollback_quietly); the watchdog keeps the newest 50 stall-*.txt dumps (STALL_DUMPS_KEEP).
• §B (1) Trace: main loop → entry/close → _get_current_conviction_and_volume → SET LOCAL → price read → except → (None, None). Detector thread → _check_release sources → _check_polymarket_price / _live_polymarket_median / _live_polymarket_extreme_outcome → _recent_prints_for_symbol → SET LOCAL → markets → prints → [] on failure → _live_gamma_yes_price (crowd witness, PM price check) or None (extreme-outcome shield). (2) Runtime: today's shapes run in ~5 ms (review probe); a lock wait now costs ≤ 15 s per read and yields the Gamma witness instead of a stalled check. (3) Lines: trading_orchestrator.py _get_current_conviction_and_volume, _write_watchdog_thread_dump, _prune_stall_dumps; trading_utils.py get_current_conviction_and_volume; earnings_detector.py _recent_prints_for_symbol + three callers; conviction_calculator.py _rollback_quietly + two handlers. (4) Limits: a ticker whose newest print is > 7 days old yields no rows (its db_eps_estimate falls back to the Gamma/consensus path) — the same lookback as the conviction reads; duplicate earnings_markets rows for one condition ID no longer duplicate prints. (5) Test tests/unit/test_finding_470_review_timeouts.py (13); on bac9f25 10 fail / 3 pass. test_earnings_detector.py / test_finding_358_print_density.py fakes now route the two-query contract and still pass on bac9f25 (same rows). Red-count method (review round 2): git worktree add --detach ~/.cursor/worktrees/polymarket-bot/b2-oldtest <parent>, git show <commit>:<test> > <test>, PYTHONDONTWRITEBYTECODE=1 python3.11 -m pytest <test> -q.
• Review round 3 (R2-4): _live_gamma_yes_price — the fallback taken after a timed-out snapshot read — runs SET LOCAL statement_timeout = CONVICTION_QUERY_TIMEOUT_MS on both of its connections (the condition-ID lookup and the persist=True INSERT), so no untimed DB statement follows a timeout. Test test_gamma_fallback_db_work_is_timed_after_a_snapshot_timeout (per-connection recorder: every connection's first statement is the timeout). R2-6: AGENT_HANDOFF now says the crowd witness and the PM price check fall back to Gamma, while the extreme-outcome shield returns None.
• Acceptance: tests above. After deploy, Tuesday 14:30 / 19:50 UTC cycles: no MAIN LOOP STALL, no heartbeat skipped during the scan, trader log stage=signals:… only if something hangs; Using robust latest price_snapshot within ~1 s of One-sided orderbook.
• Changelog: one §O execution entry; #453's entry SUPERSEDED (superseded_by #470).
• Files: src/earnings/conviction_calculator.py, src/earnings/signal_generator.py, src/earnings/trading_orchestrator.py, src/earnings/trading_utils.py, src/earnings/config.py, scripts/doc_drift_audit.py, .gitignore, tests/unit/test_finding_470_signal_stall_root_cause.py (NEW), tests/unit/test_finding_453_stall_monday_stopgap.py (rewritten), src/earnings/strategy_changelog.py, docs/OPERATIONS.md §12.2, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, this entry.

Finding 452 — Leverage alerts: page only on broker-reported, populated margin state; latch (one page + one resolve) (Sep 26, 2026)

• Severity: Medium (alert-only engine; 78 false pages, Telegram RCA RC5). Reliability plan #452, Batch B2.
• Status: IMPLEMENTED-PENDING-DEPLOY (Batch B2, branch reliability/b2).
• Root cause: TradingOrchestrator._check_leverage_alerts had no validity gate and rate-limited by clock-minute key (_last_margin_alert_key), so a persistent false condition paged once a minute:
• Aug 26 06:22–07:08: 46 pages, effective_leverage=2.50 > 2.00, source=internal_estimate, equity_base=1000, net_liquidation=18450 — get_leverage_state falls back to total_capital + PDT reserve, and total_capital is the run_earnings_trader.py startup placeholder (capital … else 1000) until the first IBKR sync;
• Sep 8 06:01–06:27: 27 pages, liquidity_buffer=0% < 25%, source=ibkr_live, net liq loaded, excess_liquidity=0, maint_margin_req=0, flat book — after the 06:00 reattach IBKR reports net liq before margin fields;
• 5 margin_state_stale pages (Aug 25–26) while syncs were failing.
• Fix: _leverage_state_validity: only source=ibkr_live, not stale, net_liquidation > 0 can page; excess liquidity and maintenance margin both 0 (or account_type None) = margin fields unpopulated → the effective-leverage check still runs, the margin-utilisation and liquidity-buffer checks do not. margin_state_stale no longer pages (unevaluable telemetry of an alert-only engine; the IBKR request-path latch covers a dead session). Latch = the persisted leverage incident from alert_policy (#451): one WARNING when a valid breach starts (text says alert-only), one leverage_resolved WARNING when a valid state shows no breach; unevaluable state neither pages nor resolves; a failed send is retried; a restart does not re-page. margin_events gets one row per incident (was one per minute). The per-minute key is removed.
• §B:
• (1) Trace: main loop (every pass, IBKR connected) → risk_manager.sync_from_ibkr() → _check_leverage_alerts → get_leverage_state → _leverage_state_validity → breach? → alert_policy.open_incident("leverage") → should_page → _record_margin_event + send_alert (→ mark_paged) / no breach + margin valid → _resolve_leverage_alert.
• (2) Runtime: after a restart the first passes are internal_estimate → silent; after the 06:00 reattach, zeroed margin fields → only the leverage check runs (0 on a flat book); a genuine 2.5× breach pages once and resolves once.
• (3) Lines: trading_orchestrator.py _leverage_state_validity, _check_leverage_alerts, _resolve_leverage_alert; __init__ (per-minute key removed).
• (4) Limits: the placeholder capital itself is unchanged (sizing reads IBKR-synced capital before any entry; out of scope); a breach that begins while margin fields are unpopulated is caught once they populate (next pass); the resolve needs a margin-valid state; margin_events count semantics change (see §E).
• (5) Test tests/unit/test_finding_452_leverage_alert_latch.py (9): Aug 26 fixture 46 min → 0 pages / 0 events; Sep 8 fixture 27 min → 0; stale → 0; genuine breach 30 min → 1 page + 1 event, then 1 resolve; restart (fresh orchestrator) → no re-page; unevaluable states mid-breach neither page nor resolve; margin breach with populated fields pages once; failed send retried once; no per-minute key. pristine parent fb237e4: 7 failed / 2 passed (the other two depend on the old code's real-clock minute key); re-run in review round 2.
• §E: scripts/leverage_live_validation_autorun.py counts margin_events of the four Phase E types for its phase_e_alert_category_seen flag: after #452 it is set only by a genuine breach (one row per incident; margin_state_stale no longer written) — the intended meaning. risk_manager.py margin_state_stale is the Phase D admission uncertainty reason, not an alert; unchanged. No other leverage/margin alert sender exists (git grep leverage_cap_breach margin_headroom_low margin_utilization_high).
• Review round 2 (Sep 26–27, 2026): the latch follows #451's corrected model — claim_page("leverage") (atomic in-flight), send_alert(..., require_telegram=True) so only a Telegram delivery counts, mark_paged(..., generation=), and leverage_resolved only when page_delivered. §B: trace unchanged except should_page → claim_page; runtime identical when Telegram works; a webhook-only success is now retried and never resolved; NEW test_page_counts_only_telegram_delivery (the file has 10 tests; fakes accept require_telegram).
• Review round 3 (Sep 26–27, 2026): the latch is a consumer of the ledger adapter (#451 round 3): _check_leverage_alerts opens the leverage incident with its category and metrics (margin event recorded once, when the incident is created) and _resolve_leverage_alert only closes it; the dispatcher sends the page and the resolve with require_telegram=True and retries both (the resolve now also Telegram-only, R2-2). Test fixture runs one dispatch pass per check (10 tests).
• Review round 4 (R3-4): expected-window deadlines are explicit per incident type — alert_policy.EXPECTED_WINDOW_KEYS = {"ibkr", "recon_guard"}; every other key, including leverage, uses only its own grace (0 s), so a valid broker-reported breach at 04:14 UTC or during the Sunday 06:00 AutoRestart pages on the first dispatch (was: 30 / 5 min later). §B (1) _check_leverage_alerts → open_incident("leverage") → event=None → deadline = opened_at → dispatch. (3) alert_policy.py EXPECTED_WINDOW_KEYS, open_incident, _expected_name. (4) The IBKR / reconciliation incidents keep their window SLAs unchanged. (5) TestNoGatewayWindowDelay (4: nightly / Sunday / weekday-AutoRestart breaches page at once; broker incidents keep 1800 s); all 4 fail on 69d33e2.
• Acceptance: tests above; after deploy, 0 leverage pages across the 04:13 / 06:00 reattaches (the Sep 8 class); a genuine ibkr_live breach pages once.
• Changelog: one §O reporting entry (alert-only; no sizing or admission change).
• Files: src/earnings/trading_orchestrator.py, tests/unit/test_finding_452_leverage_alert_latch.py (NEW), src/earnings/strategy_changelog.py, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, this entry.

Finding 451 — Trader IBKR alert policy: expected events page only past their SLA, one page + one resolve, correct lever, OD6 delivery-gated integrity dedupe (Sep 26, 2026)

• Severity: High (operator bar: "no errors in Telegram while IBKR, Polymarket and the vendors work"). Telegram RCA: 187 pages in 16 days (Sep 10–25), dominated by planned IBKR events — RC2 nightly reset 50, RC3 06:00 AutoRestart + #441 bounce 30, RC15 deploy/other restarts 6 (85 in the window, 56 within 20 min of a prod pull), RC1 recon-guard every 300 s (833 in the window); every disconnect page carried the wrong lever systemctl restart earnings-trader (client.py reconnect-FAILED and 30-min STILL DISCONNECTED pages). Reliability plan #451, Batch B2; OD6 option (c) approved Sat 26 Sep 07:11 UTC.
• Status: IMPLEMENTED-PENDING-DEPLOY (Batch B2, branch reliability/b2).
• Root cause: AlertManager is a per-process singleton with no persisted state (restart = filter reset); the IBKR client paged every 1100/1102 after a 20 s grace regardless of IBKR's schedule, paged "reconnect FAILED" as a second page, repeated STILL DISCONNECTED every 30 min, and paged RECONNECTED at CRITICAL; the #441 attach-bounce produced SHUTDOWN + STARTUP WARNINGs twice a day; the reconciliation guard paged every 300 s (in-memory cooldown). Integrity duplicated the trader's request-path page (RC6, 58 pages).
• Fix:
• NEW src/earnings/alert_policy.py: expected IBKR windows — nightly reset 23:30–01:15 ET (SLA 30 min, M2), AutoRestart 05:58–06:20 UTC (SLA 10 min; Sunday weekly 2FA 5 min); persisted incidents (logs/alert_incidents.json, cross-process fcntl lock, atomic replace, fail-open) with a deadline fixed at first open (window SLA, else the caller's grace), page-in-flight / failed-send retry, 6 h stale reset; delivery record per key:polarity; ibkr_down_lever(tcp_up); planned-restart stamp (logs/planned_restart.json, 15 min) and deploy_in_progress() (active polymarket-deploy-* unit, F-8).
• src/ibkr/client.py (alert functions only, no session behaviour change): _queue_disconnect_alert arms the ibkr_disconnect incident page for its deadline with the :4001-aware lever; the reconnect-FAILED page and the 30-min STILL DISCONNECTED reminders are replaced by re-arming the same incident (a delivered page is never repeated; a failed send is retried); _maybe_send_reconnect_alert / _note_ibkr_path_healthy resolve incidents — RECONNECTED / RESTORED page at WARNING only if the incident paged; a fresh process's first connect with a live request path (no IB() reset in flight) and every durable-health recovery (_clear_inplace_recovery_state) count as healthy. _send_request_path_alert(message, kind): incident (DEAD/RESTORED pair through the policy), operator actions (last-resort recycle, waiting on 2FA, recycle did not run, open position still dead; see review round 2 for their dedupe), info (recycle deferred ×2, recycle held, keep probing — logged into the incident, never a page). _emit_ibkr_alert records delivery.
• run_earnings_trader.py: a fresh process that cannot connect joins ibkr_disconnect (grace 120 s outside windows) and pages once at the deadline; STARTUP silent when the planned stamp is fresh (consumed); SHUTDOWN silent when stamped or a deploy unit is active (stamps for the next STARTUP). gateway_recycle.bounce_trader stamps before scheduling and clears the stamp if systemd-run fails.
• reconciliation_manager.py: the fail-closed guard (disconnected and stale-session skips) is one recon_guard incident: pages once after RECON_GUARD_PAGE_SLA_SEC (1800) or RECON_GUARD_OPEN_POSITION_SLA_SEC (300) with an open position (inside an IBKR window the window SLA applies; the market is closed at both); resolve page when reconciliation runs again. RECONCILIATION_GUARD_ALERT_COOLDOWN_SEC retired (prod .env does not set it; read-only grep Sep 26, 2026). trading_orchestrator._reconcile_positions_with_ibkr (legacy copy used by the trade reconciler) routes its disconnected page through the same incident.
• OD6: integrity_monitor._trader_already_paged — the trader:ibkr_request_path rule keeps evaluating and showing; its Telegram page is skipped only when the trader's page for the same incident generation (review round 2) and polarity was delivered within 30 min (trader dead, page not yet due, or send failed → integrity pages). memory.md quick-ref and OPERATIONS §9 row reworded in this commit.
• .gitignore: logs/alert_incidents.json*, logs/planned_restart.json* (keeps #468's clean prod checkout). OPERATIONS §8.A3 runbook.
• §B:
• (1) Trace: 1100/1300/heartbeat → _schedule_reconnect → _queue_disconnect_alert → open_incident → threading.Timer(deadline) → should_page → _send_disconnect_alert → _emit_ibkr_alert → send_alert → record_delivery / mark_paged; recovery → _attempt_reconnect_once → durable health → _clear_inplace_recovery_state → _note_ibkr_path_healthy → resolve_incident → WARNING resolve iff paged. Latch → _trip_request_path_dead → _send_request_path_alert (incident) → _arm_incident_page. Bounce → bounce_trader stamp → SIGTERM → _send_shutdown_alert (silent) → new process → connect() probe ok → incidents resolved → _send_startup_alert (silent, stamp consumed). Integrity cycle → transition → _send_integrity_alert → _trader_already_paged → skip or page.
• (2) Runtime: nightly 1100 04:13:58 UTC → incident deadline 04:43:58; health fails, #441 bounce 04:15:xx, new process connects → resolved, 0 pages (was DISCONNECTED + SHUTDOWN + STARTUP ≈ 3–4). Sunday AutoRestart hang 06:00 → one CRITICAL at 06:05 naming systemctl restart ibgateway-ibc + approve the push; no reminders; resolve when the trader reconnects. Deploy → 0 lifecycle pages. Unexpected 1100 at 15:00 UTC → page after 20 s (unchanged) + one resolve.
• (3) Lines: alert_policy.py (whole); client.py _send_request_path_alert, _arm_incident_page, _resolve_ibkr_incident, _note_ibkr_path_healthy, _emit_ibkr_alert, _send_disconnect_alert, _queue_disconnect_alert, _maybe_send_reconnect_alert, _schedule_reconnect (two former pages), connect() probe branch, _clear_inplace_recovery_state, 8 kind= call sites; run_earnings_trader.py _page_startup_disconnect_if_due, _send_startup_alert, _send_shutdown_alert, connect loop; gateway_recycle.bounce_trader; reconciliation_manager.py _page_recon_guard_if_due, _resolve_recon_guard, two guard branches; trading_orchestrator._reconcile_positions_with_ibkr; integrity_monitor._trader_already_paged + _send_integrity_alert; config.py RECON_GUARD_*.
• (4) What could prevent it / limits: (a) a genuine outage that starts inside a window pages up to the window SLA later (30 min nightly, 10 min / 5 min Sunday at 06:00) — the market is closed in both windows and hardware stops stay; #456 (B1) pages the weekend Gateway hang from IBC evidence; (b) OD6 is strict: if the trader's DEAD page is still inside its SLA, integrity pages its own transition (B1 #449 hysteresis is the complement); (c) #413's same-polarity 300 s cooldown can still hold back a second CRITICAL ibkr_request_path page within 5 min (pre-existing); (d) ~~the recon-guard page is marked delivered optimistically~~ — fixed in review round 2; (e) a stamp older than 15 min is ignored, and an operator's manual restart still pages STARTUP/SHUTDOWN (intended); (f) state-file errors fail open to paging; (g) the IBKR windows are hard-coded to IBKR's documented NA reset and our IBC AutoRestartTime 06:00 UTC — a change to either needs this module updated.
• (5) Test tests/unit/test_finding_451_trader_alert_policy.py (49): windows incl. late reset, DST (EST), Sunday, edges, naive rejected; incident deadline / reopen / in-flight / failed-send retry / stale reset / cross-process file / unreadable file; request-path leaf (in-window DEAD waits and RESTORED is silent; outside-window DEAD pages once with the lever and resolves at WARNING; deadline fire; info never, action always (superseded by review round 2); call-site kinds; no wrong-lever or STILL DISCONNECTED text left); disconnect leaf (window SLA; grace + one page + one resolve; healthy connect closes both); OD6 drill (delivered → integrity silent both polarities; trader dead / send failed / page not yet due → integrity pages; _send_integrity_alert honours the hook; other rules unaffected; rule still CRITICAL; memory wording); lever; lifecycle (planned silent + stamp consumed, deploy stamps, unplanned unchanged, stale stamp, bounce_trader stamps/clears, unit detection); startup-connect page once at the Sunday deadline; recon guard (open position 300 s outside windows, window SLA inside); replays (nightly reset + bounce → 0 pages; Sunday hang → exactly 1 page with the Gateway lever over 3 h). Mutation-checked: dropping either window, resolve-always-pages, info paging, removing the OD6 hook, ignoring delivery, ignoring the stamp, re-paging a delivered incident each fail. Existing tests moved to the new contract: test_finding_419_ibkr_inplace_recovery.py (source marker), test_ibkr_error_block_policy.py (resolve follows the incident), test_finding_420_ibkr_nightly_1100.py + test_reconciliation_guard_retry.py (guard pages once after its SLA), test_finding_404_telegram_schedule.py (guard excepts log in the helpers), test_config.py (knob swap). tests/conftest.py resets the in-memory incident store per test.
• §E: every trader send site classified — client.py (all _send_disconnect_alert / _send_request_path_alert / _emit_ibkr_alert sites routed), run_earnings_trader.py lifecycle, reconciliation_manager.py guards, trading_orchestrator.py legacy reconcile. Left paging by design (genuine or actionable): stop-fill queue full, post-reconnection reconciliation failed, reconciliation aborted (IBKR=0 / DB open), reconciliation issues, external positions, price unavailable, instant exit dropped, #399 batch-entry block, daily summary, circuit-breaker pages (risk_manager.py), earnings-date correction; leverage → #452; stall → #470. gateway_monitor.py / cron senders / GRM / integrity policy are B1 (#448/#449). Docs mentioning STILL DISCONNECTED are dated #419/#421 history.
• Review round 2 (Sep 26–27, 2026; review of bac9f25, blockers 1–6):
• Delivered = Telegram HTTP 200 (blocker 2): AlertManager.send(..., require_telegram=True) / send_alert(..., require_telegram=True) return the Telegram result only (filtered or webhook-only = False); every incident sender uses it (_emit_ibkr_alert, alert_reconciliation_guard, leverage, startup page). send()'s default return is unchanged for every other caller. Same fact as B1 #449's last-send artifact (resp.status == 200).
• One IBKR outage = one ibkr incident (blocker 3): 1100/1300/heartbeat (_queue_disconnect_alert), the startup connect failure, and the request-path latch all open ibkr; one page at its deadline, one resolve (_resolve_ibkr_incident, from RECONNECTED / RESTORED / verified-healthy). claim_page is atomic (read + mark in flight under the lock), so the disconnect timer and the request-path timer can never both page.
• Generations (blocker 4): each opening gets a generation; mark_paged ignores a result for an older generation; record_delivery(key, polarity, delivered, generation); trader_page_delivered matches down to the open incident's generation and up to the last resolved generation. A second outage inside 30 min is never suppressed by the first one's delivery.
• Recon guard (blocker 5): claim_page("recon_guard"), then a wrapper coroutine awaits alert_reconciliation_guard (now returns the Telegram result) and persists it with mark_paged(generation=); a failed send is due again next pass; the resolve pages only if page_delivered.
• Actions (blocker 6): kind="approve_2fa" (LAST-RESORT recycle, waiting on 2FA), "recycle_failed" (standing recycle did not run), "open_position" (#418) go through claim_action("ibkr", kind): the first action of an unpaged incident IS the incident page (immediate); after it, approve_2fa and recycle_failed page at most once per incident (persisted, restart-safe; a failed escalation is retried); open_position never escalates.
• Persistence failure never silences (blocker 1): _state() keeps a process-local mirror; if the file cannot be locked, read, or written, this process works from the mirror (dedupe across restarts degrades, paging does not).
• Resolve rule everywhere (IBKR, recon guard, leverage): resolve page iff the incident page was delivered.
• §B (1) Trace: symptom → open_incident("ibkr") → timer at deadline → claim_page → _emit_ibkr_alert(purpose="page") → send_alert(require_telegram=True) → mark_paged + record_delivery(down, generation); action → claim_action → page / escalate → mark_paged or mark_action; healthy → _resolve_ibkr_incident → resolve_incident (stores the resolved generation) → resolve iff page_delivered → record_delivery(up, generation). Integrity → _trader_already_paged → trader_page_delivered("ibkr", …) (generation match). Recon → claim_page → _page() awaits the send → mark_paged. (2) Runtime: nightly 1100 + bounce = 0 pages (unchanged); an out-of-window outage with a last-resort recycle = outage page + one "approve 2FA" page + one resolve; the same inside the nightly window = LAST-RESORT (as the incident page) + "recycle did not run" if it fails + one resolve. (3) Lines: alert_policy.py persisted state, incidents, claim_page, claim_action, mark_action, resolve_incident, delivery record; alerts.py AlertManager.send, send_alert, alert_reconciliation_guard; client.py IBKR_INCIDENT, _send_request_path_alert, _arm_incident_page, _resolve_ibkr_incident, _emit_ibkr_alert, _queue_disconnect_alert, _send_disconnect_alert, 4 action call sites; reconciliation_manager.py _page_recon_guard_if_due, _resolve_recon_guard; run_earnings_trader.py _page_startup_disconnect_if_due; trading_orchestrator.py _check_leverage_alerts, _resolve_leverage_alert; integrity_monitor.py TRADER_PAGED_RULES. (4) Limits: with require_telegram a send still waits for the webhook retries before returning (≤ ~3 s extra); a resolve is one attempt (its failure is recorded; OD6 then lets integrity page its resolved transition) — B1's IncidentLedger retries failed resolves; the merge adopts that (see Integration). (5) Test tests/unit/test_finding_451_trader_alert_policy.py (68, rewritten); on bac9f25 26 fail / 42 pass. On the pristine parent 0846f43 the original file fails at collection (imports alert_policy), so the round-1 red evidence was mutation runs. Red-count method (review round 2): git worktree add --detach ~/.cursor/worktrees/polymarket-bot/b2-oldtest <parent>, git show <commit>:<test> > <test>, PYTHONDONTWRITEBYTECODE=1 python3.11 -m pytest <test> -q.
• Integration with B1: B1's fix-in-progress (cursor/b1-review-fixes-aaad worktree, uncommitted src/earnings/incident_ledger.py + alerts.DeliveryResult) uses the same definition (delivered = telegram is True) and the same incident semantics (generation, deadline, 120 s in flight, failed page due again, older generation ignored, resolve owed only after a delivered page, one-shot escalation, process copy on persist failure). Merge plan: B1 lands first; on the rebase AlertManager.send takes B1's DeliveryResult body and keeps a require_telegram: bool = False keyword (redundant — truthiness already means Telegram), so B2's callers need no edit; then alert_policy's incident store becomes a thin layer over IncidentLedger (expected-window deadlines and the OD6 delivery record stay in alert_policy), adopting B1's resolve retry. tests/conftest.py: keep both autouse fixtures.
• Review round 3 (Sep 26–27, 2026; review of eaa4dbe, R2-1 / R2-2 / R2-3): B1 c1fb6c7 merged into reliability/b2 (merge 54c6f69), and B2's incident store rebuilt as a thin layer over B1's IncidentLedger — one incident model in production.
• Ledger adapter: alert_policy keeps only trader domain rules (expected-window deadlines, operator actions, page texts, ownership, OD6 record); incidents, generations, page state, escalations and resolve debt live in IncidentLedger(ALERT_INCIDENT_STATE_PATH, name="trader-alerts"). The ledger's process copy covers a vanished or unwritable file (R2-2 missing-file case). Page texts, resolve texts and the OD6 record live in a small side store ALERT_DELIVERY_STATE_PATH (logs/alert_deliveries.json, same process-copy rule).
• Dispatcher: alert_policy.dispatch() is the only sender — a daemon thread started by EarningsTrader.__init__, kicked on every open/action/resolve and run every 15 s. A failed page is due again; a failed action stays owed; a failed resolve stays owed (the ledger's resolves_due) — all retried without a second symptom (R2-2). Claims are page_due → mark_page(None) under _claim_lock (the ledger file has one writer, the trader), so two dispatch passes never double-send. The IBKR client, recon guard, leverage latch and startup path only open / act / resolve; nothing sends on the IB owner thread.
• Recovery during an in-flight page (R2-2): resolve_incident defers the close until the send result lands, so a delivered page always gets its resolve; a new symptom cancels the deferred close.
• Operator actions (R2-3): sent under ibkr_operator_action (CRITICAL; not a #413 latch category), so the same-polarity cooldown after a delivered request-path page cannot swallow "approve 2FA"; the per-incident action dedupe is the ledger escalation (mark_escalation(False) = owed, retried until delivered).
• Ownership with ibkr-monitor (R2-1): while :4001 is closed and ibkr-monitor's heartbeat (IBKR_MONITOR_HEARTBEAT_PATH, ≤ 120 s old) says live_open: false, the monitor's gateway:live incident pages the outage (its 180 s SLA, its Sunday-2FA wait, its lever) and the trader's ibkr page stands down, recorded as handled_by: ibkr-monitor for that generation (down and, on close, up). Integrity OD6 honours it. A stale / missing heartbeat or an open port hands the page back to the trader; operator actions are never deferred.
• Integrity (merge): B1's ledger dispatcher kept; _send_integrity_alert returns True ("handled") when OD6 applies, so the integrity ledger records it and never retries into a delayed duplicate.
• info messages only log (they no longer touch the incident, which would now be paged). The lever / ownership probe is the Gateway port (client socket, else a 1.5 s TCP connect; before a client exists, gateway_recycle._port_open).
• §B (1) Trace: symptom → _queue_disconnect_alert / _send_request_path_alert / _page_recon_guard_if_due / _check_leverage_alerts / _page_startup_disconnect_if_due → open_incident (window SLA → IncidentLedger.open, spec → side store) → kick → dispatcher dispatch → _dispatch_page (page_due → ownership check → mark_page(None) → send_alert(require_telegram=True) → mark_page(ok) → record_delivery(down) → deferred close) → _dispatch_actions (escalation_due → send → mark_escalation) → _dispatch_resolves (resolves_due → send → mark_resolve → record_delivery(up)). Healthy → _resolve_ibkr_incident → resolve_incident (defer if in flight) → IncidentLedger.resolve. Integrity → _dispatch → _send_integrity_alert → _trader_already_paged → trader_page_delivered (ledger generation + side store). (2) Runtime: nightly 1100 + bounce = 0 pages; an out-of-window request-path death pages within ≤ 15 s; a Telegram outage retries every 15 s until delivered; a Gateway crash (:4001 closed) = one ibkr-monitor page + one RESTORED, 0 trader, 0 integrity. (3) Lines: alert_policy.py incidents section (whole), start_dispatcher; client.py _send_request_path_alert, _resolve_ibkr_incident, _queue_disconnect_alert, _cancel_pending_disconnect_alert, _maybe_send_reconnect_alert, _gateway_port_open, __init__ probe; removed _send_disconnect_alert, _arm_incident_page, _emit_ibkr_alert; reconciliation_manager.py guard pair; trading_orchestrator.py _check_leverage_alerts, _resolve_leverage_alert; run_earnings_trader.py __init__, _page_startup_disconnect_if_due; integrity_monitor.py _send_integrity_alert; .gitignore. (4) Limits: ~~a second trader outage page within 300 s of the previous one waits for the #413 same-polarity cooldown~~ — closed in round 4 (cooldowns scoped to the incident); a page in flight when the process dies is re-sent after 120 s (possible duplicate); if ibkr-monitor owns the outage but its own page fails, B1's ledger retries it — the trader does not step in unless the heartbeat goes stale; the claim is process-atomic, not cross-process (one writer per ledger file; interface question to B1 in internal/reliability/b2-ledger-adapter-handoff-to-b1.md). (5) Test tests/unit/test_finding_451_trader_alert_policy.py (78, rewritten): ledger type; failed page / action / resolve retried without a new symptom; recovery during an in-flight page; new symptom cancels a deferred close; stale-generation result ignored; two concurrent dispatchers → one send; restart keeps debt, never re-pages; replace / temp-write / lock / unreadable / deleted-file failures still page; operator action through the real AlertManager after a delivered page; one Gateway outage across GatewayPager + trader + integrity = 1 page + 1 resolve; stale monitor / open port / actions page from the trader; OD6 handled-return; replays. Older tests migrated to the dispatcher (test_finding_418/419/420/421, test_ibkr_error_block_policy, test_reconciliation_guard_retry, test_finding_404).
• Review round 4 (Sep 26–27, 2026; review of 69d33e2, R3-5 and the B2 half of R3-1):
• Cooldowns never span incidents (R3-5): every dispatcher send carries incident_generation and incident_purpose (page / resolve / action:<kind>) in its data; AlertManager._cooldown_scope appends both to every cooldown key — the per-category rate limit, the #413 same-polarity cooldown and the CRITICAL duplicate-message cache. A new incident (or this incident's resolve) goes out on its first dispatch; only a repeat of the same send in the same incident is held back; untagged sends keep the historical category-wide keys (#413 unchanged for them). Removes the round-3 limit "a second outage page within 300 s waits for the cooldown".
• Side store after a failed write (R3-1, B2 part): once a delivery-store write fails, the process copy stays authoritative for reads and read-modify-writes until a write succeeds again (an older readable file can no longer discard a newer page spec or OD6 record). The ledger half of R3-1, the in-flight resolve durability (R3-2) and the Sunday ownership contract (R3-3) are B1's; B2 re-merges B1's fixed head.
• §B (1) Trace: dispatch → _dispatch_page / _dispatch_actions / _dispatch_resolves → _send(..., generation, purpose) → send_alert(data={incident_generation, incident_purpose}) → AlertManager._should_send / CRITICAL cache with _cooldown_scope. _store_mutate / _store_read → _store_persist_ok gate. (2) Runtime: A pages at 15:00:21, recovers, B pages at 15:01:21 on its first dispatch (was: ~15:05); a flap of three short outages = six messages; a quick recovery's resolve is not held 30 s behind its own page. (3) Lines: alerts.py _cooldown_scope, _should_send, send (CRITICAL cache + tracker keys); alert_policy.py _send + three call sites, _store_mutate, _store_read. (4) Limits: legacy untagged sends still share category-wide keys by design; while the side store cannot be written, the integrity monitor (another process) reads the older file and pages (fail-open). (5) Tests: TestCooldownScopedToIncident (6, real AlertManager: second outage inside 300 s, rapid flap, identical text of a new incident, same-incident repeat still cooled, untagged #413 kept, dispatcher tags), test_quick_recovery_resolve_is_not_held_behind_its_own_page, TestSideStoreFailedWrite; on 69d33e2 5 of these fail.
• Review round 5 (Sep 27, 2026; review of 12d0f93 R4-1 + the final-B1 checklist; B1 839f58e merged):
• Failed sends never arm a cooldown (R4-1): AlertManager.send records the per-category rate limit, the #413 same-polarity timestamp and the CRITICAL duplicate cache only after a delivered alert (Telegram HTTP 200; any channel when Telegram is not configured). A failed page / action / resolve is retried on the next dispatcher pass (15 s) with no filter; same-send dedupe stays in the ledger claim. Test: real AlertManager, first Telegram result false, second true 15 s later → exactly two transport attempts, one delivered incident (request-path, disconnect, resolve, action, WARNING category).
• On B1's final ledger: _dispatch_page claims with B1's atomic claim_page (the one-writer-per-file contract is in the ledger docstring); _pending_resolve removed — resolve() on an in-flight page persists a pending resolve that a later delivered mark_page (also from a new process) turns into an owed resolve, a failed one drops; PAGE_IN_FLIGHT_SEC comes from incident_ledger. Reopen semantics: a key that fails again before its resolve was delivered continues THAT incident (same generation, page state kept, stale resolve dropped); open_incident returns created=False for it (leverage records no second margin event) and clears the stale resolve text / handled-up record. Escalation and resolve sends remain claimed under the trader's process lock (no B1 claim API for them; one writer per ledger file).
• Ownership = the attested monitor incident (checklist 7/8): the trader reads live_incident from the ibkr-monitor heartbeat (0644) and stands down only while that incident's page is delivered, or pending / failed-and-retrying within its attested deadline + 60 s. One Gateway-down policy: the monitor's 180 s any day, and its 20-min wait only for the latched scheduled Sunday 2FA — the trader's own Sunday 5-min SLA applies only when the monitor is absent or stale. Once the monitor delivered a generation's page (recorded handled_by: ibkr-monitor + monitor_generation), failover never re-pages that generation, across trader restarts; a stale heartbeat or an overdue monitor page before delivery hands the page back to the trader.
• Pages are information (operator decision Sep 27 2026: automatic Gateway restart stays on as a last resort; the trader must never wait on Marius): the outage page's recovery note (ibkr_down_lever, follows the runtime IBKR_GATEWAY_AUTO_RECYCLE) and the client's action texts say what was detected, what the bot did automatically and how the outcome is reported; the manual lever appears only after "Fallback only". The #433 hold path (reachable only under a dated AUTO_RECYCLE=false hold) pages a recycle_needed notice once per incident. Tests: TestPagesAreInformation (4 cases × states), TestKeepProbingPagesTheLever.
• §B (1) Trace: dispatch → _dispatch_page → handled-record check → gateway_monitor_owns_outage (monitor_live_incident) → claim_page → _send → send_alert → AlertManager.send (filters; channels; arm cooldowns only if delivered) → mark_page; resolve_incident → IncidentLedger.resolve (pending if in flight) → mark_page settles it; _mark_recovery_exhausted (recycle off) → _send_request_path_alert(kind="recycle_needed") → action. (2) Runtime: a Telegram blip delays a page by one dispatcher pass (was ≤ 5 min); a weekday Gateway crash = one monitor page at ~180 s + one RESTORED, trader and integrity silent; unapproved weekly 2FA = one "NOT APPROVED" page at ~20 min; an IB() reset that cannot recover pages the operator lever once. (3) Lines: alerts.py send; alert_policy.py _unresolved_generations, _note_open, open_incident, action, resolve_incident, _close_locked, monitor_live_incident, gateway_monitor_owns_outage, _dispatch_page, record_delivery, ACTION_KINDS; client.py REQUEST_PATH_ACTIONS, _mark_recovery_exhausted. (4) Limits: a page in flight when the process dies is re-sent after 120 s (possible duplicate, fail-open); escalation / resolve claims are process-local (one writer per file). (5) Tests: TestFailedSendNeverArmsCooldown (6), TestKeepProbingPagesTheLever (2), TestMergedLedgerContract (11: claim API, two-instance claim, mid-send recovery across restart, failed page owes nothing, reopen continues, leverage reopen, failed resolve owed across restart × 3 keys, no resolve without a delivered page), TestOwnershipOnFinalLedger (10: Sunday scheduled 2FA, Sunday afternoon crash, weekday crash, weekday AutoRestart blip, monitor Telegram failure, monitor crash before / after delivery, restart, attested-only, heartbeat mode). R4-1 tests: 4 of 6 fail on 12d0f93.
• Review round 6 (Sep 27, 2026; review of abbbaf4 R5-1..R5-4; B1 2ae3f5f merged):
• Final ledger semantics: opening a key that is not open always mints a new generation and deletes that key's resolve debt (B1), so B2's reopen bookkeeping is removed — created is simply "was not open", and a new outage always pages (no stale RESTORED). Resolves go through B1's atomic claim_resolves() (prune + in-flight claim; mark_resolve settles); a recovery during an in-flight page whose sender died becomes claimable once the claim is 120 s old (R5-1) and the dispatcher sends RESTORED; expired debt is deleted (R5-2).
• R5-3 (B2 ownership): closing a monitor-owned outage no longer writes ibkr:up handled_by=ibkr-monitor. _sync_monitor_restored (every dispatcher pass) records it only when the heartbeat's last_resolved shows the SAME monitor generation (the monitor_generation stored with the handled DOWN) with resolve == "delivered"; monitor death, a failed / owed RESTORED, a dropped resolve or another generation leave integrity paging the recovery (fail-open duplicate at worst).
• Page text vs the real setting (R5-4): auto-recycle defaults ON again (B1); ibkr_down_lever and the last-resort text state that every Gateway restart is a fresh login completing once the phone 2FA is approved (#430); the #433 hold notice appears only under a dated IBKR_GATEWAY_AUTO_RECYCLE=false hold.
• Merge note: the merge commit 74f4938 recorded the unresolved memory.md and an empty AGENT_HANDOFF.md; 7f5c3a2 applies the intended resolution (content verified against both parents).
• §B (1) Trace: dispatch → _dispatch_page (claim_page) → _dispatch_actions → _dispatch_resolves (claim_resolves → _send → mark_resolve → record_delivery(up)) → _sync_monitor_restored (store DOWN handled + closed generation → heartbeat last_resolved). (2) Runtime: a Gateway crash = one monitor DOWN + one monitor RESTORED; integrity's recovery page is suppressed only after that RESTORED got HTTP 200. (3) Lines: alert_policy.py open_incident, action, _close_locked, _dispatch_resolves, monitor_last_resolved, _sync_monitor_restored, dispatch, ibkr_down_lever; client.py last-resort text. (4) Limits (accepted in round 5): action claims process-local under the one-writer contract; a page in flight at process death may be re-sent. (5) Tests: test_recovery_mid_send_survives_restart_without_the_dead_senders_result (no replayed result; claimable after 120 s), test_new_outage_while_a_resolve_is_owed_pages_as_a_new_incident, test_new_leverage_breach_after_an_owed_resolve_is_a_new_incident, TestMonitorRestoredAttestation (5: delivered same generation, monitor death, RESTORED Telegram failure, other generation, no fabricated record), Sunday test on B1's evidence rule.
• Review round 7 (Sep 27, 2026; review of 5d53721, R6-3 — OD6 reverse handback) — SUPERSEDED by round 9 (the post-send handback is removed; the outage arbiter replaces it): when the trader's page is not yet delivered, integrity pages its own trader:ibkr_request_path transition (OD6 c); once that fallback is delivered, _send_integrity_alert calls alert_policy.record_integrity_handback, which writes ibkr:<down|up> with the trader's incident generation (open incident; for up, the last resolved one if already closed) to ALERT_HANDBACK_PATH (logs/alert_handback.json, 0644, integrity is its only writer). The trader dispatcher reads it: a handed-back DOWN skips the trader's own page for that generation (not marked delivered, so no trader RESTORED becomes owed — integrity resolves its own); a handed-back RESTORED settles the trader's owed resolve (mark_resolve(..., True)) without sending. A failed integrity send writes nothing and the trader pages as usual. One physical incident = one DOWN + one RESTORED whichever pager won. §B (1) integrity _dispatch → _send_integrity_alert → delivered → record_integrity_handback → file; trader dispatch → _dispatch_page / _dispatch_resolves → integrity_handled(key, polarity, generation). (4) A race where the trader's page is already in flight when integrity delivers can still duplicate (fail-open). (5) TestIntegrityHandback (6: integrity delivers DOWN → trader skips and integrity resolves; integrity DOWN fails → trader sends; integrity delivers RESTORED → trader settles silently; integrity RESTORED fails → trader sends; generation-scoped; file 0644 + survives a trader restart). The B1-owned R6-2 / R6-4 / R6-5 and R6-6 follow with B1's next head.
• Round 8 (Sep 27, 2026; B1 c814593 merged — R6-2 counter generations, R6-4 page_unproven, R6-5 heartbeat parser): B2 now (a) reads the ibkr-monitor heartbeat only through B1's src.ibkr.monitor_heartbeat.parse_heartbeat (live ownership needs fresh valid proof; the delivered-RESTORED fact is accepted from a valid heartbeat up to 24 h old) — B2's own schema checks are gone and malformed proof is no proof; (b) passes B1's claim tokens to mark_page / mark_resolve; (c) sends ONE combined "was DOWN from … to … UTC, now RESTORED; the outage page may not have reached you (the trader restarted mid-send); no action needed" resolve when the ledger marks it page_unproven (IBKR, reconciliation guard, leverage), matching the monitor's and integrity's wording; (d) keeps the R6-3 handback keyed by B1's opaque counter generation (<opened_at>-<n>, equality only); (e) classifies a skipped RecycleResult (deferred / refused, retried by B1's backoff) structurally as information, and the "restart did not run" page says the trader retries automatically. No B1 change is needed for R6-3: the hook is already in the merged integrity_monitor._send_integrity_alert (a second call from _dispatch would be redundant). §B (1) dispatch → _dispatch_page (claim_page → mark_page(claim=)) / _dispatch_resolves (claim_resolves → handback check → _resolve_message → _unproven_resolve_message if page_unproven → mark_resolve(claim=)) → _sync_monitor_restored → monitor_last_resolved → _heartbeat_proof → parse_heartbeat. (5) Tests: TestUnprovenPageResolve (3 keys + plain resolve + claim tokens), TestHeartbeatParsedByB1 (6 malformed shapes → trader pages; valid pending proof defers; no B2 schema), test_handback_keys_use_the_counter_generation, TestRecycleResultIsStructured (2); R5-1 test now expects the combined message.
• Round 9 (Sep 27, 2026; review of 9834776, R7-1 / R7-2 / R7-3; B1 d862942 merged): a record written after a send cannot stop a second pager, so every DOWN / RESTORED send for a physical IBKR outage now takes a cross-process claim first — NEW src/earnings/outage_arbiter.py (JSON under fcntl.flock, fsync + atomic rename, ALERT_ARBITER_PATH = logs/alert_arbiter.json), used by the trader dispatcher, integrity's _send_integrity_alert (trader-paged rules) and ibkr-monitor's _page_live / _flush_resolves. Verdicts send / delivered / busy / no_down; a failed send releases the claim, a claim older than 150 s is taken over, arbiter I/O errors fail open. Invariants per outage: I1 one DOWN, I2 one RESTORED, I3 RESTORED only after a delivered DOWN (a dead claimant's DOWN → one combined "may not have reached you" RESTORED), I4 never zero, I5 membership captured at claim time — a new generation of a pager never joins an outage holding a different generation of that pager (R7-1). The round-7 post-send handback (logs/alert_handback.json) is removed. may_have_restarted (B1): the "restart did not run" page says the outcome is unknown, the weekly budget is used and there is no automatic retry. Design note: store internal/reliability/b2-outage-arbiter-design.md. §B (1) trader _dispatch_page → (monitor deferral) → outage_arbiter.claim("down","trader",gen) → send → settle; _dispatch_resolves → claim("up"); integrity _dispatch → _INTEGRITY_GENERATION → _send_integrity_alert → OD6 fast path → claim(phase,"integrity",gen,correlate={trader: gen captured now}) → send → settle; GatewayPager._page_live / _flush_resolves → claim(...,"ibkr-monitor",gen). (4) Fail-open duplicates only: arbiter unwritable; a pager joining > 30 min after the DOWN; a trader dead for the whole outage restarting after it. (5) Tests: tests/unit/test_finding_451_outage_arbiter.py (7, real concurrent processes: 4-way DOWN race, failed send taken over, dead claimant taken over after TTL, RESTORED race, real trader vs real integrity DOWN, real GatewayPager vs integrity RESTORED, R7-1 across processes), TestOneDownOneRestoredSequential (6), test_ambiguous_restart_page_says_outcome_unknown_and_budget_used.
• Round 10 (Sep 27, 2026; review of d939bb3, R8-1..R8-4; B1 1f09008 merged, which brings master 860af84):
• R8-1 — every outage page through the arbiter. The arbiter gains generation-scoped action phases action:approve_2fa / action:recycle_failed / action:recycle_needed (I6: at most one send each per outage, never a second DOWN); claim("down", ..., action=kind) claims the DOWN and the action in one critical section and settle_claims settles both. ibkr-monitor's 2FA-needed page = DOWN + approve_2fa: after another pager's delivered DOWN it goes out as the phone step only ("The outage alert for this was already sent"), not at all after a delivered approve_2fa, and it waits behind a busy DOWN (it no longer bypasses delivered / busy). The trader's action pages claim DOWN + action and its escalations claim the action phase; recon_guard joins the arbiter when its page falls due inside an open ibkr incident (decided once per generation; its RESTORED follows the same path) and pages itself otherwise. Integrity claims for systemd:ibgateway-ibc.service (integrity:ibgateway) next to trader:ibkr_request_path, and for trader:restart_tripwire only when it FAILs on the #454 attach-bounce marker (integrity:attach_bounce; its resolve only if it joined); all correlate the trader's and the monitor's generations (alert_policy.outage_correlation). Every outage page type is audited against the arbiter in the design note's table (store internal/reliability/b2-outage-arbiter-design.md).
• R8-2 — TTL bounded. ALERT_ARBITER_CLAIM_TTL_SEC must be in [120, 180] s: above the 81 s longest bounded send (15 s screenshot + Telegram and webhook, 3 × 10 s + 3 s backoff each) and at most the monitor's 180 s SLA. Unparsable / NaN / out of range → 150 s, logged ERROR; GUARD_REGISTRY flags an out-of-range .env override; .env.example documents it.
• R8-3 — open outages never pruned. Retention (24 h) applies only to settled records (RESTORED delivered, or no DOWN delivered or in flight); a live outage keeps one DOWN and one RESTORED however long it lasts.
• R8-4 — state loss. A RESTORED claim with no record (file missing, reset or corrupt mid-outage) joins the newest reconstructed outage within 30 min, or creates one, with its DOWN taken as delivered (a pager owes a RESTORED only after its own delivered DOWN): at most one RESTORED across the pagers, the first claimant sends, a failed send is taken over. A DOWN never joins a reconstructed record. Accepted edge: a DOWN claimed after the loss may repeat (duplicate over silence); that new outage then has its own one RESTORED.
• §B (1) Trace: monitor _observe_live → claim_page → _page_live → claim("down","ibkr-monitor",gen,action="approve_2fa"?) → full / action-only / none → settle_claims; trader dispatch → _dispatch_page → _outage_claimant → claim("down",claimant,gen,correlate,action) → _send → settle_claims; _dispatch_actions → claim("action:<kind>"); _dispatch_resolves → claim("up") (→ _reconstruct when unrecorded); integrity _send_integrity_alert → _outage_claimant(rule, detail, gen) → claim(phase, claimant, gen, correlate=outage_correlation()). (2) Runtime: Gateway crash with a fresh 2FA login after the trader's DOWN = 1 DOWN (trader) + 1 phone-step page (monitor) + 1 RESTORED; the trader's later LAST-RESORT approve_2fa is not repeated. (3) Lines: outage_arbiter.py (_claim_ttl_from_env, _prune/_settled, _reconstruct, _claim_action, claim, settle_claims, member_of); alert_policy.py _outage_claimant, outage_correlation, _dispatch_page, _dispatch_actions, _dispatch_resolves; gateway_pager.py _page_live; integrity_monitor.py OUTAGE_RULE_CLAIMANTS, _outage_claimant, _send_integrity_alert, GUARD_REGISTRY. (4) Limits: a recon_guard RESTORED can be the outage's one RESTORED if it resolves first (its text); B3's #454 attach-bounce tripwire still pages inside the nightly-reset window when no DOWN was delivered (B3 semantics, now deduplicated, flagged for review); fail-open duplicates as before. (5) Tests: test_finding_451_outage_arbiter.py (TTL bounds + import fallback + send-budget / SLA bracket + guard; open outage 3 days; reconstructed RESTORED with 3 real processes and a torn file; DOWN + 2FA + escalation race with 4 real processes in 3 orders; leaf-lock import check), TestEveryOutagePageThroughTheArbiter (11).
• Round 11 (Sep 27, 2026; review of 7131e35, R9-1..R9-6 + the Sunday-red suite; B1 deployed at 543fa5d, not rewritten):
• R9-1 — integrity joins before OD6. For its outage rules _send_integrity_alert claims the arbiter first, so even an outage another pager already paged gets integrity's generation as a member and integrity's RESTORED resolves that same outage (one RESTORED whichever pager resolves first). The OD6 delivery record is consulted after the claim: if the arbiter says send but the record proves the trader's delivered page (the arbiter lost it), the claim is settled delivered and nothing is sent; with the arbiter unusable OD6 still suppresses as before.
• R9-2 — explicit correlation is authoritative. _locate never falls through to the newest live outage when a correlated generation was found in a non-joinable (closed, stale, other-generation) outage: the DOWN starts its own outage; an UP reconstructs.
• R9-3 — actions. A DOWN claim with an action whose phase is in flight returns busy and claims nothing (no DOWN without the action it would carry); a page includes action text only when it owns the action claim (carries_action) — otherwise the trader and the monitor send the plain DOWN ("… the alert for its operator step was already sent"). settle_claims settles DOWN and action in one locked mutation (_settle_many); a crash inside it leaves both claims in flight together, taken over together.
• R9-4 — attach-bounce after state loss. When integrity dispatches the #454 attach-bounce opening it persists outage:integrity:attach_bounce in the integrity incident's ledger detail; the resolve record carries it (_INTEGRITY_OUTAGE_CLAIMANT), so the RESTORED always claims up and after arbiter state loss joins the reconstructed outage (member_of removed).
• R9-5 — claims never left in flight. _send_integrity_alert settles a held claim from a finally on every exit (import failure, transport exception, undelivered, an escaping exception), so another pager takes over at once instead of after the TTL.
• R9-6 — arbiter health visible. outage_arbiter.health() probes it as a pager uses it (directory / file writable, lock within 2 s, state parseable, TTL config) and reads alert_arbiter.json.health.json, where every claim / settle failure and every torn-file reset is recorded (best effort). NEW integrity rule alerting:outage_arbiter (WARNING, dashboard-only, systemd category so it runs with the DB down, outside the arbiter): FAIL when unusable now, WARN on a failure in the last hour. Registry pin 103 → 104 (b2_added).
• Sunday-red suite. Eight #418/#419/#420/#421/#437 tests reached recycle_defer_reason() with its default wall clock and failed inside the #439 Sunday 06:00–18:00 UTC quiet window. tests/unit/conftest.py weekday_recycle_clock pins that default now to Tue 15:00 UTC for those six classes; the explicit Sunday-window tests pass their own now and are unchanged; a new test runs with the real clock at Sunday 07:00.
• §B (1) Trace: integrity _dispatch → (attach-bounce: set_detail … outage:…) → _INTEGRITY_OUTAGE_CLAIMANT → _send_integrity_alert → _outage_claimant → claim → delivered/no_down handled · busy retry · OD6 → settle_claims(True) · transport in try → finally: settle_claims(delivered); arbiter claim("down", action=) → _locate (correlation authoritative) → busy action → busy; settle_claims → _settle_many (one _mutate). (4) Limits: the attach-bounce page stays (reviewer: acceptable while rare; a soak showing nightly recurrence reopens it); the health record is best effort (an unwritable directory is caught by the live probe instead). (5) Tests: TestRound9CallerRaces (11, both R9-1 orders, OD6 settle after state loss, plain-DOWN / busy-action for trader and monitor, R9-4 through the real IntegrityMonitor._dispatch with a state loss, R9-5 × 4 failures), TestRound9Arbiter (30: R9-2 × 2, every DOWN × action state combination, one-mutation settle, crash inside the settle, real-process crash inside the settle), TestArbiterHealth (5), test_quiet_window_reads_the_pinned_clock.
• Round 12 (Sep 27, 2026; review of e55bea0, R10-1 / R10-2; deployed master 543fa5d merged):
• R10-1 — fail-open keeps the action. An unusable arbiter made claim fail open without an action claim, and ibkr-monitor read that as "another pager sent the 2FA step" — the one page Marius must act on said the step was already sent. A fail-open DOWN claimed with an action now carries a fail-open action claim (carries_action); the trader and the monitor leave the action text out only for an explicit busy / delivered action claim (action_is_anothers). Tests with a real unusable arbiter file (read-only directory): the monitor's latched and non-latched 2FA pages, and the trader's action page and escalation for every action kind.
• R10-2 — health record concurrency. _note_failure now holds its own leaf fcntl lock (alert_arbiter.json.health.json.lock, given up after 1 s so a pager never blocks), fsyncs and renames atomically, and keeps the latest failure per operation plus at most 50 events from the last hour (health() reports recent_failures). Test: 6 real processes × 5 failures keep all 30 events.
• Merge: deployed master 543fa5d (B1 release, deploys 16–18: #483, #479 restart grace); conflicts only in docs/CONTEXT.md and this file, both sides kept.
• Acceptance: tests above. After deploy: M1 — zero trader pages for the 04:13 and 06:00 events and for deploys while IBC shows Login has completed and :4001 is up; trader log shows alert_policy: incident ibkr opened (expected=nightly_reset …) then resolved … no page. Drill (plan): a forced paper hang pages once with ibgateway-ibc in the text; D6 drill as tested.
• Changelog: one §O execution entry (alert semantics are live behaviour; no trading-rule change).
• Files: src/earnings/alert_policy.py (NEW), src/ibkr/client.py, src/ibkr/gateway_recycle.py, run_earnings_trader.py, src/earnings/reconciliation_manager.py, src/earnings/trading_orchestrator.py, src/earnings/config.py, src/data_tracker/integrity_monitor.py, .gitignore, tests/unit/test_finding_451_trader_alert_policy.py (NEW), tests/conftest.py, tests/unit/test_finding_419_ibkr_inplace_recovery.py, tests/unit/test_ibkr_error_block_policy.py, tests/unit/test_finding_420_ibkr_nightly_1100.py, tests/unit/test_reconciliation_guard_retry.py, tests/unit/test_finding_404_telegram_schedule.py, tests/unit/test_config.py, src/earnings/strategy_changelog.py, .cursor/memory.md, docs/OPERATIONS.md (§8.A3, §9 row, reconciliation-guard note), docs/CONTEXT.md, docs/AGENT_HANDOFF.md, this entry.

Finding 469 — PR Newswire check silently skipped for 5 min after a host boot; monotonic "never" sentinel (Sep 26, 2026)

• Severity: Medium (silent failure on the instant-exit detection path; register row 33). Reliability plan P1-f6 (close-out follow-up F-6), Batch B2.
• Status: IMPLEMENTED-PENDING-DEPLOY (Batch B2, branch reliability/b2).
• Root cause: EarningsDetector._check_prnewswire (src/earnings/earnings_detector.py) backed off with self._http_check_failures.get(cache_key, 0) > time.monotonic() - 300. time.monotonic() counts from host boot, so the default 0 is "failed at boot", not "never failed". For the first 300 s after any droplet boot the PR Newswire check returned "not released" without a log line or page. It dates from #395/#407 (per-feed key). The Batch A author found it when test_finding_395_prn_one_poller::test_two_symbols_one_urlopen failed on a low-uptime VM.
• Fix:
• _check_prnewswire: an absent key means never failed (last_failure is not None and last_failure > time.monotonic() - 300). A real failure still stamps time.monotonic() and backs off 300 s; a stamp of exactly 0.0 still counts as a failure.
• §E, same class: IBKRPositionCache._refresh_if_stale treated _last_refresh == 0.0 (never refreshed, or invalidate()) as fresh while uptime < TTL (5 s), serving an empty position map (is_position_open → False). Now 0.0 always refreshes, matching get_cache_age_seconds, which already read 0 as "never".
• §B:
• (1) Trace: detector thread (EARNINGS_POLL_ACTIVE_SEC) → check_earnings → _check_prnewswire → backoff check → _get_prnewswire_rss (30 s TTL cache, timeout=3) → parse → EarningsResult → instant-exit decision. Position cache: get_position / get_all_positions / is_position_open / get_shares → _refresh_if_stale → ib.positions().
• (2) Runtime: at 12 s uptime the old code computed 0 > 12 - 300 → True → return; the new code sees None → fetches. After a failure at t=40 s, calls up to t=340 s skip; t=340.1 s fetches.
• (3) Lines: earnings_detector.py _check_prnewswire backoff block; ibkr_position_cache.py _refresh_if_stale freshness check.
• (4) What could prevent it: nothing on the call path gates on uptime any more; _get_prnewswire_rss already tests if cached and …. The detector's other sources are unaffected (INFERRED unchanged, no shared backoff dict use).
• (5) Test tests/unit/test_finding_469_monotonic_never_sentinel.py (11 tests; time.monotonic patched, independent of real uptime): uptime 0.5/12/299 s performs the HTTP call; #395's two-symbols-one-fetch at 12 s; a release detected at 12 s; failure → exactly 300 s backoff; a 0.0 stamp still backs off; position cache refreshes at 1–2 s uptime, after invalidate(), and still caches inside the TTL; §E tripwire (no .get(…, 0) > time.monotonic() in src/, earnings_dashboard/, scripts/, strategies/). pristine parent facb218: 9 failed / 2 passed (re-run in review round 2).
• §E: every time.monotonic() site in src/, earnings_dashboard/, scripts/, strategies/, run_*.py, deploy/ reviewed. Correct already: utils/caching.py Polygon/Finnhub throttles (<= 0 → no wait), dashboard coverage cache (at is not None), orchestrator watchdog (initialised to monotonic()), btc5m jsonl_io (initialised to monotonic()), deadline-style uses (+ timeout). No perf_counter() in live code. time.time() variants cannot misfire this way (0 is 1970), and no .get(…, 0) > time.time() exists.
• Acceptance: under simulated uptime < 300 s the check performs its HTTP call; after a real failure it backs off 300 s (tests). After the next real host reboot the detector's first poll reaches PR Newswire.
• Changelog: one §O integrity entry (restores intended detection; no gate/sizing/order change). The plan listed §O as no; the operator's B2 instruction requires an entry for every live-behaviour change.
• Files: src/earnings/earnings_detector.py, src/earnings/ibkr_position_cache.py, tests/unit/test_finding_469_monotonic_never_sentinel.py (NEW), src/earnings/strategy_changelog.py, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, this entry.

Finding 482 — widen trades.token_label to match outcome (Sep 27, 2026)

• Severity: Medium (market-data completeness; no trading impact).
• Status: DEPLOYED Sep 27, 2026 02:43 UTC (deploy #15 860af84, PRs #12 and #13, master fast-forward 28b572b..860af84; unit polymarket-deploy-20260927T024258Z-2459628687, DEPLOY_EXIT rc=0; migration 042 applied on attempt 1).
• Problem: token_label stayed varchar(20) when #294/028 widened outcome to varchar(64). Collectors bind outcome into both columns; the Sep-26 incident trade (22-char outcome) poison-quarantined under #481.
• Fix: 042_widen_trades_token_label.sql — ALTER TABLE trades ALTER COLUMN token_label TYPE VARCHAR(64) (028 pattern). Integrity telegram_when nit: page on FAIL and OK (RESOLVED), not on live-trade poison WARN.
• Post-deploy: OPERATIONS § "Post-deploy: release Sep-26 live-trade poison after migration 042" — db_write_spool_admin.py release then restart polymarket-live-trades.
• §B:
• (1) Trace: deploy/deploy_from_github.sh → deploy/migrate.py MANIFEST entry main/042_widen_trades_token_label.sql (column_probe_typmod=True, :189-193) → probe_sql / _column_probe_cond (:239-280) compares format_type(atttypid, atttypmod) to the reviewed baseline character varying(64); on live varchar(20) the probe is false → psql -1 applies the ALTER and records the ledger row before any restart → live_trades_collector._write_trade binds outcome into token_label varchar(64).
• (2) Runtime: a VARCHAR widen is catalog-only on the Timescale hypertable (1047 chunks, compression off, 0 compressed chunks, no dependent views/indexes/constraints — read-only probe after deploy #14); applied under lock_timeout=5s / statement_timeout=120s.
• (3) Lines: 042_widen_trades_token_label.sql; deploy/migrate.py:239-280 (probe_sql, _column_probe_cond — format_type comparison when column_probe_typmod and the baseline type are set); :87-89 (column_probe_typmod field) and :189-193 (042 MANIFEST registration); integrity_monitor.py artifact:db_write_spool telegram_when (page on FAIL and on OK/RESOLVED, not on live-trade poison WARN).
• (4) What could prevent it: lock contention on trades (lock timeout → the migration fails and the deploy stops before restarts); a presence-only probe on live varchar(20) with baseline varchar(64) → probe_sql true → baseline_problems → MigrationError (refuses APPLY — no ledger row; column stays varchar(20); the 704d784 ship-stopper), not a silent ledger record; presence-only with a matching baseline would bootstrap-record without DDL (not this catalog); column_probe_typmod plus _column_probe_cond/probe_sql make the probe false on varchar(20) so APPLY runs; a future writer binding a longer outcome (still outcome varchar(64) = token_label varchar(64)).
• (5) Tests: tests/unit/test_finding_482_token_label_widen.py (manifest registration, SQL, typmod-probe, 029 existence-only, applies when the ledger row is missing and the column is still varchar(20)) and tests/unit/test_finding_481_live_trades_poison.py::test_db_write_spool_telegram_pages_fail_and_ok_not_warn.
• Post-deploy (deploy #15, 02:43–02:51 UTC Sep 27; ops log internal/ops-log-2026-09-27-deploy-15.md in the Project store): after the ~02:09 DigitalOcean maintenance, a read-only Step A found the DB writable and all services healthy. migrate: main/042_widen_trades_token_label.sql: applied (attempt 1), done: current=8 bootstrap=0 applied=1; trades.token_label = character varying(64). Gateway PID 1215183 unchanged; trader restarted by the deploy (expected), flat and ready. Release per OPERATIONS: live-trades stopped, the 3 quarantined payload files (2 distinct trades — one trade had two spool files under one key; a third file had appeared at 02:42 UTC before the deploy) released with operator cursor-deploy-15 and RELEASE-AFTER-VERIFICATION, copies under released/; live-trades restarted ~02:46:59. Result: the Sacramento Republic FC trade stored (trades 1, trade_dedupe 1); the Igor Ribeiro Marcondes trade not stored (the API no longer returns it — accepted by the runbook). quarantine/ empty; artifact:db_write_spool ok ("DB-write spool empty"); one CRITICAL RESOLVED page for artifact:db_write_spool at 02:51:25; no live_trades_db_write CRITICAL and no 22001 after 02:47.
• Files: src/data_tracker/migrations/042_widen_trades_token_label.sql, deploy/migrate.py, deploy/migration_baseline.json, src/data_tracker/integrity_monitor.py, docs/OPERATIONS.md, tests/unit/test_finding_481_live_trades_poison.py, tests/unit/test_finding_482_token_label_widen.py, this entry.

Finding 481 — live-trades #455 poison row spool loop + CRITICAL Telegram storm (Sep 26–27, 2026)

• Severity: High (market-data integrity + alert fatigue; no trading impact).
• Status: DEPLOYED Sep 27, 2026 00:33 UTC (deploy #14 28b572b, PR #11, master fast-forward 00607c9..28b572b; unit polymarket-deploy-20260927T003319Z-514718516, DEPLOY_EXIT rc=0; the deploy restarted polymarket-live-trades, stopped since 23:39:08 Sep 26). The incident's quarantined trades were released after #482 in deploy #15 860af84.
• Trigger: deploy #13 (00607c9, B3 #454/#455) at 23:29–23:33 UTC Sep 26. One Polymarket trade fails INSERT INTO trades with SQLSTATE 22001 because outcome (22 chars) is copied into trades.token_label varchar(20). Pre-#455 the collector logged and marked the trade seen (37× that day). Post-#455 the trade spooled, replayed, quarantined, was not marked seen, the drained branch re-armed the once-only CRITICAL, and the API re-delivered the trade → ~26 CRITICAL + 8 WARNING Telegram pages in ~5 min. Operator stopped polymarket-live-trades at 23:39 UTC.
• Root cause: public.trades.token_label is varchar(20); live_trades_collector._write_trade binds token_label from outcome (fits trades.outcome varchar(64)). Follow-up (history): done as #482 — migration 042 widened token_label to varchar(64) in deploy #15 860af84.
• Fix:
• Permanent DB errors on first save_trade → quarantine_once (one file per payload hash), mark seen, no spool, no CRITICAL.
• Replay path: quarantine permanent rows, mark seen, skip duplicate payload hashes, no quarantine CRITICAL for live_trade.
• Drained branch clears the outage alert only after successful replay (recovered > 0).
• Startup: dedupe_quarantine("live_trade") archives duplicate quarantine files by payload hash and seeds _seen_trade_ids / _poison_payload_hashes.
• artifact:db_write_spool: trader quarantine/stale pending still STATUS_FAIL + CRITICAL Telegram via telegram_when; distinct live_trade poison → STATUS_WARN only (dashboard).
• §B: (1) API re-fetch of poison trade → seen/poison-hash gate → no re-spool. (2) transient 08xxx → spool + one CRITICAL + replay ack. (3) integrity distinct poison count survives 12 duplicate prod files after dedupe. (4) trader position_open quarantine still FAIL. (5) tests/unit/test_finding_481_live_trades_poison.py.
• Post-deploy (deploy #14, 00:33–00:39 UTC Sep 27; ops log internal/ops-log-2026-09-27-deploy-14.md in the Project store): migrate: current=8 bootstrap=0 applied=0; six app services active, 0 journal err; trader ready, 0 positions, 0 breakers; ibgateway-ibc PID 1215183 unchanged, :4001 listening. No live_trades_db_write CRITICAL/WARNING after the 00:34:01 restart; one integrity RESOLVED for systemd:polymarket-live-trades.service (00:37:12). Startup dedupe: 2 poison payload files kept in quarantine/ (1 trade — same spool key; Marcondes), 10 duplicate files archived; artifact:db_write_spool WARN (integrity counter: "2 distinct live-trade poison payload(s) quarantined (#481)" — two files, not two trades). Dashboard-restart false page again: systemd:polymarket-dashboard.service read deactivating at 00:33:57 during the deploy's restart and RESOLVED at 00:37:09 — the OPEN deploy-order race (see #479); its fix, PR #9, is pending.
• Files: src/data_tracker/collectors/live_trades_collector.py, src/utils/durable_write_spool.py, src/data_tracker/integrity_monitor.py, tests/unit/test_finding_481_live_trades_poison.py, this entry.

Finding 480 — pg8000 untyped binds: covered-channel lookup always 42P18; legacy writer retired; repo tripwire (Sep 26, 2026)

• Severity: Medium integrity / zero direct trading-rule impact. The live lookup failed on every invocation and wrote false timing_channel_leak audit rows, but it runs only inside a timing-mismatch branch that always skips; the exception is caught. It cannot admit a duplicate, block a valid same-channel entry or crash the trader. The explicit integrity detector independently excludes covered rows, so the four false labels did not make coverage:channel_window_leak FAIL.
• Status: DEPLOYED Sep 26, 2026 21:56 UTC (deploy #9 0e985dd, merge of PR #5 d61d8af after the cross-family review — code and migrations approved at b693dc6, docs-only re-check at d61d8af; migration 041 sha256 1f5a6e44…).
• Root cause / production proof: SQLAlchemy's pg8000 positional dialect expands repeated named binds into distinct parameters. In CHANNEL_ALREADY_COVERED_SQL, $1 was the bare :cid IS NOT NULL; the typed condition_id = :cid occurrence became a different slot and could not type $1. The same applied to bare :symbol and :earnings_dt. A read-only production transaction (statement_timeout=5s, exact SQL, str/str/aware-datetime, SQLAlchemy 2.0.45, pg8000 1.31.5) reproduced 42P18 could not determine data type of parameter $1. Retained Sep 12–26 trader logs have four executions, all Sep 23: GIS 14:30:07, CTAS 14:30:11, PAYX 14:30:11 and GIS 19:50:09 UTC (4 warning events / 12 traceback matches in the canonical log; the error log mirrors it). All four had prior non-timing coverage: GIS depth skips, CTAS/PAYX conviction-deadband skips.
• Fix:
• CHANNEL_ALREADY_COVERED_SQL: every occurrence of cid / symbol / earnings_dt is explicitly CAST to VARCHAR / TIMESTAMPTZ; #414/#431's condition-id-first, symbol+ET-day fallback and any-non-timing semantics are unchanged.
• Legacy earnings-sync decision: retire, do not revive. sync_from_markets_table() and run_full_sync() now contain only a docstring plus raise LEGACY_SYNC_RETIRED_MESSAGE; the unreachable writer bodies are deleted. python -m src.db.earnings_sync and scripts/sync_earnings_markets.py exit 2 with the replacement (src.data_tracker.collectors.earnings_market_fetcher). Production's hourly service already uses that fetcher; its only import from EarningsMarketSync is the supported NULL-mcap backfill. The typed update constant remains solely for real-driver regression proof and cannot write through either retired entry point. This avoids reactivating a January-broken writer that could change earnings_date without the mandatory earnings_date_audit row (#349 / memory §F). The dashboard's genuine-empty-table warning and its test point operators at the active fetcher module, not the retired command.
• tests/integration/test_batch_execution.py: allocation / shares casts both binds to NUMERIC, closing the test-only PostgreSQL 42725 unknown / unknown.
• Repo-wide source tripwire parses every non-docstring Python string containing a bind (including SQL fragments that do not start with a statement keyword) plus every .sql file. It rejects bare bind null tests, bind-to-bind comparison/arithmetic, bare SELECT/RETURNING, bare grouping/sort binds, ambiguous date_trunc('…', :bind) / EXTRACT(… FROM :bind), and SQLAlchemy's :bind::type mis-parse. SQL comments and the non-SQL strategy-changelog data file are excluded; explicit CAST(:bind AS type) cases are negative controls.
• Real-pg8000 integration tests are cloud-marked (integration, db, dedicated finding480_test* URL): exact channel query with condition-id and NULL fallback shapes; exact earnings update with confirmed/unconfirmed rows; AST-extracted batch-allocation INSERT.
• Data correction choice — migration 041: relabel rather than preserve known-false audit labels. 041_channel_leak_false_rows.sql updates only opportunity-log ids 9381 GIS, 9382 CTAS, 9383 PAYX and 9387 GIS from timing_channel_leak to timing_channel_mismatch. Every UPDATE carries the exact prod before-values (id, microsecond timestamp, condition id, symbol, channel, resolved timing, earnings timestamp, stage and reason), then GET DIAGNOSTICS ROW_COUNT + <> 1 RAISE. It is a DATA_MANIFEST migration: never schema-probed/bootstrap-adopted, one psql -1 transaction with its ledger row, checksum-immutable, and later deploys no-op from the ledger. #477 is merged at 4d80e11; the manifest is numerically pinned as 040 then 041 and both ship in the same data-migration deploy. This is preferable to a note because the rows are proven false, are part of the research audit trail, and can be corrected without deleting history or risking a broad update.
• §B:
• (1) Trace: BMO/AMC cycle → _get_signals_for_timing → timing differs → classifier says deadline passed → _already_covered_on_resolved_channel → typed lookup → covered row returns true → resolve_timing_mismatch_reason writes timing_channel_mismatch → opportunity row → unconditional continue. Same-channel signal generation never enters this branch. Legacy module/script → explicit retired guard → exit before engine access. Live hourly sync → earnings_market_fetcher (unchanged) → supported backfill helper only. Deploy → Runner.run schema MANIFEST → DATA_MANIFEST 040 → 041 → ledgers.
• (2) Runtime simulation: a covered wrong-channel probe now performs one SELECT and records mismatch; an uncovered probe returns no row and still records a true leak; DB outage still catches and conservatively keeps a leak; NULL fallback keys parse and execute under pg8000. Any legacy manual invocation fails before a connection, so it cannot bypass earnings_date_audit. A 040 or 041 mismatch rolls back that complete migration and stops before restarts; 041 mismatch rolls back all four relabels. At 4 AM, weekends and restart the query behavior is identical; no state is held in memory.
• (3) Lines: signal_generator.py CHANNEL_ALREADY_COVERED_SQL / _already_covered_on_resolved_channel; earnings_sync.py retired guards and typed legacy constant; scripts/sync_earnings_markets.py guard; migrations 040/041; deploy/migrate.py DATA_MANIFEST / Runner.run; the #477/#480 migration tests and #480 bind tests.
• (4) What could prevent it: an uncast occurrence would restore the positional-slot failure (fragment-aware source tripwire); a driver-only parser difference would escape mocks (real-pg8000 tests); a changed prod row would make 041 raise rather than mutate the wrong row; a lost ledger would make the exact before-value guards refuse a second correction rather than double-apply. The handoff's exact read-only pre-check must return 4 before deploy. Real-pg8000 tests require the dedicated cloud DB and skip without its URL.
• (5) Acceptance after merging #477 and deployed master 614a29d: full unit suite 2,541 passed (14 warnings); scripts/doc_drift_audit.py CLEAN (12 checks, 0 violations). Review VM on pinned pg8000/PostgreSQL 16: all three #480 real-driver tests passed and migration 041 passed exact/apply, ledger rerun, drift rollback, already-relabelled fail-closed and look-alike isolation cases through the real runner. Local #477/#480 integration files collect 8 tests and skip without their dedicated URLs. Post-deploy (not this branch): pre-check returns 4; 040 then 041 log their one-row notices; the four ids read timing_channel_mismatch; no new covered-channel lookup warning/42P18 appears; integrity remains 0 explicit + 0 swept.
• §E: exhaustive executable-string + fragment sweep found exactly the two original runtime bare-null sites and one bind-to-bind arithmetic test; all are cast or retired. No executable bind-to-bind comparison, :bind IS NULL, ambiguous date_trunc/EXTRACT, pyformat or $n equivalent remains. earnings_market_fetcher.py's old pattern is a comment documenting its prior Python-side fix; ::date < :entry_d is a PostgreSQL cast plus a typed column comparison. Typed COALESCE(:bind, column) / column IS DISTINCT FROM :bind sites obtain type from the column and are outside this defect. The tripwire now scans the ~13 fragment literals too; explicit casts remain clean.
• Post-deploy (21:59–22:05 UTC, deploy #9 — covers #477/#480): acceptance met.
• Pre-deploy gate, read-only at 21:56:07 UTC: the 041 query returned 4; each of 040's three guarded WHERE clauses matched exactly 1 row; schema_migrations had no 040/041 row.
• Unit polymarket-deploy-20260926T215623Z-326053163 21:56:26 → 21:59:54 UTC, DEPLOY_EXIT rc=0; pull f28c0da..0e985dd, 31 files; hygiene 0/0/0; needrestart OK (14). migrate: main/040_pnl_fields_bbw_snow.sql then main/041_channel_leak_false_rows.sql, each applied on attempt 1 in one transaction with its ledger row — three Finding #477 040 and four Finding #480 041 NOTICE lines (before → after) — done: current=5 bootstrap=0 applied=2. Close-persist contract holds; / HTTP 200.
• Rows (read-only): BBW trade_audit_log 274 gross/net/realized −282.10; trading_positions 307 exit 34.66, realized −282.10, pct −11.604183; SNOW 75 net/gross/realized −1.16. Opportunity-log 9381 GIS, 9382 CTAS, 9383 PAYX and 9387 GIS read timing_channel_mismatch (0 leak rows left among them). Ledger: 040 and 041 applied by 0e985dd at 21:56:30.

• 477 detector (_load_pnl_consistency_rows + _evaluate_pnl_consistency, read-only, full history): 214 rows, 0 mismatches, 3 legacy pre-epoch twins (ids 4, 9, 10), so nothing to page. The scheduled 7-day and full-history reconcile runs had not run yet at this record (UNVERIFIED until the next daily run).

• Phase-1 PNL_EFFECTIVE_SQL under TRADE_METRICS_FILTER_SQL (closed, entry ≥ 2026-08-21 ET): n=13, −$922.2826 — equal to scripts/phase1_pnl_record.json, which no longer needs its BBW override to match the DB.
• Integrity 87 rules: 85 ok / 2 warn (table sizes); the restarted monitor's first cycle had 0 transitions and nothing paged; security:db_client_activity ok. The #479 sampler-restart false page did not recur (the old monitor's last cycle ran at 21:58:48; the sampler restarted at 21:59:43).
• Services: the 6 app services restarted; ibgateway-ibc 1215183, xvfb 1215180, x11vnc 1215184 and :4001 java 1537985 unchanged; root crontab sha256 unchanged (99928f0f…22c7). Trader reconnected 21:56:52 (same-thread req* ok (#440); reconciliation in sync); ready, 0 breakers, 0 positions, 0 stops today.
• Logs 21:56:26 → 22:05 UTC: 0 42P18 / covered-channel lookup warnings; the only ERROR line is the connect-time IBKR 2107; 0 journal err. The covered-channel path runs in the BMO/AMC cycles, which a Saturday does not exercise.
• Changelog: one §O integrity entry (#480): live opportunity-log classification changes from false leak to mismatch; no strategy/admission change and no history marker.
• Files: src/earnings/signal_generator.py, src/db/earnings_sync.py, src/db/__init__.py, scripts/sync_earnings_markets.py, scripts/verify_retry_backoff_and_block.py, tests/integration/test_batch_execution.py, deploy/migrate.py, src/data_tracker/migrations/041_channel_leak_false_rows.sql (NEW), tests/unit/test_finding_480_pg8000_bind_typing.py (NEW), tests/unit/test_finding_480_migration_041.py (NEW), tests/integration/test_finding_480_pg8000.py (NEW), tests/unit/test_finding_414_channel_leak_already_admitted.py, tests/unit/test_finding_444_migration_ledger.py, tests/unit/test_finding_474_bootstrap_baseline.py, src/earnings/strategy_changelog.py, docs/OPERATIONS.md, docs/DATABASE_DIRECTIVES.md, docs/DATA_ARCHITECTURE.md, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, this entry.

Finding 477 — Exit-price corrections left gross_pnl / net_pnl_after_financing and the trading_positions twin stale (BBW#274, SNOW#75) (Sep 26, 2026)

• Severity: High. Reporting (PNL_EFFECTIVE_SQL), the daily-loss breaker input (risk_manager.py COALESCE(net_pnl_after_financing, realized_pnl)) and the integrity day-P&L all read net_pnl_after_financing before realized_pnl. BBW's stale net understated the Phase-1 loss by $160.58.
• Status: DEPLOYED Sep 26, 2026 21:56 UTC (deploy #9 0e985dd, shipped with #480 in PR #5; data correction = migration 040 (sha256 6446315f…), applied by deploy/migrate.py in deploy #9 at 21:56:30 UTC; deploy-wide checks under #480 Post-deploy).
• Problem (bounded read-only prod SELECTs on the droplet, Sep 26 2026 ~17:55 UTC):
• trade_audit_log 274 BBW LONG 62 @ 39.2100: exit_price 34.6600, realized_pnl −282.10 (broker fill (34.66 − 39.21) × 62), but gross_pnl = net_pnl_after_financing = −121.52, the $37.25 hardware-stop stamp (37.25 − 39.21) × 62. financing_cost_total 0; exit_commission NULL; total_commission 0.3628.
• trading_positions 307 (same symbol + entry_time): exit_price 37.25, realized_pnl −121.52, realized_pnl_pct −4.998725. Never corrected.
• trade_audit_log 75 SNOW SHORT 2 @ 165.8700 (Feb 25, testing era): exit_price 166.4500, realized_pnl = gross_pnl = −1.16, but net_pnl_after_financing = −9.96 = (165.87 − 170.85) × 2, the $170.85 stop stamp. Twin 108 is already correct (166.45 / −1.16).
• Root cause: trade_reconciler._exit_price_correction_params + the exec-link UPDATE (#423) rewrote only exit_price, realized_pnl, realized_pnl_pct. The close paths (position_manager.persist_position_close_with_exit_details, the orchestrator execution-ledger close) write realized_pnl, gross_pnl and net_pnl_after_financing together from one :pnl, and write the twin. The correction had its own partial copy. SNOW#75 is the same class through a different writer: the Feb 26 hand repair (row notes "REPAIR 2026-02-26 … not stop_loss at 170.85") set exit/realized, #332 later fixed gross_pnl, and nothing fixed net.
• SNOW class answer (question 3): SNOW is not a commission double-count. No closed row has net_pnl_after_financing ≈ realized_pnl − financing − total_commission (scan n=0), and no row has realized_pnl ≈ gross_pnl − total_commission (n=0). So COALESCE(net_pnl_after_financing, realized_pnl) − total_commission never double-counts through the net column. SNOW is the stale-stop-stamp class. The full-history scan finds exactly two members: |gross − realized| > 0.005 → {274}; |net − (realized − financing)| > 0.005 → {75, 274}. Both are fixed by 040. SNOW's canonical net −10.9604 → −2.1604 (+$8.80, all-time, not Phase 1).
• A different class, proposed, NOT fixed (evidence only):
• January launch-era rows whose realized_pnl is itself net of a commission: T#10 (price arithmetic 5.32, realized 4.32), SBUX#11 (−14.07 / −15.07) and XOM#19 (−0.84 / −2.84).
• On those rows the canonical formula does double-count the commission: $4.00 in total. T#10's twin (5.32) agrees with the price arithmetic; SBUX/XOM twins agree with the audit rows.
• Twin mismatches before the #370 epoch: MTB#4 (twin realized 0.00, exit NULL, reconciliation:not_in_ibkr), TXN#9 (audit 12.39 vs twin 42.76 at the same exit 208.03; TXN is a named outlier) and T#10.
• Fixing them needs the January IBKR statements (Flex unavailable, IBKR 1001). They are outside Phase 1, and stats read trade_audit_log only (#353). The detector reports these twins as legacy.
• Also observed, not this finding (UNVERIFIED cause): 34 closed rows have realized_pnl ≠ (exit − entry) × shares by > $0.02, mostly cents. Phase-1 members: HPQ#271 −100.86 vs −100.45, CPB#285 −107.91 vs −107.415, KR#292 −94.00 vs −93.80. realized_pnl is the reference for every derived field, so this class is not flagged.
• Fix (writer):
• NEW src/earnings/pnl_fields.py:
  • AUDIT_PNL_ASSIGNMENTS is the one SQL fragment for realized_pnl / realized_pnl_pct / gross_pnl = :pnl / net_pnl_after_financing = :pnl − Σ margin_ledger financing;
  • price_pnl() is the one price formula;
  • rewrite_exit_price(conn, audit_id, exit_price) writes the audit row via the fragment, plus the twin matched on (symbol, entry_time) when exactly one twin matches.
• position_manager primary close now uses the fragment. The SQL is byte-equivalent (the inline gross_pnl = :gross_pnl with gross_pnl = pnl and the inline net expression moved into the fragment).
• trade_reconciler exec-link UPDATE now sets only link fields. When the fill differs from the stamped exit and the link UPDATE won (rowcount 1), it calls rewrite_exit_price.
• scripts/backfill_ibkr_executions_from_statement_csv.py rewrote entry_price / direction / exit_price with no P&L. It now re-derives through rewrite_exit_price per row.
• Fix (data) — migration main/040_pnl_fields_bbw_snow.sql in NEW DATA_MANIFEST:
• (a) deploy/migrate.py applies DATA_MANIFEST entries once, after every MANIFEST entry, via the existing apply(): psql -1 -f file -c <ledger INSERT>, so the file and its ledger row are one transaction. They are never probed or bootstrapped. sha256 drift fails closed.
• (b) The file does SET LOCAL lock_timeout '5s', statement_timeout '60s' and client_min_messages = notice. PGOPTIONS already sets warning, which would hide the log lines.
• (c) All three UPDATEs are one DO statement, with no transaction control and no exception handler.
• (d) Every WHERE carries the exact prod before-values: ids, symbol, direction, status, shares, prices, realized, pct, gross, net, financing, total_commission, exit_commission IS NULL, the exit exec id, and for the twin entry_time.
• (e) Each UPDATE is followed by GET DIAGNOSTICS n = ROW_COUNT; IF n <> 1 THEN RAISE EXCEPTION, which rolls back all UPDATEs and the ledger row. migrate.py then exits 1 before restarts, with no retry (not a lock timeout).
• (f) One RAISE NOTICE per row with before → after, echoed into /var/log/polymarket/deploy-migrations.log.

• Exact changes:

  row	column	before	after
  trade_audit_log 274 BBW	gross_pnl	−121.52	−282.10
  trade_audit_log 274 BBW	net_pnl_after_financing	−121.52	−282.10
  trade_audit_log 274 BBW	notes	NULL	Finding #477: gross_pnl/net_pnl_after_financing -121.52 -> -282.10 (stop stamp -> broker fill)
  trading_positions 307 BBW	exit_price	37.25	34.66
  trading_positions 307 BBW	realized_pnl	−121.52	−282.10
  trading_positions 307 BBW	realized_pnl_pct	−4.998725	−11.604183
  trade_audit_log 75 SNOW	net_pnl_after_financing	−9.96	−1.16
  trade_audit_log 75 SNOW	notes	existing	existing + \| Finding #477: net_pnl_after_financing -9.96 -> -1.16 (stop stamp -> repaired fill)

  Plus updated_at = NOW() on the three rows. realized_pnl, exit_price (audit), every commission column and exit_commission (stays NULL, never invented) are unchanged. - Prod pre-check (read-only, Sep 26 ~18:11 UTC): each of the three WHERE clauses, lifted programmatically from the file, matches exactly 1 row. Prod is PostgreSQL 16.15, and main/040 is absent from schema_migrations. - Fix (detection): trade_reconciler.reconcile_trades now also runs _load_pnl_consistency_rows + _evaluate_pnl_consistency. It covers closed rows exited or updated_at inside the window (7-day timer) or all rows (--full-history timer). It flags gross_pnl_stale (|gross − realized| > 0.01), net_pnl_stale (|net − (realized − financing_cost_total)| > 0.01) and twin_pnl_mismatch (exactly-one twin whose realized_pnl or exit_price differs by > 0.01). - NULL gross/net are skipped. - Twin mismatches before EXEC_LINKAGE_EPOCH_UTC are legacy gaps (#370 pattern). - Mismatches are trade issues, so the run persists status=error, exits 2 and sends the existing trade_reconciliation alert. - The tolerance is one cent: stored P&L is NUMERIC(.,2) and the twin's exit_price NUMERIC(10,2) (EB#77 −0.505 stored −0.51 stays clean). - After 040 the full-history run has 0 new issues; MTB#4 / TXN#9 / T#10 are legacy. - Placement / merge plan: - Detection lives in trade_reconciler.py, not integrity_monitor.py. B1 (+385 lines), B2 (+34) and B3 (+112) all edit the monitor; B3 touches trade_reconciler.py at one line (824), far from these hunks. - migrate.py: B3 inserts 039 inside MANIFEST (after 038) and follow-ups rewrites the baseline/bootstrap code. DATA_MANIFEST sits after MANIFEST's close, and Runner.run() / __init__ are untouched by both, so 039 and 040 apply in either deploy order. --check-baseline / --capture-baseline (follow-ups) iterate MANIFEST only. - test_finding_444 edits are two lines (_runner(data_manifest=()), the registration set) outside every B3 / follow-ups hunk. - FINDINGS / changelog / CONTEXT / HANDOFF append at the shared insertion points: add/add, keep both. - A later integrity rule, after B1+B2 land, can call _evaluate_pnl_consistency; not needed for the alert. - §B: - (1) Trace: - correction: polymarket-trade-reconcile(.full).service → main() → _auto_fix_trade_exec_links → _find_execution_match / log candidates → INSERT ibkr_executions → link UPDATE → rewrite_exit_price → audit UPDATE via AUDIT_PNL_ASSIGNMENTS + twin UPDATE → commit; - close: ReconciliationManager / exit paths → persist_position_close_with_exit_details → twin + audit UPDATE via the same fragment; - detection: main() → reconcile_trades → _evaluate_pnl_consistency → _persist_job_result (status) + _send_alert_if_needed → exit 2; - data: deploy_from_github.sh → deploy/migrate.py → Runner.run() → DATA_MANIFEST loop → apply() → psql -1 -f 040 -c INSERT schema_migrations. - (2) Runtime: - BBW replay (unit fake): the audit UPDATE binds exit_price 34.66 and pnl −282.10, and the twin UPDATE binds the same. - Live Postgres 16.15 (throwaway local cluster), via the real Runner and psql: 040 applied; 274 gross/net −282.10, exit commission NULL; 307 34.66 / −282.10 / −11.604183; 75 net −1.16 with the REPAIR note kept; three NOTICE lines in the runner output; the second run current=2 applied=0 with byte-identical rows. - SNOW before-value drifted to −9.95: rc 1, BBW rows unchanged, no ledger row. - Ledger row deleted, then rerun: rc 1 (id=274 matched 0 rows), no double apply. - Hand psql -1 with the twin row locked and no PGOPTIONS: lock timeout in < 20 s, nothing changed. - (3) Lines: src/earnings/pnl_fields.py; src/earnings/position_manager.py primary close {AUDIT_PNL_ASSIGNMENTS}; src/data_tracker/collectors/trade_reconciler.py _exit_price_correction_params, exec-link UPDATE + rewrite_exit_price, _load_pnl_consistency_rows, _evaluate_pnl_consistency, reconcile_trades; deploy/migrate.py DATA_MANIFEST + run() loop; src/data_tracker/migrations/040_pnl_fields_bbw_snow.sql. - (4) What could prevent it: - The twin has no FK. The (symbol, entry_time) match covers 213/214 closed rows, and the twin write is skipped unless exactly one row matches. - A 040 before-value that drifts before deploy fails the deploy closed (old code keeps running). Re-read the rows and ship 041. - The correction runs only when the exec link is new (ibkr_exit_exec_id IS NULL), so BBW is never re-touched by code; hence 040. - The #340 fallback close still leaves net NULL (margin-ledger aggregates are primary-only by design). Reporting falls back to realized_pnl, the detector skips NULL, and the tripwire pins exactly one such writer. - (5) Tests (fail on pre-#477 code: 10 of 12 in tests/unit/test_finding_477_pnl_fields.py; the other two are regression guards): - tests/unit/test_finding_477_pnl_fields.py (12): the correction writes gross/net/twin; no write when the link race is lost or the fill equals the stamp; the shared fragment is used by the close path; a same-pattern tripwire over every active-code UPDATE trade_audit_log literal (realized_pnl ⇒ gross + net, except the one #340 fallback; exit_price ⇒ P&L or rewrite_exit_price); the detector on the prod rows; the reconcile_trades → issues → status=error path. - tests/unit/test_finding_477_migration_040.py (16): one test per safety property: single transaction / timeouts / exactly one row / exact before-values / ledger idempotent / log line. Mutation-checked: dropping -1, adding COMMIT, splitting the DO, adding an exception handler, removing either timeout, removing or weakening a row guard (> 1), removing a before-value, setting exit_commission, removing the ledger check, removing a NOTICE or the notice level each fail (14/14). - tests/integration/test_finding_477_postgres.py (5, PNL477_TEST_DATABASE_URL): the live runs in (2). - §E (exhaustive grep \b(realized_pnl|exit_price)\s*= + UPDATE trade_audit_log / trading_positions over src, scripts, earnings_dashboard, deploy): - (a) Fixed: the trade_reconciler correction; the statement-CSV backfill script. - (b) Uses the fragment: position_manager primary close. reconciliation_manager ledger close delegates to it (persist_position_close_with_exit_details). - (c) Deliberately left: - the position_manager fallback close (#340: realized + gross, net NULL on the open row); - the orchestrator execution-ledger close (trading_orchestrator.py ~1745–1850). It already writes all three from :pnl plus the twin; B3 makes it unreachable (delegates to ReconciliationManager); editing it would conflict with B3's rewrite and the docs-sweep comment edits there. - (d) Stamp sources, consistent by construction: exit_manager.py ~701 and trading_orchestrator.py ~4130 default exit_price to the stop when fill details are missing, then call the close path. The later correction is the repair. - (e) Not trade-table writers: kwargs into the close path (exit_manager, trading_orchestrator, reconciliation_manager, scripts/close_persist_contract_check.py); hypothetical_exit_price (counterfactual tables); analysis/simulator loaders; src/ibkr/client.py local realized_pnl; archive/scripts/fix_database.py (archive). - Readers of net: earnings_dashboard/app.py PNL_EFFECTIVE_SQL, risk_manager.py daily P&L and integrity_monitor.py day P&L. They are correct once 040 applies; the Aug 27 historical day-P&L changes, with no live effect. - Impact after deploy: Phase-1 known net −$922.2826 before BBW's missing exit commission (reviewer-confirmed; the pre-review docs-sweep figure −$761.7026 + (−282.10 − (−121.52)) = −$922.2826). All-time PNL_EFFECTIVE −$151.78 (BBW −160.58, SNOW +8.80). Playbook §1 / strategy report numbers regenerate after 040 (§K/N re-run after material DB changes). - Changelog: one §O integrity entry; #423's entry AMENDED (superseded_by #477). - Files: NEW src/earnings/pnl_fields.py, NEW src/data_tracker/migrations/040_pnl_fields_bbw_snow.sql, src/earnings/position_manager.py, src/data_tracker/collectors/trade_reconciler.py, scripts/backfill_ibkr_executions_from_statement_csv.py, deploy/migrate.py, tests/unit/test_finding_444_migration_ledger.py, NEW tests/unit/test_finding_477_pnl_fields.py, NEW tests/unit/test_finding_477_migration_040.py, NEW tests/integration/test_finding_477_postgres.py, src/earnings/strategy_changelog.py, docs/OPERATIONS.md, docs/DATA_ARCHITECTURE.md, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, this entry.

Finding 467 — Public dashboard defense in depth: secret-value response filter, FastAPI attack surface removed, IBKR/wallet/path identifiers masked (Sep 26, 2026)

• Severity: Critical (live-credential leak; the dashboard is intentionally public). Reliability plan — "dashboard open to the internet" item. The operator decided the dashboard and its /docs/* pages stay open to others, and that secrets must never appear in docs. Sibling of #457 (committed-secret scrub on reliability/cred-scrub, with its own guard test over tracked files) — the two together are the only controls that keep secrets off the public dashboard.
• Status: DEPLOYED Sep 26, 2026 22:19 UTC (deploy #11 3138aca, merge of PR #4 d576b7d after the GPT round-3 approval; detached unit polymarket-deploy-20260926T221858Z-2508114717 22:19:01 → 22:22:24 UTC, DEPLOY_EXIT rc=0).
• Trigger: /docs/operations rendered docs/OPERATIONS.md verbatim, and that doc had the full DATABASE_URL (including its password) pasted into it. An internet scanner harvested it on Sep 14. This finding is the runtime guard: no dashboard RESPONSE can leak a secret or a sensitive identifier again, regardless of how it gets into the response body (a doc, a log excerpt reflected into an error message, a stray f-string) — independent of #457's scrub of what's currently committed.
• Fix — three parts, same finding:
• Response filter (earnings_dashboard/secret_redaction.py, own module, one line in app.py: install_secret_redaction(app) right after app = FastAPI(...)).
  • Redact by VALUE first. At install time (= app startup — prod's real secrets are already in os.environ by then, sourced from .env via the systemd unit's EnvironmentFile=) the module snapshots every secret-looking value out of the process environment: name ends in KEY/TOKEN, name contains SECRET/PASS/PASSPHRASE/PRIVATE_KEY, the password part of DATABASE_URL and the whole DATABASE_URL string, and the password segment of ANY env var whose value happens to be a URL with embedded credentials (covers DATABASE_URL without a special case). Values only — names are never logged, and the collected values themselves are never logged either.
  • Redact by SHAPE as a backstop: postgres URL credentials (any scheme://user:[REDACTED]@ — covers postgres and every other scheme with one pattern), DigitalOcean AVNS_ passwords, Telegram bot tokens — these fire even when the exact value isn't in the CURRENT environment (a stale/rotated credential left in an old doc).
  • Every response type, including FastAPI's own defaults: a pure-ASGI SecretRedactionMiddleware wraps send for every request — HTML, JSON, plain text (covers the btc5m subtree's separate PlainTextResponse docs route too), 404s. A catch-all Exception handler (also installed by install_secret_redaction()) exists for the true-unhandled-exception case, but — corrected post-review, see below — its response does NOT actually pass through SecretRedactionMiddleware: Starlette hands any handler registered for Exception/500 straight to ServerErrorMiddleware itself, which sits outside all user middleware and sends via the ORIGINAL send it was given. Safety on this one path is single-point: the handler calls redact_text() on the message BEFORE building the response, with no ASGI-level backstop behind it.
  • Bounded cost: a capped, longest-first literal alternation (≤500 literals); shape patterns with bounded quantifiers (no catastrophic backtracking); responses ≤16 MiB are redacted as one buffered unit with a corrected Content-Length; larger/streaming responses switch to bounded chunk-wise redaction, carrying back max(longest current literal, longest shape-pattern match, 4 KiB) so an in-progress match's visible-so-far portion is always fully inside the carried tail, and Content-Length is dropped rather than guessed. Non-text content types pass through unscanned. (Post-review fix — see below: the ORIGINAL shipped version split the raw window before redacting, which could cut a secret in half at the split point.)
• Attack surface removed (app.py):
  • FastAPI's own auto-generated schema/docs disabled via the constructor (openapi_url=None, docs_url=None, redoc_url=None) — /openapi.json, /docs, /docs/oauth2-redirect, /redoc all 404. The dashboard's OWN /docs/{doc_name} markdown route is untouched (different route, registered separately).
  • Three always-inert unauthenticated write endpoints deleted outright: POST /api/testing-prod/confirm, POST /api/testing-prod/reset, POST /api/set-trading-mode. Each began orchestrator = get_orchestrator(); if not orchestrator: return {"error": ...} — and get_orchestrator() always returns None (the trader runs as its own systemd service; the dashboard is DB/file read-only by design). save_persisted_mode() deleted too — its only caller was the removed set-trading-mode endpoint. GET /api/testing-prod/status (read-only) is unaffected. Stale UI references fixed: dashboard.html's "Manual Confirmation Required" panel no longer names the removed API; docs/OPERATIONS.md/docs/API_REFERENCE.md/docs/CONTEXT.md "Change Trading Mode" instructions point at the file-based path only.
• Identifier masking (earnings_dashboard/identifier_masking.py): IBKR account ids, order/execution ids, wallet addresses, and server-side hostnames/IPs/paths are masked or truncated at the source — where each payload is assembled — not by a second blind response filter; performance and position data (P&L, shares, prices, direction) are untouched. Every site found by an exhaustive repo-wide grep for account/order_id/wallet/host/execution across earnings_dashboard/ and both modular dashboard subtrees:
  • /api/ibkr-status: status.live.account_id / status.paper.account_id (from gateway_monitor.get_gateway_status()) truncated in place — the dashboard JS reads these fields verbatim into the DOM, so the truncated value keeps rendering there instead of going blank.
  • get_active_positions_from_db(): stop_order_id / ibkr_stop_order_ids masked at the two dict-literal sites, AFTER the real DB-vs-IBKR match/mismatch comparison runs on the unmasked values (hardware-stop reconciliation is unaffected). One fix here covers FOUR consumers that all read from the same list: /api/positions, /api/live-positions, /api/state, the main / dashboard HTML, and the ibkr_reality.stop_order_issues overlay.
  • btc5m subtree (strategies/btc5m/dashboard/routes.py + btc5m_tab.html): /api/btc5m/state and /api/btc5m/live-sample were serving the maker lab's real wallet addresses and the execution host's real hostname/IP/filesystem path (facts.py: WALLET_MAIN, WALLET_TITUS, EXECUTION_HOST="[REDACTED]", EXECUTION_HOST_IP="[REDACTED]", EXECUTION_TREE_PATH="[REDACTED]") in both the top-level wallets/execution_tree JSON fields AND nested inside sample_2026_08_17.wallet/.host; recorder_status.raw_root (a DB-sourced filesystem path) too. btc5m_tab.html also had the hostname hardcoded in static prose. Local, dependency-free mask helpers (_mask_wallet / _mask_location) — not an import from earnings_dashboard.identifier_masking: the btc5m README's "Modularity guarantee" requires the subtree delete cleanly with 5 file reverts elsewhere, so it must never import back from the mounting app. Belt-and-suspenders: the same 5 raw constant values were also added to the response filter's known-identifier literal set (secret_redaction._collect_known_identifier_literals), so they're redacted even if they appear in the subtree's own markdown docs served via /api/btc5m/docs/{slug} — verified against the REAL docs/LIVE_SAMPLE_2026-08-17.md, which does contain the raw wallet/host/IP in prose.
  • etfstack subtree (strategies/etfstack/dashboard/routes.py + etfstack_tab.html) — found by the same sweep, same isolation rule, same treatment: /api/etfstack/state (paper_account, latest_signal.paper_account_id), /api/etfstack/positions (ibkr_account per row), /api/etfstack/trades (ibkr_order_id per row) masked via local _mask_account_id / _mask_order_id. etfstack_tab.html's static "0.5 — Paper Gateway" changelog row had the literal default paper account DU…91 hardcoded in prose — replaced with the generic DUP* reference already used elsewhere in the same file.
• §B:
• Trace: app.py module import → app = FastAPI(...) (auto-docs disabled) → install_secret_redaction(app) (middleware + exception handler registered) → every route handler runs inside ExceptionMiddleware inside SecretRedactionMiddleware inside ServerErrorMiddleware → SecretRedactionMiddleware.__call__ wraps send in a per-request _ResponseBodyRedactor → every http.response.start/http.response.body ASGI message passes through it before reaching uvicorn. Identifier masking: get_active_positions_from_db() computes stop_order_id_value/ibkr_stop_ids, uses them for the real stop_match comparison, THEN masks them into the dict it appends to positions → that list is state["positions"] → get_ibkr_reality_overlay(state["positions"]) reads the already-masked value.
• Simulate: planted DATABASE_URL with a fake password, appended it to a copy of docs/OPERATIONS.md, hit /docs/operations — body contains [REDACTED], not the password, not the full URL. Added a throwaway route directly to the live app object returning the same planted secret as HTML, JSON, plain text, and via an unhandled RuntimeError — all four came back redacted, including the 500 (proving the catch-all-exception-handler path, not just the happy path). Ran every parameter-free GET route (56 total; _parameter_free_get_routes() walks app.routes) with the same planted secret active and with requests.get patched to fail instantly (keeps the sweep bounded, ~3-6 s) — zero leaks. Faked a connected IBKR status (account_id="U2…13") and confirmed /api/ibkr-status returns "U2…13", never the raw id. Confirmed via git diff-driven mutation checks (temporarily reverting the registration line, the openapi_url=None block, and the account-id masking block, one at a time) that each removal fails the matching test, then restored the file byte-identical (diff confirmed).
• Lines: earnings_dashboard/secret_redaction.py (new, ~380 lines) · earnings_dashboard/identifier_masking.py (new, ~65 lines) · earnings_dashboard/app.py: import block (~L58-61), app = FastAPI(...) (~L5344-5356), get_active_positions_from_db dict literals (~L3593-3604 — masking AFTER the comparison at ~L3527-3529), save_persisted_mode deleted (was ~L4902-4909), /api/ibkr-status (~L6555-6564), the three removed endpoints (was ~L6813-6899) · strategies/btc5m/dashboard/routes.py (local mask helpers + 4 call sites) + btc5m_tab.html (1 line) · strategies/etfstack/dashboard/routes.py (local mask helpers + 4 call sites) + etfstack_tab.html (1 line).
• Constraints: the filter cannot redact a secret that ISN'T collected (an env var whose name doesn't match KEY/TOKEN/SECRET/PASS/PASSPHRASE/PRIVATE_KEY and isn't URL-shaped, and doesn't match a shape pattern) — this is why identifier masking is a SEPARATE, structural mechanism rather than folding IBKR account ids into the same literal set (an account id has no fixed env-derivable source at redaction-install time; it only exists once IBKR is connected). Superseded by the cross-family review, see below: the original version of this bullet deferred doc-prose mentions of the real production account number to #457; the review overruled that scope boundary (decision D covers every public page, docs included) and this finding now masks it on the rendered /docs/* copy directly.
• Test: tests/unit/test_finding_467_dashboard_hardening.py — 74 tests (10 classes): env-literal collection rules, generic shape patterns (fire independent of current env), no-false-redaction on hashes/prices/normal URLs, filter registration (static + live app.user_middleware + catch-all handler present), filter-applies-to-any-route (4 trap-route tests: HTML/JSON/plain-text/unhandled-exception), the real /docs/operations + /api/btc5m/docs/readme planted-secret tests, the full 56-route sweep for both a planted secret and the known btc5m/etfstack identifiers (with a bounded-time assertion), streaming/chunk-boundary/binary-passthrough tests, auto-docs-404 + custom-docs-still-works, write-endpoints-404-or-405 + no-stale-UI-reference, and per-subtree identifier-masking + no-cross-import (modularity) tests for both btc5m and etfstack. Every removal/reversion above was mutation-checked to fail the matching test before the file was restored.
• §E: exhaustive grep for account/order_id/orderId/execution/wallet/host/raw_root across earnings_dashboard/app.py, strategies/btc5m/dashboard/, strategies/etfstack/dashboard/, and every template under both — every hit is one of: (a) a performance/position field staying visible (P&L, shares, market_value — deliberately unmasked, tested in TestPerformanceAndPositionsStayVisible), (b) our own internal DB row id used for legitimate UI wiring (trading_positions.id for the chart-modal link, etfstack_backtest_runs.run_id, etfstack_rebalance_log.order_trade_id — an FK to our own trades table, not an IBKR id) — left alone, or (c) handled above. Zero unhandled (b)-class matches remain.
• Post-deploy (22:23–22:28 UTC, deploy #11 — covers #467 and the #479 restart-grace fix): acceptance met.
• Deploy: pull 0e985dd..3138aca, 16 files; hygiene 0/0/0; needrestart OK (14); migrate: current=7 bootstrap=0 applied=0; close-persist contract holds; / HTTP 200. Pre-merge: pytest tests/unit 2680 passed; drift audit CLEAN (12).
• Render: /, all 27 /docs/* pages, the 26 parameter-free /api/* views the dashboard templates fetch (plus /api/state and /api/btc5m/live-sample) and all 12 btc5m doc slugs return 200. Two non-200s were not views: /api/etfstack/killswitch is the pre-existing POST-only toggle (GET 405; not called), and 7 btc5m archive_sections ids are not doc slugs (404 by the unchanged DOC_SLUGS lookup).
• Masking, over those 72 URLs: 0 raw U2…13 (docs show U2…13), 0 [hidden], 0 [hidden], 0 other 40-hex addresses; of the known wallet values, WALLET_MAIN, WALLET_TITUS and FUNDER_ADDRESS were absent, but that scan did not check WALLET_NAGI (see Known gap below), so its "0 of 3 known wallets" figure missed the one leak. Other server paths (/var/log/polymarket, /root/, /etc/systemd) still appear in doc prose, which is outside the masked set by design (exact known values only).
• Secrets: 0 occurrences of any of the 15 current .env secret values and 0 known-leak shapes on any of the 72 URLs.
• Attack surface: GET /openapi.json, /docs, /docs/oauth2-redirect and /redoc return 404; the three removed endpoints return 404 on GET (no route; a POST-only route would return 405).
• Dashboard journal since its 22:22:12 restart: 0 5xx, 0 error/exception lines.
• Integrity 87: 85 ok / 2 warn (table sizes); nothing paged; trader ready, 0 breakers, 0 positions; Gateway stack and :4001 unchanged; root crontab unchanged (99928f0f…22c7).
• Known gap — OPEN (found by the GPT sweep review, confirmed on prod 22:52–22:57 UTC): GET /api/btc5m/docs/doc_map (HTTP 200) returns the raw WALLET_NAGI value once. It is the six-character prefix of the third-party benchmark maker's wallet — not a full address and not Marius's wallet — which #467 already treats as sensitive (/api/btc5m/state masks it as nagi_prefix via _mask_wallet()). Cause: _collect_known_identifier_literals() and the route-sweep test's known values include WALLET_MAIN and WALLET_TITUS but not WALLET_NAGI, and test_every_btc5m_doc_slug_is_clean checks account/server identifiers, not btc5m wallets. The value is not written here because /docs/findings is public. Code follow-up: the #467 author is fixing it on a new PR, which ships as its own deploy after GPT approval.
• Widened identifier scan (22:52–22:57 UTC, prod 2ecb02c, dashboard code unchanged since deploy #11): every public GET route — all 44 parameter-free routes including the JSON APIs, every /docs/* page and every btc5m DOC_SLUGS doc, 82 URLs — against 10 known values (account id, server IP, deploy path, every WALLET_* / EXECUTION_* constant in strategies/btc5m/src/facts.py, and the .env wallet address), any 40-hex address, and the 15 .env secret values. Result: only WALLET_NAGI on /api/btc5m/docs/doc_map (×1); everything else absent; 0 40-hex addresses; 0 secrets. Not covered: /api/trade-chart-data (needs query parameters; 422 without) and the etfstack backtest run routes (no run ids listed).
• Pre-deploy note: the pre-check sent POST to the three write endpoints on the old code (22:17:38, and to the two testing-prod ones again at 22:18:03). They returned {"error":"Orchestrator not available"} (the old get_orchestrator() always returned None) and 422 for set-trading-mode; nothing ran, and trader mode, pause state and breakers were unchanged.
• Changelog: none (§O no — response-shaping and attack-surface hardening; no live trading behaviour changes; masking runs strictly after existing comparison/reconciliation logic).
• Files: earnings_dashboard/secret_redaction.py (NEW), earnings_dashboard/identifier_masking.py (NEW), earnings_dashboard/app.py, earnings_dashboard/templates/dashboard.html, strategies/btc5m/dashboard/routes.py, strategies/btc5m/dashboard/templates/btc5m_tab.html, strategies/etfstack/dashboard/routes.py, strategies/etfstack/dashboard/templates/etfstack_tab.html, docs/OPERATIONS.md, docs/API_REFERENCE.md, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, tests/unit/test_finding_467_dashboard_hardening.py (NEW, 74 tests), this entry.

Cross-family review (71dd002, docs/reliability/review-467-dashboard.md): CHANGES REQUIRED — 4 blockers, all fixed same finding: 1. Streaming chunk-boundary leak (high, reproducible). _ResponseBodyRedactor._emit_chunk split the RAW window into an emit/carry pair BEFORE redacting it — a secret straddling that split point had each half scanned independently, so neither half matched the full literal and it shipped raw (the reviewer's exact repro: _MAX_BUFFER_BYTES=16, overlap=8, one 24-byte chunk b'xxxxxxxxxxxxSecretValue1'). Fixed: redact the WHOLE combined window first, THEN split the already-safe result; the carry-back margin is max(longest current literal, 788 [longest shape-pattern match], 4096) via new _SecretStore.max_match_length() — an in-progress match's visible-so-far portion is, by construction, always shorter than the margin, so it's never in the emitted half. Regression: TestStreamingChunkBoundaryRegression sweeps every junk_len around the exact danger band (margin − secret_len, margin) using the reviewer's own margin-shrinking technique; mutation-checked against a reverted split-before-redact implementation (14 of 32 parametrized cases correctly fail against it, 0 against the fix). 2. Public /docs/{doc_name} pages exposed identifiers (high). #457 removes committed secrets, not non-secret identifiers, and decision D covers every public page — docs included; deferring the account number to #457 was the wrong scope boundary. Fix: new identifier_masking.mask_known_public_identifiers() — exact-value replacement (never a path-shaped regex, so Markdown links/code fences survive) of the real IBKR account number (→ U2…13) and the droplet IP + [hidden] (→ [hidden]) on the raw doc text, applied before Markdown conversion in serve_documentation() (source .md files on disk stay untouched) and on btc5m's plaintext docs route via a local, dependency-free duplicate (modularity). Same 3 values also joined the response filter's known-identifier literal set as a backstop. Tests parametrize over the review's own reported routes (operations/agent-handoff/readme/strategy) plus every doc_map slug (26, discovered from the route's own 404 body) and every btc5m doc slug (12). 3. Bare/short/URL-encoded DATABASE_URL passwords (medium). The 8-char minimum literal length silently dropped a short password entirely, and only the URL-as-appears (still percent-encoded) form was ever collected. Fixed: no minimum length for a value with known provenance (name-matched or URL-password-extracted) — literals under 8 chars are anchored to token boundaries ((?<![A-Za-z0-9])...(?![A-Za-z0-9])) instead of being dropped, so they can't flood false-positives as an unanchored substring match either; new _url_password_variants() collects the raw/as-appears form, the URL-decoded (bare) form, and a canonical re-encoded form. 4. The full-route sweep was not mutation-sensitive (medium). It only checked whether a planted secret happened to leak — nothing in this app's NORMAL (no live DB/IBKR in tests) request handling embeds one, so the test passed with the filter removed (reviewer's own mutation check: 44 of 44 routes still zero-leak with SecretRedactionMiddleware deleted from the live app). Fixed: test_every_route_response_traverses_the_redactor instruments redact_bytes directly (call-count before/after each request) to prove the POSITIVE the leak-scan can only assume — every route's response actually reaches the redactor — plus an explicit mutation test using the reviewer's exact method (remove the middleware, rebuild the stack), which reproduces the same "44 of 44 never traverse" result.

Bonus finding made while building the blocker #4 instrumentation (not reviewer-flagged): the catch-all Exception handler's response does NOT pass through SecretRedactionMiddleware — Starlette's build_middleware_stack() hands any handler registered for Exception/500 straight to ServerErrorMiddleware, outside all user middleware, sent via the original send. The module docstring previously claimed the opposite. Corrected the docs; the path stays safe because the handler redacts BEFORE building its response (single-point, not defense-in-depth, for this one path) — added a dedicated test proving both facts (zero redact_bytes calls, zero leak).

Post-review: full unit suite 2348/2348 PASS (125 tests in the #467 file, +51 from this round); scripts/doc_drift_audit.py clean.

Cross-family review round 2 (05c02ed, docs/reliability/review-467-dashboard.md): CHANGES REQUIRED — 1 blocker, fixed same finding: 1. Multibyte/Unicode secrets still leaked across a streaming chunk boundary. Round 1's fix sized the carry-back margin from len(v) — Python character count — and redact_bytes() decoded each window to str with errors="replace" before matching. Both are wrong for a non-ASCII literal: (a) a 1500-character emoji secret is 6000 UTF-8 bytes, so a margin of "1500" was 4x too small to hold its visible-but-incomplete tail, and independently (b) decoding a window that ends mid-way through a multi-byte UTF-8 sequence permanently replaces that sequence's trailing bytes with U+FFFD before the rest of the character ever arrives in the next chunk — no margin size fixes a corruption that already happened. Reviewer's exact repro: "🔐" * 1500 (6000 bytes) split into a 5000-byte chunk and a 1000-byte chunk — old code reported max_match_length()==1500, leaked the raw secret, no [REDACTED] anywhere in the output. Fixed by matching secrets as UTF-8 bytes throughout the streaming path, never decoding a window at all: new _build_literal_pattern_bytes(), _GENERIC_PATTERNS_BYTES, _redact_url_credentials_bytes(), _apply_generic_patterns_bytes(), and _SecretStore.redact_bytes() (bytes in, bytes out, no str round-trip); _SecretStore._max_len_bytes is now max(788, len(v.encode("utf-8")) for v in literals) — measured in encoded bytes, not characters. redact_text() (str → str, used only by the exception handler and other complete-in-memory-string call sites, which have no chunk boundary to split) is unchanged. _emit_chunk's redact-before-split invariant from round 1 is unchanged; only what it redacts with, and how it sizes the margin, changed. Regression: new TestStreamingMultibyteBoundaryRegression (10 tests) — direct assertions that the margin is byte-sized for 2-/3-/4-byte characters (é/€/🔐), the reviewer's exact 1500×🔐/5000-1000-byte-split repro run against the REAL (un-overridden) margin, an offset sweep generalizing TestStreamingChunkBoundaryRegression's ASCII sweep to all three character widths (every byte offset around the danger band, including offsets that land mid-character), and two targeted mutation checks — one reverting only the margin formula to character-count, one reverting only redact_bytes to the old decode-based implementation — each confirmed to reproduce a real, unredacted leak. Mutation-checked against the actual pre-fix file (git show HEAD:earnings_dashboard/secret_redaction.py, i.e. the round-1-only version): 3 of the 10 new tests correctly fail against it (test_margin_is_measured_in_encoded_bytes_not_characters, the 🔐-width case of test_margin_scales_with_byte_width_for_each_character_class, test_reviewer_exact_repro_1500_emoji_split_mid_stream); file restored byte-identical afterward (diff confirmed). Post-round-2: full unit suite 2619/2619 PASS (135 tests in the #467 file, +10 from this round); scripts/doc_drift_audit.py clean (12 checks, 0 violations).

Finding 479 — Database access was unattributed and non-durable; add per-service identities plus query-free activity evidence (Sep 26, 2026)

• Severity: High — after the Sep-14/Sep-26 credential exposure, retained evidence could not prove whether the harvested doadmin credential was used. pg_stat_activity was point-in-time only, all ten application sessions had blank application_name, and the database owner could alter in-database audit rows.
• Status: DEPLOYED Sep 26, 2026 20:55 UTC (deploy #8 614a29d, fast-forward a695050..614a29d after the cross-family review; detached unit polymarket-deploy-20260926T205457Z-3090020335, DEPLOY_EXIT rc=0). Marius approved hardening item E. The deploy installed and enabled polymarket-db-activity-sampler.service, which started 20:58:07 UTC, after the OD8 doadmin reset, on the new .env. Sampler-grace race hardening (PR #8 094a00f): DEPLOYED Sep 26, 2026 22:39 UTC (deploy #12 2ecb02c). Sampler restart-grace fix (PR #7 295fe28): DEPLOYED Sep 26, 2026 22:19 UTC (deploy #11 3138aca). Post-deploy evaluator fix: DEPLOYED Sep 26, 2026 21:30 UTC (deploy #10 f28c0da, merge of PR #6 155f0eb after Opus review; detached unit polymarket-deploy-20260926T213008Z-1418529119 21:30:11 → 21:33:44 UTC, DEPLOY_EXIT rc=0).
• Post-deploy defect (history, Sep 26, 2026 20:58–21:33 UTC): after deploy #8 the integrity rule security:db_client_activity could not evaluate (Rule.evaluate passed the engine to _check_db_activity_status(path)), so the per-new-address CRITICAL page was inactive; the sampler, log and latch kept working and no foreign client appeared (all 143 db_activity.log samples 20:58:08–21:33:28 UTC have empty foreign_client_addrs / untrusted_client_addrs: 142 before the sampler restart, through 21:33:22, and the restarted sampler's first sample at 21:33:28). Fixed DEPLOYED Sep 26, 2026 21:30 UTC (deploy #10 f28c0da) — see Post-deploy defect and fix below.
• Root cause: connection creation was fragmented across SQLAlchemy, psycopg2, and pg_dump; service units supplied no identity; no host-side process sampled pg_stat_activity; the integrity panel had no client-address rule.
• Fix:
• src/db/connection.py is the single startup-identity helper for SQLAlchemy, psycopg2, and libpq subprocesses. Known entrypoint filenames derive stable production identities in code; DB_APPLICATION_NAME in an installed unit is an override, not a deployment dependency. Thus the seven hand-installed units whose repo twins are not reinstalled by a normal deploy still identify correctly after their normal restart/next run. Startup packets carry identity from connection establishment, not a later SET.
• polymarket-db-activity-sampler.service samples all rows in pg_stat_activity (all databases plus NULL-database provider workers) every 15 seconds under a 3-second statement timeout, 5-second connect timeout, and read-only transaction. One rotated JSON line per snapshot records only client_addr, usename, application_name, backend_start, state, and server-side query class. It never selects/emits query text or parameters and touches no application table/hypertable.
• The sampler atomically latches every globally routable foreign address and its first/last-seen/user/application evidence in a query-free sidecar. Evidence remains until it has been successfully paged and then quiet for 24 hours, so a read-and-disconnect session cannot disappear between five-minute monitor cycles. Integrity rule security:db_client_activity stays WARN while any latch exists (true since deploy #10 f28c0da; inactive 20:58–21:33 UTC after deploy #8 — Post-deploy defect above). The monitor dispatches one CRITICAL data_integrity page per new address independent of rule status transitions; CRITICAL bypasses the shared 30-second category limiter, and failed sends are not acknowledged and retry next cycle. No RESOLVED page is sent: disappearance is not proof the access was benign. Malformed/ambiguous addresses and sampler staleness remain panel-only.
• The existing /var/log/polymarket/*.log logrotate policy bounds the activity log daily/at 500 MiB, retains 14 compressed rotations, and the deploy script owns installation/startup of the new unit.
• §B:
• Trigger → effect: entrypoint-derived identity (unit env may override) → shared helper adds startup application_name → PostgreSQL activity/connection logs attribute the session. Sampler → read-only all-database catalog query → JSONL + persistent foreign latch → integrity monitor → visible WARN plus per-new-address CRITICAL page (live since deploy #10 f28c0da; inactive 20:58–21:33 UTC after deploy #8).
• Runtime: normal sessions from [hidden], local workers, or RFC1918/ULA/link-local networks stay OK. A global address seen by any 15-second sample is latched, survives clean later samples and WARN→WARN states, and is retried until its unique CRITICAL page succeeds. A session shorter than the 15-second interval can fall entirely between samples; DigitalOcean's forwarded connection/disconnection logs are the complete connection record.
• Code: src/db/connection.py::{get_application_name,create_db_engine,connect_psycopg2,postgres_subprocess_env}; src/data_tracker/db_activity_sampler.py::{ACTIVITY_SQL,_merge_foreign_seen,run_once}; src/data_tracker/integrity_monitor.py::{_check_db_activity_status,_send_foreign_client_alert,IntegrityMonitor._dispatch_alerts}.
• Blockers/failure modes: sampler down/stale → panel WARN plus the separate systemd liveness rule; malformed status JSON → panel WARN; database unreachable → no fabricated evidence and systemd retry; unsuccessful Telegram send remains unacknowledged; log growth bounded by the existing glob; no query/parameter text leaves PostgreSQL.
• Acceptance: every hand-installed and deploy-managed production entrypoint derives its intended identity without relying on unit reinstall; all touched scripts execute --help from a foreign working directory with PYTHONPATH absent; two consecutive snapshots preserve a disconnected foreign address; a second address pages during an existing WARN; all databases are sampled; the page is CRITICAL; /api/integrity renders the WARN.
• §E same-pattern sweep: all active/installed earnings, collector, dashboard, integrity, reconciliation, backup, resolver, archive, ETF, BTC recorder, and scheduled backfill connection paths use the helper. Runtime subprocess tests—not source greps alone—cover every touched script entrypoint. Known production entrypoints derive identities in code; unit-file names are tested only as optional overrides.
• Owner-thread invariant (#440): review caught that normalizing the +pg8000 URL had revived the #418 _db_has_open_position() fallback as a real psycopg2 round-trip on the IB owner thread. That fallback previously failed immediately in production and is now a no-I/O compatibility stub: _has_open_position_for_alert() uses only the orchestrator's in-memory probe and otherwise returns false. The older weekly last-resort Gateway-recycle path still reaches gateway_recycle.py::db_open_position_count on the owner thread; that pre-existing call belongs to B1's Gateway-recycle fixes, not #479.
• Tests: tests/unit/test_finding_479_db_activity_logging.py covers subprocess entrypoints, derived identities, all-database SQL, persistent foreign latches, WARN→WARN/new-address paging, CRITICAL rate-limit bypass, failed-send retry, conditional _send_integrity_alert, and the owner-thread no-DB invariant.
• Post-deploy (21:02–21:05 UTC, deploy #8 — covers #457/#478/#479): acceptance met except the #479 evaluator defect (history above; fixed in deploy #10 f28c0da).
• Prod HEAD 614a29d == origin/master; git status --porcelain 0 lines (also with --untracked-files=all). Detached unit polymarket-deploy-20260926T205457Z-3090020335 20:55:00 → 20:58:21 UTC, DEPLOY_EXIT rc=0; hygiene 0/0/0; host-reference ok; needrestart coverage OK (14); migrate: current=5 bootstrap=0 applied=0; close-persist contract holds; / HTTP 200.
• The 6 app services restarted; ibgateway-ibc 1215183, xvfb 1215180, x11vnc 1215184 and :4001 java 1537985 unchanged (NRestarts 0). Root crontab sha256 unchanged (99928f0f…22c7, the cron-a state).
• Trader reconnected 20:55:25 (same-thread req* ok after connect (#440); reconciliation in sync 20:55:32); effective_trading_state=ready, active_breakers=0, testing_prod_paused=False, 0 stops today, 0 positions (IBKR 0 / DB 0). Integrity 87 rules: 84 ok / 2 warn (two table sizes) / 1 unknown (security:db_client_activity, the #479 evaluator defect, fixed in deploy #10 f28c0da).
• Logs 20:54:57 → 21:05 UTC: the connect-time IBKR 2107 (informational) and, until deploy #10 f28c0da, the #479 rule's TypeError each 5-minute monitor cycle; 0 journal err lines across 9 units.

• 479: pg_stat_activity (read-only) shows earnings-trader, polymarket-dashboard (2), polymarket-collector, polymarket-live-trades, polymarket-websocket, earnings-conviction-tracker, polymarket-integrity-monitor and db-activity-sampler, all from [hidden]; every other session is DigitalOcean-internal with a NULL address; no blank-named droplet session and no foreign address. Sampler active and enabled (NRestarts 0); db_activity.log growing; status sidecar session_count=20, foreign_seen={}; error log empty. security:db_client_activity read unknown at that point (the evaluator defect, fixed in deploy #10 f28c0da).

• Changelog: none (§O N/A: observability only; the review-introduced owner-thread behavior change was removed before deployment; no strategy, sizing, order, admission, or exit behavior changes).
• Files: src/db/connection.py; sampler + integrity monitor; production connection call sites; DB-capable systemd units; deploy installer; tests/unit/test_finding_479_db_activity_logging.py; this entry, CONTEXT, and AGENT_HANDOFF.
• Post-deploy defect and fix (Sep 26, 2026, deploy #8):
• Observed: the sampler was healthy and no foreign client was present, but security:db_client_activity returned UNKNOWN and logged a traceback every five-minute integrity cycle. Because foreign paging consumes this rule's detail, a real new foreign client would have gone unpaged.
• Root cause: Rule.evaluate(engine) invokes every registered evaluator with the database engine. _check_db_activity_status(path=None, ...) is a file reader whose first positional argument is a path, but _build_rules() registered it directly. The engine was passed as path, and Path(engine) raised TypeError. Earlier tests called the helper directly with a real path and never exercised the registered rule through Rule.evaluate.
• Fix: register evaluator=lambda _engine: _check_db_activity_status(), explicitly adapting the engine-based rule contract to the engine-independent file check.
• Class closure: test_registered_db_activity_rule_evaluates_through_monitor_contract drives this exact registered rule through Rule.evaluate(fake_engine). test_every_registered_rule_accepts_monitor_engine_argument pins the registry at exactly 87 rules and requires security:db_client_activity, then drives every entry through the same path. It catches a TypeError escaping an evaluator into Rule.evaluate—including the deployed wrong-first-argument defect; it does not claim to detect every possible kind of callable-signature drift.
• Acceptance: targeted #479 tests, the complete unit suite, and documentation drift audit pass; after deploy, security:db_client_activity is OK (or a truthful WARN), no evaluator traceback recurs, and a latched foreign address remains eligible for its CRITICAL page.
• Deployed: DEPLOYED Sep 26, 2026 21:30 UTC (deploy #10 f28c0da; unit polymarket-deploy-20260926T213008Z-1418529119 21:30:11 → 21:33:44 UTC, DEPLOY_EXIT rc=0; pull 614a29d..f28c0da, 3 files; hygiene 0/0/0; needrestart OK (14); migrate: current=5 bootstrap=0 applied=0; close-persist contract holds; / HTTP 200). Pre-merge: pytest tests/unit 2486 passed; drift audit CLEAN (12).
• Post-deploy (21:33–21:39 UTC): acceptance met. The restarted monitor (21:33:32) evaluated security:db_client_activity = ok ("database clients trusted (19 sessions)") on its first cycle (21:33:34) and again on cycle 1 (21:38:43). The monitor log's evaluator raised / Traceback count went 7 → 8: the 8th is the old process's last cycle (21:33:26), before the restart; the new process logged none. No foreign-client page; no db_activity_alerted.json; sidecar foreign_seen={}, untrusted=[]. Sampler active (restarted 21:33:28), log growing, error log empty. Integrity 87 rules: 85 ok / 2 warn (table sizes) / 0 unknown.
• Side effect seen at this deploy (history; fixed in deploy #11 3138aca — see Post-deploy false page and fix below): the deploy restarts the sampler just before the integrity monitor. The old monitor's last cycle ran at that moment, systemd:polymarket-db-activity-sampler.service saw deactivating, and it sent a CRITICAL "Data Integrity TRIGGERED" Telegram at 21:33:29; the new monitor sent "RESOLVED" at 21:33:37. Any deploy can repeat this if the old monitor's cycle lands in that window.
• Post-deploy false page and fix (Sep 26, 2026, deploy #10):
• Observed: deploy #10 restarted polymarket-db-activity-sampler.service while the old integrity monitor was finishing its last cycle. The generic always-active service rule sampled the unit mid-restart, sent a false CRITICAL "Data Integrity TRIGGERED" at 21:33:29 UTC, then RESOLVED eight seconds later. The sampler itself recovered and security:db_client_activity was OK; no foreign client was present.
• Root cause: the sampler used the generic instantaneous systemctl is-active rule, and its sidecar freshness rule had no knowledge of a deliberate service restart. A normal deploy creates a brief, expected non-active/stale window before the integrity monitor itself restarts.
• Fix: remove the sampler from the generic always-active list and register the same rule name with a sampler-specific evaluator. A non-active unit whose systemd StateChangeTimestampMonotonic is within 120 seconds returns OK with explicit restart-grace detail. Missing, unreadable, or stale sampler status gets the same bounded grace. This is isolated to the sampler and does not change the generic Rule/transition machinery, keeping it compatible with Reliability B1's later deploy deferral and two-cycle policy.
• Genuine outage: after 120 seconds, a non-active sampler returns FAIL on the CRITICAL service rule and remains page-eligible; stale/missing activity status returns WARN. Restart tolerance therefore suppresses only the deployment race, not a sampler that stays down.
• Tests: test_sampler_service_and_freshness_tolerate_short_restart drives both registered rules at 30 seconds after a systemd state change and requires OK/restart-grace. test_sampler_genuinely_down_still_fails_and_pages drives them at 300 seconds and requires CRITICAL FAIL plus freshness WARN.
• Deployed: DEPLOYED Sep 26, 2026 22:19 UTC (deploy #11 3138aca, merge of PR #7 295fe28 after Opus review; unit polymarket-deploy-20260926T221858Z-2508114717).
• Post-deploy (22:22–22:28 UTC): no page. The deploy restarted the sampler at 22:22:10. The old monitor's last cycle ran at 22:19:55 and the new monitor's cycle 0 at 22:22:13–22:22:16, both with 0 transitions and 0 Telegram-eligible; systemd:polymarket-db-activity-sampler.service reads ok ("active") and security:db_client_activity ok. No monitor cycle landed inside the sampler's restart window this time, so the 120 s grace path was not exercised on prod; its tests cover it.
• Review hardening: the service evaluator now takes one consistent systemctl show snapshot containing ActiveState, StateChangeTimestampMonotonic, and NRestarts; an already-recovered active state is OK, eliminating the old is-active→show race. Three or more automatic restarts — systemd's NRestarts, counted since the last manual (re)start (a deploy's systemctl restart resets it) — with the last state change within five minutes override grace and FAIL CRITICAL, so a long-running crash loop remains visible. Because the count is cumulative since the last manual start, not a five-minute window, after three scattered automatic restarts with no deploy in between every later single crash pages once (FAIL for up to 300 s, then RESOLVED): loud, not silent (review nit 1). A readable stale sidecar with an ambiguous/untrusted address stays WARN during service grace; client evidence is evaluated before freshness grace.
• Review tests: test_sampler_service_uses_one_consistent_recovered_state_read reproduces the reviewer's stop→active race and proves only show is called; test_sampler_restart_loop_overrides_restart_grace proves NRestarts=3 fails while the last state change is inside five minutes; test_sampler_grace_keeps_ambiguous_address_warning proves stale ambiguous evidence is not hidden.
• Deployed: DEPLOYED Sep 26, 2026 22:39 UTC (deploy #12 2ecb02c, merge of PR #8 094a00f after Opus review; unit polymarket-deploy-20260926T223934Z-2361616640 22:39:37 → 22:43:08 UTC, DEPLOY_EXIT rc=0). Pre-merge: pytest tests/unit 2683 passed; drift audit CLEAN (12).
• Post-deploy (22:43–22:48 UTC): sampler checks met; one unrelated false page. systemctl show polymarket-db-activity-sampler: ActiveState=active, NRestarts=0 (restarted by the deploy 22:42:43). Monitor log after the pre-deploy baseline: 0 TRIGGERED for the sampler rule, 0 evaluator raised, 0 tracebacks. After cycle 1 (22:47:54): systemd:polymarket-db-activity-sampler.service ok ("active", detail n_restarts: 0, restart_loop: false) and security:db_client_activity ok ("database clients trusted (20 sessions)"). Status file 4–12 s old, untrusted/foreign/foreign_seen empty, no db_activity_alerted.json.
• False page at this deploy — OPEN follow-up (not the sampler): the deploy restarts the integrity monitor (22:42:45) and then force-restarts the dashboard; the dashboard took 7 s to stop (deactivating 22:42:45 → 22:42:52), the new monitor's immediate cycle 0 read it at 22:42:48, and systemd:polymarket-dashboard.service sent a CRITICAL "Data Integrity TRIGGERED" Telegram; cycle 1 sent "RESOLVED" at 22:47:54. Same race class as the sampler's, from the deploy script's restart order (integrity monitor before the final dashboard restart) — it predates this PR and missed in deploys #9 and #11 only by timing. Seen again at deploy #14 (systemd:polymarket-dashboard.service deactivating 00:33:57, RESOLVED 00:37:09 Sep 27); it did not page at deploy #13. The fix, PR #9, is pending.
• Review hardening: the service evaluator now takes one consistent systemctl show snapshot containing ActiveState, StateChangeTimestampMonotonic, and NRestarts; an already-recovered active state is OK, eliminating the old is-active→show race. A readable stale sidecar with an ambiguous/untrusted address stays WARN during service grace; client evidence is evaluated before freshness grace. History: deploy #12 initially treated the lifetime NRestarts >= 3 plus a recent state change as a loop; the sliding-window correction below supersedes that over-broad rule.
• Review tests: test_sampler_service_uses_one_consistent_recovered_state_read reproduces the reviewer's stop→active race and proves only show is called; test_sampler_restart_loop_overrides_restart_grace proves NRestarts=3 fails inside the five-minute window; test_sampler_grace_keeps_ambiguous_address_warning proves stale ambiguous evidence is not hidden.
• Post-deploy cross-unit false page and fix (Sep 26, 2026, deploy #12):
• Observed: the deploy restarted the new integrity monitor before its final dashboard restart. The dashboard took seven seconds to stop (22:42:45–22:42:52 UTC); monitor cycle 0 sampled it at 22:42:48, sent a false CRITICAL systemd:polymarket-dashboard.service TRIGGERED page, then RESOLVED it at 22:47:54. Deploys #9/#11 avoided the same race only by timing.
• Root cause: PR #8's bounded grace was sampler-specific. The dashboard, trader, collectors, IB Gateway and monitor self-check still used one instantaneous systemctl is-active sample, so any deliberate service restart could transition CRITICAL.
• Monitor fix: every always-on service rule and the monitor self-check now use the same single systemctl show snapshot (ActiveState, StateChangeTimestampMonotonic, NRestarts). activating, deactivating, or inactive states within 120 seconds are OK/restart-grace; recovered active is immediately OK.
• Real-outage bound: a service still non-active after 120 seconds FAILs CRITICAL on the first subsequent five-minute cycle, so dashboard/trader outages page within at most 420 seconds.
• Sliding crash-loop detection: the monitor stores each unit's first NRestarts as a baseline, records only positive deltas, clears history when systemd resets the counter, and FAILs CRITICAL at three newly observed automatic restarts inside a rolling 15-minute window. This catches a real tight loop quickly without adding an integrity page to every trader stall restart after the lifetime counter reaches three.
• Deploy ownership / own-deploy scope: the PR #9 stop/trap/start edits are removed; B3's rollback trap remains untouched, there is no SIGKILL monitoring gap, and deploy-wide alert suppression belongs to B1's deferral window. This monitor-side fix protects its own deploy's final dashboard restart because the new monitor code is already running then. Earlier service restarts occur before the monitor restart and still use the old process until B1 lands.
• Tests: the exact deploy #12 sequence (dashboard deactivating at +3 seconds, then active) remains OK; dashboard and trader inactive at 121 seconds are CRITICAL FAIL; three restarts in three minutes FAIL; three lifetime restarts spread ten minutes apart remain OK.

Finding 478 — C-Q2 / G-RS decided (keep all four detection values and the Polymarket price source); the overdue Sep 13 date reminder is replaced by a dashboard-only count trigger (Sep 26, 2026)

• Severity: Low (records + one alert-only integrity rule). No trading-rule, gate, sizing, stop or order-path change.
• Status: DEPLOYED Sep 26, 2026 20:55 UTC (deploy #8 614a29d, after the cross-family review; deploy-wide checks under #479 Post-deploy). Operator decisions #8 and #9, Sep 26, 2026 17:34 UTC.
• Trigger: contract:cq2_grs_detection_rejudgment (#389) had been WARN since its Sep 13 due date. The re-judgment it asked for (Rev §II.5 C-Q2, §II.7 G-RS; G-002/G-003/G-004) ran Sep 26, 2026 on 5.7 weeks of clean post-#358 prints (earnings dates Aug 17 – Sep 25: 69 resolved markets, 194,780 prints, 23 real trades). It used a replay at #358 parity with the Gamma fallback bounded, and GPT-5.6 reviewed it (v2 approved with nits).
• Result the operator decided on:
• #8 — keep 0.98/0.02, 0.25/0.90, 300 s and 60 s (config.py POLYMARKET_*, unchanged) and the Polymarket price source:
  • 0 false exits: 0 of 60 right-way positions in the full population, 0 of 24 in the live entry-gate lens;
  • BOX and BBW, the only wrong positions held through the release gate, were stopped 1.6 and 0.8 min before the crowd crossed 0.25;
  • no priced benefit, so the source is a cautious default with unproven value;
  • looser thresholds add false exits (0.35/0.80: COST, a 12.7 min whipsaw); shorter windows gain only one out-of-gate case (GAP), which fails the gauntlet;
  • on clean prints genuine verdicts sustained ≥122.7 min, with 0 whipsaws at 0.25/0.90.
• #9 — count trigger instead of a date: the re-judgment is underpowered at n=2, so it re-runs when the cohort reaches 12 or on 2026-12-15, whichever is first. Dashboard/integrity WARN, not a Telegram page.
• Root cause of the fix need: the #389 rule could only be date-based (_check_contract_date(due_date=_CQ2_GRS_DATE)). Once past its date it returned WARN every day forever, with no way to encode "re-judge when there is enough evidence".
• Fix:
• NEW src/data_tracker/contract_cq2_count.py:
  • cq2_wrong_held_sql(canonical_filter) defines the cohort: unique trade_audit_log positions passing the canonical filter, closed, entered on or after the #358 deploy (Aug 16, 2026 20:22 UTC). The market (earnings_markets by condition_id) must have a final pm_resolution, the position must have been open at the detector's release-window gate (#337: earnings date 04:00 ET for BMO/unknown timing, 15:30 ET for AMC), and the held direction must be contra the resolution.
  • The canonical filter is _CANONICAL_TRADE_FILTER_SQL, identical to TRADE_METRICS_FILTER_SQL and applied verbatim inside the trade subquery.
  • check_cq2_count_trigger() returns warn at CQ2_COUNT_TRIGGER=12 or on/after CQ2_BACKSTOP_DATE=2026-12-15, otherwise ok with "n/12 … days out". The detail lists every cohort position for the race statistic.
  • It reads two plain tables (no hypertable), bounded by the cohort epoch.
• integrity_monitor.py:
  • _CQ2_GRS_DATE is deleted.
  • A thin _check_contract_cq2_count wrapper does a local import, per the file's convention.
  • The rule keeps its name, category contract and WARNING severity, but now has telegram_reminder=False. So _send_integrity_alert returns before any send, and the WARN shows only on the dashboard integrity panel.
  • The C-Q3 date reminder is untouched.
• Records, per R2 (decision state change + new finding + retiring a mechanism):
  • Rev §I timeline, §II.5 C-Q2, the §II.7 intro and G-RS row, and Theme G rows G-002/G-003/G-004;
  • IMPLEMENTATION_PLAN_2026_08.md C-Q2/G-RS rows;
  • build_revalidation_357(): C-Q2 DECIDED Sep 26, 2026 — KEEP (#478; no trading change), the G-RS monitor criterion, future work, the A-8 note;
  • the Playbook banner and the STRATEGY.md detection row, whose stale "last 5 prints" statistic is corrected to the #358 fresh-window median;
  • CONTEXT, AGENT_HANDOFF §11, and a §O entry (#389 → AMENDED);
  • drift-audit C2 now bans the retired "C-Q2/G-RS … Sep 13" reminder claim in current-state docs.
• Today's count (read-only prod run of the exact rule SQL, Sep 26, 2026): 2 — trade_audit_log ids 267 (BOX, LONG, exit 2026-08-25 20:05 UTC) and 274 (BBW, LONG, exit 2026-08-27 10:52 UTC). The rule reads ok "2/12 … (backstop 2026-12-15, 80 days out)".
• §B:
• (1) Trace: IntegrityMonitor cycle → _build_rules() → Rule.evaluate → _check_contract_cq2_count → check_cq2_count_trigger → one SELECT → status row (dashboard /api/integrity). On a transition, _send_integrity_alert returns at its severity/telegram_reminder gate (WARNING ∉ TELEGRAM_SEVERITIES, telegram_reminder=False).
• (2) Runtime: the generated SQL ran read-only on prod (psql, default_transaction_read_only=on, 20 s timeout) and returned BOX and BBW in 3 s. The same cohort matched sql/40_count_trigger.sql from the analysis.
• (3) Lines: src/data_tracker/contract_cq2_count.py (cq2_wrong_held_sql, check_cq2_count_trigger); integrity_monitor.py _check_contract_cq2_count and the contract:cq2_grs_detection_rejudgment Rule.
• (4) What could prevent it:
  • Markets that never get pm_resolution (DY, CPB, KR today) never count. That is by design: truth is the market's own resolution (#372/#382).
  • earnings_date rewrites (#442) move the gate, so a position entered for a pre-rewrite date that never spanned the real release is excluded (CBRL id 293).
  • condition_id is unique in earnings_markets (1,353/1,353), so the join cannot double-count; DISTINCT guards it anyway.
  • A DB error yields unknown, never fail.
• (5) Test tests/unit/test_finding_478_cq2_count_trigger.py (20 tests):
  • 2 positions on Sep 26 → ok; 11 positions on Dec 14 → ok; 12 → warn; Dec 15 with 0 → warn; both reasons reported; DB error → unknown;
  • registered dashboard-only (telegram_reminder is False), with a Telegram-path test in which a control reminder does send;
  • _CQ2_GRS_DATE and date(2026, 9, 13) gone;
  • the executed SQL equals the builder, the canonical filter is verbatim in the trade subquery, the epoch bound and closed filter are present, no hypertable is read, the release gate matches EarningsDetector._RELEASE_WINDOW_*, and the query is resolved and contra only.
  • the C2 stale literal fires on a synthetic current-state claim and skips history lines.
  • Against the old reminder (master's integrity_monitor.py), 11 of the 20 fail.
  • Updated: test_finding_389_contract_reminders.py (the C-Q2 rule is dashboard-only; the date is retired) and test_finding_357_revalidation.py (C-Q2 reads DECIDED).
• §E:
• C-Q3 (_CQ3_DATE, Oct 12) is the only other date-based contract reminder. It is deliberately unchanged: the operator decided only C-Q2.
• Repo-wide grep for C-Q2|G-RS|cq2|CQ2|G-00[234] outside archive/, packs/ and strategies/: every current-state match was updated here. Intentional history left as is: the #389/#358/#360/#361 FINDINGS narratives, Rev §10 (preserved verbatim), the frozen scripts/analysis_357/* reports, the build_revalidation_357 docstring's Aug 16 note, and the implementation-queue item "started the ~4-week clean-print clock".
• Evidence: analysis c-q2-rejudgment-2026-09.md v2 and its GPT-5.6 review in the Project store (docs/analysis/), with scripts, SQL and outputs under internal/analysis/c-q2-rejudgment-2026-09/. The #358-parity replay for the repo is #475 (reliability/followups).
• Post-deploy (20:58 UTC): acceptance met. contract:cq2_grs_detection_rejudgment reads ok: "C-Q2/G-RS re-judgment: 2/12 wrong-held positions (backstop 2026-12-15, 80 days out)" (BOX, BBW). The overdue Sep 13 WARN is gone from the integrity panel.
• Files: NEW src/data_tracker/contract_cq2_count.py, src/data_tracker/integrity_monitor.py, NEW tests/unit/test_finding_478_cq2_count_trigger.py, tests/unit/test_finding_389_contract_reminders.py, tests/unit/test_finding_357_revalidation.py, scripts/strategy_performance_analysis.py, scripts/doc_drift_audit.py, src/earnings/strategy_changelog.py, docs/STRATEGY_REVALIDATION_2026_08.md, docs/IMPLEMENTATION_PLAN_2026_08.md, docs/STRATEGY.md, docs/STRATEGY_PLAYBOOK.md, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, docs/FINDINGS.md.

Finding 476 — Cron alert scripts silently dead since Feb 15; root crontab unmanaged; a Gateway PID-killer on cron (Sep 26, 2026)

• Severity: High (ops / silent failure). ibkr_health_monitor.sh, ibkr_restart_warning.sh, sunday_2fa_reminder.sh and ibkr_schedule_audit.sh were git 100644 from creation. Root's crontab exec'd them directly, so every run since the Feb 15, 2026 ~08:42 UTC checkout failed with Permission denied (288/day for the health monitor; 3 Sunday reminders; the daily audit). No Sunday 2FA reminder has arrived since Feb 15. The crontab lived only on the host. It also ran a check_trader_health.sh that was never in git (No such file every 5 min), a Jan 8 one-shot exit_reminder.sh, and /etc/cron.d/monitor_23_45 (a Jan 28 one-shot). gateway_restart_manager.sh (100755 since f7dff71, Mon-Sat 05:58) SIGTERM/SIGKILLed any "newer" install4j.ibgateway.GWClient PID. That violates OD5 (page-only, no Gateway lifecycle), and it sent a daily false "✅ Session preserved" (RC10). Does not place or cancel orders. Hardware stops unaffected. Evidence: Project store internal/reliability/prod-cron-gateway-check-2026-09-26.md (read-only probe 17:25-17:30 UTC); Telegram RCA F2/RC10; B1 review blockers 3 and 4.
• Status: IMPLEMENTED-PENDING-DEPLOY (reliability Batch B1). The operator's crontab-only mitigation "cron a" has been live since 17:38 UTC Sep 26: the four scripts run via /bin/bash, and the restart-manager line is commented out (backup /root/crontab.pre-cron-a-20260926T173824Z).
• Root cause: (1) no mode check anywhere; the only mode fix (f7dff71, #437) flipped just the restart manager, an §E miss. (2) Nothing owned the crontab: deploy_ibkr_monitoring.sh scp'd scripts past git and printed lines to paste into crontab -e. (3) The restart manager's premise was obsolete: IBC's AutoRestart restarts the session inside the same JVM, so there is never a second Gateway process. It logged 0 kills in 15 runs Sep 10-26, and its pattern matches neither IBC JVM (ibcalpha.ibc.IbcGateway). But a "fixed" pattern would see live + paper as two Gateways and kill the higher PID, which could be live.
• Fix:
• NEW deploy/cron/root.crontab is the only definition of root's crontab. It holds SHELL=/bin/bash, the one Sunday reminder (50 5 * * 0) and the 04:00 audit, both called as /bin/bash [hidden]/scripts/… (the cron-a form, so a lost exec bit cannot silence them). sunday_2fa_reminder.sh and ibkr_schedule_audit.sh are also 100755 in git (git update-index --chmod=+x).
• NEW deploy/install_crontab.py: --check (multiset compare: 0 match / 1 drift / 2 error), --dry-run and --install. The install is a transaction (review R2 blocker 3):
  • Every live line is validated before any change. Only exact normalized legacy lines (KNOWN_LEGACY_LINES: the cron-a and pre-cron-a forms) are retired, and managed lines are kept. Any other line is an operator job, preserved verbatim under a "preserved host lines" block, so a deploy never deletes one; --check then reports it as EXTRA. A line that runs a retired script together with anything else (… ibkr_health_monitor.sh && /root/job.sh) refuses with rc 3 and nothing changed, including /etc/cron.d. Round 3 (majors 3/minor 1): detection tokenizes the command with shlex (punctuation-aware) and matches the retired basenames after normpath, so scripts//x.sh, quoted 'x.sh', cd … && ./x.sh, bash -c "…/x.sh" and ../scripts/x.sh all refuse; an untokenizable line refuses too. The basename match is exact (x.sh.bak is an operator job). Only the five schedule fields are normalized; the command payload is compared byte-exact.
  • An exclusive non-blocking fcntl lock (/run/lock/polymarket-crontab.lock, CRONTAB_LOCK_PATH) serializes installers; a second one exits rc 2 without touching anything (Round 3 major 4).
  • A new 0600, collision-safe backup (mkstemp: /var/backups/polymarket/crontab.pre-deploy-<ts>-<random>), then the live crontab is re-read: if it changed since planning (a concurrent crontab -e), abort rc 2 with nothing written. Then an atomic crontab <file> swap, then a multiset re-check.
  • If crontab <file> fails and the live crontab is unchanged, rc 2 "crontab unchanged" (no restore needed). On any other install error or verification mismatch the backup is re-installed and verified (rc 2); if the restore fails too, rc 4 names the backup.
  • Only after a verified install, /etc/cron.d/monitor_23_45 is moved aside, and only when it is exactly the Jan 28 one-shot, re-validated at move time; the destination is a fresh 0600 mkstemp file. If it changed or the move fails, it stays in place with a WARNING and --check reports it as EXTRA (Round 3 minor).
  • deploy_from_github.sh runs --install right after the host gates, before migrations and restarts, and any non-zero rc stops the deploy.
  • The pre-pull pre-flight is enforced, not a manual step (Round 3 major 5): deploy/remote_deploy.sh makes git fetch fatal, then, when origin/master carries deploy/install_crontab.py, extracts the candidate installer and root.crontab with git show into a mktemp -d and runs --dry-run against the live crontab. A non-zero rc stops with "crontab pre-flight (#476) failed rc=… — nothing pulled, nothing restarted".
  • One pinned target (Round 4 blocker 1): the unit fetches once and resolves ONE commit (origin/master, or remote_deploy.sh --sha <40-hex>, which must be on origin/master and a fast-forward of HEAD, else rc 7). The crontab pre-flight reads <sha>:deploy/install_crontab.py / <sha>:deploy/cron/root.crontab; the checkout moves with git merge --ff-only <sha> (no second fetch or pull), and deploy_from_github.sh receives DEPLOY_TARGET_SHA, uses it for hygiene, never fetches or pulls on that path, and stops before any gate unless HEAD == <sha>. A manual deploy_from_github.sh run fetches once and pins the same way (pre-flight included). A push landing mid-deploy cannot slip in. --poll <id> is format-checked (Round 4 minor).
  • Round 4 majors 4–6: the command is parsed only up to cron's unescaped % (the rest is stdin); a dynamic token ($VAR, ${x}.sh, $(…), backticks, globs) refuses when it could expand to a retired basename (a bare $X could be anything, so it refuses). Operator lines are reported as [operator job '<schedule>' runs <program> … sha256:<12>] / [env NAME=… sha256:<12>] in the installer output, --check and the dashboard: never arguments or assignment values. After a failed install the backup is restored only over this install's own output; a crontab that is neither the old one nor ours (another writer) is left alone with rc 4 naming the backup. Residual: crontab has no compare-and-swap, so an operator crontab -e in the milliseconds between the re-read and the swap can still be replaced — do not edit root's crontab during a deploy (OPERATIONS §8.F).
• Retired and deleted: gateway_restart_manager.sh (the kill loop is gone with the file, not re-patterned); ibkr_health_monitor.sh (live :4001 paging is ibkr-monitor, #448; running both would double-page every outage); ibkr_restart_warning.sh and the 05:55 duplicate reminder (plan acceptance: one reminder at 05:50); the uninstalled --critical market-hours line (it would re-page every 15 min during an outage); and the check_trader_health.sh / exit_reminder.sh lines (never in git; trader liveness is the Healthchecks earnings-trader slug, #448).
• scripts/ibkr_schedule_audit.sh checks install_crontab.py --check, the ibgateway-ibc/ibkr-monitor units and IBC AutoRestartTime against scripts/ibkr_schedule.env. That file now holds only the restart time; cron times live in the crontab file (R1). NEW integrity rule cron:schedule_audit (WARNING, dashboard-only) runs the audit every cycle, so drift is no longer silent.
• deploy_ibkr_monitoring.sh is now a read-only check (no scp/chmod/crontab edits). scripts/ibkr_diagnostics.py uses --check.
• Drift audit C2 gets the retired names as stale literals (R2 "retiring a mechanism"). They are scanned over the current-state docs plus OPERATIONS, IBKR_CONTINUITY and STATE_MACHINES.
• §B:
• (1) Trace: deploy → install_crontab.py --install → crontab -l → diff of active lines vs deploy/cron/root.crontab → unknown-line gate → backup → crontab <file> → --check. Cron 05:50 Sun → /bin/bash sunday_2fa_reminder.sh → /bin/bash send_telegram.sh (#449) → HTTP 200 → last-send artifact. Integrity cycle → _check_cron_schedule_audit → /bin/bash ibkr_schedule_audit.sh → install_crontab.py --check → cron:schedule_audit row.
• (2) Runtime on the first B1 deploy: the live crontab is cron a. Every active line runs a known script, so it is replaced. The backup holds cron a. Active lines go 9 → 3 (SHELL, reminder, audit). The commented restart-manager line and the empty comment headers go with the old file. monitor_23_45 moves to /var/backups/polymarket/. Sun 05:50: one reminder. A hand-added host line is preserved verbatim and cron:schedule_audit WARNs EXTRA … until it moves into the repo file. A chained retired-script line → rc 3, nothing changed, deploy stops before any restart. A failed or partial crontab write → backup re-installed and verified, rc 2, deploy stops.
• (3) Lines: deploy/cron/root.crontab (NEW); deploy/install_crontab.py (NEW) plan/install/check/_private_file/_verified/_retire_one_shot; deploy/deploy_from_github.sh crontab block; scripts/ibkr_schedule_audit.sh; scripts/ibkr_schedule.env; scripts/sunday_2fa_reminder.sh; deploy_ibkr_monitoring.sh; scripts/ibkr_diagnostics.py check_cron_jobs; integrity_monitor.py _check_cron_schedule_audit + rule; scripts/doc_drift_audit.py RETIRED_MECHANISM_LITERALS.
• (4) Constraints: never delete an operator cron line; never re-introduce a PID-kill path; do not "fix" the restart-manager pattern; the deploy must not revert cron a to direct exec; the audit is dashboard-only (no Telegram); memory.md IBKR contracts #421-#441 untouched.
• (5) Round 4: test_finding_476_managed_crontab.py is 50 (12 fail on 2ae3f5f: %, ?h, glob, ${x}, $(…), backtick and bare-variable refusals; operator secrets never in notes, --check or the refusal; the never-restore-over-another-writer case). test_finding_444_detached_deploy.py is 24 (11 fail on 2ae3f5f: master moving mid-deploy cannot change what is deployed; the pre-flight reads <sha>:; --sha deployed exactly; not-on-master refuses rc 7 before touching the checkout; bad --sha/--poll refused locally; the deploy script never fetches or pulls on the pinned path).
• (5) Round 3: tests/unit/test_finding_476_managed_crontab.py is 37, and 10 fail on 839f58e: five shell-equivalent retired invocations refuse (//, quoted basename, cd && ./, ../scripts/, untokenizable); the payload is compared exactly; a concurrent edit aborts without overwriting; installs are serialized; a changed one-shot is revalidated and left visible (WARNING + EXTRA); install failure leaves the crontab unchanged. Regression guards: bash -c "…" refuses; x.sh.bak is not a retired basename. tests/unit/test_finding_444_detached_deploy.py (+2, both fail on 839f58e): a pre-flight refusal stops before the pull; a pass continues to the pull. Before Round 3 (26; Round 2 +9 new, and 10 cases fail on 3a6555d: operator line preserved; chained line refuses before any mutation incl. cron.d; install failure → restore; partial write → rollback; restore failure → rc 4; backups 0600 + unique; dry-run mutates nothing; duplicate line is drift; deploy installs before migrations and fails hard): every scheduled script exists, is git 100755 and is called via /bin/bash; send_telegram.sh is 100755; the retired files are gone and unscheduled; one Sunday reminder; no scheduled script contains kill/pkill/systemctl start|stop|restart (the same scan matches the facb218 restart manager); cron a is replaced and backed up; an unknown line refuses with the crontab unchanged; the current file is a no-op; --check reports drift; the one-shot is retired only when exact; the audit uses the installer; the env file has no cron times; the deploy installs; deploy_ibkr_monitoring.sh is read-only; the integrity rule is WARNING and reads the audit. On e94dd0d the file does not exist and the modes are 100644, so the suite fails. tests/unit/test_doc_drift_audit.py test_c2_retired_mechanism_fires_unless_history.
• §E: (b) every root-cron entry is now in deploy/cron/root.crontab. The #449 caller list is the reminder only. Crontab blocks in STRATEGY §8.5/8.6, IBKR_CONTINUITY, OPERATIONS §Cron Jobs/§8.F, STATE_MACHINES (daily cycle), EDGE_CASES 4.6 and CONTEXT (session preservation) are rewritten or history-marked. (c) systemd timers are unchanged (their units are already in git). (a) history: FINDINGS #437/#438 mention the restart manager as then-live; docs/packs/ are frozen; the HANDOFF dated log (below "Recently completed") is left as history.
• Acceptance: after the deploy: crontab -l equals deploy/cron/root.crontab; /var/backups/polymarket/crontab.pre-deploy-* holds cron a; bash scripts/ibkr_schedule_audit.sh prints PASS; /api/integrity cron:schedule_audit ok; no Permission denied / No such file in cron logs; Sun Oct 4 05:50 one reminder arrives; no process on the host runs a Gateway kill.
• Changelog: one §O integrity entry.
• Files: deploy/cron/root.crontab (NEW), deploy/install_crontab.py (NEW), deploy/deploy_from_github.sh, deploy_ibkr_monitoring.sh, scripts/ibkr_schedule_audit.sh (mode 100755), scripts/sunday_2fa_reminder.sh (mode 100755), scripts/ibkr_schedule.env, scripts/ibkr_diagnostics.py, scripts/gateway_restart_manager.sh (DELETED), scripts/ibkr_health_monitor.sh (DELETED), scripts/ibkr_restart_warning.sh (DELETED), src/data_tracker/integrity_monitor.py, scripts/doc_drift_audit.py, tests/unit/test_finding_476_managed_crontab.py (NEW), tests/unit/test_doc_drift_audit.py, src/earnings/strategy_changelog.py, docs/STRATEGY.md, docs/IBKR_CONTINUITY.md, docs/OPERATIONS.md, docs/STATE_MACHINES.md, docs/EDGE_CASES.md, docs/CONTEXT.md, docs/AGENT_HANDOFF.md, this entry.