Critical fixes identified in code review:
1. Add warnings column migration to _migrate_schema()
- Checks if warnings column exists in jobs table
- Adds column via ALTER TABLE if missing
- Ensures existing databases get new column on upgrade
2. Document CHECK constraint limitation
- Added docstring explaining ALTER TABLE cannot add CHECK constraints
- Notes that "downloading_data" status requires fresh DB or manual migration
3. Add comprehensive migration tests
- test_migration_adds_warnings_column: Verifies warnings column migration
- test_migration_adds_simulation_run_id_column: Tests existing migration
- Both tests include cleanup to prevent cross-test contamination
4. Update test fixtures and expectations
- Updated clean_db fixture to delete from all 9 tables
- Fixed table count assertions (6 -> 9 tables)
- Updated expected columns in schema tests
All 21 database tests now pass.
Add support for:
- downloading_data job status for visibility during data prep
- warnings TEXT column for storing job-level warnings (JSON array)
Co-Authored-By: Claude <noreply@anthropic.com>
Add automatic schema migration to handle existing databases that don't
have the simulation_run_id column in the positions table.
Problem:
- v0.3.0-alpha.3 databases lack simulation_run_id column
- CREATE TABLE IF NOT EXISTS doesn't add new columns to existing tables
- Index creation fails with "no such column: simulation_run_id"
Solution:
- Add _migrate_schema() function to detect and migrate old schemas
- Check if positions table exists and inspect its columns
- ALTER TABLE to add simulation_run_id if missing
- Run migration before creating indexes
This allows seamless upgrades from alpha.3 to alpha.4 without manual
database deletion or migration scripts.
Fixes docker compose startup error:
sqlite3.OperationalError: no such column: simulation_run_id
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Phase 1 of v0.3.0 date range and on-demand download implementation.
Database changes:
- Add price_data table (OHLCV data, replaces merged.jsonl)
- Add price_data_coverage table (track downloaded date ranges)
- Add simulation_runs table (soft delete support)
- Add simulation_run_id to positions table
- Add comprehensive indexes for new tables
New modules:
- api/price_data_manager.py - Priority-based download manager
- Coverage gap detection
- Smart download prioritization (maximize date completion)
- Rate limit handling with retry logic
- Alpha Vantage integration
Configuration:
- configs/nasdaq100_symbols.json - NASDAQ 100 constituent list
Next steps (not yet implemented):
- Migration script for merged.jsonl -> price_data
- Update API models (start_date/end_date)
- Update tools/price_tools.py to read from database
- Simulation run tracking implementation
- API integration
- Tests and documentation
This is work in progress for the v0.3.0 release.
