Move database initialization logic from shell script to Python application
lifespan, following separation of concerns and improving maintainability.
Benefits:
- Single source of truth for database initialization (api/main.py lifespan)
- Better testability - Python code vs shell scripts
- Clearer logging with structured messages
- Easier to debug and maintain
- Infrastructure (entrypoint.sh) focuses on service orchestration
- Application (api/main.py) owns its data layer
Changes:
- Removed database init from entrypoint.sh
- Enhanced lifespan function with detailed logging
- Simplified entrypoint script (now 4 steps instead of 5)
- All tests pass (28/28 API endpoint tests)
- Fix lifespan function to access db_path from create_app scope via closure
- Prevents "no such table: jobs" error by ensuring database initialization runs
- Previous version tried to access app.state.db_path before it was set
The issue was that app.state is set after FastAPI instantiation, but the
lifespan function needs the db_path during startup. Using closure allows
the lifespan function to capture db_path from the create_app function scope.
- Add database initialization to API lifespan event handler
- DEV mode: Reset database on startup (unless PRESERVE_DEV_DATA=true)
- PROD mode: Ensure database schema exists
- Migrate from deprecated @app.on_event to modern lifespan context manager
- Fixes 400 error "Another simulation job is already running" on fresh container starts
This ensures the dev database is reset when the API server starts in dev mode,
preventing stale "running" or "pending" jobs from blocking new job creation.
When the trigger simulation API receives an empty models list ([]),
it now correctly falls back to enabled models from config instead
of running with no models.
Changes:
- Update condition to check for both None and empty list
- Add test case for empty models list behavior
- Update API documentation to clarify this behavior
All 28 integration tests pass.
Move data preparation to background worker:
- Fast endpoint response (<1s)
- No blocking downloads
- Worker handles data download and filtering
- Maintains backwards compatibility
Co-Authored-By: Claude <noreply@anthropic.com>
- Add FastAPI @app.on_event("startup") handler to display warning
- Previously only appeared when running directly (not via uvicorn)
- Add DEPLOYMENT_MODE and PRESERVE_DEV_DATA to docker-compose.yml
- Update CHANGELOG.md with fix documentation
Fixes issue where dev mode banner wasn't visible in Docker logs
because uvicorn imports app without executing __main__ block.
BREAKING CHANGE: end_date is now required and cannot be null/empty
New Features:
- Resume mode: Set start_date to null to continue from last completed date per model
- Idempotent by default: Skip already-completed dates with replace_existing=false
- Per-model independence: Each model resumes from its own last completed date
- Cold start handling: If no data exists in resume mode, runs only end_date as single day
API Changes:
- start_date: Now optional (null enables resume mode)
- end_date: Now REQUIRED (cannot be null or empty string)
- replace_existing: New optional field (default: false for idempotent behavior)
Implementation:
- Added JobManager.get_last_completed_date_for_model() method
- Added JobManager.get_completed_model_dates() method
- Updated create_job() to support model_day_filter for selective task creation
- Fixed bug with start_date=None in price data checks
Documentation:
- Updated API_REFERENCE.md with complete examples and behavior matrix
- Updated QUICK_START.md with resume mode examples
- Updated docs/user-guide/using-the-api.md
- Added CHANGELOG_NEW_API.md with migration guide
- Updated all integration tests for new schema
- Updated client library examples (Python, TypeScript)
Migration:
- Old: {"start_date": "2025-01-16"}
- New: {"start_date": "2025-01-16", "end_date": "2025-01-16"}
- Resume: {"start_date": null, "end_date": "2025-01-31"}
See CHANGELOG_NEW_API.md for complete details.
Add comprehensive warning display when server starts in development mode
to ensure users are aware of simulated AI calls and data handling.
Changes:
- Add log_dev_mode_startup_warning() function in deployment_config.py
- Display warning on main.py startup when DEPLOYMENT_MODE=DEV
- Display warning on API server startup (api/main.py)
- Warning shows AI simulation status and data persistence behavior
- Provides clear instructions for switching to PROD mode
The warning is highly visible and informs users that:
- AI API calls are simulated (no costs incurred)
- Data may be reset between runs (based on PRESERVE_DEV_DATA)
- System is using isolated dev database and paths
Co-Authored-By: Claude <noreply@anthropic.com>
Phase 2 progress - API integration complete.
API Changes:
- Replace date_range (List[str]) with start_date/end_date (str)
- Add automatic end_date defaulting to start_date for single day
- Add date format validation
- Integrate PriceDataManager for on-demand downloads
- Add rate limit handling (trusts provider, no pre-config)
- Validate date ranges with configurable max days (MAX_SIMULATION_DAYS)
New Modules:
- api/date_utils.py - Date validation and expansion utilities
- scripts/migrate_price_data.py - Migration script for merged.jsonl
API Flow:
1. Validate date range (start <= end, max 30 days, not future)
2. Check missing price data coverage
3. Download missing data if AUTO_DOWNLOAD_PRICE_DATA=true
4. Priority-based download (maximize date completion)
5. Create job with available trading dates
6. Graceful handling of partial data (rate limits)
Configuration:
- AUTO_DOWNLOAD_PRICE_DATA (default: true)
- MAX_SIMULATION_DAYS (default: 30)
- No rate limit configuration needed
Still TODO:
- Update tools/price_tools.py to read from database
- Implement simulation run tracking
- Update .env.example
- Comprehensive testing
- Documentation updates
Breaking Changes:
- API request format changed (date_range -> start_date/end_date)
- This completes v0.3.0 preparation
Makes config_path an internal server detail rather than an API parameter.
Changes:
- Remove config_path from SimulateTriggerRequest
- Add config_path parameter to create_app() with default
- Store in app.state.config_path for internal use
- Update trigger endpoint to use internal config path
- Change missing config error from 400 to 500 (server error)
API calls now only need to specify date_range (and optionally models):
POST /simulate/trigger
{"date_range": ["2025-01-16"]}
The server uses configs/default_config.json by default.
This simplifies the API and hides implementation details from clients.
Changed the API to respect the 'enabled' field in model configurations,
rather than requiring models to be explicitly specified in API requests.
Changes:
- Make 'models' parameter optional in POST /simulate/trigger
- If models not provided, read config and use enabled models
- If models provided, use as explicit override (for testing)
- Raise error if no enabled models found and none specified
- Update response message to show model count
Behavior:
- Default: Only runs models with "enabled": true in config
- Override: Can still specify models in request for manual testing
- Safety: Prevents accidental execution of disabled/expensive models
Example before (required):
POST /simulate/trigger
{"config_path": "...", "date_range": [...], "models": ["gpt-4"]}
Example after (optional):
POST /simulate/trigger
{"config_path": "...", "date_range": [...]}
# Uses models where enabled: true
This makes the config file the source of truth for which models
should run, while still allowing ad-hoc overrides for testing.
