Added dev database files and dev_agent_data directory to gitignore
to prevent runtime dev data from being committed to the repository.
Patterns added:
- data/jobs_dev.db
- data/*_dev.db
- data/dev_agent_data/
This ensures dev mode runtime data remains local and doesn't pollute
version control.
Added comprehensive implementation plan for development mode feature
and test configuration used during verification.
Files:
- docs/plans/2025-11-01-dev-mode-mock-ai.md: Complete 12-task plan
- configs/test_dev_mode.json: Test configuration for dev mode
These files document the feature implementation process and provide
reference configurations for testing.
Fix critical bug where dev mode was initializing the production database
path instead of the dev database path. The initialize_dev_database() call
now correctly uses get_db_path() to resolve to data/jobs_dev.db.
Impact:
- Before: DEV mode would reset data/jobs.db (production database)
- After: DEV mode correctly resets data/jobs_dev.db (dev database)
Testing:
- Verified database isolation between dev and prod
- Confirmed PRESERVE_DEV_DATA flag works correctly
- Validated dev mode banner and deployment mode detection
Documentation:
- Added comprehensive manual verification results
- Documented all test cases and outcomes
- Recorded fix details for future reference
Task: Task 12 - Manual Verification and Final Testing
Plan: docs/plans/2025-11-01-dev-mode-mock-ai.md
Add comprehensive development mode documentation to README.md, API_REFERENCE.md, and CLAUDE.md:
README.md:
- New "Development Mode" section after Configuration
- Quick start guide with environment variables
- Explanation of DEV vs PROD mode behavior
- Mock AI behavior and stock rotation details
- Environment variables reference
- Use cases and limitations
API_REFERENCE.md:
- New "Deployment Mode" section after health check
- Response format with deployment_mode fields
- DEV mode behavior explanation
- Health check example with deployment fields
- Use cases for testing and CI/CD
CLAUDE.md:
- New "Development Mode" subsection in Important Implementation Details
- Deployment modes overview
- DEV mode characteristics
- Implementation details with file references
- Testing commands and mock behavior notes
All sections explain:
- DEPLOYMENT_MODE environment variable (PROD/DEV)
- PRESERVE_DEV_DATA flag for dev data persistence
- Mock AI provider with deterministic stock rotation
- Separate dev database and data paths
- Use cases for development and testing
Add comprehensive design document for rebranding project from AI-Trader
to AI-Trader-Server. Includes 4-phase approach with validation
checkpoints, naming conventions, and success criteria.
Co-Authored-By: Claude <noreply@anthropic.com>
Reorganize documentation into user-focused, developer-focused, and deployment-focused sections.
**New structure:**
- Root: README.md (streamlined), QUICK_START.md, API_REFERENCE.md
- docs/user-guide/: configuration, API usage, integrations, troubleshooting
- docs/developer/: contributing, development setup, testing, architecture
- docs/deployment/: Docker deployment, production checklist, monitoring
- docs/reference/: environment variables, MCP tools, data formats
**Changes:**
- Streamline README.md from 831 to 469 lines
- Create QUICK_START.md for 5-minute onboarding
- Create API_REFERENCE.md as single source of truth for API
- Remove 9 outdated specification docs (v0.2.0 API design)
- Remove DOCKER_API.md (content consolidated into new structure)
- Remove docs/plans/ directory with old design documents
- Update CLAUDE.md with documentation structure guide
- Remove orchestration-specific references
**Benefits:**
- Clear entry points for different audiences
- No content duplication
- Better discoverability through logical hierarchy
- All content reflects current v0.3.0 API
Remove config_path from request examples throughout README.md as it is
not a per-request parameter. Config file path is set when initializing
the API server, not with each API call.
Changes:
- Remove config_path from all curl examples
- Remove config_path from TypeScript integration example
- Remove config_path from Python integration example
- Update parameter documentation to clarify config_path is server init only
- Add note that detail level control is not yet implemented in v0.3.0
- Clarify server configuration is set via CONFIG_PATH env var at startup
API Request Parameters (v0.3.0):
- start_date (required)
- end_date (optional, defaults to start_date)
- models (optional, defaults to all enabled models from config)
Server Configuration:
- Set via CONFIG_PATH environment variable or create_app() parameter
- Default: configs/default_config.json
- Contains model definitions and agent settings
Co-Authored-By: Claude <noreply@anthropic.com>
Correct all code examples in Integration Examples and Advanced API
Usage sections to use complete, valid JSON with all required fields.
Changes:
- TypeScript: Fix body type to 'any' and use proper property assignment
- Python: Fix variable overwriting, use unique names for examples
- On-Demand Downloads: Replace '...' with complete JSON examples
- Detail Levels: Add complete curl examples with all required fields
- Concurrent Job Prevention: Show complete API calls with proper JSON
All curl examples now include:
- Content-Type header
- Proper JSON formatting
- All required fields (config_path, start_date, models)
Co-Authored-By: Claude <noreply@anthropic.com>
Update API documentation to reflect start_date/end_date parameters
instead of date_range arrays. Add comprehensive API reference with
validation rules, error handling, and advanced usage patterns.
Changes:
- Replace date_range arrays with start_date/end_date parameters
- Document optional end_date (defaults to start_date for single day)
- Add complete parameter documentation for POST /simulate/trigger
- Add validation rules (date format, range limits, model selection)
- Add error response examples with HTTP status codes
- Document job and model-day status values
- Add Advanced API Usage section:
- On-demand price data download behavior
- Detail levels (summary vs full)
- Concurrent job prevention
- Update Quick Start curl examples
- Update Integration Examples (TypeScript and Python)
- Update Latest Updates section with v0.3.0 improvements
Co-Authored-By: Claude <noreply@anthropic.com>
Add automatic schema migration to handle existing databases that don't
have the simulation_run_id column in the positions table.
Problem:
- v0.3.0-alpha.3 databases lack simulation_run_id column
- CREATE TABLE IF NOT EXISTS doesn't add new columns to existing tables
- Index creation fails with "no such column: simulation_run_id"
Solution:
- Add _migrate_schema() function to detect and migrate old schemas
- Check if positions table exists and inspect its columns
- ALTER TABLE to add simulation_run_id if missing
- Run migration before creating indexes
This allows seamless upgrades from alpha.3 to alpha.4 without manual
database deletion or migration scripts.
Fixes docker compose startup error:
sqlite3.OperationalError: no such column: simulation_run_id
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add comprehensive web dashboard interface to planned features for v0.4.0.
Web UI Features:
- Job management dashboard
* View/monitor active, pending, and completed jobs
* Start new simulations with form-based configuration
* Real-time job progress monitoring
* Cancel running jobs
- Results visualization
* Performance charts (P&L over time, cumulative returns)
* Position history timeline
* Model comparison views
* Trade log explorer with filtering
- Configuration management
* Model configuration editor
* Date range selection with calendar picker
* Price data coverage visualization
- Technical implementation
* Modern frontend framework (React, Vue.js, or Svelte)
* Real-time updates via WebSocket or Server-Sent Events
* Responsive design for mobile access
* Chart library for visualizations
* Single container deployment alongside API
The web UI will provide an accessible interface for users who prefer
graphical interaction over API calls, while maintaining the same
functionality available through the REST API.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Remove all future releases (v0.5.0-v0.7.0) and infrastructure/enhancement
sections from roadmap. Focus exclusively on v0.4.0 planned features.
v0.4.0 - Enhanced Simulation Management remains with:
- Resume/continue API for advancing from last completed date
- Position history tracking and analysis
- Advanced performance metrics (Sharpe, Sortino, drawdown, win rate)
- Price data management endpoints
Removed sections:
- v0.5.0 Real-Time Trading Support
- v0.6.0 Multi-Strategy & Portfolio Management
- v0.7.0 Alternative Data & Advanced Features
- Future Enhancements (infrastructure, data, UI, AI/ML, integration, testing)
Keep roadmap focused on near-term deliverables with clear scope.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Create ROADMAP.md documenting planned features across multiple releases.
Planned releases:
- v0.4.0: Enhanced simulation management
* Resume/continue API for advancing from last completed date
* Position history tracking and analysis
* Advanced performance metrics (Sharpe, Sortino, drawdown)
* Price data management endpoints
- v0.5.0: Real-time trading support
* Live market data integration
* Real-time simulation mode
* Scheduled automation
* WebSocket price feeds
- v0.6.0: Multi-strategy & portfolio management
* Strategy composition and ensembles
* Advanced risk controls
* Portfolio-level optimization
* Dynamic allocation
- v0.7.0: Alternative data & advanced features
* News and sentiment analysis
* Market regime detection
* Custom indicators
* Event-driven strategies
Future enhancements:
- Kubernetes deployment and cloud provider support
- Alternative databases (PostgreSQL, TimescaleDB)
- Web UI dashboard with real-time visualization
- Model training and reinforcement learning
- Webhook notifications and plugin system
- Performance and chaos testing
Key feature: Resume API in v0.4.0
- POST /simulate/resume - Continue from last completed date
- POST /simulate/continue - Extend existing simulations
- Automatic detection of completion state per model
- Support for daily incremental updates
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Clarify that v0.3.0 is the first version with REST API functionality,
and remove misleading "API Request Format Changed" entries that implied
the API existed in v0.2.0.
Key improvements:
- Remove "API Request Format Changed" from Changed section (API is new)
- Remove "Model Selection" and "API Interface" items (API design, not changes)
- Clarify batch mode removal context (v0.2.0 had batch, v0.3.0 adds API)
- Update test counts to reflect new tests (175 total, up from 102)
- Add coverage details for new test files (date_utils, price_data_manager)
- Update test execution time estimate (~12 seconds for full suite)
Breaking changes now correctly identify what changed from v0.2.0:
- Batch execution replaced with REST API (new capability)
- Price data storage moved from JSONL to SQLite (migration required)
- Configuration variables added/removed for new features
v0.2.0 was Docker-focused with batch execution
v0.3.0 adds REST API, on-demand downloads, and database storage
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add 64 new tests covering date utilities, price data management, and
on-demand download workflows with 100% coverage for date_utils and 85%
coverage for price_data_manager.
New test files:
- tests/unit/test_date_utils.py (22 tests)
* Date range expansion and validation
* Max simulation days configuration
* Chronological ordering and boundary checks
* 100% coverage of api/date_utils.py
- tests/unit/test_price_data_manager.py (33 tests)
* Initialization and configuration
* Symbol date retrieval and coverage detection
* Priority-based download ordering
* Rate limit and error handling
* Data storage and coverage tracking
* 85% coverage of api/price_data_manager.py
- tests/integration/test_on_demand_downloads.py (10 tests)
* End-to-end download workflows
* Rate limit handling with graceful degradation
* Coverage tracking and gap detection
* Data validation and filtering
Code improvements:
- Add DownloadError exception class for non-rate-limit failures
- Update all ValueError raises to DownloadError for consistency
- Add API key validation at download start
- Improve response validation to check for Meta Data
Test coverage:
- 64 tests passing (54 unit + 10 integration)
- api/date_utils.py: 100% coverage
- api/price_data_manager.py: 85% coverage
- Validates priority-first download strategy
- Confirms graceful rate limit handling
- Verifies database storage and retrieval
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Phase 2 progress - API integration complete.
API Changes:
- Replace date_range (List[str]) with start_date/end_date (str)
- Add automatic end_date defaulting to start_date for single day
- Add date format validation
- Integrate PriceDataManager for on-demand downloads
- Add rate limit handling (trusts provider, no pre-config)
- Validate date ranges with configurable max days (MAX_SIMULATION_DAYS)
New Modules:
- api/date_utils.py - Date validation and expansion utilities
- scripts/migrate_price_data.py - Migration script for merged.jsonl
API Flow:
1. Validate date range (start <= end, max 30 days, not future)
2. Check missing price data coverage
3. Download missing data if AUTO_DOWNLOAD_PRICE_DATA=true
4. Priority-based download (maximize date completion)
5. Create job with available trading dates
6. Graceful handling of partial data (rate limits)
Configuration:
- AUTO_DOWNLOAD_PRICE_DATA (default: true)
- MAX_SIMULATION_DAYS (default: 30)
- No rate limit configuration needed
Still TODO:
- Update tools/price_tools.py to read from database
- Implement simulation run tracking
- Update .env.example
- Comprehensive testing
- Documentation updates
Breaking Changes:
- API request format changed (date_range -> start_date/end_date)
- This completes v0.3.0 preparation
Phase 1 of v0.3.0 date range and on-demand download implementation.
Database changes:
- Add price_data table (OHLCV data, replaces merged.jsonl)
- Add price_data_coverage table (track downloaded date ranges)
- Add simulation_runs table (soft delete support)
- Add simulation_run_id to positions table
- Add comprehensive indexes for new tables
New modules:
- api/price_data_manager.py - Priority-based download manager
- Coverage gap detection
- Smart download prioritization (maximize date completion)
- Rate limit handling with retry logic
- Alpha Vantage integration
Configuration:
- configs/nasdaq100_symbols.json - NASDAQ 100 constituent list
Next steps (not yet implemented):
- Migration script for merged.jsonl -> price_data
- Update API models (start_date/end_date)
- Update tools/price_tools.py to read from database
- Simulation run tracking implementation
- API integration
- Tests and documentation
This is work in progress for the v0.3.0 release.
Makes config_path an internal server detail rather than an API parameter.
Changes:
- Remove config_path from SimulateTriggerRequest
- Add config_path parameter to create_app() with default
- Store in app.state.config_path for internal use
- Update trigger endpoint to use internal config path
- Change missing config error from 400 to 500 (server error)
API calls now only need to specify date_range (and optionally models):
POST /simulate/trigger
{"date_range": ["2025-01-16"]}
The server uses configs/default_config.json by default.
This simplifies the API and hides implementation details from clients.
Changed the API to respect the 'enabled' field in model configurations,
rather than requiring models to be explicitly specified in API requests.
Changes:
- Make 'models' parameter optional in POST /simulate/trigger
- If models not provided, read config and use enabled models
- If models provided, use as explicit override (for testing)
- Raise error if no enabled models found and none specified
- Update response message to show model count
Behavior:
- Default: Only runs models with "enabled": true in config
- Override: Can still specify models in request for manual testing
- Safety: Prevents accidental execution of disabled/expensive models
Example before (required):
POST /simulate/trigger
{"config_path": "...", "date_range": [...], "models": ["gpt-4"]}
Example after (optional):
POST /simulate/trigger
{"config_path": "...", "date_range": [...]}
# Uses models where enabled: true
This makes the config file the source of truth for which models
should run, while still allowing ad-hoc overrides for testing.
Complete rewrite of README.md to reflect the new REST API service
architecture and remove batch mode references.
Changes:
- Focus on REST API deployment and usage
- Updated architecture diagram showing FastAPI → Worker → Database flow
- Comprehensive API endpoint documentation with examples
- Docker-first quick start guide
- Integration examples (Windmill.dev, Python client)
- Database schema documentation
- Simplified configuration guide
- Updated project structure
- Removed batch mode references
- Removed web UI mentions
The new README positions AI-Trader as an API service for autonomous
trading simulations, not a standalone batch application.
Key additions:
- Complete API reference (/trigger, /status, /results, /health)
- Integration patterns for external orchestration
- Database querying examples
- Testing and validation procedures
- Production deployment guidance
The web UI (docs/index.html, portfolio.html) exists but is not served
in API mode. Removing the port configuration to eliminate confusion.
Changes:
- Remove port 8888 mapping from docker-compose.yml
- Remove WEB_HTTP_PORT from .env.example
- Update Dockerfile EXPOSE to only port 8080
- Update CHANGELOG.md to document removal
Technical details:
- Web UI static files remain in docs/ folder (legacy from batch mode)
- These were designed for JSONL file format, not the new SQLite database
- No web server was ever started in entrypoint.sh for API mode
- Port 8888 was exposed but nothing listened on it
Result:
- Cleaner configuration (1 fewer port mapping)
- Only REST API (8080) is exposed
- Eliminates user confusion about non-functional web UI
Fixed line 70-71 where a literal newline in the bash script was
breaking YAML parsing. Changed from:
TAGS="$TAGS
ghcr.io/..."
To:
TAGS="${TAGS}"$'\n'"ghcr.io/..."
This uses bash's ANSI-C quoting syntax to properly insert a newline
within a single YAML line, avoiding the syntax error.
Consolidated the configuration simplification changes (RUNTIME_ENV_PATH
removal, API_PORT cleanup, MCP port removal) into the v0.3.0 release
notes under the 'Changed - Configuration' section.
This ensures all v0.3.0 changes are documented together in a single
release entry rather than split across Unreleased and v0.3.0 sections.
MCP services are completely internal to the container and accessed
only via localhost. They should not be configurable or exposed.
Changes:
- Remove MATH_HTTP_PORT, SEARCH_HTTP_PORT, TRADE_HTTP_PORT,
GETPRICE_HTTP_PORT from docker-compose.yml environment
- Remove MCP service port mappings from docker-compose.yml
- Remove MCP port configuration from .env.example
- Update README.md to remove MCP port configuration
- Update CLAUDE.md to clarify MCP services use fixed internal ports
- Update CHANGELOG.md with these simplifications
Technical details:
- MCP services hardcode to ports 8000-8003 via os.getenv() defaults
- Services only accessed via localhost URLs within container:
- http://localhost:8000/mcp (math)
- http://localhost:8001/mcp (search)
- http://localhost:8002/mcp (trade)
- http://localhost:8003/mcp (price)
- No external access needed or desired for these services
- Only API (8080) and web dashboard (8888) should be exposed
Benefits:
- Simpler configuration (4 fewer environment variables)
- Reduced attack surface (4 fewer exposed ports)
- Clearer architecture (internal vs external services)
- Prevents accidental misconfiguration of internal services
The API_PORT variable was incorrectly included in the container's
environment section. It should only be used for host port mapping
in docker-compose.yml, not passed into the container.
Changes:
- Remove API_PORT from environment section in docker-compose.yml
- Container always uses port 8080 internally (hardcoded in entrypoint.sh)
- API_PORT in .env/.env.example only controls the host-side mapping:
ports: "${API_PORT:-8080}:8080" (host:container)
Why this matters:
- Prevents confusion about whether API_PORT changes internal port
- Clarifies that entrypoint.sh hardcodes --port 8080
- Simplifies container environment (one less unused variable)
- More explicit about the port mapping behavior
No functional change - the container was already ignoring this variable.
Simplifies deployment configuration by removing the RUNTIME_ENV_PATH
environment variable, which is no longer needed for API mode.
Changes:
- Remove RUNTIME_ENV_PATH from docker-compose.yml
- Remove RUNTIME_ENV_PATH from .env.example
- Update CLAUDE.md to reflect API-managed runtime configs
- Update README.md to remove RUNTIME_ENV_PATH from config examples
- Update CHANGELOG.md with this simplification
Technical details:
- API mode dynamically creates isolated runtime config files via
RuntimeConfigManager (data/runtime_env_{job_id}_{model}_{date}.json)
- tools/general_tools.py already handles missing RUNTIME_ENV_PATH
gracefully, returning empty dict and warning on writes
- No functional impact - all tests pass without this variable set
- Reduces configuration complexity for new deployments
Breaking change: None - variable was vestigial from batch mode era
Updates the docker-release.yml workflow to distinguish between
stable releases and pre-releases (alpha, beta, rc versions).
Changes:
- Add pre-release detection logic to extract version step
- Create new "Generate Docker tags" step to conditionally build tag list
- Only apply 'latest' tag for stable releases
- Pre-releases are tagged with version number only
- Update "Image published" message to reflect pre-release status
Example behavior:
- v0.3.0 -> tags: 0.3.0, latest
- v0.3.0-alpha -> tags: 0.3.0-alpha (NOT latest)
- v1.0.0-rc1 -> tags: 1.0.0-rc1 (NOT latest)
This prevents pre-release versions from overwriting the stable
'latest' tag, allowing users to safely pull the latest stable
version while still providing access to pre-release versions by
explicit version tag.
- Document port conflict resolution in TESTING_GUIDE.md
- Add example for custom API_PORT in .env.example
- Explain container vs host port architecture
- Provide solutions for common port conflict scenarios
- Read API_PORT from .env file if it exists
- Construct API_BASE_URL using configured port
- Display which URL is being tested
- Consistent with validate_docker_build.sh behavior
- Read API_PORT from .env file if it exists
- Use configured port instead of hardcoded 8080
- Display which port is being tested
- Fixes validation when API_PORT is customized
- Add curl for Docker health checks and diagnostics
- Add procps for process monitoring (ps command)
- Required for validation scripts to work properly
- Minimal size increase (~5MB) for critical debugging tools
- Move trap setup before uvicorn (was after, never executed)
- Use exec to replace bash with uvicorn process (better signal handling)
- Ensures uvicorn stays running as PID 1 in container
