Remove all future releases (v0.5.0-v0.7.0) and infrastructure/enhancement
sections from roadmap. Focus exclusively on v0.4.0 planned features.
v0.4.0 - Enhanced Simulation Management remains with:
- Resume/continue API for advancing from last completed date
- Position history tracking and analysis
- Advanced performance metrics (Sharpe, Sortino, drawdown, win rate)
- Price data management endpoints
Removed sections:
- v0.5.0 Real-Time Trading Support
- v0.6.0 Multi-Strategy & Portfolio Management
- v0.7.0 Alternative Data & Advanced Features
- Future Enhancements (infrastructure, data, UI, AI/ML, integration, testing)
Keep roadmap focused on near-term deliverables with clear scope.
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Create ROADMAP.md documenting planned features across multiple releases.
Planned releases:
- v0.4.0: Enhanced simulation management
* Resume/continue API for advancing from last completed date
* Position history tracking and analysis
* Advanced performance metrics (Sharpe, Sortino, drawdown)
* Price data management endpoints
- v0.5.0: Real-time trading support
* Live market data integration
* Real-time simulation mode
* Scheduled automation
* WebSocket price feeds
- v0.6.0: Multi-strategy & portfolio management
* Strategy composition and ensembles
* Advanced risk controls
* Portfolio-level optimization
* Dynamic allocation
- v0.7.0: Alternative data & advanced features
* News and sentiment analysis
* Market regime detection
* Custom indicators
* Event-driven strategies
Future enhancements:
- Kubernetes deployment and cloud provider support
- Alternative databases (PostgreSQL, TimescaleDB)
- Web UI dashboard with real-time visualization
- Model training and reinforcement learning
- Webhook notifications and plugin system
- Performance and chaos testing
Key feature: Resume API in v0.4.0
- POST /simulate/resume - Continue from last completed date
- POST /simulate/continue - Extend existing simulations
- Automatic detection of completion state per model
- Support for daily incremental updates
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Clarify that v0.3.0 is the first version with REST API functionality,
and remove misleading "API Request Format Changed" entries that implied
the API existed in v0.2.0.
Key improvements:
- Remove "API Request Format Changed" from Changed section (API is new)
- Remove "Model Selection" and "API Interface" items (API design, not changes)
- Clarify batch mode removal context (v0.2.0 had batch, v0.3.0 adds API)
- Update test counts to reflect new tests (175 total, up from 102)
- Add coverage details for new test files (date_utils, price_data_manager)
- Update test execution time estimate (~12 seconds for full suite)
Breaking changes now correctly identify what changed from v0.2.0:
- Batch execution replaced with REST API (new capability)
- Price data storage moved from JSONL to SQLite (migration required)
- Configuration variables added/removed for new features
v0.2.0 was Docker-focused with batch execution
v0.3.0 adds REST API, on-demand downloads, and database storage
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add 64 new tests covering date utilities, price data management, and
on-demand download workflows with 100% coverage for date_utils and 85%
coverage for price_data_manager.
New test files:
- tests/unit/test_date_utils.py (22 tests)
* Date range expansion and validation
* Max simulation days configuration
* Chronological ordering and boundary checks
* 100% coverage of api/date_utils.py
- tests/unit/test_price_data_manager.py (33 tests)
* Initialization and configuration
* Symbol date retrieval and coverage detection
* Priority-based download ordering
* Rate limit and error handling
* Data storage and coverage tracking
* 85% coverage of api/price_data_manager.py
- tests/integration/test_on_demand_downloads.py (10 tests)
* End-to-end download workflows
* Rate limit handling with graceful degradation
* Coverage tracking and gap detection
* Data validation and filtering
Code improvements:
- Add DownloadError exception class for non-rate-limit failures
- Update all ValueError raises to DownloadError for consistency
- Add API key validation at download start
- Improve response validation to check for Meta Data
Test coverage:
- 64 tests passing (54 unit + 10 integration)
- api/date_utils.py: 100% coverage
- api/price_data_manager.py: 85% coverage
- Validates priority-first download strategy
- Confirms graceful rate limit handling
- Verifies database storage and retrieval
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Phase 2 progress - API integration complete.
API Changes:
- Replace date_range (List[str]) with start_date/end_date (str)
- Add automatic end_date defaulting to start_date for single day
- Add date format validation
- Integrate PriceDataManager for on-demand downloads
- Add rate limit handling (trusts provider, no pre-config)
- Validate date ranges with configurable max days (MAX_SIMULATION_DAYS)
New Modules:
- api/date_utils.py - Date validation and expansion utilities
- scripts/migrate_price_data.py - Migration script for merged.jsonl
API Flow:
1. Validate date range (start <= end, max 30 days, not future)
2. Check missing price data coverage
3. Download missing data if AUTO_DOWNLOAD_PRICE_DATA=true
4. Priority-based download (maximize date completion)
5. Create job with available trading dates
6. Graceful handling of partial data (rate limits)
Configuration:
- AUTO_DOWNLOAD_PRICE_DATA (default: true)
- MAX_SIMULATION_DAYS (default: 30)
- No rate limit configuration needed
Still TODO:
- Update tools/price_tools.py to read from database
- Implement simulation run tracking
- Update .env.example
- Comprehensive testing
- Documentation updates
Breaking Changes:
- API request format changed (date_range -> start_date/end_date)
- This completes v0.3.0 preparation
Phase 1 of v0.3.0 date range and on-demand download implementation.
Database changes:
- Add price_data table (OHLCV data, replaces merged.jsonl)
- Add price_data_coverage table (track downloaded date ranges)
- Add simulation_runs table (soft delete support)
- Add simulation_run_id to positions table
- Add comprehensive indexes for new tables
New modules:
- api/price_data_manager.py - Priority-based download manager
- Coverage gap detection
- Smart download prioritization (maximize date completion)
- Rate limit handling with retry logic
- Alpha Vantage integration
Configuration:
- configs/nasdaq100_symbols.json - NASDAQ 100 constituent list
Next steps (not yet implemented):
- Migration script for merged.jsonl -> price_data
- Update API models (start_date/end_date)
- Update tools/price_tools.py to read from database
- Simulation run tracking implementation
- API integration
- Tests and documentation
This is work in progress for the v0.3.0 release.
Makes config_path an internal server detail rather than an API parameter.
Changes:
- Remove config_path from SimulateTriggerRequest
- Add config_path parameter to create_app() with default
- Store in app.state.config_path for internal use
- Update trigger endpoint to use internal config path
- Change missing config error from 400 to 500 (server error)
API calls now only need to specify date_range (and optionally models):
POST /simulate/trigger
{"date_range": ["2025-01-16"]}
The server uses configs/default_config.json by default.
This simplifies the API and hides implementation details from clients.
Changed the API to respect the 'enabled' field in model configurations,
rather than requiring models to be explicitly specified in API requests.
Changes:
- Make 'models' parameter optional in POST /simulate/trigger
- If models not provided, read config and use enabled models
- If models provided, use as explicit override (for testing)
- Raise error if no enabled models found and none specified
- Update response message to show model count
Behavior:
- Default: Only runs models with "enabled": true in config
- Override: Can still specify models in request for manual testing
- Safety: Prevents accidental execution of disabled/expensive models
Example before (required):
POST /simulate/trigger
{"config_path": "...", "date_range": [...], "models": ["gpt-4"]}
Example after (optional):
POST /simulate/trigger
{"config_path": "...", "date_range": [...]}
# Uses models where enabled: true
This makes the config file the source of truth for which models
should run, while still allowing ad-hoc overrides for testing.
Complete rewrite of README.md to reflect the new REST API service
architecture and remove batch mode references.
Changes:
- Focus on REST API deployment and usage
- Updated architecture diagram showing FastAPI → Worker → Database flow
- Comprehensive API endpoint documentation with examples
- Docker-first quick start guide
- Integration examples (Windmill.dev, Python client)
- Database schema documentation
- Simplified configuration guide
- Updated project structure
- Removed batch mode references
- Removed web UI mentions
The new README positions AI-Trader as an API service for autonomous
trading simulations, not a standalone batch application.
Key additions:
- Complete API reference (/trigger, /status, /results, /health)
- Integration patterns for external orchestration
- Database querying examples
- Testing and validation procedures
- Production deployment guidance
The web UI (docs/index.html, portfolio.html) exists but is not served
in API mode. Removing the port configuration to eliminate confusion.
Changes:
- Remove port 8888 mapping from docker-compose.yml
- Remove WEB_HTTP_PORT from .env.example
- Update Dockerfile EXPOSE to only port 8080
- Update CHANGELOG.md to document removal
Technical details:
- Web UI static files remain in docs/ folder (legacy from batch mode)
- These were designed for JSONL file format, not the new SQLite database
- No web server was ever started in entrypoint.sh for API mode
- Port 8888 was exposed but nothing listened on it
Result:
- Cleaner configuration (1 fewer port mapping)
- Only REST API (8080) is exposed
- Eliminates user confusion about non-functional web UI
Fixed line 70-71 where a literal newline in the bash script was
breaking YAML parsing. Changed from:
TAGS="$TAGS
ghcr.io/..."
To:
TAGS="${TAGS}"$'\n'"ghcr.io/..."
This uses bash's ANSI-C quoting syntax to properly insert a newline
within a single YAML line, avoiding the syntax error.
Consolidated the configuration simplification changes (RUNTIME_ENV_PATH
removal, API_PORT cleanup, MCP port removal) into the v0.3.0 release
notes under the 'Changed - Configuration' section.
This ensures all v0.3.0 changes are documented together in a single
release entry rather than split across Unreleased and v0.3.0 sections.
MCP services are completely internal to the container and accessed
only via localhost. They should not be configurable or exposed.
Changes:
- Remove MATH_HTTP_PORT, SEARCH_HTTP_PORT, TRADE_HTTP_PORT,
GETPRICE_HTTP_PORT from docker-compose.yml environment
- Remove MCP service port mappings from docker-compose.yml
- Remove MCP port configuration from .env.example
- Update README.md to remove MCP port configuration
- Update CLAUDE.md to clarify MCP services use fixed internal ports
- Update CHANGELOG.md with these simplifications
Technical details:
- MCP services hardcode to ports 8000-8003 via os.getenv() defaults
- Services only accessed via localhost URLs within container:
- http://localhost:8000/mcp (math)
- http://localhost:8001/mcp (search)
- http://localhost:8002/mcp (trade)
- http://localhost:8003/mcp (price)
- No external access needed or desired for these services
- Only API (8080) and web dashboard (8888) should be exposed
Benefits:
- Simpler configuration (4 fewer environment variables)
- Reduced attack surface (4 fewer exposed ports)
- Clearer architecture (internal vs external services)
- Prevents accidental misconfiguration of internal services
The API_PORT variable was incorrectly included in the container's
environment section. It should only be used for host port mapping
in docker-compose.yml, not passed into the container.
Changes:
- Remove API_PORT from environment section in docker-compose.yml
- Container always uses port 8080 internally (hardcoded in entrypoint.sh)
- API_PORT in .env/.env.example only controls the host-side mapping:
ports: "${API_PORT:-8080}:8080" (host:container)
Why this matters:
- Prevents confusion about whether API_PORT changes internal port
- Clarifies that entrypoint.sh hardcodes --port 8080
- Simplifies container environment (one less unused variable)
- More explicit about the port mapping behavior
No functional change - the container was already ignoring this variable.
Simplifies deployment configuration by removing the RUNTIME_ENV_PATH
environment variable, which is no longer needed for API mode.
Changes:
- Remove RUNTIME_ENV_PATH from docker-compose.yml
- Remove RUNTIME_ENV_PATH from .env.example
- Update CLAUDE.md to reflect API-managed runtime configs
- Update README.md to remove RUNTIME_ENV_PATH from config examples
- Update CHANGELOG.md with this simplification
Technical details:
- API mode dynamically creates isolated runtime config files via
RuntimeConfigManager (data/runtime_env_{job_id}_{model}_{date}.json)
- tools/general_tools.py already handles missing RUNTIME_ENV_PATH
gracefully, returning empty dict and warning on writes
- No functional impact - all tests pass without this variable set
- Reduces configuration complexity for new deployments
Breaking change: None - variable was vestigial from batch mode era
Updates the docker-release.yml workflow to distinguish between
stable releases and pre-releases (alpha, beta, rc versions).
Changes:
- Add pre-release detection logic to extract version step
- Create new "Generate Docker tags" step to conditionally build tag list
- Only apply 'latest' tag for stable releases
- Pre-releases are tagged with version number only
- Update "Image published" message to reflect pre-release status
Example behavior:
- v0.3.0 -> tags: 0.3.0, latest
- v0.3.0-alpha -> tags: 0.3.0-alpha (NOT latest)
- v1.0.0-rc1 -> tags: 1.0.0-rc1 (NOT latest)
This prevents pre-release versions from overwriting the stable
'latest' tag, allowing users to safely pull the latest stable
version while still providing access to pre-release versions by
explicit version tag.
- Document port conflict resolution in TESTING_GUIDE.md
- Add example for custom API_PORT in .env.example
- Explain container vs host port architecture
- Provide solutions for common port conflict scenarios
- Read API_PORT from .env file if it exists
- Construct API_BASE_URL using configured port
- Display which URL is being tested
- Consistent with validate_docker_build.sh behavior
- Read API_PORT from .env file if it exists
- Use configured port instead of hardcoded 8080
- Display which port is being tested
- Fixes validation when API_PORT is customized
- Add curl for Docker health checks and diagnostics
- Add procps for process monitoring (ps command)
- Required for validation scripts to work properly
- Minimal size increase (~5MB) for critical debugging tools
- Move trap setup before uvicorn (was after, never executed)
- Use exec to replace bash with uvicorn process (better signal handling)
- Ensures uvicorn stays running as PID 1 in container
- Add retry logic (up to 15 attempts over 30 seconds)
- Add comprehensive diagnostics on failure
- Test endpoint from inside container to isolate networking issues
- Show recent logs if health check fails
- Better error messages for troubleshooting
Removes dual-mode deployment complexity, focusing on REST API service only.
Changes:
- Removed batch mode from docker-compose.yml (now single ai-trader service)
- Deleted scripts/test_batch_mode.sh validation script
- Renamed entrypoint-api.sh to entrypoint.sh (now default)
- Simplified Dockerfile (single entrypoint, removed CMD)
- Updated validation scripts to use 'ai-trader' service name
- Updated documentation (README.md, TESTING_GUIDE.md, CHANGELOG.md)
Benefits:
- Eliminates port conflicts between batch and API services
- Simpler configuration and deployment
- API-first architecture aligned with Windmill integration
- Reduced maintenance complexity
Breaking Changes:
- Batch mode no longer available
- All simulations must use REST API endpoints
API_PORT env var now only controls host port mapping in docker-compose.yml.
Container always binds uvicorn to port 8080 internally for consistency
with health checks and documentation.
Update changelog with comprehensive release notes including:
- All features added during alpha testing phase
- Configuration improvements and new documentation
- Bug fixes and stability improvements
- Corrected release date to 2025-10-31
Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Changed merge_jsonl.py to use os.getcwd() instead of os.path.dirname(__file__)
to ensure merged.jsonl is written to the working directory where data files exist,
not to the script's installation directory.
Root cause:
- Dockerfile copies scripts to /app/scripts/ for volume compatibility
- entrypoint.sh runs: cd /app/data && python /app/scripts/merge_jsonl.py
- Old logic used script directory (/app/scripts/), ignoring working directory
- This caused merged.jsonl to be created in /app/scripts/ instead of /app/data/
- Since /app/data/ is volume-mounted, merged file was not visible to host
Solution:
- Scripts now respect current working directory (Unix philosophy)
- Works correctly with volume mounts and script relocation
- Tested in both local and Docker directory structure scenarios
Fixes the issue where merged.jsonl was missing from mounted data volume.
Streamline the data preparation workflow by having get_daily_price.py
automatically invoke merge_jsonl.py after fetching all stock prices.
Changes:
- Modified get_daily_price.py to call merge_jsonl.py automatically
- Updated entrypoint.sh to remove redundant merge_jsonl.py call
- Updated main.sh to remove redundant merge_jsonl.py call
- Fixed import order for linting compliance
Benefits:
- Single command now handles both fetching and merging
- Ensures merged.jsonl is always created after price updates
- Simplifies Docker container startup process
- Prevents missing merged.jsonl errors in production
Add VOLUME_PATH environment variable to customize storage location for
all persistent data (data/, logs/, configs/ subdirectories).
Changes:
- .env.example: Add VOLUME_PATH variable with documentation
- docker-compose.yml: Use ${VOLUME_PATH:-.} for all volume mounts
- docs/DOCKER.md: Document custom volume location feature
Benefits:
- Store data outside project directory (e.g., on separate disk)
- Easier backup/restore (single directory)
- Clean separation of code and data
- Support for absolute or relative paths
Usage:
# In .env file
VOLUME_PATH=/home/user/trading-data
# Results in mounts:
/home/user/trading-data/data:/app/data
/home/user/trading-data/logs:/app/logs
/home/user/trading-data/configs:/app/configs
Defaults to current directory (.) if not set for backward compatibility.
Preserve existing merged.jsonl to avoid data loss and API rate limits.
Only fetch new data if merged.jsonl is missing or empty.
Problem:
- Entrypoint always fetched fresh data from Alpha Vantage on every start
- Overwrote existing mounted data directory
- Caused API rate limit issues and data inconsistencies
- Lost historical data needed for backtesting specific date ranges
Solution:
- Check if merged.jsonl exists and has content before fetching
- Display stock count when using existing data
- Provide manual refresh instructions for when updates are needed
Benefits:
- Faster container startup (no API calls if data exists)
- Avoids Alpha Vantage rate limits (5 calls/min, 500/day)
- Preserves user's existing historical datasets
- Enables reliable backtesting with consistent data
To refresh data: rm data/merged.jsonl && docker-compose restart
Implement automatic detection of custom_config.json for simpler Docker usage.
No environment variables or command-line arguments needed for most users.
Changes:
- entrypoint.sh: Add smart config detection (custom_config.json > CLI arg > default_config.json)
- docker-compose.yml: Add configs volume mount for editing without rebuilds
- docs/DOCKER.md: Update documentation with simplified workflow
- .gitignore: Add custom_config.json to prevent accidental commits
Usage:
cp configs/default_config.json configs/custom_config.json
nano configs/custom_config.json
docker-compose up # Automatically uses custom_config.json
Simplifies user experience by following convention over configuration principle.
Root cause: Python adds script directory (not working directory)
to sys.path. Services in /app/agent_tools/ couldn't import from
/app/tools/ because sys.path[0] was /app/agent_tools.
Solution: Set PYTHONPATH=/app when starting services so /app is
always in the Python import path.
Fixes: ModuleNotFoundError: No module named 'tools' in price.log
Add comment explaining OPENAI_API_BASE can be left empty
to use the default OpenAI endpoint, or set to a custom
proxy URL if needed.
Sets default value to empty instead of placeholder URL.
- Clarify OPENAI_API_BASE can be left empty for default endpoint
- Increase initial wait time from 3s to 5s
- Add health checking for all MCP services (ports 8000-8003)
- Retry up to 10 times with 1s intervals
- Install netcat for port checking
- Display warnings if services aren't ready
Helps diagnose and prevent MCP client initialization errors.
Run MCP service manager from /app instead of /app/agent_tools
to ensure services can import from tools module.
Changes:
- entrypoint.sh: Run start_mcp_services.py from /app directory
- start_mcp_services.py: Use absolute paths for service scripts
- start_mcp_services.py: Set working directory to /app for services
Fixes ModuleNotFoundError: No module named 'tools' in price.log
Replace verbose JSON logging with concise status messages:
- Success: '✓ Fetched SYMBOL'
- Error: '⚠️ SYMBOL: API rate limit or error - [message]'
Prevents logs from being flooded with full JSON responses
for 100+ stock symbols during startup.
Update all GitHub URLs in README from HKUDS/AI-Trader to
Xe138/AI-Trader including:
- Clone URLs (2 locations)
- Support & Community links (Discussions, Issues)
- Badge URLs (stars, forks)
Add validation at startup to check required environment variables:
- OPENAI_API_KEY
- ALPHAADVANTAGE_API_KEY
- JINA_API_KEY
If any are missing, display clear error message with setup
instructions and exit immediately (no restart loop).
Change restart policy from 'unless-stopped' to 'on-failure:3'
to limit restart attempts and prevent endless loops on
configuration errors.
Move get_daily_price.py and merge_jsonl.py to /app/scripts
to prevent volume mount from overlaying the scripts.
Scripts run from /app/data context to output correctly.
Fixes container startup error where scripts were not found
after ./data volume mount replaced container's /app/data.
Add WEB_HTTP_PORT environment variable to configure
the host port for the web interface (port 8888).
Container still uses 8888 internally, but host port
can be customized via .env file.
