Commit Graph

117 Commits

Author SHA1 Message Date
eac2e781f7 docs: clarify API_PORT usage in .env.example
Added detailed comments explaining that container always uses port 8080
internally and API_PORT only controls host port mapping.
2025-10-31 12:42:41 -04:00
77da47a40d fix: use hardcoded port 8080 internally in container
API_PORT env var now only controls host port mapping in docker-compose.yml.
Container always binds uvicorn to port 8080 internally for consistency
with health checks and documentation.
2025-10-31 12:42:28 -04:00
c63cdffd0e fix: enable local Docker builds for development and testing 2025-10-31 12:37:41 -04:00
1c8e59340e fix: make entrypoint-api.sh executable for Docker 2025-10-31 12:34:51 -04:00
fb9583b374 feat: transform to REST API service with SQLite persistence (v0.3.0)
Major architecture transformation from batch-only to API service with
database persistence for Windmill integration.

## REST API Implementation
- POST /simulate/trigger - Start simulation jobs
- GET /simulate/status/{job_id} - Monitor job progress
- GET /results - Query results with filters (job_id, date, model)
- GET /health - Service health checks

## Database Layer
- SQLite persistence with 6 tables (jobs, job_details, positions,
  holdings, reasoning_logs, tool_usage)
- Foreign key constraints with cascade deletes
- Replaces JSONL file storage

## Backend Components
- JobManager: Job lifecycle management with concurrency control
- RuntimeConfigManager: Thread-safe isolated runtime configs
- ModelDayExecutor: Single model-day execution engine
- SimulationWorker: Date-sequential, model-parallel orchestration

## Testing
- 102 unit and integration tests (85% coverage)
- Database: 98% coverage
- Job manager: 98% coverage
- API endpoints: 81% coverage
- Pydantic models: 100% coverage
- TDD approach throughout

## Docker Deployment
- Dual-mode: API server (persistent) + batch (one-time)
- Health checks with 30s interval
- Volume persistence for database and logs
- Separate entrypoints for each mode

## Validation Tools
- scripts/validate_docker_build.sh - Build validation
- scripts/test_api_endpoints.sh - Complete API testing
- scripts/test_batch_mode.sh - Batch mode validation
- DOCKER_API.md - Deployment guide
- TESTING_GUIDE.md - Testing procedures

## Configuration
- API_PORT environment variable (default: 8080)
- Backwards compatible with existing configs
- FastAPI, uvicorn, pydantic>=2.0 dependencies

Co-Authored-By: AI Assistant <noreply@example.com>
2025-10-31 11:47:10 -04:00
5da02b4ba0 docs: update CHANGELOG.md for v0.2.0 release
Update changelog with comprehensive release notes including:
- All features added during alpha testing phase
- Configuration improvements and new documentation
- Bug fixes and stability improvements
- Corrected release date to 2025-10-31

Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
v0.2.0
2025-10-31 00:28:13 -04:00
11509ba8c7 fix: merge script now writes to current directory for volume compatibility
Changed merge_jsonl.py to use os.getcwd() instead of os.path.dirname(__file__)
to ensure merged.jsonl is written to the working directory where data files exist,
not to the script's installation directory.

Root cause:
- Dockerfile copies scripts to /app/scripts/ for volume compatibility
- entrypoint.sh runs: cd /app/data && python /app/scripts/merge_jsonl.py
- Old logic used script directory (/app/scripts/), ignoring working directory
- This caused merged.jsonl to be created in /app/scripts/ instead of /app/data/
- Since /app/data/ is volume-mounted, merged file was not visible to host

Solution:
- Scripts now respect current working directory (Unix philosophy)
- Works correctly with volume mounts and script relocation
- Tested in both local and Docker directory structure scenarios

Fixes the issue where merged.jsonl was missing from mounted data volume.
v0.2.0-alpha.11
2025-10-31 00:03:39 -04:00
1785f9b06f feat: automate merged.jsonl creation during price fetching
Streamline the data preparation workflow by having get_daily_price.py
automatically invoke merge_jsonl.py after fetching all stock prices.

Changes:
- Modified get_daily_price.py to call merge_jsonl.py automatically
- Updated entrypoint.sh to remove redundant merge_jsonl.py call
- Updated main.sh to remove redundant merge_jsonl.py call
- Fixed import order for linting compliance

Benefits:
- Single command now handles both fetching and merging
- Ensures merged.jsonl is always created after price updates
- Simplifies Docker container startup process
- Prevents missing merged.jsonl errors in production
v0.2.0-alpha.10
2025-10-30 23:48:12 -04:00
55206549c7 feat: add configurable volume path for persistent data
Add VOLUME_PATH environment variable to customize storage location for
all persistent data (data/, logs/, configs/ subdirectories).

Changes:
- .env.example: Add VOLUME_PATH variable with documentation
- docker-compose.yml: Use ${VOLUME_PATH:-.} for all volume mounts
- docs/DOCKER.md: Document custom volume location feature

Benefits:
- Store data outside project directory (e.g., on separate disk)
- Easier backup/restore (single directory)
- Clean separation of code and data
- Support for absolute or relative paths

Usage:
  # In .env file
  VOLUME_PATH=/home/user/trading-data

  # Results in mounts:
  /home/user/trading-data/data:/app/data
  /home/user/trading-data/logs:/app/logs
  /home/user/trading-data/configs:/app/configs

Defaults to current directory (.) if not set for backward compatibility.
v0.2.0-alpha.9
2025-10-30 23:40:21 -04:00
9e05ce0891 fix: prevent price data overwrite on container restart
Preserve existing merged.jsonl to avoid data loss and API rate limits.
Only fetch new data if merged.jsonl is missing or empty.

Problem:
- Entrypoint always fetched fresh data from Alpha Vantage on every start
- Overwrote existing mounted data directory
- Caused API rate limit issues and data inconsistencies
- Lost historical data needed for backtesting specific date ranges

Solution:
- Check if merged.jsonl exists and has content before fetching
- Display stock count when using existing data
- Provide manual refresh instructions for when updates are needed

Benefits:
- Faster container startup (no API calls if data exists)
- Avoids Alpha Vantage rate limits (5 calls/min, 500/day)
- Preserves user's existing historical datasets
- Enables reliable backtesting with consistent data

To refresh data: rm data/merged.jsonl && docker-compose restart
v0.2.0-alpha.8
2025-10-30 23:27:58 -04:00
2f2c1d6ea2 feat: simplify Docker config file selection with convention over configuration
Implement automatic detection of custom_config.json for simpler Docker usage.
No environment variables or command-line arguments needed for most users.

Changes:
- entrypoint.sh: Add smart config detection (custom_config.json > CLI arg > default_config.json)
- docker-compose.yml: Add configs volume mount for editing without rebuilds
- docs/DOCKER.md: Update documentation with simplified workflow
- .gitignore: Add custom_config.json to prevent accidental commits

Usage:
  cp configs/default_config.json configs/custom_config.json
  nano configs/custom_config.json
  docker-compose up  # Automatically uses custom_config.json

Simplifies user experience by following convention over configuration principle.
v0.2.0-alpha.7
2025-10-30 22:34:22 -04:00
e9c571402a fix: set PYTHONPATH for MCP services to resolve import errors
Root cause: Python adds script directory (not working directory)
to sys.path. Services in /app/agent_tools/ couldn't import from
/app/tools/ because sys.path[0] was /app/agent_tools.

Solution: Set PYTHONPATH=/app when starting services so /app is
always in the Python import path.

Fixes: ModuleNotFoundError: No module named 'tools' in price.log
v0.2.0-alpha.6
2025-10-30 21:47:58 -04:00
79d14444ed docs: add data cache reuse design document
Captures design for staleness-based data refresh to avoid
re-fetching all 103 NASDAQ tickers on every container restart.

Key features:
- Check all daily_prices_*.json files for staleness
- Configurable MAX_DATA_AGE_DAYS threshold (default: 7)
- Bash wrapper logic in entrypoint.sh
- Handles edge cases (partial data, missing files, rate limits)
2025-10-30 21:46:05 -04:00
12ecb1e6b6 docs: clarify OPENAI_API_BASE can be left empty
Add comment explaining OPENAI_API_BASE can be left empty
to use the default OpenAI endpoint, or set to a custom
proxy URL if needed.

Sets default value to empty instead of placeholder URL.
v0.2.0-alpha.5
2025-10-30 21:34:16 -04:00
203b60b252 Revert "fix: improve MCP service startup reliability"
This reverts commit d70362b9d4.
2025-10-30 21:33:12 -04:00
d70362b9d4 fix: improve MCP service startup reliability
- Clarify OPENAI_API_BASE can be left empty for default endpoint
- Increase initial wait time from 3s to 5s
- Add health checking for all MCP services (ports 8000-8003)
- Retry up to 10 times with 1s intervals
- Install netcat for port checking
- Display warnings if services aren't ready

Helps diagnose and prevent MCP client initialization errors.
2025-10-30 21:32:42 -04:00
5ec7977b47 fix: resolve module import error for MCP services
Run MCP service manager from /app instead of /app/agent_tools
to ensure services can import from tools module.

Changes:
- entrypoint.sh: Run start_mcp_services.py from /app directory
- start_mcp_services.py: Use absolute paths for service scripts
- start_mcp_services.py: Set working directory to /app for services

Fixes ModuleNotFoundError: No module named 'tools' in price.log
2025-10-30 21:22:42 -04:00
8142f38ab9 docs: add API key registration URLs to .env.example
Add inline comments with URLs for API key registration:
- OPENAI_API_KEY: https://platform.openai.com/api-keys
- ALPHAADVANTAGE_API_KEY: https://www.alphavantage.co/support/#api-key
- JINA_API_KEY: https://jina.ai/

Makes it easier for users to find where to get their API keys.
v0.2.0-alpha.4
2025-10-30 21:17:21 -04:00
595a659fe7 fix: reduce log flooding during data fetch
Replace verbose JSON logging with concise status messages:
- Success: '✓ Fetched SYMBOL'
- Error: '⚠️  SYMBOL: API rate limit or error - [message]'

Prevents logs from being flooded with full JSON responses
for 100+ stock symbols during startup.
2025-10-30 21:15:59 -04:00
a4bc4fd0de fix: update repository URLs to Xe138/AI-Trader fork
Update all GitHub URLs in README from HKUDS/AI-Trader to
Xe138/AI-Trader including:
- Clone URLs (2 locations)
- Support & Community links (Discussions, Issues)
- Badge URLs (stars, forks)
2025-10-30 21:06:28 -04:00
4666e09385 fix: prevent restart loop on missing API keys
Add validation at startup to check required environment variables:
- OPENAI_API_KEY
- ALPHAADVANTAGE_API_KEY
- JINA_API_KEY

If any are missing, display clear error message with setup
instructions and exit immediately (no restart loop).

Change restart policy from 'unless-stopped' to 'on-failure:3'
to limit restart attempts and prevent endless loops on
configuration errors.
v0.2.0-alpha.3
2025-10-30 21:00:12 -04:00
0c58baed42 fix: separate data scripts from volume mount directory
Move get_daily_price.py and merge_jsonl.py to /app/scripts
to prevent volume mount from overlaying the scripts.
Scripts run from /app/data context to output correctly.

Fixes container startup error where scripts were not found
after ./data volume mount replaced container's /app/data.
v0.2.0-alpha.2
2025-10-30 20:53:44 -04:00
2d1c356199 feat: add configurable web interface host port
Add WEB_HTTP_PORT environment variable to configure
the host port for the web interface (port 8888).
Container still uses 8888 internally, but host port
can be customized via .env file.
2025-10-30 20:51:21 -04:00
ffa158224d fix: use fixed internal ports with configurable host ports
Internal container ports are now fixed at 8000-8003.
Host ports are configurable via .env variables.
This prevents port conflicts inside the container while
allowing users to map to different host ports if needed.
2025-10-30 20:50:37 -04:00
6825a60b20 feat: use pre-built image by default in docker-compose
Pull ghcr.io/xe138/ai-trader:latest by default instead of building.
Add commented build option for local development.
Makes deployment faster for end users.
2025-10-30 20:47:11 -04:00
9e308ee243 fix: use generic template for draft releases
Replace hardcoded release notes with generic template.
User can complete the release notes before publishing.
Includes TODO markers for summary and changelog content.
v0.2.0-alpha
2025-10-30 20:40:30 -04:00
90ea10c7ef feat: add automatic draft release creation to workflow
Automatically create GitHub draft release with comprehensive
release notes when Docker build succeeds. Includes:
- Quick start instructions for Docker Compose and pre-built images
- Full changelog for v0.2.0
- Documentation links
- Automatic pre-release detection for alpha/beta/rc tags
2025-10-30 20:38:59 -04:00
77ce1b2b11 fix: correct Dockerfile FROM AS casing
Use uppercase AS keyword to match FROM keyword casing,
following Docker best practices and eliminating build warning.
2025-10-30 20:36:40 -04:00
2585045be1 fix: convert repository owner to lowercase for Docker tags
Docker tags require lowercase repository names. Convert
github.repository_owner to lowercase before using in tags.
2025-10-30 20:34:43 -04:00
9d5f449b1c fix: validate GITHUB_REF is a tag in docker-release workflow
Add validation to ensure workflow only processes tag pushes.
Prevents invalid Docker tags when workflow runs on non-tag refs.
2025-10-30 20:31:55 -04:00
e4b7e197d3 Update repository URLs for fork
Change HKUDS/AI-Trader to Xe138/AI-Trader
Update all documentation links and references
2025-10-30 20:26:18 -04:00
928f5fb53f Update CHANGELOG for v0.2.0 release
Set version to 0.2.0 for Docker deployment feature
Release date: 2025-10-30
Update comparison links for proper version tracking
2025-10-30 20:24:46 -04:00
46582d38bb Add CHANGELOG.md
Document all changes including Docker deployment feature
Follow Keep a Changelog format with semantic versioning
Include template for future releases
2025-10-30 20:22:10 -04:00
147fab11d5 Remove deprecated version tag from docker-compose.yml
Docker Compose no longer requires version specification
Maintains compatibility with modern docker-compose versions
2025-10-30 20:20:49 -04:00
85fc925b4e Add release process documentation
Complete guide for creating releases:
- Tag creation and push process
- GitHub Actions automation workflow
- Verification and testing steps
- Troubleshooting common issues
2025-10-30 19:26:56 -04:00
156d0cab21 Add Docker build test results
Local build verification completed successfully
Image builds without errors
Configuration parses correctly
2025-10-30 19:19:00 -04:00
4e7c4ccf99 Update CLAUDE.md with Docker commands
Add Docker build and run commands
Include release process for Docker images
Document GitHub Actions automation
2025-10-30 19:05:23 -04:00
9dfc8f1ea6 Add GitHub Actions workflow for Docker builds
Triggers on release tags (v*) and manual dispatch
Builds and pushes to GitHub Container Registry
Tags with both version and latest
Uses build caching for faster builds
2025-10-30 19:01:09 -04:00
6af1808c2f Add Docker deployment section to README
Include quick start with Docker Compose
Add pre-built image usage instructions
Link to detailed Docker documentation
2025-10-30 18:44:04 -04:00
733b6a4b1c Add Docker deployment documentation
Comprehensive guide including:
- Quick start instructions
- Configuration options
- Usage examples and volume persistence
- Troubleshooting common issues
- Pre-built image usage
2025-10-30 18:41:27 -04:00
592b105b19 Update .env.example with Docker configuration
Add Docker-specific paths and documentation
Include all required API keys and MCP ports
Show default values for optional settings
2025-10-30 18:39:01 -04:00
4af00d3709 Add docker-compose configuration
Mounts data and logs volumes for persistence
Injects environment variables from .env file
Exposes all MCP service ports and web dashboard
Auto-restart on failure
2025-10-30 18:36:25 -04:00
e9baa818a4 Add entrypoint script for container startup
Sequential execution: data fetch → MCP services → trading agent
Handles graceful shutdown of background services
Supports custom config file as argument
2025-10-30 18:33:40 -04:00
f25bd87c26 Add Dockerfile for containerization
Multi-stage build with Python 3.10-slim base
Exposes MCP service ports and web dashboard
Uses entrypoint.sh for sequential startup
2025-10-30 18:30:38 -04:00
794028e910 Add .dockerignore for Docker builds
Excludes git history, Python cache, secrets, and runtime data
2025-10-30 18:28:15 -04:00
efc2d754d6 Add Docker deployment implementation plan
Comprehensive 12-task plan including:
- Dockerfile and docker-compose.yml creation
- entrypoint.sh sequential startup script
- GitHub Actions CI/CD workflow
- Complete documentation (DOCKER.md, RELEASING.md)
- Testing and validation steps
2025-10-30 18:26:47 -04:00
5225950703 Add .worktrees/ to .gitignore
Prevents worktree contents from being tracked in repository
2025-10-30 18:23:14 -04:00
6794a65ef6 Add Docker deployment design document
Comprehensive design for Docker containerization including:
- Single monolithic container approach for dev/testing
- docker-compose.yml with environment variable injection
- Sequential startup via entrypoint.sh script
- GitHub Actions workflow for automated image builds on release tags
- Documentation strategy and implementation checklist
2025-10-30 18:22:03 -04:00
c15e25f826 Add CLAUDE.md for repository guidance
Provides comprehensive guidance for working with the AI-Trader codebase including:
- Development commands for setup, data preparation, and running simulations
- Architecture overview of agent system, MCP toolchain, and data flow
- Configuration system with multi-layered priority
- Data formats for positions and price data
- Implementation details and common troubleshooting steps
2025-10-30 18:03:24 -04:00
tianyufan
1c7a9621d0 update License 2025-10-30 19:38:54 +08:00