Commit Graph

16 Commits

Author SHA1 Message Date
1bfcdd78b8 feat: complete v0.3.0 database migration and configuration
Final phase of v0.3.0 implementation - all core features complete.

Price Tools Migration:
- Update get_open_prices() to query price_data table
- Update get_yesterday_open_and_close_price() to query database
- Remove merged.jsonl file I/O (replaced with SQLite queries)
- Maintain backward-compatible function signatures
- Add db_path parameter (default: data/jobs.db)

Configuration:
- Add AUTO_DOWNLOAD_PRICE_DATA to .env.example (default: true)
- Add MAX_SIMULATION_DAYS to .env.example (default: 30)
- Document new configuration options

Documentation:
- Comprehensive CHANGELOG updates for v0.3.0
- Document all breaking changes (API format, data storage, config)
- Document new features (on-demand downloads, date ranges, database)
- Document migration path (scripts/migrate_price_data.py)
- Clear upgrade instructions

Breaking Changes (v0.3.0):
1. API request format: date_range -> start_date/end_date
2. Data storage: merged.jsonl -> price_data table
3. Config variables: removed RUNTIME_ENV_PATH, MCP ports, WEB_HTTP_PORT
4. Added AUTO_DOWNLOAD_PRICE_DATA, MAX_SIMULATION_DAYS

Migration Steps:
1. Run: python scripts/migrate_price_data.py
2. Update API clients to use new date format
3. Update .env with new variables
4. Remove old config variables

Status: v0.3.0 implementation complete
Ready for: Testing, deployment, and release
2025-10-31 16:44:46 -04:00
246dbd1b34 refactor: remove unused web UI port configuration
The web UI (docs/index.html, portfolio.html) exists but is not served
in API mode. Removing the port configuration to eliminate confusion.

Changes:
- Remove port 8888 mapping from docker-compose.yml
- Remove WEB_HTTP_PORT from .env.example
- Update Dockerfile EXPOSE to only port 8080
- Update CHANGELOG.md to document removal

Technical details:
- Web UI static files remain in docs/ folder (legacy from batch mode)
- These were designed for JSONL file format, not the new SQLite database
- No web server was ever started in entrypoint.sh for API mode
- Port 8888 was exposed but nothing listened on it

Result:
- Cleaner configuration (1 fewer port mapping)
- Only REST API (8080) is exposed
- Eliminates user confusion about non-functional web UI
2025-10-31 14:54:10 -04:00
d587a5f213 refactor: remove unnecessary MCP service port configuration
MCP services are completely internal to the container and accessed
only via localhost. They should not be configurable or exposed.

Changes:
- Remove MATH_HTTP_PORT, SEARCH_HTTP_PORT, TRADE_HTTP_PORT,
  GETPRICE_HTTP_PORT from docker-compose.yml environment
- Remove MCP service port mappings from docker-compose.yml
- Remove MCP port configuration from .env.example
- Update README.md to remove MCP port configuration
- Update CLAUDE.md to clarify MCP services use fixed internal ports
- Update CHANGELOG.md with these simplifications

Technical details:
- MCP services hardcode to ports 8000-8003 via os.getenv() defaults
- Services only accessed via localhost URLs within container:
  - http://localhost:8000/mcp (math)
  - http://localhost:8001/mcp (search)
  - http://localhost:8002/mcp (trade)
  - http://localhost:8003/mcp (price)
- No external access needed or desired for these services
- Only API (8080) and web dashboard (8888) should be exposed

Benefits:
- Simpler configuration (4 fewer environment variables)
- Reduced attack surface (4 fewer exposed ports)
- Clearer architecture (internal vs external services)
- Prevents accidental misconfiguration of internal services
2025-10-31 14:41:07 -04:00
849e7bffa2 refactor: remove unnecessary RUNTIME_ENV_PATH environment variable
Simplifies deployment configuration by removing the RUNTIME_ENV_PATH
environment variable, which is no longer needed for API mode.

Changes:
- Remove RUNTIME_ENV_PATH from docker-compose.yml
- Remove RUNTIME_ENV_PATH from .env.example
- Update CLAUDE.md to reflect API-managed runtime configs
- Update README.md to remove RUNTIME_ENV_PATH from config examples
- Update CHANGELOG.md with this simplification

Technical details:
- API mode dynamically creates isolated runtime config files via
  RuntimeConfigManager (data/runtime_env_{job_id}_{model}_{date}.json)
- tools/general_tools.py already handles missing RUNTIME_ENV_PATH
  gracefully, returning empty dict and warning on writes
- No functional impact - all tests pass without this variable set
- Reduces configuration complexity for new deployments

Breaking change: None - variable was vestigial from batch mode era
2025-10-31 14:37:00 -04:00
483eca7c77 docs: add port configuration troubleshooting
- Document port conflict resolution in TESTING_GUIDE.md
- Add example for custom API_PORT in .env.example
- Explain container vs host port architecture
- Provide solutions for common port conflict scenarios
2025-10-31 14:18:48 -04:00
eac2e781f7 docs: clarify API_PORT usage in .env.example
Added detailed comments explaining that container always uses port 8080
internally and API_PORT only controls host port mapping.
2025-10-31 12:42:41 -04:00
fb9583b374 feat: transform to REST API service with SQLite persistence (v0.3.0)
Major architecture transformation from batch-only to API service with
database persistence for Windmill integration.

## REST API Implementation
- POST /simulate/trigger - Start simulation jobs
- GET /simulate/status/{job_id} - Monitor job progress
- GET /results - Query results with filters (job_id, date, model)
- GET /health - Service health checks

## Database Layer
- SQLite persistence with 6 tables (jobs, job_details, positions,
  holdings, reasoning_logs, tool_usage)
- Foreign key constraints with cascade deletes
- Replaces JSONL file storage

## Backend Components
- JobManager: Job lifecycle management with concurrency control
- RuntimeConfigManager: Thread-safe isolated runtime configs
- ModelDayExecutor: Single model-day execution engine
- SimulationWorker: Date-sequential, model-parallel orchestration

## Testing
- 102 unit and integration tests (85% coverage)
- Database: 98% coverage
- Job manager: 98% coverage
- API endpoints: 81% coverage
- Pydantic models: 100% coverage
- TDD approach throughout

## Docker Deployment
- Dual-mode: API server (persistent) + batch (one-time)
- Health checks with 30s interval
- Volume persistence for database and logs
- Separate entrypoints for each mode

## Validation Tools
- scripts/validate_docker_build.sh - Build validation
- scripts/test_api_endpoints.sh - Complete API testing
- scripts/test_batch_mode.sh - Batch mode validation
- DOCKER_API.md - Deployment guide
- TESTING_GUIDE.md - Testing procedures

## Configuration
- API_PORT environment variable (default: 8080)
- Backwards compatible with existing configs
- FastAPI, uvicorn, pydantic>=2.0 dependencies

Co-Authored-By: AI Assistant <noreply@example.com>
2025-10-31 11:47:10 -04:00
55206549c7 feat: add configurable volume path for persistent data
Add VOLUME_PATH environment variable to customize storage location for
all persistent data (data/, logs/, configs/ subdirectories).

Changes:
- .env.example: Add VOLUME_PATH variable with documentation
- docker-compose.yml: Use ${VOLUME_PATH:-.} for all volume mounts
- docs/DOCKER.md: Document custom volume location feature

Benefits:
- Store data outside project directory (e.g., on separate disk)
- Easier backup/restore (single directory)
- Clean separation of code and data
- Support for absolute or relative paths

Usage:
  # In .env file
  VOLUME_PATH=/home/user/trading-data

  # Results in mounts:
  /home/user/trading-data/data:/app/data
  /home/user/trading-data/logs:/app/logs
  /home/user/trading-data/configs:/app/configs

Defaults to current directory (.) if not set for backward compatibility.
2025-10-30 23:40:21 -04:00
12ecb1e6b6 docs: clarify OPENAI_API_BASE can be left empty
Add comment explaining OPENAI_API_BASE can be left empty
to use the default OpenAI endpoint, or set to a custom
proxy URL if needed.

Sets default value to empty instead of placeholder URL.
2025-10-30 21:34:16 -04:00
203b60b252 Revert "fix: improve MCP service startup reliability"
This reverts commit d70362b9d4.
2025-10-30 21:33:12 -04:00
d70362b9d4 fix: improve MCP service startup reliability
- Clarify OPENAI_API_BASE can be left empty for default endpoint
- Increase initial wait time from 3s to 5s
- Add health checking for all MCP services (ports 8000-8003)
- Retry up to 10 times with 1s intervals
- Install netcat for port checking
- Display warnings if services aren't ready

Helps diagnose and prevent MCP client initialization errors.
2025-10-30 21:32:42 -04:00
8142f38ab9 docs: add API key registration URLs to .env.example
Add inline comments with URLs for API key registration:
- OPENAI_API_KEY: https://platform.openai.com/api-keys
- ALPHAADVANTAGE_API_KEY: https://www.alphavantage.co/support/#api-key
- JINA_API_KEY: https://jina.ai/

Makes it easier for users to find where to get their API keys.
2025-10-30 21:17:21 -04:00
2d1c356199 feat: add configurable web interface host port
Add WEB_HTTP_PORT environment variable to configure
the host port for the web interface (port 8888).
Container still uses 8888 internally, but host port
can be customized via .env file.
2025-10-30 20:51:21 -04:00
ffa158224d fix: use fixed internal ports with configurable host ports
Internal container ports are now fixed at 8000-8003.
Host ports are configurable via .env variables.
This prevents port conflicts inside the container while
allowing users to map to different host ports if needed.
2025-10-30 20:50:37 -04:00
592b105b19 Update .env.example with Docker configuration
Add Docker-specific paths and documentation
Include all required API keys and MCP ports
Show default values for optional settings
2025-10-30 18:39:01 -04:00
tianyufan
df5c25c98d init update 2025-10-24 00:35:21 +08:00