mirror of
https://github.com/Xe138/AI-Trader.git
synced 2026-04-07 03:07:24 -04:00
Compare commits
6 Commits
v0.3.0-alp
...
v0.3.0
| Author | SHA1 | Date | |
|---|---|---|---|
| 1095798320 | |||
| e590cdc13b | |||
| c74747d1d4 | |||
| 96f6b78a93 | |||
| 6c395f740d | |||
| 618943b278 |
84
CHANGELOG.md
84
CHANGELOG.md
@@ -7,13 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
### Fixed
|
## [0.3.0] - 2025-11-03
|
||||||
- **Dev Mode Warning in Docker** - DEV mode startup warning now displays correctly in Docker logs
|
|
||||||
- Added FastAPI `@app.on_event("startup")` handler to trigger warning on API server startup
|
|
||||||
- Previously only appeared when running `python api/main.py` directly (not via uvicorn)
|
|
||||||
- Docker compose now includes `DEPLOYMENT_MODE` and `PRESERVE_DEV_DATA` environment variables
|
|
||||||
|
|
||||||
## [0.3.0] - 2025-10-31
|
### Added - Development & Testing Features
|
||||||
|
- **Development Mode** - Mock AI provider for cost-free testing
|
||||||
|
- `DEPLOYMENT_MODE=DEV` enables mock AI responses with deterministic stock rotation
|
||||||
|
- Isolated dev database (`trading_dev.db`) separate from production data
|
||||||
|
- `PRESERVE_DEV_DATA=true` option to prevent dev database reset on startup
|
||||||
|
- No AI API costs during development and testing
|
||||||
|
- All API responses include `deployment_mode` field
|
||||||
|
- Startup warning displayed when running in DEV mode
|
||||||
|
- **Config Override System** - Docker configuration merging
|
||||||
|
- Place custom configs in `user-configs/` directory
|
||||||
|
- Startup merges user config with default config
|
||||||
|
- Comprehensive validation with clear error messages
|
||||||
|
- Volume mount: `./user-configs:/app/user-configs`
|
||||||
|
|
||||||
|
### Added - Enhanced API Features
|
||||||
|
- **Async Price Download** - Non-blocking data preparation
|
||||||
|
- `POST /simulate/trigger` no longer blocks on price downloads
|
||||||
|
- New job status: `downloading_data` during data preparation
|
||||||
|
- Warnings field in status response for download issues
|
||||||
|
- Better user experience for large date ranges
|
||||||
|
- **Resume Mode** - Idempotent simulation execution
|
||||||
|
- Jobs automatically skip already-completed model-days
|
||||||
|
- Safe to re-run jobs without duplicating work
|
||||||
|
- `status="skipped"` for already-completed executions
|
||||||
|
- Error-free job completion when partial results exist
|
||||||
|
- **Reasoning Logs API** - Access AI decision-making history
|
||||||
|
- `GET /reasoning` endpoint for querying reasoning logs
|
||||||
|
- Filter by job_id, model_name, date, include_full_conversation
|
||||||
|
- Includes conversation history and tool usage
|
||||||
|
- Database-only storage (no JSONL files)
|
||||||
|
- AI-powered summary generation for reasoning sessions
|
||||||
|
- **Job Skip Status** - Enhanced job status tracking
|
||||||
|
- New status: `skipped` for already-completed model-days
|
||||||
|
- Better differentiation between pending, running, and skipped
|
||||||
|
- Accurate job completion detection
|
||||||
|
|
||||||
### Added - Price Data Management & On-Demand Downloads
|
### Added - Price Data Management & On-Demand Downloads
|
||||||
- **SQLite Price Data Storage** - Replaced JSONL files with relational database
|
- **SQLite Price Data Storage** - Replaced JSONL files with relational database
|
||||||
@@ -83,13 +113,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
- Windmill integration patterns and examples
|
- Windmill integration patterns and examples
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
- **Project Rebrand** - AI-Trader renamed to AI-Trader-Server
|
||||||
|
- Updated all documentation for new project name
|
||||||
|
- Updated Docker images to ghcr.io/xe138/ai-trader-server
|
||||||
|
- Updated GitHub Actions workflows
|
||||||
|
- Updated README, CHANGELOG, and all user guides
|
||||||
- **Architecture** - Transformed from batch-only to API-first service with database persistence
|
- **Architecture** - Transformed from batch-only to API-first service with database persistence
|
||||||
- **Data Storage** - Migrated from JSONL files to SQLite relational database
|
- **Data Storage** - Migrated from JSONL files to SQLite relational database
|
||||||
- Price data now stored in `price_data` table instead of `merged.jsonl`
|
- Price data now stored in `price_data` table instead of `merged.jsonl`
|
||||||
- Tools/price_tools.py updated to query database
|
- Tools/price_tools.py updated to query database
|
||||||
- Position data remains in database (already migrated in earlier versions)
|
- Position data fully migrated to database-only storage (removed JSONL dependencies)
|
||||||
|
- Trade tools now read/write from database tables with lazy context injection
|
||||||
- **Deployment** - Simplified to single API-only Docker service (REST API is new in v0.3.0)
|
- **Deployment** - Simplified to single API-only Docker service (REST API is new in v0.3.0)
|
||||||
|
- **Logging** - Removed duplicate MCP service log files for cleaner output
|
||||||
- **Configuration** - Simplified environment variable configuration
|
- **Configuration** - Simplified environment variable configuration
|
||||||
|
- **Added:** `DEPLOYMENT_MODE` (PROD/DEV) for environment control
|
||||||
|
- **Added:** `PRESERVE_DEV_DATA` (default: false) to keep dev data between runs
|
||||||
- **Added:** `AUTO_DOWNLOAD_PRICE_DATA` (default: true) - Enable on-demand downloads
|
- **Added:** `AUTO_DOWNLOAD_PRICE_DATA` (default: true) - Enable on-demand downloads
|
||||||
- **Added:** `MAX_SIMULATION_DAYS` (default: 30) - Maximum date range size
|
- **Added:** `MAX_SIMULATION_DAYS` (default: 30) - Maximum date range size
|
||||||
- **Added:** `API_PORT` for host port mapping (default: 8080, customizable for port conflicts)
|
- **Added:** `API_PORT` for host port mapping (default: 8080, customizable for port conflicts)
|
||||||
@@ -137,6 +176,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
- **Monitoring** - Health checks and status tracking
|
- **Monitoring** - Health checks and status tracking
|
||||||
- **Persistence** - SQLite database survives container restarts
|
- **Persistence** - SQLite database survives container restarts
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **Context Injection** - Runtime parameters correctly injected into MCP tools
|
||||||
|
- ContextInjector always overrides AI-provided parameters (defense-in-depth)
|
||||||
|
- Hidden context parameters from AI tool schema to prevent hallucination
|
||||||
|
- Resolved database locking issues with concurrent tool calls
|
||||||
|
- Proper async handling of tool reloading after context injection
|
||||||
|
- **Simulation Re-runs** - Prevent duplicate execution of completed model-days
|
||||||
|
- Fixed job hanging when re-running partially completed simulations
|
||||||
|
- `_execute_date()` now skips already-completed model-days
|
||||||
|
- Job completion status correctly reflects skipped items
|
||||||
|
- **Agent Initialization** - Correct parameter passing in API mode
|
||||||
|
- Fixed BaseAgent initialization parameters in ModelDayExecutor
|
||||||
|
- Resolved async execution and position storage issues
|
||||||
|
- **Database Reliability** - Various improvements for concurrent access
|
||||||
|
- Fixed column existence checks before creating indexes
|
||||||
|
- Proper database path resolution in dev mode (prevents recursive _dev suffix)
|
||||||
|
- Module-level database initialization for uvicorn reliability
|
||||||
|
- Fixed database locking during concurrent writes
|
||||||
|
- Improved error handling in buy/sell functions
|
||||||
|
- **Configuration** - Improved config handling
|
||||||
|
- Use enabled field from config to determine which models run
|
||||||
|
- Use config models when empty models list provided
|
||||||
|
- Correct handling of merged runtime configs in containers
|
||||||
|
- Proper get_db_path() usage to pass base database path
|
||||||
|
- **Docker** - Various deployment improvements
|
||||||
|
- Removed non-existent data scripts from Dockerfile
|
||||||
|
- Proper respect for dev mode in entrypoint database initialization
|
||||||
|
- Correct closure usage to capture db_path in lifespan context manager
|
||||||
|
|
||||||
### Breaking Changes
|
### Breaking Changes
|
||||||
- **Batch Mode Removed** - All simulations now run through REST API
|
- **Batch Mode Removed** - All simulations now run through REST API
|
||||||
- v0.2.0 used sequential batch execution via Docker entrypoint
|
- v0.2.0 used sequential batch execution via Docker entrypoint
|
||||||
@@ -147,7 +215,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
- `merged.jsonl` no longer used (replaced by `price_data` table)
|
- `merged.jsonl` no longer used (replaced by `price_data` table)
|
||||||
- Automatic on-demand downloads eliminate need for manual data fetching
|
- Automatic on-demand downloads eliminate need for manual data fetching
|
||||||
- **Configuration Variables Changed**
|
- **Configuration Variables Changed**
|
||||||
- Added: `AUTO_DOWNLOAD_PRICE_DATA`, `MAX_SIMULATION_DAYS`, `API_PORT`
|
- Added: `DEPLOYMENT_MODE`, `PRESERVE_DEV_DATA`, `AUTO_DOWNLOAD_PRICE_DATA`, `MAX_SIMULATION_DAYS`, `API_PORT`
|
||||||
- Removed: `RUNTIME_ENV_PATH`, MCP service ports, `WEB_HTTP_PORT`
|
- Removed: `RUNTIME_ENV_PATH`, MCP service ports, `WEB_HTTP_PORT`
|
||||||
- MCP services now use fixed internal ports (not exposed to host)
|
- MCP services now use fixed internal ports (not exposed to host)
|
||||||
|
|
||||||
|
|||||||
@@ -49,14 +49,16 @@ class ContextInjector:
|
|||||||
"""
|
"""
|
||||||
# Inject context parameters for trade tools
|
# Inject context parameters for trade tools
|
||||||
if request.name in ["buy", "sell"]:
|
if request.name in ["buy", "sell"]:
|
||||||
# Add signature and today_date to args if not present
|
# Debug: Log self attributes BEFORE injection
|
||||||
if "signature" not in request.args:
|
print(f"[ContextInjector.__call__] ENTRY: id={id(self)}, self.signature={self.signature}, self.today_date={self.today_date}, self.job_id={self.job_id}, self.session_id={self.session_id}")
|
||||||
request.args["signature"] = self.signature
|
print(f"[ContextInjector.__call__] Args BEFORE injection: {request.args}")
|
||||||
if "today_date" not in request.args:
|
|
||||||
request.args["today_date"] = self.today_date
|
# ALWAYS inject/override context parameters (don't trust AI-provided values)
|
||||||
if "job_id" not in request.args and self.job_id:
|
request.args["signature"] = self.signature
|
||||||
|
request.args["today_date"] = self.today_date
|
||||||
|
if self.job_id:
|
||||||
request.args["job_id"] = self.job_id
|
request.args["job_id"] = self.job_id
|
||||||
if "session_id" not in request.args and self.session_id:
|
if self.session_id:
|
||||||
request.args["session_id"] = self.session_id
|
request.args["session_id"] = self.session_id
|
||||||
|
|
||||||
# Debug logging
|
# Debug logging
|
||||||
|
|||||||
@@ -82,24 +82,13 @@ def get_current_position_from_db(job_id: str, model: str, date: str) -> Tuple[Di
|
|||||||
conn.close()
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
@mcp.tool()
|
def _buy_impl(symbol: str, amount: int, signature: str = None, today_date: str = None,
|
||||||
def buy(symbol: str, amount: int, signature: str = None, today_date: str = None,
|
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
|
||||||
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
|
|
||||||
"""
|
"""
|
||||||
Buy stock function - writes to SQLite database.
|
Internal buy implementation - accepts injected context parameters.
|
||||||
|
|
||||||
Args:
|
This function is not exposed to the AI model. It receives runtime context
|
||||||
symbol: Stock symbol (e.g., "AAPL", "MSFT")
|
(signature, today_date, job_id, session_id) from the ContextInjector.
|
||||||
amount: Number of shares to buy (positive integer)
|
|
||||||
signature: Model signature (injected by ContextInjector)
|
|
||||||
today_date: Trading date YYYY-MM-DD (injected by ContextInjector)
|
|
||||||
job_id: Job UUID (injected by ContextInjector)
|
|
||||||
session_id: Trading session ID (injected by ContextInjector)
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dict[str, Any]:
|
|
||||||
- Success: {"CASH": amount, symbol: quantity, ...}
|
|
||||||
- Failure: {"error": message, ...}
|
|
||||||
"""
|
"""
|
||||||
# Validate required parameters
|
# Validate required parameters
|
||||||
if not job_id:
|
if not job_id:
|
||||||
@@ -206,8 +195,29 @@ def buy(symbol: str, amount: int, signature: str = None, today_date: str = None,
|
|||||||
|
|
||||||
|
|
||||||
@mcp.tool()
|
@mcp.tool()
|
||||||
def sell(symbol: str, amount: int, signature: str = None, today_date: str = None,
|
def buy(symbol: str, amount: int, signature: str = None, today_date: str = None,
|
||||||
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
|
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Buy stock shares.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
symbol: Stock symbol (e.g., "AAPL", "MSFT", "GOOGL")
|
||||||
|
amount: Number of shares to buy (positive integer)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dict[str, Any]:
|
||||||
|
- Success: {"CASH": remaining_cash, "SYMBOL": shares, ...}
|
||||||
|
- Failure: {"error": error_message, ...}
|
||||||
|
|
||||||
|
Note: signature, today_date, job_id, session_id are automatically injected by the system.
|
||||||
|
Do not provide these parameters - they will be added automatically.
|
||||||
|
"""
|
||||||
|
# Delegate to internal implementation
|
||||||
|
return _buy_impl(symbol, amount, signature, today_date, job_id, session_id)
|
||||||
|
|
||||||
|
|
||||||
|
def _sell_impl(symbol: str, amount: int, signature: str = None, today_date: str = None,
|
||||||
|
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
|
||||||
"""
|
"""
|
||||||
Sell stock function - writes to SQLite database.
|
Sell stock function - writes to SQLite database.
|
||||||
|
|
||||||
@@ -327,6 +337,28 @@ def sell(symbol: str, amount: int, signature: str = None, today_date: str = None
|
|||||||
conn.close()
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
@mcp.tool()
|
||||||
|
def sell(symbol: str, amount: int, signature: str = None, today_date: str = None,
|
||||||
|
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
|
||||||
|
"""
|
||||||
|
Sell stock shares.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
symbol: Stock symbol (e.g., "AAPL", "MSFT", "GOOGL")
|
||||||
|
amount: Number of shares to sell (positive integer)
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dict[str, Any]:
|
||||||
|
- Success: {"CASH": remaining_cash, "SYMBOL": shares, ...}
|
||||||
|
- Failure: {"error": error_message, ...}
|
||||||
|
|
||||||
|
Note: signature, today_date, job_id, session_id are automatically injected by the system.
|
||||||
|
Do not provide these parameters - they will be added automatically.
|
||||||
|
"""
|
||||||
|
# Delegate to internal implementation
|
||||||
|
return _sell_impl(symbol, amount, signature, today_date, job_id, session_id)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
port = int(os.getenv("TRADE_HTTP_PORT", "8002"))
|
port = int(os.getenv("TRADE_HTTP_PORT", "8002"))
|
||||||
mcp.run(transport="streamable-http", port=port)
|
mcp.run(transport="streamable-http", port=port)
|
||||||
|
|||||||
@@ -90,7 +90,7 @@ class SimulationWorker:
|
|||||||
logger.info(f"Starting job {self.job_id}: {len(date_range)} dates, {len(models)} models")
|
logger.info(f"Starting job {self.job_id}: {len(date_range)} dates, {len(models)} models")
|
||||||
|
|
||||||
# NEW: Prepare price data (download if needed)
|
# NEW: Prepare price data (download if needed)
|
||||||
available_dates, warnings = self._prepare_data(date_range, models, config_path)
|
available_dates, warnings, completion_skips = self._prepare_data(date_range, models, config_path)
|
||||||
|
|
||||||
if not available_dates:
|
if not available_dates:
|
||||||
error_msg = "No trading dates available after price data preparation"
|
error_msg = "No trading dates available after price data preparation"
|
||||||
@@ -100,7 +100,7 @@ class SimulationWorker:
|
|||||||
# Execute available dates only
|
# Execute available dates only
|
||||||
for date in available_dates:
|
for date in available_dates:
|
||||||
logger.info(f"Processing date {date} with {len(models)} models")
|
logger.info(f"Processing date {date} with {len(models)} models")
|
||||||
self._execute_date(date, models, config_path)
|
self._execute_date(date, models, config_path, completion_skips)
|
||||||
|
|
||||||
# Job completed - determine final status
|
# Job completed - determine final status
|
||||||
progress = self.job_manager.get_job_progress(self.job_id)
|
progress = self.job_manager.get_job_progress(self.job_id)
|
||||||
@@ -145,7 +145,8 @@ class SimulationWorker:
|
|||||||
"error": error_msg
|
"error": error_msg
|
||||||
}
|
}
|
||||||
|
|
||||||
def _execute_date(self, date: str, models: List[str], config_path: str) -> None:
|
def _execute_date(self, date: str, models: List[str], config_path: str,
|
||||||
|
completion_skips: Dict[str, Set[str]] = None) -> None:
|
||||||
"""
|
"""
|
||||||
Execute all models for a single date in parallel.
|
Execute all models for a single date in parallel.
|
||||||
|
|
||||||
@@ -153,14 +154,24 @@ class SimulationWorker:
|
|||||||
date: Trading date (YYYY-MM-DD)
|
date: Trading date (YYYY-MM-DD)
|
||||||
models: List of model signatures to execute
|
models: List of model signatures to execute
|
||||||
config_path: Path to configuration file
|
config_path: Path to configuration file
|
||||||
|
completion_skips: {model: {dates}} of already-completed model-days to skip
|
||||||
|
|
||||||
Uses ThreadPoolExecutor to run all models concurrently for this date.
|
Uses ThreadPoolExecutor to run all models concurrently for this date.
|
||||||
Waits for all models to complete before returning.
|
Waits for all models to complete before returning.
|
||||||
|
Skips models that have already completed this date.
|
||||||
"""
|
"""
|
||||||
|
if completion_skips is None:
|
||||||
|
completion_skips = {}
|
||||||
|
|
||||||
with ThreadPoolExecutor(max_workers=self.max_workers) as executor:
|
with ThreadPoolExecutor(max_workers=self.max_workers) as executor:
|
||||||
# Submit all model executions for this date
|
# Submit all model executions for this date
|
||||||
futures = []
|
futures = []
|
||||||
for model in models:
|
for model in models:
|
||||||
|
# Skip if this model-day was already completed
|
||||||
|
if date in completion_skips.get(model, set()):
|
||||||
|
logger.debug(f"Skipping {model} on {date} (already completed)")
|
||||||
|
continue
|
||||||
|
|
||||||
future = executor.submit(
|
future = executor.submit(
|
||||||
self._execute_model_day,
|
self._execute_model_day,
|
||||||
date,
|
date,
|
||||||
@@ -397,7 +408,10 @@ class SimulationWorker:
|
|||||||
config_path: Path to configuration file
|
config_path: Path to configuration file
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Tuple of (available_dates, warnings)
|
Tuple of (available_dates, warnings, completion_skips)
|
||||||
|
- available_dates: Dates to process
|
||||||
|
- warnings: Warning messages
|
||||||
|
- completion_skips: {model: {dates}} of already-completed model-days
|
||||||
"""
|
"""
|
||||||
from api.price_data_manager import PriceDataManager
|
from api.price_data_manager import PriceDataManager
|
||||||
|
|
||||||
@@ -456,7 +470,7 @@ class SimulationWorker:
|
|||||||
self.job_manager.update_job_status(self.job_id, "running")
|
self.job_manager.update_job_status(self.job_id, "running")
|
||||||
logger.info(f"Job {self.job_id}: Starting execution - {len(dates_to_process)} dates, {len(models)} models")
|
logger.info(f"Job {self.job_id}: Starting execution - {len(dates_to_process)} dates, {len(models)} models")
|
||||||
|
|
||||||
return dates_to_process, warnings
|
return dates_to_process, warnings, completion_skips
|
||||||
|
|
||||||
def get_job_info(self) -> Dict[str, Any]:
|
def get_job_info(self) -> Dict[str, Any]:
|
||||||
"""
|
"""
|
||||||
|
|||||||
Reference in New Issue
Block a user