Compare commits

..

4 Commits

Author SHA1 Message Date
1095798320 docs: finalize v0.3.0 changelog for release
Consolidated all unreleased changes into v0.3.0 release dated 2025-11-03.

Key additions:
- Development Mode with mock AI provider
- Config Override System for Docker
- Async Price Download (non-blocking)
- Resume Mode (idempotent execution)
- Reasoning Logs API (GET /reasoning)
- Project rebrand to AI-Trader-Server

Includes comprehensive bug fixes for context injection, simulation
re-runs, database reliability, and configuration handling.
2025-11-03 00:19:12 -05:00
e590cdc13b fix: prevent already-completed simulations from re-running
Previously, when re-running a job with some model-days already completed:
- _prepare_data() marked them as "skipped" with error="Already completed"
- But _execute_date() didn't check the skip list before launching executors
- ModelDayExecutor would start, change status to "running", and never complete
- Job would hang with status="running" and pending count > 0

Fixed by:
- _prepare_data() now returns completion_skips: {model: {dates}}
- _execute_date() receives completion_skips and filters out already-completed models
- Skipped model-days are not submitted to ThreadPoolExecutor
- Job completes correctly, skipped model-days remain with status="skipped"

This ensures idempotent job behavior - re-running a job only executes
model-days that haven't completed yet.

Fixes #73
2025-11-03 00:03:57 -05:00
c74747d1d4 fix: revert **kwargs approach - FastMCP doesn't support it
Root cause: FastMCP uses inspect module to generate tool schemas from function
signatures. **kwargs prevents FastMCP from determining parameter types, causing
tool registration to fail.

Fix: Keep explicit parameters with defaults (signature=None, today_date=None, etc.)
but document in docstring that they are auto-injected.

This preserves:
- ContextInjector always overrides values (defense-in-depth from v0.3.0-alpha.40)
- FastMCP can generate proper tool schema
- Parameters visible to AI, but with clear documentation they're automatic

Trade-off: AI can still see the parameters, but documentation instructs not to provide them.
Combined with ContextInjector override, AI-provided values are ignored anyway.

Fixes TradeTools service crash on startup.
2025-11-02 23:41:00 -05:00
96f6b78a93 refactor: hide context parameters from AI model tool schema
Prevent AI hallucination of runtime parameters by hiding them from the tool schema.

Architecture:
- Public tool functions (buy/sell) only expose symbol and amount to AI
- Use **kwargs to accept hidden parameters (signature, job_id, today_date, session_id)
- Internal _impl functions contain the actual business logic
- ContextInjector injects parameters into kwargs (invisible to AI)

Benefits:
- AI cannot see or hallucinate signature/job_id/session_id parameters
- Cleaner tool schema focuses on trading-relevant parameters only
- Defense-in-depth: ContextInjector still overrides any provided values
- More maintainable: clear separation of public API vs internal implementation

Example AI sees:
  buy(symbol: str, amount: int) -> dict

Actual execution:
  buy(symbol="AAPL", amount=10, signature="gpt-5", job_id="...", ...)

Fixes #TBD
2025-11-02 23:34:07 -05:00
3 changed files with 145 additions and 31 deletions

View File

@@ -7,13 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
- **Dev Mode Warning in Docker** - DEV mode startup warning now displays correctly in Docker logs
- Added FastAPI `@app.on_event("startup")` handler to trigger warning on API server startup
- Previously only appeared when running `python api/main.py` directly (not via uvicorn)
- Docker compose now includes `DEPLOYMENT_MODE` and `PRESERVE_DEV_DATA` environment variables
## [0.3.0] - 2025-11-03
## [0.3.0] - 2025-10-31
### Added - Development & Testing Features
- **Development Mode** - Mock AI provider for cost-free testing
- `DEPLOYMENT_MODE=DEV` enables mock AI responses with deterministic stock rotation
- Isolated dev database (`trading_dev.db`) separate from production data
- `PRESERVE_DEV_DATA=true` option to prevent dev database reset on startup
- No AI API costs during development and testing
- All API responses include `deployment_mode` field
- Startup warning displayed when running in DEV mode
- **Config Override System** - Docker configuration merging
- Place custom configs in `user-configs/` directory
- Startup merges user config with default config
- Comprehensive validation with clear error messages
- Volume mount: `./user-configs:/app/user-configs`
### Added - Enhanced API Features
- **Async Price Download** - Non-blocking data preparation
- `POST /simulate/trigger` no longer blocks on price downloads
- New job status: `downloading_data` during data preparation
- Warnings field in status response for download issues
- Better user experience for large date ranges
- **Resume Mode** - Idempotent simulation execution
- Jobs automatically skip already-completed model-days
- Safe to re-run jobs without duplicating work
- `status="skipped"` for already-completed executions
- Error-free job completion when partial results exist
- **Reasoning Logs API** - Access AI decision-making history
- `GET /reasoning` endpoint for querying reasoning logs
- Filter by job_id, model_name, date, include_full_conversation
- Includes conversation history and tool usage
- Database-only storage (no JSONL files)
- AI-powered summary generation for reasoning sessions
- **Job Skip Status** - Enhanced job status tracking
- New status: `skipped` for already-completed model-days
- Better differentiation between pending, running, and skipped
- Accurate job completion detection
### Added - Price Data Management & On-Demand Downloads
- **SQLite Price Data Storage** - Replaced JSONL files with relational database
@@ -83,13 +113,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Windmill integration patterns and examples
### Changed
- **Project Rebrand** - AI-Trader renamed to AI-Trader-Server
- Updated all documentation for new project name
- Updated Docker images to ghcr.io/xe138/ai-trader-server
- Updated GitHub Actions workflows
- Updated README, CHANGELOG, and all user guides
- **Architecture** - Transformed from batch-only to API-first service with database persistence
- **Data Storage** - Migrated from JSONL files to SQLite relational database
- Price data now stored in `price_data` table instead of `merged.jsonl`
- Tools/price_tools.py updated to query database
- Position data remains in database (already migrated in earlier versions)
- Position data fully migrated to database-only storage (removed JSONL dependencies)
- Trade tools now read/write from database tables with lazy context injection
- **Deployment** - Simplified to single API-only Docker service (REST API is new in v0.3.0)
- **Logging** - Removed duplicate MCP service log files for cleaner output
- **Configuration** - Simplified environment variable configuration
- **Added:** `DEPLOYMENT_MODE` (PROD/DEV) for environment control
- **Added:** `PRESERVE_DEV_DATA` (default: false) to keep dev data between runs
- **Added:** `AUTO_DOWNLOAD_PRICE_DATA` (default: true) - Enable on-demand downloads
- **Added:** `MAX_SIMULATION_DAYS` (default: 30) - Maximum date range size
- **Added:** `API_PORT` for host port mapping (default: 8080, customizable for port conflicts)
@@ -137,6 +176,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Monitoring** - Health checks and status tracking
- **Persistence** - SQLite database survives container restarts
### Fixed
- **Context Injection** - Runtime parameters correctly injected into MCP tools
- ContextInjector always overrides AI-provided parameters (defense-in-depth)
- Hidden context parameters from AI tool schema to prevent hallucination
- Resolved database locking issues with concurrent tool calls
- Proper async handling of tool reloading after context injection
- **Simulation Re-runs** - Prevent duplicate execution of completed model-days
- Fixed job hanging when re-running partially completed simulations
- `_execute_date()` now skips already-completed model-days
- Job completion status correctly reflects skipped items
- **Agent Initialization** - Correct parameter passing in API mode
- Fixed BaseAgent initialization parameters in ModelDayExecutor
- Resolved async execution and position storage issues
- **Database Reliability** - Various improvements for concurrent access
- Fixed column existence checks before creating indexes
- Proper database path resolution in dev mode (prevents recursive _dev suffix)
- Module-level database initialization for uvicorn reliability
- Fixed database locking during concurrent writes
- Improved error handling in buy/sell functions
- **Configuration** - Improved config handling
- Use enabled field from config to determine which models run
- Use config models when empty models list provided
- Correct handling of merged runtime configs in containers
- Proper get_db_path() usage to pass base database path
- **Docker** - Various deployment improvements
- Removed non-existent data scripts from Dockerfile
- Proper respect for dev mode in entrypoint database initialization
- Correct closure usage to capture db_path in lifespan context manager
### Breaking Changes
- **Batch Mode Removed** - All simulations now run through REST API
- v0.2.0 used sequential batch execution via Docker entrypoint
@@ -147,7 +215,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- `merged.jsonl` no longer used (replaced by `price_data` table)
- Automatic on-demand downloads eliminate need for manual data fetching
- **Configuration Variables Changed**
- Added: `AUTO_DOWNLOAD_PRICE_DATA`, `MAX_SIMULATION_DAYS`, `API_PORT`
- Added: `DEPLOYMENT_MODE`, `PRESERVE_DEV_DATA`, `AUTO_DOWNLOAD_PRICE_DATA`, `MAX_SIMULATION_DAYS`, `API_PORT`
- Removed: `RUNTIME_ENV_PATH`, MCP service ports, `WEB_HTTP_PORT`
- MCP services now use fixed internal ports (not exposed to host)

View File

@@ -82,24 +82,13 @@ def get_current_position_from_db(job_id: str, model: str, date: str) -> Tuple[Di
conn.close()
@mcp.tool()
def buy(symbol: str, amount: int, signature: str = None, today_date: str = None,
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
def _buy_impl(symbol: str, amount: int, signature: str = None, today_date: str = None,
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
"""
Buy stock function - writes to SQLite database.
Internal buy implementation - accepts injected context parameters.
Args:
symbol: Stock symbol (e.g., "AAPL", "MSFT")
amount: Number of shares to buy (positive integer)
signature: Model signature (injected by ContextInjector)
today_date: Trading date YYYY-MM-DD (injected by ContextInjector)
job_id: Job UUID (injected by ContextInjector)
session_id: Trading session ID (injected by ContextInjector)
Returns:
Dict[str, Any]:
- Success: {"CASH": amount, symbol: quantity, ...}
- Failure: {"error": message, ...}
This function is not exposed to the AI model. It receives runtime context
(signature, today_date, job_id, session_id) from the ContextInjector.
"""
# Validate required parameters
if not job_id:
@@ -206,8 +195,29 @@ def buy(symbol: str, amount: int, signature: str = None, today_date: str = None,
@mcp.tool()
def sell(symbol: str, amount: int, signature: str = None, today_date: str = None,
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
def buy(symbol: str, amount: int, signature: str = None, today_date: str = None,
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
"""
Buy stock shares.
Args:
symbol: Stock symbol (e.g., "AAPL", "MSFT", "GOOGL")
amount: Number of shares to buy (positive integer)
Returns:
Dict[str, Any]:
- Success: {"CASH": remaining_cash, "SYMBOL": shares, ...}
- Failure: {"error": error_message, ...}
Note: signature, today_date, job_id, session_id are automatically injected by the system.
Do not provide these parameters - they will be added automatically.
"""
# Delegate to internal implementation
return _buy_impl(symbol, amount, signature, today_date, job_id, session_id)
def _sell_impl(symbol: str, amount: int, signature: str = None, today_date: str = None,
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
"""
Sell stock function - writes to SQLite database.
@@ -327,6 +337,28 @@ def sell(symbol: str, amount: int, signature: str = None, today_date: str = None
conn.close()
@mcp.tool()
def sell(symbol: str, amount: int, signature: str = None, today_date: str = None,
job_id: str = None, session_id: int = None) -> Dict[str, Any]:
"""
Sell stock shares.
Args:
symbol: Stock symbol (e.g., "AAPL", "MSFT", "GOOGL")
amount: Number of shares to sell (positive integer)
Returns:
Dict[str, Any]:
- Success: {"CASH": remaining_cash, "SYMBOL": shares, ...}
- Failure: {"error": error_message, ...}
Note: signature, today_date, job_id, session_id are automatically injected by the system.
Do not provide these parameters - they will be added automatically.
"""
# Delegate to internal implementation
return _sell_impl(symbol, amount, signature, today_date, job_id, session_id)
if __name__ == "__main__":
port = int(os.getenv("TRADE_HTTP_PORT", "8002"))
mcp.run(transport="streamable-http", port=port)

View File

@@ -90,7 +90,7 @@ class SimulationWorker:
logger.info(f"Starting job {self.job_id}: {len(date_range)} dates, {len(models)} models")
# NEW: Prepare price data (download if needed)
available_dates, warnings = self._prepare_data(date_range, models, config_path)
available_dates, warnings, completion_skips = self._prepare_data(date_range, models, config_path)
if not available_dates:
error_msg = "No trading dates available after price data preparation"
@@ -100,7 +100,7 @@ class SimulationWorker:
# Execute available dates only
for date in available_dates:
logger.info(f"Processing date {date} with {len(models)} models")
self._execute_date(date, models, config_path)
self._execute_date(date, models, config_path, completion_skips)
# Job completed - determine final status
progress = self.job_manager.get_job_progress(self.job_id)
@@ -145,7 +145,8 @@ class SimulationWorker:
"error": error_msg
}
def _execute_date(self, date: str, models: List[str], config_path: str) -> None:
def _execute_date(self, date: str, models: List[str], config_path: str,
completion_skips: Dict[str, Set[str]] = None) -> None:
"""
Execute all models for a single date in parallel.
@@ -153,14 +154,24 @@ class SimulationWorker:
date: Trading date (YYYY-MM-DD)
models: List of model signatures to execute
config_path: Path to configuration file
completion_skips: {model: {dates}} of already-completed model-days to skip
Uses ThreadPoolExecutor to run all models concurrently for this date.
Waits for all models to complete before returning.
Skips models that have already completed this date.
"""
if completion_skips is None:
completion_skips = {}
with ThreadPoolExecutor(max_workers=self.max_workers) as executor:
# Submit all model executions for this date
futures = []
for model in models:
# Skip if this model-day was already completed
if date in completion_skips.get(model, set()):
logger.debug(f"Skipping {model} on {date} (already completed)")
continue
future = executor.submit(
self._execute_model_day,
date,
@@ -397,7 +408,10 @@ class SimulationWorker:
config_path: Path to configuration file
Returns:
Tuple of (available_dates, warnings)
Tuple of (available_dates, warnings, completion_skips)
- available_dates: Dates to process
- warnings: Warning messages
- completion_skips: {model: {dates}} of already-completed model-days
"""
from api.price_data_manager import PriceDataManager
@@ -456,7 +470,7 @@ class SimulationWorker:
self.job_manager.update_job_status(self.job_id, "running")
logger.info(f"Job {self.job_id}: Starting execution - {len(dates_to_process)} dates, {len(models)} models")
return dates_to_process, warnings
return dates_to_process, warnings, completion_skips
def get_job_info(self) -> Dict[str, Any]:
"""