Compare commits

...

9 Commits

Author SHA1 Message Date
396a2747d3 fix: resolve async execution and position storage issues
- Fix async call in model_day_executor.py by wrapping with asyncio.run()
  Resolves RuntimeWarning where run_trading_session coroutine was never awaited
- Remove register_agent() call in API mode to prevent file-based position storage
  Position data is now stored exclusively in SQLite database (jobs.db)
- Update test mocks to use AsyncMock for async run_trading_session method

This fixes production deployment issues:
1. Trading sessions now execute properly (async bug)
2. No position files created, database-only storage
3. All tests pass

Closes issue with no trades being executed in production
2025-11-02 17:16:55 -05:00
71ec53db45 fix: read merged runtime config in containerized mode
The FastAPI app now checks for /tmp/runtime_config.json (created by
entrypoint.sh config merger) before falling back to the default config.

This allows user-provided configs mounted at /app/user-configs/ to
properly override default values in containerized deployments.

Fixes issue where custom user configs were not being applied.
2025-11-02 16:16:58 -05:00
b6bd949d23 config: enable all 5 models in default config
Updated default_config.json to enable all trading models:
- claude-sonnet-4-5
- deepseek-chat-v3.1
- qwen3-max
- gemini-2.5-flash
- gpt-5

This ensures that when models=[] is passed to the API, all 5 models
will run instead of just gpt-5.
2025-11-02 16:10:38 -05:00
b5f18ac0f3 chore: remove diagnostic logging code
Cleaned up all diagnostic print statements added during debugging.
The root cause (non-idempotent get_db_path) has been fixed, so the
extensive instrumentation is no longer needed.

Changes:
- Removed all diagnostic prints from api/main.py (lifespan and module-level)
- Removed all diagnostic prints from api/database.py (get_db_connection and initialize_dev_database)
- Kept essential user-facing messages (PRESERVE_DEV_DATA notice, database creation messages)

All 28 integration tests pass.
2025-11-02 16:00:34 -05:00
90b6ad400d fix: make get_db_path() idempotent to prevent recursive _dev suffix
Root Cause:
The get_db_path() function was being called multiple times in the
initialization chain, causing recursive suffix addition:
  data/jobs.db -> data/jobs_dev.db -> data/jobs_dev_dev.db

This resulted in tables being created in data/jobs_dev_dev.db while
the application tried to access data/jobs_dev.db (empty database).

Fix:
Added idempotency check in get_db_path() to detect and skip
transformation if path already contains "_dev.db" suffix.

Evidence from Diagnostics:
- alpha.20 logs showed: Input='data/jobs_dev.db', Resolved='data/jobs_dev_dev.db'
- Tables were created in jobs_dev_dev.db but accessed from jobs_dev.db
- This caused "no such table: jobs" errors despite successful initialization

All 28 integration tests pass with this fix.

Fixes #6
2025-11-02 15:52:25 -05:00
6e4b2a4cc5 debug: add connection-level diagnostics to trace database access
Enhanced diagnostics to trace database path resolution and table existence
at connection time. This will help identify if get_db_connection() is
resolving paths correctly and accessing the right database file.

Added diagnostics to:
- get_db_connection(): Show input path, resolved path, file existence, and tables found
- initialize_dev_database(): Verify tables exist after creation

This will reveal whether the path resolution is working correctly or if
there's a timing/caching issue with database file access.
2025-11-02 15:46:36 -05:00
18bd4d169d debug: add comprehensive diagnostic logging for database initialization
Following systematic debugging methodology after 5 failed fix attempts.
Adding extensive print-based diagnostics to trace execution flow in Docker.

Instrumentation added to:
- api/main.py: Module import, app creation, lifespan function, module-level init
- api/database.py: initialize_dev_database() entry/exit and decision points

This diagnostic version will help identify:
1. Whether module-level code executes in Docker
2. Which initialization layer is failing
3. Database paths being resolved
4. Environment variable values

Tests confirmed passing with diagnostic logging.
2025-11-02 15:41:47 -05:00
8b91c75b32 fix: add module-level database initialization for uvicorn reliability
Add database initialization at module load time to ensure it runs
regardless of how uvicorn handles the lifespan context manager.

Issue: The lifespan function wasn't being triggered consistently when
uvicorn loads the app module, causing "no such table: jobs" errors.

Solution: Initialize database when the module is imported (after app
creation), providing a reliable fallback that works in all deployment
scenarios.

This provides defense-in-depth:
1. Lifespan function (ideal path)
2. Module-level initialization (fallback/guarantee)

Both paths check deployment mode and call the appropriate init function.
2025-11-02 15:36:12 -05:00
bdb3f6a6a2 refactor: move database initialization from entrypoint to application
Move database initialization logic from shell script to Python application
lifespan, following separation of concerns and improving maintainability.

Benefits:
- Single source of truth for database initialization (api/main.py lifespan)
- Better testability - Python code vs shell scripts
- Clearer logging with structured messages
- Easier to debug and maintain
- Infrastructure (entrypoint.sh) focuses on service orchestration
- Application (api/main.py) owns its data layer

Changes:
- Removed database init from entrypoint.sh
- Enhanced lifespan function with detailed logging
- Simplified entrypoint script (now 4 steps instead of 5)
- All tests pass (28/28 API endpoint tests)
2025-11-02 15:32:53 -05:00
6 changed files with 59 additions and 39 deletions

View File

@@ -116,7 +116,7 @@ class HealthResponse(BaseModel):
def create_app(
db_path: str = "data/jobs.db",
config_path: str = "configs/default_config.json"
config_path: str = "/tmp/runtime_config.json" if Path("/tmp/runtime_config.json").exists() else "configs/default_config.json"
) -> FastAPI:
"""
Create FastAPI application instance.
@@ -135,19 +135,27 @@ def create_app(
from api.database import initialize_dev_database, initialize_database
# Startup - use closure to access db_path from create_app scope
logger.info("🚀 FastAPI application starting...")
logger.info("📊 Initializing database...")
if is_dev_mode():
# Initialize dev database (reset unless PRESERVE_DEV_DATA=true)
logger.info(" 🔧 DEV mode detected - initializing dev database")
dev_db_path = get_db_path(db_path)
initialize_dev_database(dev_db_path)
log_dev_mode_startup_warning()
else:
# Ensure production database schema exists
logger.info(" 🏭 PROD mode - ensuring database schema exists")
initialize_database(db_path)
logger.info("✅ Database initialized")
logger.info("🌐 API server ready to accept requests")
yield
# Shutdown (if needed in future)
pass
logger.info("🛑 FastAPI application shutting down...")
app = FastAPI(
title="AI-Trader Simulation API",
@@ -514,11 +522,30 @@ def create_app(
# Create default app instance
app = create_app()
# Ensure database is initialized when module is loaded
# This handles cases where lifespan might not be triggered properly
logger.info("🔧 Module-level database initialization check...")
from tools.deployment_config import is_dev_mode, get_db_path
from api.database import initialize_dev_database, initialize_database
_db_path = app.state.db_path
if is_dev_mode():
logger.info(" 🔧 DEV mode - initializing dev database at module load")
_dev_db_path = get_db_path(_db_path)
initialize_dev_database(_dev_db_path)
else:
logger.info(" 🏭 PROD mode - ensuring database exists at module load")
initialize_database(_db_path)
logger.info("✅ Module-level database initialization complete")
if __name__ == "__main__":
import uvicorn
# Note: Database initialization happens in startup_event()
# DEV mode warning will be displayed there as well
# Note: Database initialization happens in lifespan AND at module load
# for maximum reliability
uvicorn.run(app, host="0.0.0.0", port=8080)

View File

@@ -11,6 +11,7 @@ This module provides:
import logging
import os
import asyncio
from typing import Dict, Any, Optional, List, TYPE_CHECKING
from pathlib import Path
@@ -118,7 +119,7 @@ class ModelDayExecutor:
# Run trading session
logger.info(f"Running trading session for {self.model_sig} on {self.date}")
session_result = agent.run_trading_session(self.date)
session_result = asyncio.run(agent.run_trading_session(self.date))
# Persist results to SQLite
self._write_results_to_db(agent, session_result)
@@ -211,8 +212,10 @@ class ModelDayExecutor:
init_date=config.get("date_range", {}).get("init_date", "2025-10-13")
)
# Register agent (creates initial position if needed)
agent.register_agent()
# Note: In API mode, we don't call register_agent() because:
# - Position data is stored in SQLite database, not files
# - Database initialization is handled by JobManager
# - File-based position tracking is only for standalone/CLI mode
return agent

View File

@@ -6,30 +6,28 @@
},
"models": [
{
"name": "claude-3.7-sonnet",
"basemodel": "anthropic/claude-3.7-sonnet",
"signature": "claude-3.7-sonnet",
"enabled": false,
"openai_base_url": "Optional: YOUR_OPENAI_BASE_URL,you can write them in .env file",
"openai_api_key": "Optional: YOUR_OPENAI_API_KEY,you can write them in .env file"
"name": "claude-sonnet-4-5",
"basemodel": "anthropic/claude-sonnet-4.5",
"signature": "claude-sonnet-4.5",
"enabled": true
},
{
"name": "deepseek-chat-v3.1",
"basemodel": "deepseek/deepseek-chat-v3.1",
"signature": "deepseek-chat-v3.1",
"enabled": false
"enabled": true
},
{
"name": "qwen3-max",
"basemodel": "qwen/qwen3-max",
"signature": "qwen3-max",
"enabled": false
"enabled": true
},
{
"name": "gemini-2.5-flash",
"basemodel": "google/gemini-2.5-flash",
"signature": "gemini-2.5-flash",
"enabled": false
"enabled": true
},
{
"name": "gpt-5",

View File

@@ -36,24 +36,7 @@ fi
echo "✅ Environment variables validated"
# Step 1: Initialize database (respecting dev/prod mode)
echo "📊 Initializing database..."
python -c "
from tools.deployment_config import is_dev_mode, get_db_path
from api.database import initialize_dev_database, initialize_database
db_path = 'data/jobs.db'
if is_dev_mode():
print(' 🔧 DEV mode detected - initializing dev database')
dev_db_path = get_db_path(db_path)
initialize_dev_database(dev_db_path)
else:
print(' 🏭 PROD mode - initializing production database')
initialize_database(db_path)
"
echo "✅ Database initialized"
# Step 2: Merge and validate configuration
# Step 1: Merge and validate configuration
echo "🔧 Merging and validating configuration..."
python -c "from tools.config_merger import merge_and_validate; merge_and_validate()" || {
echo "❌ Configuration validation failed"
@@ -62,7 +45,7 @@ python -c "from tools.config_merger import merge_and_validate; merge_and_validat
export CONFIG_PATH=/tmp/runtime_config.json
echo "✅ Configuration validated and merged"
# Step 3: Start MCP services in background
# Step 2: Start MCP services in background
echo "🔧 Starting MCP services..."
cd /app
python agent_tools/start_mcp_services.py &
@@ -71,11 +54,11 @@ MCP_PID=$!
# Setup cleanup trap before starting uvicorn
trap "echo '🛑 Stopping services...'; kill $MCP_PID 2>/dev/null; exit 0" EXIT SIGTERM SIGINT
# Step 4: Wait for services to initialize
# Step 3: Wait for services to initialize
echo "⏳ Waiting for MCP services to start..."
sleep 3
# Step 5: Start FastAPI server with uvicorn (this blocks)
# Step 4: Start FastAPI server with uvicorn (this blocks)
# Note: Container always uses port 8080 internally
# The API_PORT env var only affects the host port mapping in docker-compose.yml
echo "🌐 Starting FastAPI server on port 8080..."

View File

@@ -14,7 +14,7 @@ Tests verify:
import pytest
import json
from unittest.mock import Mock, patch, MagicMock
from unittest.mock import Mock, patch, MagicMock, AsyncMock
from pathlib import Path
@@ -29,7 +29,8 @@ def create_mock_agent(positions=None, last_trade=None, current_prices=None,
mock_agent.get_current_prices.return_value = current_prices or {}
mock_agent.get_reasoning_steps.return_value = reasoning_steps or []
mock_agent.get_tool_usage.return_value = tool_usage or {}
mock_agent.run_trading_session.return_value = session_result or {"success": True}
# run_trading_session is async, so use AsyncMock
mock_agent.run_trading_session = AsyncMock(return_value=session_result or {"success": True})
return mock_agent

View File

@@ -69,8 +69,16 @@ def get_db_path(base_db_path: str) -> str:
Example:
PROD: "data/trading.db" -> "data/trading.db"
DEV: "data/trading.db" -> "data/trading_dev.db"
Note:
This function is idempotent - calling it multiple times on the same
path will not add multiple _dev suffixes.
"""
if is_dev_mode():
# Check if already has _dev suffix (idempotent)
if "_dev.db" in base_db_path:
return base_db_path
# Insert _dev before .db extension
if base_db_path.endswith(".db"):
return base_db_path[:-3] + "_dev.db"