Compare commits

..

10 Commits

Author SHA1 Message Date
71ec53db45 fix: read merged runtime config in containerized mode
The FastAPI app now checks for /tmp/runtime_config.json (created by
entrypoint.sh config merger) before falling back to the default config.

This allows user-provided configs mounted at /app/user-configs/ to
properly override default values in containerized deployments.

Fixes issue where custom user configs were not being applied.
2025-11-02 16:16:58 -05:00
b6bd949d23 config: enable all 5 models in default config
Updated default_config.json to enable all trading models:
- claude-sonnet-4-5
- deepseek-chat-v3.1
- qwen3-max
- gemini-2.5-flash
- gpt-5

This ensures that when models=[] is passed to the API, all 5 models
will run instead of just gpt-5.
2025-11-02 16:10:38 -05:00
b5f18ac0f3 chore: remove diagnostic logging code
Cleaned up all diagnostic print statements added during debugging.
The root cause (non-idempotent get_db_path) has been fixed, so the
extensive instrumentation is no longer needed.

Changes:
- Removed all diagnostic prints from api/main.py (lifespan and module-level)
- Removed all diagnostic prints from api/database.py (get_db_connection and initialize_dev_database)
- Kept essential user-facing messages (PRESERVE_DEV_DATA notice, database creation messages)

All 28 integration tests pass.
2025-11-02 16:00:34 -05:00
90b6ad400d fix: make get_db_path() idempotent to prevent recursive _dev suffix
Root Cause:
The get_db_path() function was being called multiple times in the
initialization chain, causing recursive suffix addition:
  data/jobs.db -> data/jobs_dev.db -> data/jobs_dev_dev.db

This resulted in tables being created in data/jobs_dev_dev.db while
the application tried to access data/jobs_dev.db (empty database).

Fix:
Added idempotency check in get_db_path() to detect and skip
transformation if path already contains "_dev.db" suffix.

Evidence from Diagnostics:
- alpha.20 logs showed: Input='data/jobs_dev.db', Resolved='data/jobs_dev_dev.db'
- Tables were created in jobs_dev_dev.db but accessed from jobs_dev.db
- This caused "no such table: jobs" errors despite successful initialization

All 28 integration tests pass with this fix.

Fixes #6
2025-11-02 15:52:25 -05:00
6e4b2a4cc5 debug: add connection-level diagnostics to trace database access
Enhanced diagnostics to trace database path resolution and table existence
at connection time. This will help identify if get_db_connection() is
resolving paths correctly and accessing the right database file.

Added diagnostics to:
- get_db_connection(): Show input path, resolved path, file existence, and tables found
- initialize_dev_database(): Verify tables exist after creation

This will reveal whether the path resolution is working correctly or if
there's a timing/caching issue with database file access.
2025-11-02 15:46:36 -05:00
18bd4d169d debug: add comprehensive diagnostic logging for database initialization
Following systematic debugging methodology after 5 failed fix attempts.
Adding extensive print-based diagnostics to trace execution flow in Docker.

Instrumentation added to:
- api/main.py: Module import, app creation, lifespan function, module-level init
- api/database.py: initialize_dev_database() entry/exit and decision points

This diagnostic version will help identify:
1. Whether module-level code executes in Docker
2. Which initialization layer is failing
3. Database paths being resolved
4. Environment variable values

Tests confirmed passing with diagnostic logging.
2025-11-02 15:41:47 -05:00
8b91c75b32 fix: add module-level database initialization for uvicorn reliability
Add database initialization at module load time to ensure it runs
regardless of how uvicorn handles the lifespan context manager.

Issue: The lifespan function wasn't being triggered consistently when
uvicorn loads the app module, causing "no such table: jobs" errors.

Solution: Initialize database when the module is imported (after app
creation), providing a reliable fallback that works in all deployment
scenarios.

This provides defense-in-depth:
1. Lifespan function (ideal path)
2. Module-level initialization (fallback/guarantee)

Both paths check deployment mode and call the appropriate init function.
2025-11-02 15:36:12 -05:00
bdb3f6a6a2 refactor: move database initialization from entrypoint to application
Move database initialization logic from shell script to Python application
lifespan, following separation of concerns and improving maintainability.

Benefits:
- Single source of truth for database initialization (api/main.py lifespan)
- Better testability - Python code vs shell scripts
- Clearer logging with structured messages
- Easier to debug and maintain
- Infrastructure (entrypoint.sh) focuses on service orchestration
- Application (api/main.py) owns its data layer

Changes:
- Removed database init from entrypoint.sh
- Enhanced lifespan function with detailed logging
- Simplified entrypoint script (now 4 steps instead of 5)
- All tests pass (28/28 API endpoint tests)
2025-11-02 15:32:53 -05:00
3502a7ffa8 fix: respect dev mode in entrypoint database initialization
- Update entrypoint.sh to check DEPLOYMENT_MODE before initializing database
- DEV mode: calls initialize_dev_database() which resets the database
- PROD mode: calls initialize_database() which preserves existing data
- Adds clear logging to show which mode is being used

This ensures the dev database is properly reset on container startup,
matching the behavior of the lifespan function in api/main.py.
2025-11-02 15:30:11 -05:00
68d9f241e1 fix: use closure to capture db_path in lifespan context manager
- Fix lifespan function to access db_path from create_app scope via closure
- Prevents "no such table: jobs" error by ensuring database initialization runs
- Previous version tried to access app.state.db_path before it was set

The issue was that app.state is set after FastAPI instantiation, but the
lifespan function needs the db_path during startup. Using closure allows
the lifespan function to capture db_path from the create_app function scope.
2025-11-02 15:24:29 -05:00
4 changed files with 53 additions and 25 deletions

View File

@@ -116,7 +116,7 @@ class HealthResponse(BaseModel):
def create_app(
db_path: str = "data/jobs.db",
config_path: str = "configs/default_config.json"
config_path: str = "/tmp/runtime_config.json" if Path("/tmp/runtime_config.json").exists() else "configs/default_config.json"
) -> FastAPI:
"""
Create FastAPI application instance.
@@ -134,20 +134,28 @@ def create_app(
from tools.deployment_config import is_dev_mode, get_db_path
from api.database import initialize_dev_database, initialize_database
# Startup
# Startup - use closure to access db_path from create_app scope
logger.info("🚀 FastAPI application starting...")
logger.info("📊 Initializing database...")
if is_dev_mode():
# Initialize dev database (reset unless PRESERVE_DEV_DATA=true)
dev_db_path = get_db_path(app.state.db_path)
logger.info(" 🔧 DEV mode detected - initializing dev database")
dev_db_path = get_db_path(db_path)
initialize_dev_database(dev_db_path)
log_dev_mode_startup_warning()
else:
# Ensure production database schema exists
initialize_database(app.state.db_path)
logger.info(" 🏭 PROD mode - ensuring database schema exists")
initialize_database(db_path)
logger.info("✅ Database initialized")
logger.info("🌐 API server ready to accept requests")
yield
# Shutdown (if needed in future)
pass
logger.info("🛑 FastAPI application shutting down...")
app = FastAPI(
title="AI-Trader Simulation API",
@@ -514,11 +522,30 @@ def create_app(
# Create default app instance
app = create_app()
# Ensure database is initialized when module is loaded
# This handles cases where lifespan might not be triggered properly
logger.info("🔧 Module-level database initialization check...")
from tools.deployment_config import is_dev_mode, get_db_path
from api.database import initialize_dev_database, initialize_database
_db_path = app.state.db_path
if is_dev_mode():
logger.info(" 🔧 DEV mode - initializing dev database at module load")
_dev_db_path = get_db_path(_db_path)
initialize_dev_database(_dev_db_path)
else:
logger.info(" 🏭 PROD mode - ensuring database exists at module load")
initialize_database(_db_path)
logger.info("✅ Module-level database initialization complete")
if __name__ == "__main__":
import uvicorn
# Note: Database initialization happens in startup_event()
# DEV mode warning will be displayed there as well
# Note: Database initialization happens in lifespan AND at module load
# for maximum reliability
uvicorn.run(app, host="0.0.0.0", port=8080)

View File

@@ -6,30 +6,28 @@
},
"models": [
{
"name": "claude-3.7-sonnet",
"basemodel": "anthropic/claude-3.7-sonnet",
"signature": "claude-3.7-sonnet",
"enabled": false,
"openai_base_url": "Optional: YOUR_OPENAI_BASE_URL,you can write them in .env file",
"openai_api_key": "Optional: YOUR_OPENAI_API_KEY,you can write them in .env file"
"name": "claude-sonnet-4-5",
"basemodel": "anthropic/claude-sonnet-4.5",
"signature": "claude-sonnet-4.5",
"enabled": true
},
{
"name": "deepseek-chat-v3.1",
"basemodel": "deepseek/deepseek-chat-v3.1",
"signature": "deepseek-chat-v3.1",
"enabled": false
"enabled": true
},
{
"name": "qwen3-max",
"basemodel": "qwen/qwen3-max",
"signature": "qwen3-max",
"enabled": false
"enabled": true
},
{
"name": "gemini-2.5-flash",
"basemodel": "google/gemini-2.5-flash",
"signature": "gemini-2.5-flash",
"enabled": false
"enabled": true
},
{
"name": "gpt-5",

View File

@@ -36,12 +36,7 @@ fi
echo "✅ Environment variables validated"
# Step 1: Initialize database
echo "📊 Initializing database..."
python -c "from api.database import initialize_database; initialize_database('data/jobs.db')"
echo "✅ Database initialized"
# Step 2: Merge and validate configuration
# Step 1: Merge and validate configuration
echo "🔧 Merging and validating configuration..."
python -c "from tools.config_merger import merge_and_validate; merge_and_validate()" || {
echo "❌ Configuration validation failed"
@@ -50,7 +45,7 @@ python -c "from tools.config_merger import merge_and_validate; merge_and_validat
export CONFIG_PATH=/tmp/runtime_config.json
echo "✅ Configuration validated and merged"
# Step 3: Start MCP services in background
# Step 2: Start MCP services in background
echo "🔧 Starting MCP services..."
cd /app
python agent_tools/start_mcp_services.py &
@@ -59,11 +54,11 @@ MCP_PID=$!
# Setup cleanup trap before starting uvicorn
trap "echo '🛑 Stopping services...'; kill $MCP_PID 2>/dev/null; exit 0" EXIT SIGTERM SIGINT
# Step 4: Wait for services to initialize
# Step 3: Wait for services to initialize
echo "⏳ Waiting for MCP services to start..."
sleep 3
# Step 5: Start FastAPI server with uvicorn (this blocks)
# Step 4: Start FastAPI server with uvicorn (this blocks)
# Note: Container always uses port 8080 internally
# The API_PORT env var only affects the host port mapping in docker-compose.yml
echo "🌐 Starting FastAPI server on port 8080..."

View File

@@ -69,8 +69,16 @@ def get_db_path(base_db_path: str) -> str:
Example:
PROD: "data/trading.db" -> "data/trading.db"
DEV: "data/trading.db" -> "data/trading_dev.db"
Note:
This function is idempotent - calling it multiple times on the same
path will not add multiple _dev suffixes.
"""
if is_dev_mode():
# Check if already has _dev suffix (idempotent)
if "_dev.db" in base_db_path:
return base_db_path
# Insert _dev before .db extension
if base_db_path.endswith(".db"):
return base_db_path[:-3] + "_dev.db"