Commit Graph

9 Commits

Author SHA1 Message Date
e968434062 fix: reload tools after context injection and prevent database locking
Critical fixes for ContextInjector and database concurrency:

1. ContextInjector Not Working:
   - Made set_context() async to reload tools after recreating MCP client
   - Tools from old client (without interceptor) were still being used
   - Now tools are reloaded from new client with interceptor active
   - This ensures buy/sell calls properly receive injected parameters

2. Database Locking:
   - Closed main connection before _write_results_to_db() opens new one
   - SQLite doesn't handle concurrent write connections well
   - Prevents "database is locked" error during position writes

Changes:
- agent/base_agent/base_agent.py:
  - async def set_context() instead of def set_context()
  - Added: self.tools = await self.client.get_tools()
- api/model_day_executor.py:
  - await agent.set_context(context_injector)
  - conn.close() before _write_results_to_db()

Root Cause:
When recreating the MCP client with tool_interceptors, the old tools
were still cached in self.tools and being passed to the AI agent.
The interceptor was never invoked, so job_id/signature/date were missing.
2025-11-02 22:42:17 -05:00
027b4bd8e4 refactor: implement database-only position tracking with lazy context injection
This commit migrates the system to database-only position storage,
eliminating file-based position.jsonl dependencies and fixing
ContextInjector initialization timing issues.

Key Changes:

1. ContextInjector Lifecycle Refactor:
   - Remove ContextInjector creation from BaseAgent.__init__()
   - Add BaseAgent.set_context() method for post-initialization injection
   - Update ModelDayExecutor to create ContextInjector with correct trading day date
   - Ensures ContextInjector receives actual trading date instead of init_date
   - Includes session_id injection for proper database linking

2. Database Position Functions:
   - Implement get_today_init_position_from_db() for querying previous positions
   - Implement add_no_trade_record_to_db() for no-trade day handling
   - Both functions query SQLite directly (positions + holdings tables)
   - Handle first trading day case with initial cash return
   - Include comprehensive error handling and logging

3. System Integration:
   - Update get_agent_system_prompt() to use database queries
   - Update _handle_trading_result() to write no-trade records to database
   - Remove dependencies on position.jsonl file reading/writing
   - Use deployment_config for automatic prod/dev database resolution

Data Flow:
- ModelDayExecutor creates runtime config and trading session
- Agent initialized without context
- ContextInjector created with (signature, date, job_id, session_id)
- Context injected via set_context()
- System prompt queries database for yesterday's position
- Trade tools write directly to database
- No-trade handler creates database records

Fixes:
- ContextInjector no longer receives None values
- No FileNotFoundError for missing position.jsonl files
- Database is single source of truth for position tracking
- Session linking maintained across all position records

Design: docs/plans/2025-02-11-database-position-tracking-design.md
2025-11-02 22:20:01 -05:00
fcfdf36c1c fix: use 'no_trade' action type for initial position
Changed action_type from 'init' to 'no_trade' in _initialize_starting_position()
to comply with database CHECK constraint that only allows 'buy', 'sell', or 'no_trade'.

Fixes sqlite3.IntegrityError during position initialization.
2025-11-02 21:49:57 -05:00
019c84fca8 refactor: migrate trade tools from file-based to SQLite position storage
Complete rewrite of position management in MCP trade tools:

**Trade Tools (agent_tools/tool_trade.py)**
- Replace file-based position.jsonl reads with SQLite queries
- Add get_current_position_from_db() to query positions and holdings tables
- Rewrite buy() and sell() to write directly to database
- Calculate portfolio value and P&L metrics in tools
- Accept job_id and session_id parameters via ContextInjector
- Return errors with proper context for debugging
- Use deployment-aware database path resolution

**Context Injection (agent/context_injector.py)**
- Add job_id and session_id to constructor
- Inject job_id and session_id into buy/sell tool calls
- Support optional parameters (None in standalone mode)

**BaseAgent (agent/base_agent/base_agent.py)**
- Read JOB_ID from runtime config
- Pass job_id to ContextInjector during initialization
- Enable automatic context injection for API mode

**ModelDayExecutor (api/model_day_executor.py)**
- Add _initialize_starting_position() method
- Create initial position record before agent runs
- Load initial_cash from config
- Update context_injector.session_id after session creation
- Link positions to sessions automatically

**Architecture Changes:**
- Eliminates file-based position tracking entirely
- Single source of truth: SQLite database
- Positions automatically linked to trading sessions
- Concurrent execution safe (no file system conflicts)
- Deployment mode aware (prod vs dev databases)

This completes the migration to database-only position storage.
File-based position.jsonl is no longer used or created.

Fixes context injection errors in concurrent simulations.
2025-11-02 21:36:57 -05:00
dbd8f0141c fix: initialize agent properly in API mode
Fixed critical bug where ModelDayExecutor._initialize_agent() created
a BaseAgent but never called agent.initialize(), leaving self.model=None.
This caused 'NoneType' object has no attribute 'bind' error when
run_trading_session() tried to create the langchain agent.

Changes:
- Made _initialize_agent() async
- Added await agent.initialize() call
- Updated call site to await async function

Now properly initializes MCP client, tools, and AI model before
executing trading sessions, matching the working CLI mode pattern.
2025-11-02 19:39:43 -05:00
555f0e7b66 feat: store reasoning logs with sessions in model_day_executor
- Add _create_trading_session() method to create session records
- Add async _store_reasoning_logs() to store conversation with AI summaries
- Add async _update_session_summary() to generate overall session summary
- Modify execute() -> execute_async() with async workflow
- Add execute_sync() wrapper and keep execute() as sync entry point
- Update _write_results_to_db() to accept and use session_id parameter
- Modify positions INSERT to include session_id foreign key
- Remove old reasoning_logs code block (obsolete schema)
- Add comprehensive unit tests for all new functionality

All tests pass. Session-based reasoning storage now integrated.
2025-11-02 18:03:41 -05:00
396a2747d3 fix: resolve async execution and position storage issues
- Fix async call in model_day_executor.py by wrapping with asyncio.run()
  Resolves RuntimeWarning where run_trading_session coroutine was never awaited
- Remove register_agent() call in API mode to prevent file-based position storage
  Position data is now stored exclusively in SQLite database (jobs.db)
- Update test mocks to use AsyncMock for async run_trading_session method

This fixes production deployment issues:
1. Trading sessions now execute properly (async bug)
2. No position files created, database-only storage
3. All tests pass

Closes issue with no trades being executed in production
2025-11-02 17:16:55 -05:00
34d3317571 fix: correct BaseAgent initialization parameters in ModelDayExecutor
Fixed incorrect parameter passing to BaseAgent.__init__():
- Changed model_name to basemodel (correct parameter name)
- Removed invalid config parameter
- Properly mapped all configuration values to BaseAgent parameters

This resolves simulation job failures with error:
"BaseAgent.__init__() got an unexpected keyword argument 'model_name'"

Fixes initialization of trading agents in API simulation jobs.
2025-11-02 09:00:09 -05:00
fb9583b374 feat: transform to REST API service with SQLite persistence (v0.3.0)
Major architecture transformation from batch-only to API service with
database persistence for Windmill integration.

## REST API Implementation
- POST /simulate/trigger - Start simulation jobs
- GET /simulate/status/{job_id} - Monitor job progress
- GET /results - Query results with filters (job_id, date, model)
- GET /health - Service health checks

## Database Layer
- SQLite persistence with 6 tables (jobs, job_details, positions,
  holdings, reasoning_logs, tool_usage)
- Foreign key constraints with cascade deletes
- Replaces JSONL file storage

## Backend Components
- JobManager: Job lifecycle management with concurrency control
- RuntimeConfigManager: Thread-safe isolated runtime configs
- ModelDayExecutor: Single model-day execution engine
- SimulationWorker: Date-sequential, model-parallel orchestration

## Testing
- 102 unit and integration tests (85% coverage)
- Database: 98% coverage
- Job manager: 98% coverage
- API endpoints: 81% coverage
- Pydantic models: 100% coverage
- TDD approach throughout

## Docker Deployment
- Dual-mode: API server (persistent) + batch (one-time)
- Health checks with 30s interval
- Volume persistence for database and logs
- Separate entrypoints for each mode

## Validation Tools
- scripts/validate_docker_build.sh - Build validation
- scripts/test_api_endpoints.sh - Complete API testing
- scripts/test_batch_mode.sh - Batch mode validation
- DOCKER_API.md - Deployment guide
- TESTING_GUIDE.md - Testing procedures

## Configuration
- API_PORT environment variable (default: 8080)
- Backwards compatible with existing configs
- FastAPI, uvicorn, pydantic>=2.0 dependencies

Co-Authored-By: AI Assistant <noreply@example.com>
2025-10-31 11:47:10 -04:00