Healthcheck now runs once per hour instead of every 30 seconds,
reducing log spam while still maintaining startup verification
during the 40s start_period.
Benefits:
- Minimal log noise (1 check/hour vs every 30s)
- Maintains startup verification
- Compatible with Docker orchestration tools
The planned v0.4.0 "Simplified Simulation Control" features were
already implemented in v0.3.0:
- POST /simulate/trigger with optional start_date (auto-resume when null)
- Required end_date parameter for target date
- replace_existing flag (equivalent to force_resimulate)
- Automatic detection of last completed date per model
- Idempotent behavior by default
Updated version history to reflect v1.0.0 as the next planned release
after v0.3.0.
Reduce visual noise during development by logging healthcheck requests
at DEBUG level when DEPLOYMENT_MODE=DEV. Production mode continues to
log healthchecks at INFO level for proper observability.
Changes:
- Modified /health endpoint to check deployment mode
- DEV mode: logs at DEBUG level (only visible with DEBUG logging)
- PROD mode: logs at INFO level (maintains current behavior)
Co-Authored-By: Claude <noreply@anthropic.com>
Critical bug fixes for position tracking:
- Fixed cash reset between trading days
- Fixed positions lost over weekends
- Fixed profit calculation accuracy
Plus standardized testing infrastructure.
- Implementation plan with 9 tasks covering bug fixes and testing
- Summary report documenting root causes, solution, and verification
- Both documents provide comprehensive reference for future maintainers
Add comprehensive suite of testing scripts for different workflows:
- test.sh: Interactive menu for all testing operations
- quick_test.sh: Fast unit test feedback (~10-30s)
- run_tests.sh: Main test runner with full configuration options
- coverage_report.sh: Coverage analysis with HTML/JSON/terminal reports
- ci_test.sh: CI/CD optimized testing with JUnit/coverage XML output
Features:
- Colored terminal output with clear error messages
- Consistent option flags across all scripts
- Support for test markers (unit, integration, e2e, slow, etc.)
- Parallel execution support
- Coverage thresholds (default: 85%)
- Virtual environment and dependency checks
Documentation:
- Update CLAUDE.md with testing section and examples
- Expand docs/developer/testing.md with comprehensive guide
- Add scripts/README.md with quick reference
All scripts are tested and executable. This standardizes the testing
process for local development, CI/CD, and pull request workflows.
Document the critical bug fixes for position tracking:
- Cash reset to initial value each day
- Positions lost over weekends
- Incorrect profit calculations treating trades as losses
Update database schema documentation to explain the corrected
profit calculation logic that compares to start-of-day portfolio
value instead of previous day's final value.
Previously, profit calculations compared portfolio value to the previous
day's final value. This caused trades to appear as losses since buying
stocks decreases cash and increases stock value equally (net zero change).
Now profit calculations compare to the start-of-day portfolio value
(action_id=0 for current date), which accurately reflects gains/losses
from price movements and trading decisions.
Changes:
- agent_tools/tool_trade.py: Fixed profit calc in _buy_impl() and _sell_impl()
- tools/price_tools.py: Fixed profit calc in add_no_trade_record_to_db()
Test: test_profit_calculation_accuracy now passes
- Updated create_mock_agent() to remove references to deleted methods (get_positions, get_last_trade, get_current_prices)
- Replaced position/holdings write tests with initial position creation test
- Added set_context AsyncMock to properly test async agent flow
- Skipped deprecated tests that verified removed _write_results_to_db() and _calculate_portfolio_value() methods
- All model_day_executor tests now pass (11 passed, 3 skipped)
- Removed call to _write_results_to_db() in execute_async()
- Deleted entire _write_results_to_db() method (lines 435-531)
- Deleted helper method _calculate_portfolio_value() (lines 533-557)
- Position tracking now exclusively handled by trade tools
This method was calling non-existent methods (get_positions(), get_last_trade(),
get_current_prices()) on BaseAgent, resulting in corrupt records with cash=0
and holdings=[]. Removal fixes bugs where cash resets to initial value and
positions are lost over weekends.
- Create tests/unit/test_position_tracking_bugs.py with three test cases
- test_cash_not_reset_between_days: Tests that cash carries over between days
- test_positions_persist_over_weekend: Tests that positions persist across non-trading days
- test_profit_calculation_accuracy: Tests that profit calculations are accurate
Note: These tests currently PASS, which indicates either:
1. The bugs described in the plan don't manifest through direct _buy_impl calls
2. The bugs only occur when going through ModelDayExecutor._write_results_to_db()
3. The trade tools are working correctly, but ModelDayExecutor creates corrupt records
The tests validate the CORRECT behavior. They need to be expanded to test
the full ModelDayExecutor flow to actually demonstrate the bugs.
Consolidated all unreleased changes into v0.3.0 release dated 2025-11-03.
Key additions:
- Development Mode with mock AI provider
- Config Override System for Docker
- Async Price Download (non-blocking)
- Resume Mode (idempotent execution)
- Reasoning Logs API (GET /reasoning)
- Project rebrand to AI-Trader-Server
Includes comprehensive bug fixes for context injection, simulation
re-runs, database reliability, and configuration handling.
Previously, when re-running a job with some model-days already completed:
- _prepare_data() marked them as "skipped" with error="Already completed"
- But _execute_date() didn't check the skip list before launching executors
- ModelDayExecutor would start, change status to "running", and never complete
- Job would hang with status="running" and pending count > 0
Fixed by:
- _prepare_data() now returns completion_skips: {model: {dates}}
- _execute_date() receives completion_skips and filters out already-completed models
- Skipped model-days are not submitted to ThreadPoolExecutor
- Job completes correctly, skipped model-days remain with status="skipped"
This ensures idempotent job behavior - re-running a job only executes
model-days that haven't completed yet.
Fixes#73
Root cause: FastMCP uses inspect module to generate tool schemas from function
signatures. **kwargs prevents FastMCP from determining parameter types, causing
tool registration to fail.
Fix: Keep explicit parameters with defaults (signature=None, today_date=None, etc.)
but document in docstring that they are auto-injected.
This preserves:
- ContextInjector always overrides values (defense-in-depth from v0.3.0-alpha.40)
- FastMCP can generate proper tool schema
- Parameters visible to AI, but with clear documentation they're automatic
Trade-off: AI can still see the parameters, but documentation instructs not to provide them.
Combined with ContextInjector override, AI-provided values are ignored anyway.
Fixes TradeTools service crash on startup.
Prevent AI hallucination of runtime parameters by hiding them from the tool schema.
Architecture:
- Public tool functions (buy/sell) only expose symbol and amount to AI
- Use **kwargs to accept hidden parameters (signature, job_id, today_date, session_id)
- Internal _impl functions contain the actual business logic
- ContextInjector injects parameters into kwargs (invisible to AI)
Benefits:
- AI cannot see or hallucinate signature/job_id/session_id parameters
- Cleaner tool schema focuses on trading-relevant parameters only
- Defense-in-depth: ContextInjector still overrides any provided values
- More maintainable: clear separation of public API vs internal implementation
Example AI sees:
buy(symbol: str, amount: int) -> dict
Actual execution:
buy(symbol="AAPL", amount=10, signature="gpt-5", job_id="...", ...)
Fixes #TBD
Root cause: AI models were hallucinating signature/job_id/today_date values
and passing them in tool calls. The ContextInjector was checking
"if param not in request.args" before injecting, which failed when AI
provided (incorrect) values.
Fix: Always override context parameters, never trust AI-provided values.
Evidence from logs:
- ContextInjector had correct values (self.signature=gpt-5, job_id=6dabd9e6...)
- But AI was passing signature=None or hallucinated values like "fundamental-bot-v1"
- After injection, args showed the AI's (wrong) values, not the interceptor's
This ensures runtime context is ALWAYS injected regardless of what the AI sends.
Fixes #TBD
Log ContextInjector instance ID and attribute values at entry to __call__()
to diagnose why attributes appear as None during tool invocation despite
being set correctly during set_context().
This will reveal whether:
- Multiple ContextInjector instances exist
- Attributes are being overwritten/cleared
- Wrong instance is being invoked
Add instrumentation at component boundaries to trace where ContextInjector values become None:
- ModelDayExecutor: Log ContextInjector creation and set_context() invocation
- BaseAgent.set_context(): Log entry, client creation, tool reload, completion
- Includes object IDs to verify instance identity across boundaries
Part of systematic debugging investigation for issue #TBD.
Critical fixes for ContextInjector and database concurrency:
1. ContextInjector Not Working:
- Made set_context() async to reload tools after recreating MCP client
- Tools from old client (without interceptor) were still being used
- Now tools are reloaded from new client with interceptor active
- This ensures buy/sell calls properly receive injected parameters
2. Database Locking:
- Closed main connection before _write_results_to_db() opens new one
- SQLite doesn't handle concurrent write connections well
- Prevents "database is locked" error during position writes
Changes:
- agent/base_agent/base_agent.py:
- async def set_context() instead of def set_context()
- Added: self.tools = await self.client.get_tools()
- api/model_day_executor.py:
- await agent.set_context(context_injector)
- conn.close() before _write_results_to_db()
Root Cause:
When recreating the MCP client with tool_interceptors, the old tools
were still cached in self.tools and being passed to the AI agent.
The interceptor was never invoked, so job_id/signature/date were missing.
The get_db_path() function requires a base_db_path argument
to properly resolve PROD vs DEV database paths. Updated all
calls to pass "data/jobs.db" as the base path.
Changes:
- agent_tools/tool_trade.py: Fix 3 occurrences (lines 33, 113, 236)
- tools/price_tools.py: Fix 2 occurrences in new database functions
- Remove unused get_db_path import from tool_trade.py
This fixes TypeError when running simulations:
get_db_path() missing 1 required positional argument: 'base_db_path'
The get_db_connection() function internally calls get_db_path()
to resolve the correct database path based on DEPLOYMENT_MODE.
This commit migrates the system to database-only position storage,
eliminating file-based position.jsonl dependencies and fixing
ContextInjector initialization timing issues.
Key Changes:
1. ContextInjector Lifecycle Refactor:
- Remove ContextInjector creation from BaseAgent.__init__()
- Add BaseAgent.set_context() method for post-initialization injection
- Update ModelDayExecutor to create ContextInjector with correct trading day date
- Ensures ContextInjector receives actual trading date instead of init_date
- Includes session_id injection for proper database linking
2. Database Position Functions:
- Implement get_today_init_position_from_db() for querying previous positions
- Implement add_no_trade_record_to_db() for no-trade day handling
- Both functions query SQLite directly (positions + holdings tables)
- Handle first trading day case with initial cash return
- Include comprehensive error handling and logging
3. System Integration:
- Update get_agent_system_prompt() to use database queries
- Update _handle_trading_result() to write no-trade records to database
- Remove dependencies on position.jsonl file reading/writing
- Use deployment_config for automatic prod/dev database resolution
Data Flow:
- ModelDayExecutor creates runtime config and trading session
- Agent initialized without context
- ContextInjector created with (signature, date, job_id, session_id)
- Context injected via set_context()
- System prompt queries database for yesterday's position
- Trade tools write directly to database
- No-trade handler creates database records
Fixes:
- ContextInjector no longer receives None values
- No FileNotFoundError for missing position.jsonl files
- Database is single source of truth for position tracking
- Session linking maintained across all position records
Design: docs/plans/2025-02-11-database-position-tracking-design.md
Created comprehensive design document addressing:
- ContextInjector initialization timing issues
- Migration from file-based to database-only position tracking
- Complete data flow and integration strategy
- Testing and validation approach
Design resolves two critical simulation failures:
1. ContextInjector receiving None values for trade tool parameters
2. FileNotFoundError when accessing position.jsonl files
Ready for implementation.
Changed action_type from 'init' to 'no_trade' in _initialize_starting_position()
to comply with database CHECK constraint that only allows 'buy', 'sell', or 'no_trade'.
Fixes sqlite3.IntegrityError during position initialization.
Complete rewrite of position management in MCP trade tools:
**Trade Tools (agent_tools/tool_trade.py)**
- Replace file-based position.jsonl reads with SQLite queries
- Add get_current_position_from_db() to query positions and holdings tables
- Rewrite buy() and sell() to write directly to database
- Calculate portfolio value and P&L metrics in tools
- Accept job_id and session_id parameters via ContextInjector
- Return errors with proper context for debugging
- Use deployment-aware database path resolution
**Context Injection (agent/context_injector.py)**
- Add job_id and session_id to constructor
- Inject job_id and session_id into buy/sell tool calls
- Support optional parameters (None in standalone mode)
**BaseAgent (agent/base_agent/base_agent.py)**
- Read JOB_ID from runtime config
- Pass job_id to ContextInjector during initialization
- Enable automatic context injection for API mode
**ModelDayExecutor (api/model_day_executor.py)**
- Add _initialize_starting_position() method
- Create initial position record before agent runs
- Load initial_cash from config
- Update context_injector.session_id after session creation
- Link positions to sessions automatically
**Architecture Changes:**
- Eliminates file-based position tracking entirely
- Single source of truth: SQLite database
- Positions automatically linked to trading sessions
- Concurrent execution safe (no file system conflicts)
- Deployment mode aware (prod vs dev databases)
This completes the migration to database-only position storage.
File-based position.jsonl is no longer used or created.
Fixes context injection errors in concurrent simulations.
Add proper exception handling around get_latest_position() calls in both
buy() and sell() functions. Previously, exceptions were caught but code
continued execution with undefined variables, causing "variable referenced
before assignment" errors.
Now returns error dict with context when position lookup fails.
Related to context injection implementation for concurrent simulations.
The interceptor __call__ method must be async and follow the proper
signature: async __call__(request, handler) -> result
Previous implementation was synchronous and had wrong signature, causing
'object function can't be used in await expression' error.
Resolves issue where MCP trade tools couldn't access SIGNATURE and TODAY_DATE
during concurrent API simulations, causing "SIGNATURE environment variable is
not set" errors.
Problem:
- MCP services run as separate HTTP processes
- Multiple simulations execute concurrently via ThreadPoolExecutor
- Environment variables from executor process not accessible to MCP services
Solution:
- Add ContextInjector that implements ToolCallInterceptor
- Automatically injects signature and today_date into buy/sell tool calls
- Trade tools accept optional parameters, falling back to config/env
- BaseAgent creates interceptor and updates today_date per session
Changes:
- agent/context_injector.py: New interceptor for context injection
- agent/base_agent/base_agent.py: Create and use ContextInjector
- agent_tools/tool_trade.py: Add optional signature/today_date parameters
Benefits:
- Supports concurrent multi-model simulations
- Maintains backward compatibility with CLI mode
- AI model unaware of injected parameters
Remove redundant log file creation for MCP services since output is
already captured by Docker logs. This simplifies deployment by removing
unnecessary volume mounts and file management.
Changes:
- Remove logs/ directory creation from Dockerfile
- Remove logs/ volume mount from docker-compose.yml
- Update start_mcp_services.py to send output to DEVNULL
- Update documentation to reflect changes (DOCKER.md, docs/DOCKER.md)
- Update .env.example to remove logs/ from volume description
Users can still view MCP service output via 'docker logs' or
'docker-compose logs -f'. Trading session logs in data/agent_data/
remain unchanged.
Fixed critical bug where ModelDayExecutor._initialize_agent() created
a BaseAgent but never called agent.initialize(), leaving self.model=None.
This caused 'NoneType' object has no attribute 'bind' error when
run_trading_session() tried to create the langchain agent.
Changes:
- Made _initialize_agent() async
- Added await agent.initialize() call
- Updated call site to await async function
Now properly initializes MCP client, tools, and AI model before
executing trading sessions, matching the working CLI mode pattern.
Fixes startup error 'no such column: session_id' that occurs when
_create_indexes() tries to create indexes on columns that don't exist yet.
The issue occurred when initializing a database from scratch:
1. _migrate_schema() adds session_id column to positions table
2. _create_indexes() tries to create index on session_id
3. But on fresh databases, positions table was created without session_id
4. Migration runs after table creation, before index creation
5. Index creation fails because column doesn't exist yet
Solution: Check if columns exist before creating indexes on them.
This ensures the database can be initialized both:
- Fresh (CREATE TABLE without session_id, then ALTER TABLE, then CREATE INDEX)
- Migrated (ALTER TABLE adds column, then CREATE INDEX)
Tested: All 21 database tests passing
The get_daily_price.py, get_interdaily_price.py, and merge_jsonl.py
scripts don't exist in the repository. Removed the RUN command that
tries to copy these files to /app/scripts/.
This fixes the Docker build failure:
ERROR: process did not complete successfully: exit code: 1
The API server doesn't need these data preparation scripts as it
operates on existing database records.
Complete implementation of reasoning logs retrieval system that
replaces JSONL file-based logging with database-only storage.
Database Changes:
- Add trading_sessions table (one record per model-day)
- Add reasoning_logs table (conversation history with summaries)
- Add session_id column to positions table
- Add indexes for query performance
Agent Changes:
- Add conversation history tracking to BaseAgent
- Add AI-powered summary generation using same model
- Remove JSONL logging code (_log_message, _setup_logging)
- Preserve in-memory conversation tracking
ModelDayExecutor Changes:
- Create trading session at start of execution
- Store reasoning logs with AI-generated summaries
- Update session summary after completion
- Link positions to sessions via session_id
API Changes:
- Add GET /reasoning endpoint with filters (job_id, date, model)
- Support include_full_conversation parameter
- Return both summaries and full conversation on demand
- Include deployment mode info in responses
Documentation:
- Add complete API reference for GET /reasoning
- Add design document with architecture details
- Add implementation guide with step-by-step tasks
- Update Python and TypeScript client examples
Testing:
- Add 6 tests for conversation history tracking
- Add 4 tests for summary generation
- Add 5 tests for model_day_executor integration
- Add 8 tests for GET /reasoning endpoint
- Add 9 integration tests for E2E flow
- Update existing tests for schema changes
All 32 new feature tests passing. Total: 285 tests passing.
- Remove _log_message() and _setup_logging() methods
- Remove all calls to logging methods in run_trading_session()
- Update log_path parameter docstring for clarity
- Update integration test to verify conversation history instead of JSONL files
- Reasoning logs now stored exclusively in database via model_day_executor
- Conversation history tracking preserved in memory
Related: Task 6 of reasoning logs API feature
- Add Pydantic models for reasoning API responses
- Implement GET /reasoning with job_id, date, model filters
- Support include_full_conversation parameter
- Add comprehensive unit tests (8 tests)
- Return deployment mode info in responses
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add _create_trading_session() method to create session records
- Add async _store_reasoning_logs() to store conversation with AI summaries
- Add async _update_session_summary() to generate overall session summary
- Modify execute() -> execute_async() with async workflow
- Add execute_sync() wrapper and keep execute() as sync entry point
- Update _write_results_to_db() to accept and use session_id parameter
- Modify positions INSERT to include session_id foreign key
- Remove old reasoning_logs code block (obsolete schema)
- Add comprehensive unit tests for all new functionality
All tests pass. Session-based reasoning storage now integrated.
- Add conversation_history instance variable to BaseAgent.__init__
- Create _capture_message() method to capture messages with timestamps
- Create get_conversation_history() method to retrieve conversation
- Create clear_conversation_history() method to reset history
- Modify run_trading_session() to capture user prompts and AI responses
- Add comprehensive unit tests for conversation tracking
- Fix datetime deprecation warning by using timezone-aware datetime
All tests pass successfully.
- Fix async call in model_day_executor.py by wrapping with asyncio.run()
Resolves RuntimeWarning where run_trading_session coroutine was never awaited
- Remove register_agent() call in API mode to prevent file-based position storage
Position data is now stored exclusively in SQLite database (jobs.db)
- Update test mocks to use AsyncMock for async run_trading_session method
This fixes production deployment issues:
1. Trading sessions now execute properly (async bug)
2. No position files created, database-only storage
3. All tests pass
Closes issue with no trades being executed in production
The FastAPI app now checks for /tmp/runtime_config.json (created by
entrypoint.sh config merger) before falling back to the default config.
This allows user-provided configs mounted at /app/user-configs/ to
properly override default values in containerized deployments.
Fixes issue where custom user configs were not being applied.
Updated default_config.json to enable all trading models:
- claude-sonnet-4-5
- deepseek-chat-v3.1
- qwen3-max
- gemini-2.5-flash
- gpt-5
This ensures that when models=[] is passed to the API, all 5 models
will run instead of just gpt-5.
Cleaned up all diagnostic print statements added during debugging.
The root cause (non-idempotent get_db_path) has been fixed, so the
extensive instrumentation is no longer needed.
Changes:
- Removed all diagnostic prints from api/main.py (lifespan and module-level)
- Removed all diagnostic prints from api/database.py (get_db_connection and initialize_dev_database)
- Kept essential user-facing messages (PRESERVE_DEV_DATA notice, database creation messages)
All 28 integration tests pass.
Root Cause:
The get_db_path() function was being called multiple times in the
initialization chain, causing recursive suffix addition:
data/jobs.db -> data/jobs_dev.db -> data/jobs_dev_dev.db
This resulted in tables being created in data/jobs_dev_dev.db while
the application tried to access data/jobs_dev.db (empty database).
Fix:
Added idempotency check in get_db_path() to detect and skip
transformation if path already contains "_dev.db" suffix.
Evidence from Diagnostics:
- alpha.20 logs showed: Input='data/jobs_dev.db', Resolved='data/jobs_dev_dev.db'
- Tables were created in jobs_dev_dev.db but accessed from jobs_dev.db
- This caused "no such table: jobs" errors despite successful initialization
All 28 integration tests pass with this fix.
Fixes#6
Enhanced diagnostics to trace database path resolution and table existence
at connection time. This will help identify if get_db_connection() is
resolving paths correctly and accessing the right database file.
Added diagnostics to:
- get_db_connection(): Show input path, resolved path, file existence, and tables found
- initialize_dev_database(): Verify tables exist after creation
This will reveal whether the path resolution is working correctly or if
there's a timing/caching issue with database file access.
Following systematic debugging methodology after 5 failed fix attempts.
Adding extensive print-based diagnostics to trace execution flow in Docker.
Instrumentation added to:
- api/main.py: Module import, app creation, lifespan function, module-level init
- api/database.py: initialize_dev_database() entry/exit and decision points
This diagnostic version will help identify:
1. Whether module-level code executes in Docker
2. Which initialization layer is failing
3. Database paths being resolved
4. Environment variable values
Tests confirmed passing with diagnostic logging.
