mirror of
https://github.com/Xe138/AI-Trader.git
synced 2026-04-02 01:27:24 -04:00
Compare commits
12 Commits
v0.3.0-alp
...
v0.3.1
| Author | SHA1 | Date | |
|---|---|---|---|
| 4c30478520 | |||
| 090875d6f2 | |||
| 0669bd1bab | |||
| fe86dceeac | |||
| 923cdec5ca | |||
| 84320ab8a5 | |||
| 9be14a1602 | |||
| 6cb56f85ec | |||
| c47798d3c3 | |||
| 179cbda67b | |||
| 1095798320 | |||
| e590cdc13b |
107
CHANGELOG.md
107
CHANGELOG.md
@@ -7,13 +7,66 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
- **Dev Mode Warning in Docker** - DEV mode startup warning now displays correctly in Docker logs
|
||||
- Added FastAPI `@app.on_event("startup")` handler to trigger warning on API server startup
|
||||
- Previously only appeared when running `python api/main.py` directly (not via uvicorn)
|
||||
- Docker compose now includes `DEPLOYMENT_MODE` and `PRESERVE_DEV_DATA` environment variables
|
||||
## [0.3.1] - 2025-11-03
|
||||
|
||||
## [0.3.0] - 2025-10-31
|
||||
### Fixed
|
||||
- **Critical:** Fixed position tracking bugs causing cash reset and positions lost over weekends
|
||||
- Removed redundant `ModelDayExecutor._write_results_to_db()` that created corrupt records with cash=0 and holdings=[]
|
||||
- Fixed profit calculation to compare against start-of-day portfolio value instead of previous day's final value
|
||||
- Positions now correctly carry over between trading days and across weekends
|
||||
- Profit/loss calculations now accurately reflect trading gains/losses without treating trades as losses
|
||||
|
||||
### Changed
|
||||
- Position tracking now exclusively handled by trade tools (`buy()`, `sell()`) and `add_no_trade_record_to_db()`
|
||||
- Daily profit calculation compares to start-of-day (action_id=0) portfolio value for accurate P&L tracking
|
||||
|
||||
### Added
|
||||
- Standardized testing scripts for different workflows:
|
||||
- `scripts/test.sh` - Interactive menu for all testing operations
|
||||
- `scripts/quick_test.sh` - Fast unit test feedback (~10-30s)
|
||||
- `scripts/run_tests.sh` - Main test runner with full configuration options
|
||||
- `scripts/coverage_report.sh` - Coverage analysis with HTML/JSON/terminal reports
|
||||
- `scripts/ci_test.sh` - CI/CD optimized testing with JUnit/coverage XML output
|
||||
- Comprehensive testing documentation in `docs/developer/testing.md`
|
||||
- Test coverage requirement: 85% minimum (currently at 89.86%)
|
||||
|
||||
## [0.3.0] - 2025-11-03
|
||||
|
||||
### Added - Development & Testing Features
|
||||
- **Development Mode** - Mock AI provider for cost-free testing
|
||||
- `DEPLOYMENT_MODE=DEV` enables mock AI responses with deterministic stock rotation
|
||||
- Isolated dev database (`trading_dev.db`) separate from production data
|
||||
- `PRESERVE_DEV_DATA=true` option to prevent dev database reset on startup
|
||||
- No AI API costs during development and testing
|
||||
- All API responses include `deployment_mode` field
|
||||
- Startup warning displayed when running in DEV mode
|
||||
- **Config Override System** - Docker configuration merging
|
||||
- Place custom configs in `user-configs/` directory
|
||||
- Startup merges user config with default config
|
||||
- Comprehensive validation with clear error messages
|
||||
- Volume mount: `./user-configs:/app/user-configs`
|
||||
|
||||
### Added - Enhanced API Features
|
||||
- **Async Price Download** - Non-blocking data preparation
|
||||
- `POST /simulate/trigger` no longer blocks on price downloads
|
||||
- New job status: `downloading_data` during data preparation
|
||||
- Warnings field in status response for download issues
|
||||
- Better user experience for large date ranges
|
||||
- **Resume Mode** - Idempotent simulation execution
|
||||
- Jobs automatically skip already-completed model-days
|
||||
- Safe to re-run jobs without duplicating work
|
||||
- `status="skipped"` for already-completed executions
|
||||
- Error-free job completion when partial results exist
|
||||
- **Reasoning Logs API** - Access AI decision-making history
|
||||
- `GET /reasoning` endpoint for querying reasoning logs
|
||||
- Filter by job_id, model_name, date, include_full_conversation
|
||||
- Includes conversation history and tool usage
|
||||
- Database-only storage (no JSONL files)
|
||||
- AI-powered summary generation for reasoning sessions
|
||||
- **Job Skip Status** - Enhanced job status tracking
|
||||
- New status: `skipped` for already-completed model-days
|
||||
- Better differentiation between pending, running, and skipped
|
||||
- Accurate job completion detection
|
||||
|
||||
### Added - Price Data Management & On-Demand Downloads
|
||||
- **SQLite Price Data Storage** - Replaced JSONL files with relational database
|
||||
@@ -83,13 +136,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- Windmill integration patterns and examples
|
||||
|
||||
### Changed
|
||||
- **Project Rebrand** - AI-Trader renamed to AI-Trader-Server
|
||||
- Updated all documentation for new project name
|
||||
- Updated Docker images to ghcr.io/xe138/ai-trader-server
|
||||
- Updated GitHub Actions workflows
|
||||
- Updated README, CHANGELOG, and all user guides
|
||||
- **Architecture** - Transformed from batch-only to API-first service with database persistence
|
||||
- **Data Storage** - Migrated from JSONL files to SQLite relational database
|
||||
- Price data now stored in `price_data` table instead of `merged.jsonl`
|
||||
- Tools/price_tools.py updated to query database
|
||||
- Position data remains in database (already migrated in earlier versions)
|
||||
- Position data fully migrated to database-only storage (removed JSONL dependencies)
|
||||
- Trade tools now read/write from database tables with lazy context injection
|
||||
- **Deployment** - Simplified to single API-only Docker service (REST API is new in v0.3.0)
|
||||
- **Logging** - Removed duplicate MCP service log files for cleaner output
|
||||
- **Configuration** - Simplified environment variable configuration
|
||||
- **Added:** `DEPLOYMENT_MODE` (PROD/DEV) for environment control
|
||||
- **Added:** `PRESERVE_DEV_DATA` (default: false) to keep dev data between runs
|
||||
- **Added:** `AUTO_DOWNLOAD_PRICE_DATA` (default: true) - Enable on-demand downloads
|
||||
- **Added:** `MAX_SIMULATION_DAYS` (default: 30) - Maximum date range size
|
||||
- **Added:** `API_PORT` for host port mapping (default: 8080, customizable for port conflicts)
|
||||
@@ -137,6 +199,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- **Monitoring** - Health checks and status tracking
|
||||
- **Persistence** - SQLite database survives container restarts
|
||||
|
||||
### Fixed
|
||||
- **Context Injection** - Runtime parameters correctly injected into MCP tools
|
||||
- ContextInjector always overrides AI-provided parameters (defense-in-depth)
|
||||
- Hidden context parameters from AI tool schema to prevent hallucination
|
||||
- Resolved database locking issues with concurrent tool calls
|
||||
- Proper async handling of tool reloading after context injection
|
||||
- **Simulation Re-runs** - Prevent duplicate execution of completed model-days
|
||||
- Fixed job hanging when re-running partially completed simulations
|
||||
- `_execute_date()` now skips already-completed model-days
|
||||
- Job completion status correctly reflects skipped items
|
||||
- **Agent Initialization** - Correct parameter passing in API mode
|
||||
- Fixed BaseAgent initialization parameters in ModelDayExecutor
|
||||
- Resolved async execution and position storage issues
|
||||
- **Database Reliability** - Various improvements for concurrent access
|
||||
- Fixed column existence checks before creating indexes
|
||||
- Proper database path resolution in dev mode (prevents recursive _dev suffix)
|
||||
- Module-level database initialization for uvicorn reliability
|
||||
- Fixed database locking during concurrent writes
|
||||
- Improved error handling in buy/sell functions
|
||||
- **Configuration** - Improved config handling
|
||||
- Use enabled field from config to determine which models run
|
||||
- Use config models when empty models list provided
|
||||
- Correct handling of merged runtime configs in containers
|
||||
- Proper get_db_path() usage to pass base database path
|
||||
- **Docker** - Various deployment improvements
|
||||
- Removed non-existent data scripts from Dockerfile
|
||||
- Proper respect for dev mode in entrypoint database initialization
|
||||
- Correct closure usage to capture db_path in lifespan context manager
|
||||
|
||||
### Breaking Changes
|
||||
- **Batch Mode Removed** - All simulations now run through REST API
|
||||
- v0.2.0 used sequential batch execution via Docker entrypoint
|
||||
@@ -147,7 +238,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- `merged.jsonl` no longer used (replaced by `price_data` table)
|
||||
- Automatic on-demand downloads eliminate need for manual data fetching
|
||||
- **Configuration Variables Changed**
|
||||
- Added: `AUTO_DOWNLOAD_PRICE_DATA`, `MAX_SIMULATION_DAYS`, `API_PORT`
|
||||
- Added: `DEPLOYMENT_MODE`, `PRESERVE_DEV_DATA`, `AUTO_DOWNLOAD_PRICE_DATA`, `MAX_SIMULATION_DAYS`, `API_PORT`
|
||||
- Removed: `RUNTIME_ENV_PATH`, MCP service ports, `WEB_HTTP_PORT`
|
||||
- MCP services now use fixed internal ports (not exposed to host)
|
||||
|
||||
|
||||
@@ -1,265 +0,0 @@
|
||||
# API Schema Update - Resume Mode & Idempotent Behavior
|
||||
|
||||
## Summary
|
||||
|
||||
Updated the `/simulate/trigger` endpoint to support three new use cases:
|
||||
1. **Resume mode**: Continue simulations from last completed date per model
|
||||
2. **Idempotent behavior**: Skip already-completed dates by default
|
||||
3. **Explicit date ranges**: Clearer API contract with required `end_date`
|
||||
|
||||
## Breaking Changes
|
||||
|
||||
### Request Schema
|
||||
|
||||
**Before:**
|
||||
```json
|
||||
{
|
||||
"start_date": "2025-10-01", // Required
|
||||
"end_date": "2025-10-02", // Optional (defaulted to start_date)
|
||||
"models": ["gpt-5"] // Optional
|
||||
}
|
||||
```
|
||||
|
||||
**After:**
|
||||
```json
|
||||
{
|
||||
"start_date": "2025-10-01", // Optional (null for resume mode)
|
||||
"end_date": "2025-10-02", // REQUIRED (cannot be null/empty)
|
||||
"models": ["gpt-5"], // Optional
|
||||
"replace_existing": false // NEW: Optional (default: false)
|
||||
}
|
||||
```
|
||||
|
||||
### Key Changes
|
||||
|
||||
1. **`end_date` is now REQUIRED**
|
||||
- Cannot be `null` or empty string
|
||||
- Must always be provided
|
||||
- For single-day simulation, set `start_date` == `end_date`
|
||||
|
||||
2. **`start_date` is now OPTIONAL**
|
||||
- Can be `null` or omitted to enable resume mode
|
||||
- When `null`, each model resumes from its last completed date
|
||||
- If no data exists (cold start), uses `end_date` as single-day simulation
|
||||
|
||||
3. **NEW `replace_existing` field**
|
||||
- `false` (default): Skip already-completed model-days (idempotent)
|
||||
- `true`: Re-run all dates even if previously completed
|
||||
|
||||
## Use Cases
|
||||
|
||||
### 1. Explicit Date Range
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/simulate/trigger \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"start_date": "2025-10-01",
|
||||
"end_date": "2025-10-31",
|
||||
"models": ["gpt-5"]
|
||||
}'
|
||||
```
|
||||
|
||||
### 2. Single Date
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/simulate/trigger \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"start_date": "2025-10-15",
|
||||
"end_date": "2025-10-15",
|
||||
"models": ["gpt-5"]
|
||||
}'
|
||||
```
|
||||
|
||||
### 3. Resume Mode (NEW)
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/simulate/trigger \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"start_date": null,
|
||||
"end_date": "2025-10-31",
|
||||
"models": ["gpt-5"]
|
||||
}'
|
||||
```
|
||||
|
||||
**Behavior:**
|
||||
- Model "gpt-5" last completed: `2025-10-15`
|
||||
- Will simulate: `2025-10-16` through `2025-10-31`
|
||||
- If no data exists: Will simulate only `2025-10-31`
|
||||
|
||||
### 4. Idempotent Simulation (NEW)
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/simulate/trigger \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"start_date": "2025-10-01",
|
||||
"end_date": "2025-10-31",
|
||||
"models": ["gpt-5"],
|
||||
"replace_existing": false
|
||||
}'
|
||||
```
|
||||
|
||||
**Behavior:**
|
||||
- Checks database for already-completed dates
|
||||
- Only simulates dates that haven't been completed yet
|
||||
- Returns error if all dates already completed
|
||||
|
||||
### 5. Force Replace
|
||||
```bash
|
||||
curl -X POST http://localhost:8080/simulate/trigger \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"start_date": "2025-10-01",
|
||||
"end_date": "2025-10-31",
|
||||
"models": ["gpt-5"],
|
||||
"replace_existing": true
|
||||
}'
|
||||
```
|
||||
|
||||
**Behavior:**
|
||||
- Re-runs all dates regardless of completion status
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Files Modified
|
||||
|
||||
1. **`api/main.py`**
|
||||
- Updated `SimulateTriggerRequest` Pydantic model
|
||||
- Added validators for `end_date` (required)
|
||||
- Added validators for `start_date` (optional, can be null)
|
||||
- Added resume logic per model
|
||||
- Added idempotent filtering logic
|
||||
- Fixed bug with `start_date=None` in price data checks
|
||||
|
||||
2. **`api/job_manager.py`**
|
||||
- Added `get_last_completed_date_for_model(model)` method
|
||||
- Added `get_completed_model_dates(models, start_date, end_date)` method
|
||||
- Updated `create_job()` to accept `model_day_filter` parameter
|
||||
|
||||
3. **`tests/integration/test_api_endpoints.py`**
|
||||
- Updated all tests to use new schema
|
||||
- Added tests for resume mode
|
||||
- Added tests for idempotent behavior
|
||||
- Added tests for validation rules
|
||||
|
||||
4. **Documentation Updated**
|
||||
- `API_REFERENCE.md` - Complete API documentation with examples
|
||||
- `QUICK_START.md` - Updated getting started examples
|
||||
- `docs/user-guide/using-the-api.md` - Updated user guide
|
||||
- Client library examples (Python, TypeScript)
|
||||
|
||||
### Database Schema
|
||||
|
||||
No changes to database schema. New functionality uses existing tables:
|
||||
- `job_details` table tracks completion status per model-day
|
||||
- Unique index on `(job_id, date, model)` ensures no duplicates
|
||||
|
||||
### Per-Model Independence
|
||||
|
||||
Each model maintains its own completion state:
|
||||
```
|
||||
Model A: last_completed_date = 2025-10-15
|
||||
Model B: last_completed_date = 2025-10-10
|
||||
|
||||
Request: start_date=null, end_date=2025-10-31
|
||||
|
||||
Result:
|
||||
- Model A simulates: 2025-10-16 through 2025-10-31 (16 days)
|
||||
- Model B simulates: 2025-10-11 through 2025-10-31 (21 days)
|
||||
```
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### For API Clients
|
||||
|
||||
**Old Code:**
|
||||
```python
|
||||
# Single day (old)
|
||||
client.trigger_simulation(start_date="2025-10-15")
|
||||
```
|
||||
|
||||
**New Code:**
|
||||
```python
|
||||
# Single day (new) - MUST provide end_date
|
||||
client.trigger_simulation(start_date="2025-10-15", end_date="2025-10-15")
|
||||
|
||||
# Or use resume mode
|
||||
client.trigger_simulation(start_date=None, end_date="2025-10-31")
|
||||
```
|
||||
|
||||
### Validation Changes
|
||||
|
||||
**Will Now Fail:**
|
||||
```json
|
||||
{
|
||||
"start_date": "2025-10-01",
|
||||
"end_date": "" // ❌ Empty string rejected
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"start_date": "2025-10-01",
|
||||
"end_date": null // ❌ Null rejected
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"start_date": "2025-10-01" // ❌ Missing end_date
|
||||
}
|
||||
```
|
||||
|
||||
**Will Work:**
|
||||
```json
|
||||
{
|
||||
"end_date": "2025-10-31" // ✓ start_date omitted = resume mode
|
||||
}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"start_date": null,
|
||||
"end_date": "2025-10-31" // ✓ Explicit null = resume mode
|
||||
}
|
||||
```
|
||||
|
||||
## Benefits
|
||||
|
||||
1. **Daily Automation**: Resume mode perfect for cron jobs
|
||||
- No need to calculate "yesterday's date"
|
||||
- Just provide today as end_date
|
||||
|
||||
2. **Idempotent by Default**: Safe to re-run
|
||||
- Accidentally trigger same date? No problem, it's skipped
|
||||
- Explicit `replace_existing=true` when you want to re-run
|
||||
|
||||
3. **Per-Model Independence**: Flexible deployment
|
||||
- Can add new models without re-running old ones
|
||||
- Models can progress at different rates
|
||||
|
||||
4. **Clear API Contract**: No ambiguity
|
||||
- `end_date` always required
|
||||
- `start_date=null` clearly means "resume"
|
||||
- Default behavior is safe (idempotent)
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
⚠️ **This is a BREAKING CHANGE** for clients that:
|
||||
- Rely on `end_date` defaulting to `start_date`
|
||||
- Don't explicitly provide `end_date`
|
||||
|
||||
**Migration:** Update all API calls to explicitly provide `end_date`.
|
||||
|
||||
## Testing
|
||||
|
||||
Run integration tests:
|
||||
```bash
|
||||
pytest tests/integration/test_api_endpoints.py -v
|
||||
```
|
||||
|
||||
All tests updated to cover:
|
||||
- Single-day simulation
|
||||
- Date ranges
|
||||
- Resume mode (cold start and with existing data)
|
||||
- Idempotent behavior
|
||||
- Validation rules
|
||||
56
CLAUDE.md
56
CLAUDE.md
@@ -327,6 +327,55 @@ DEPLOYMENT_MODE=DEV python main.py configs/default_config.json
|
||||
|
||||
## Testing Changes
|
||||
|
||||
### Automated Test Scripts
|
||||
|
||||
The project includes standardized test scripts for different workflows:
|
||||
|
||||
```bash
|
||||
# Quick feedback during development (unit tests only, ~10-30 seconds)
|
||||
bash scripts/quick_test.sh
|
||||
|
||||
# Full test suite with coverage (before commits/PRs)
|
||||
bash scripts/run_tests.sh
|
||||
|
||||
# Generate coverage report with HTML output
|
||||
bash scripts/coverage_report.sh -o
|
||||
|
||||
# CI/CD optimized testing (for automation)
|
||||
bash scripts/ci_test.sh -f -m 85
|
||||
|
||||
# Interactive menu (recommended for beginners)
|
||||
bash scripts/test.sh
|
||||
```
|
||||
|
||||
**Common test script options:**
|
||||
```bash
|
||||
# Run only unit tests
|
||||
bash scripts/run_tests.sh -t unit
|
||||
|
||||
# Run with custom markers
|
||||
bash scripts/run_tests.sh -m "unit and not slow"
|
||||
|
||||
# Fail fast on first error
|
||||
bash scripts/run_tests.sh -f
|
||||
|
||||
# Run tests in parallel
|
||||
bash scripts/run_tests.sh -p
|
||||
|
||||
# Skip coverage reporting (faster)
|
||||
bash scripts/run_tests.sh -n
|
||||
```
|
||||
|
||||
**Available test markers:**
|
||||
- `unit` - Fast, isolated unit tests
|
||||
- `integration` - Tests with real dependencies
|
||||
- `e2e` - End-to-end tests (requires Docker)
|
||||
- `slow` - Tests taking >10 seconds
|
||||
- `performance` - Performance benchmarks
|
||||
- `security` - Security tests
|
||||
|
||||
### Manual Testing Workflow
|
||||
|
||||
When modifying agent behavior or adding tools:
|
||||
1. Create test config with short date range (2-3 days)
|
||||
2. Set `max_steps` low (e.g., 10) to iterate faster
|
||||
@@ -334,6 +383,13 @@ When modifying agent behavior or adding tools:
|
||||
4. Verify position updates in `position/position.jsonl`
|
||||
5. Use `main.sh` only for full end-to-end testing
|
||||
|
||||
### Test Coverage
|
||||
|
||||
- **Minimum coverage:** 85%
|
||||
- **Target coverage:** 90%
|
||||
- **Configuration:** `pytest.ini`
|
||||
- **Coverage reports:** `htmlcov/index.html`, `coverage.xml`, terminal output
|
||||
|
||||
See [docs/developer/testing.md](docs/developer/testing.md) for complete testing guide.
|
||||
|
||||
## Documentation Structure
|
||||
|
||||
44
ROADMAP.md
44
ROADMAP.md
@@ -61,6 +61,15 @@ curl -X POST http://localhost:5000/simulate/to-date \
|
||||
|
||||
**Focus:** Comprehensive testing, documentation, and production readiness
|
||||
|
||||
#### API Consolidation & Improvements
|
||||
- **Endpoint Refactoring** - Simplify API surface before v1.0
|
||||
- Merge results and reasoning endpoints:
|
||||
- Current: `/jobs/{job_id}/results` and `/jobs/{job_id}/reasoning/{model_name}` are separate
|
||||
- Consolidated: Single endpoint with query parameters to control response
|
||||
- `/jobs/{job_id}/results?include_reasoning=true&model=<model_name>`
|
||||
- Benefits: Fewer endpoints, more consistent API design, easier to use
|
||||
- Maintains backward compatibility with legacy endpoints (deprecated but functional)
|
||||
|
||||
#### Testing & Validation
|
||||
- **Comprehensive Test Suite** - Full coverage of core functionality
|
||||
- Unit tests for all agent components
|
||||
@@ -93,10 +102,37 @@ curl -X POST http://localhost:5000/simulate/to-date \
|
||||
- File system error handling (disk full, permission errors)
|
||||
- Comprehensive error messages with troubleshooting guidance
|
||||
- Logging improvements:
|
||||
- Structured logging with consistent format
|
||||
- Log rotation and size management
|
||||
- Error classification (user error vs. system error)
|
||||
- Debug mode for detailed diagnostics
|
||||
- **Configurable Log Levels** - Environment-based logging control
|
||||
- `LOG_LEVEL` environment variable (DEBUG, INFO, WARNING, ERROR, CRITICAL)
|
||||
- Per-component log level configuration (API, agents, MCP tools, database)
|
||||
- Default production level: INFO, development level: DEBUG
|
||||
- **Structured Logging** - Consistent, parseable log format
|
||||
- JSON-formatted logs option for production (machine-readable)
|
||||
- Human-readable format for development
|
||||
- Consistent fields: timestamp, level, component, message, context
|
||||
- Correlation IDs for request tracing across components
|
||||
- **Log Clarity & Organization** - Improve log readability
|
||||
- Clear log prefixes per component: `[API]`, `[AGENT]`, `[MCP]`, `[DB]`
|
||||
- Reduce noise: consolidate repetitive messages, rate-limit verbose logs
|
||||
- Action-oriented messages: "Starting simulation job_id=123" vs "Job started"
|
||||
- Include relevant context: model name, date, symbols in trading logs
|
||||
- Progress indicators for long operations (e.g., "Processing date 15/30")
|
||||
- **Log Rotation & Management** - Prevent disk space issues
|
||||
- Automatic log rotation by size (default: 10MB per file)
|
||||
- Retention policy (default: 30 days)
|
||||
- Separate log files per component (api.log, agents.log, mcp.log)
|
||||
- Archive old logs with compression
|
||||
- **Error Classification** - Distinguish error types
|
||||
- User errors (invalid input, configuration issues): WARN level
|
||||
- System errors (API failures, database errors): ERROR level
|
||||
- Critical failures (MCP service down, data corruption): CRITICAL level
|
||||
- Include error codes for programmatic handling
|
||||
- **Debug Mode** - Enhanced diagnostics for troubleshooting
|
||||
- `DEBUG=true` environment variable
|
||||
- Detailed request/response logging (sanitize API keys)
|
||||
- MCP tool call/response logging with timing
|
||||
- Database query logging with execution time
|
||||
- Memory and resource usage tracking
|
||||
|
||||
#### Performance & Scalability
|
||||
- **Performance Optimization** - Ensure efficient resource usage
|
||||
|
||||
@@ -141,20 +141,25 @@ def _buy_impl(symbol: str, amount: int, signature: str = None, today_date: str =
|
||||
except KeyError:
|
||||
pass # Symbol price not available, skip
|
||||
|
||||
# Get previous portfolio value for P&L calculation
|
||||
# Get start-of-day portfolio value (action_id=0 for today) for P&L calculation
|
||||
cursor.execute("""
|
||||
SELECT portfolio_value
|
||||
FROM positions
|
||||
WHERE job_id = ? AND model = ? AND date < ?
|
||||
ORDER BY date DESC, action_id DESC
|
||||
WHERE job_id = ? AND model = ? AND date = ? AND action_id = 0
|
||||
LIMIT 1
|
||||
""", (job_id, signature, today_date))
|
||||
|
||||
row = cursor.fetchone()
|
||||
previous_value = row[0] if row else 10000.0 # Default initial value
|
||||
|
||||
daily_profit = portfolio_value - previous_value
|
||||
daily_return_pct = (daily_profit / previous_value * 100) if previous_value > 0 else 0
|
||||
if row:
|
||||
# Compare to start of day (action_id=0)
|
||||
start_of_day_value = row[0]
|
||||
daily_profit = portfolio_value - start_of_day_value
|
||||
daily_return_pct = (daily_profit / start_of_day_value * 100) if start_of_day_value > 0 else 0
|
||||
else:
|
||||
# First action of first day - no baseline yet
|
||||
daily_profit = 0.0
|
||||
daily_return_pct = 0.0
|
||||
|
||||
# Step 6: Write to positions table
|
||||
created_at = datetime.utcnow().isoformat() + "Z"
|
||||
@@ -284,20 +289,25 @@ def _sell_impl(symbol: str, amount: int, signature: str = None, today_date: str
|
||||
except KeyError:
|
||||
pass
|
||||
|
||||
# Get previous portfolio value
|
||||
# Get start-of-day portfolio value (action_id=0 for today) for P&L calculation
|
||||
cursor.execute("""
|
||||
SELECT portfolio_value
|
||||
FROM positions
|
||||
WHERE job_id = ? AND model = ? AND date < ?
|
||||
ORDER BY date DESC, action_id DESC
|
||||
WHERE job_id = ? AND model = ? AND date = ? AND action_id = 0
|
||||
LIMIT 1
|
||||
""", (job_id, signature, today_date))
|
||||
|
||||
row = cursor.fetchone()
|
||||
previous_value = row[0] if row else 10000.0
|
||||
|
||||
daily_profit = portfolio_value - previous_value
|
||||
daily_return_pct = (daily_profit / previous_value * 100) if previous_value > 0 else 0
|
||||
if row:
|
||||
# Compare to start of day (action_id=0)
|
||||
start_of_day_value = row[0]
|
||||
daily_profit = portfolio_value - start_of_day_value
|
||||
daily_return_pct = (daily_profit / start_of_day_value * 100) if start_of_day_value > 0 else 0
|
||||
else:
|
||||
# First action of first day - no baseline yet
|
||||
daily_profit = 0.0
|
||||
daily_return_pct = 0.0
|
||||
|
||||
# Step 6: Write to positions table
|
||||
created_at = datetime.utcnow().isoformat() + "Z"
|
||||
|
||||
@@ -713,6 +713,14 @@ def create_app(
|
||||
Returns:
|
||||
Health status and timestamp
|
||||
"""
|
||||
from tools.deployment_config import is_dev_mode
|
||||
|
||||
# Log at DEBUG in dev mode, INFO in prod mode
|
||||
if is_dev_mode():
|
||||
logger.debug("Health check")
|
||||
else:
|
||||
logger.info("Health check")
|
||||
|
||||
try:
|
||||
# Test database connection
|
||||
conn = get_db_connection(app.state.db_path)
|
||||
|
||||
@@ -158,13 +158,13 @@ class ModelDayExecutor:
|
||||
# Update session summary
|
||||
await self._update_session_summary(cursor, session_id, conversation, agent)
|
||||
|
||||
# Commit and close connection before _write_results_to_db opens a new one
|
||||
# Commit and close connection
|
||||
conn.commit()
|
||||
conn.close()
|
||||
conn = None # Mark as closed
|
||||
|
||||
# Store positions (pass session_id) - this opens its own connection
|
||||
self._write_results_to_db(agent, session_id)
|
||||
# Note: Positions are written by trade tools (buy/sell) or no_trade_record
|
||||
# No need to write positions here - that was creating duplicate/corrupt records
|
||||
|
||||
# Update status to completed
|
||||
self.job_manager.update_job_detail_status(
|
||||
@@ -431,127 +431,3 @@ class ModelDayExecutor:
|
||||
total_messages = ?
|
||||
WHERE id = ?
|
||||
""", (session_summary, completed_at, len(conversation), session_id))
|
||||
|
||||
def _write_results_to_db(self, agent, session_id: int) -> None:
|
||||
"""
|
||||
Write execution results to SQLite.
|
||||
|
||||
Args:
|
||||
agent: Trading agent instance
|
||||
session_id: Trading session ID (for linking positions)
|
||||
|
||||
Writes to:
|
||||
- positions: Position record with action and P&L (linked to session)
|
||||
- holdings: Current portfolio holdings
|
||||
- tool_usage: Tool usage stats (if available)
|
||||
"""
|
||||
conn = get_db_connection(self.db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
try:
|
||||
# Get current positions and trade info
|
||||
positions = agent.get_positions() if hasattr(agent, 'get_positions') else {}
|
||||
last_trade = agent.get_last_trade() if hasattr(agent, 'get_last_trade') else None
|
||||
|
||||
# Calculate portfolio value
|
||||
current_prices = agent.get_current_prices() if hasattr(agent, 'get_current_prices') else {}
|
||||
total_value = self._calculate_portfolio_value(positions, current_prices)
|
||||
|
||||
# Get previous value for P&L calculation
|
||||
cursor.execute("""
|
||||
SELECT portfolio_value
|
||||
FROM positions
|
||||
WHERE job_id = ? AND model = ? AND date < ?
|
||||
ORDER BY date DESC
|
||||
LIMIT 1
|
||||
""", (self.job_id, self.model_sig, self.date))
|
||||
|
||||
row = cursor.fetchone()
|
||||
previous_value = row[0] if row else 10000.0 # Initial portfolio value
|
||||
|
||||
daily_profit = total_value - previous_value
|
||||
daily_return_pct = (daily_profit / previous_value * 100) if previous_value > 0 else 0
|
||||
|
||||
# Determine action_id (sequence number for this model)
|
||||
cursor.execute("""
|
||||
SELECT COALESCE(MAX(action_id), 0) + 1
|
||||
FROM positions
|
||||
WHERE job_id = ? AND model = ?
|
||||
""", (self.job_id, self.model_sig))
|
||||
|
||||
action_id = cursor.fetchone()[0]
|
||||
|
||||
# Insert position record
|
||||
action_type = last_trade.get("action") if last_trade else "no_trade"
|
||||
symbol = last_trade.get("symbol") if last_trade else None
|
||||
amount = last_trade.get("amount") if last_trade else None
|
||||
price = last_trade.get("price") if last_trade else None
|
||||
cash = positions.get("CASH", 0.0)
|
||||
|
||||
from datetime import datetime
|
||||
created_at = datetime.utcnow().isoformat() + "Z"
|
||||
|
||||
cursor.execute("""
|
||||
INSERT INTO positions (
|
||||
job_id, date, model, action_id, action_type, symbol,
|
||||
amount, price, cash, portfolio_value, daily_profit, daily_return_pct,
|
||||
session_id, created_at
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""", (
|
||||
self.job_id, self.date, self.model_sig, action_id, action_type,
|
||||
symbol, amount, price, cash, total_value,
|
||||
daily_profit, daily_return_pct, session_id, created_at
|
||||
))
|
||||
|
||||
position_id = cursor.lastrowid
|
||||
|
||||
# Insert holdings
|
||||
for symbol, quantity in positions.items():
|
||||
cursor.execute("""
|
||||
INSERT INTO holdings (position_id, symbol, quantity)
|
||||
VALUES (?, ?, ?)
|
||||
""", (position_id, symbol, float(quantity)))
|
||||
|
||||
# Insert tool usage (if available)
|
||||
if hasattr(agent, 'get_tool_usage') and hasattr(agent, 'get_tool_usage'):
|
||||
tool_usage = agent.get_tool_usage()
|
||||
for tool_name, count in tool_usage.items():
|
||||
cursor.execute("""
|
||||
INSERT INTO tool_usage (
|
||||
job_id, date, model, tool_name, call_count
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?)
|
||||
""", (self.job_id, self.date, self.model_sig, tool_name, count))
|
||||
|
||||
conn.commit()
|
||||
logger.debug(f"Wrote results to DB for {self.model_sig} on {self.date}")
|
||||
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
def _calculate_portfolio_value(
|
||||
self,
|
||||
positions: Dict[str, float],
|
||||
current_prices: Dict[str, float]
|
||||
) -> float:
|
||||
"""
|
||||
Calculate total portfolio value.
|
||||
|
||||
Args:
|
||||
positions: Current holdings (symbol: quantity)
|
||||
current_prices: Current market prices (symbol: price)
|
||||
|
||||
Returns:
|
||||
Total portfolio value in dollars
|
||||
"""
|
||||
total = 0.0
|
||||
|
||||
for symbol, quantity in positions.items():
|
||||
if symbol == "CASH":
|
||||
total += quantity
|
||||
else:
|
||||
price = current_prices.get(symbol, 0.0)
|
||||
total += quantity * price
|
||||
|
||||
return total
|
||||
|
||||
@@ -90,7 +90,7 @@ class SimulationWorker:
|
||||
logger.info(f"Starting job {self.job_id}: {len(date_range)} dates, {len(models)} models")
|
||||
|
||||
# NEW: Prepare price data (download if needed)
|
||||
available_dates, warnings = self._prepare_data(date_range, models, config_path)
|
||||
available_dates, warnings, completion_skips = self._prepare_data(date_range, models, config_path)
|
||||
|
||||
if not available_dates:
|
||||
error_msg = "No trading dates available after price data preparation"
|
||||
@@ -100,7 +100,7 @@ class SimulationWorker:
|
||||
# Execute available dates only
|
||||
for date in available_dates:
|
||||
logger.info(f"Processing date {date} with {len(models)} models")
|
||||
self._execute_date(date, models, config_path)
|
||||
self._execute_date(date, models, config_path, completion_skips)
|
||||
|
||||
# Job completed - determine final status
|
||||
progress = self.job_manager.get_job_progress(self.job_id)
|
||||
@@ -145,7 +145,8 @@ class SimulationWorker:
|
||||
"error": error_msg
|
||||
}
|
||||
|
||||
def _execute_date(self, date: str, models: List[str], config_path: str) -> None:
|
||||
def _execute_date(self, date: str, models: List[str], config_path: str,
|
||||
completion_skips: Dict[str, Set[str]] = None) -> None:
|
||||
"""
|
||||
Execute all models for a single date in parallel.
|
||||
|
||||
@@ -153,14 +154,24 @@ class SimulationWorker:
|
||||
date: Trading date (YYYY-MM-DD)
|
||||
models: List of model signatures to execute
|
||||
config_path: Path to configuration file
|
||||
completion_skips: {model: {dates}} of already-completed model-days to skip
|
||||
|
||||
Uses ThreadPoolExecutor to run all models concurrently for this date.
|
||||
Waits for all models to complete before returning.
|
||||
Skips models that have already completed this date.
|
||||
"""
|
||||
if completion_skips is None:
|
||||
completion_skips = {}
|
||||
|
||||
with ThreadPoolExecutor(max_workers=self.max_workers) as executor:
|
||||
# Submit all model executions for this date
|
||||
futures = []
|
||||
for model in models:
|
||||
# Skip if this model-day was already completed
|
||||
if date in completion_skips.get(model, set()):
|
||||
logger.debug(f"Skipping {model} on {date} (already completed)")
|
||||
continue
|
||||
|
||||
future = executor.submit(
|
||||
self._execute_model_day,
|
||||
date,
|
||||
@@ -397,7 +408,10 @@ class SimulationWorker:
|
||||
config_path: Path to configuration file
|
||||
|
||||
Returns:
|
||||
Tuple of (available_dates, warnings)
|
||||
Tuple of (available_dates, warnings, completion_skips)
|
||||
- available_dates: Dates to process
|
||||
- warnings: Warning messages
|
||||
- completion_skips: {model: {dates}} of already-completed model-days
|
||||
"""
|
||||
from api.price_data_manager import PriceDataManager
|
||||
|
||||
@@ -456,7 +470,7 @@ class SimulationWorker:
|
||||
self.job_manager.update_job_status(self.job_id, "running")
|
||||
logger.info(f"Job {self.job_id}: Starting execution - {len(dates_to_process)} dates, {len(models)} models")
|
||||
|
||||
return dates_to_process, warnings
|
||||
return dates_to_process, warnings, completion_skips
|
||||
|
||||
def get_job_info(self) -> Dict[str, Any]:
|
||||
"""
|
||||
|
||||
@@ -64,6 +64,38 @@ CREATE TABLE positions (
|
||||
);
|
||||
```
|
||||
|
||||
**Column Descriptions:**
|
||||
|
||||
| Column | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| id | INTEGER | Primary key, auto-incremented |
|
||||
| job_id | TEXT | Foreign key to jobs table |
|
||||
| date | TEXT | Trading date (YYYY-MM-DD) |
|
||||
| model | TEXT | Model signature/identifier |
|
||||
| action_id | INTEGER | Sequential action ID for the day (0 = start-of-day baseline) |
|
||||
| action_type | TEXT | Type of action: 'no_trade', 'buy', or 'sell' |
|
||||
| symbol | TEXT | Stock symbol (null for no_trade) |
|
||||
| amount | INTEGER | Number of shares traded (null for no_trade) |
|
||||
| price | REAL | Price per share (null for no_trade) |
|
||||
| cash | REAL | Cash balance after action |
|
||||
| portfolio_value | REAL | Total portfolio value (cash + holdings value) |
|
||||
| daily_profit | REAL | **Daily profit/loss compared to start-of-day portfolio value (action_id=0).** Calculated as: `current_portfolio_value - start_of_day_portfolio_value`. This shows the actual gain/loss from price movements and trading decisions, not affected by merely buying/selling stocks. |
|
||||
| daily_return_pct | REAL | **Daily return percentage compared to start-of-day portfolio value.** Calculated as: `(daily_profit / start_of_day_portfolio_value) * 100` |
|
||||
| created_at | TEXT | ISO 8601 timestamp with 'Z' suffix |
|
||||
|
||||
**Important Notes:**
|
||||
|
||||
- **Position tracking flow:** Positions are written by trade tools (`buy()`, `sell()` in `agent_tools/tool_trade.py`) and no-trade records (`add_no_trade_record_to_db()` in `tools/price_tools.py`). Each trade creates a new position record.
|
||||
|
||||
- **Action ID sequence:**
|
||||
- `action_id=0`: Start-of-day position (created by `ModelDayExecutor._initialize_starting_position()` on first day only)
|
||||
- `action_id=1+`: Each trade or no-trade action increments the action_id
|
||||
|
||||
- **Profit calculation:** Daily profit is calculated by comparing current portfolio value to the **start-of-day** portfolio value (action_id=0 for the current date). This ensures that:
|
||||
- Buying stocks doesn't show as a loss (cash ↓, stock value ↑ equally)
|
||||
- Selling stocks doesn't show as a gain (cash ↑, stock value ↓ equally)
|
||||
- Only actual price movements and strategic trading show as profit/loss
|
||||
|
||||
### holdings
|
||||
Portfolio holdings breakdown per position.
|
||||
|
||||
|
||||
@@ -1,15 +1,310 @@
|
||||
# Testing Guide
|
||||
|
||||
Guide for testing AI-Trader-Server during development.
|
||||
This guide covers running tests for the AI-Trader project, including unit tests, integration tests, and end-to-end tests.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Interactive test menu (recommended for local development)
|
||||
bash scripts/test.sh
|
||||
|
||||
# Quick unit tests (fast feedback)
|
||||
bash scripts/quick_test.sh
|
||||
|
||||
# Full test suite with coverage
|
||||
bash scripts/run_tests.sh
|
||||
|
||||
# Generate coverage report
|
||||
bash scripts/coverage_report.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Automated Testing
|
||||
## Test Scripts Overview
|
||||
|
||||
### 1. `test.sh` - Interactive Test Helper
|
||||
|
||||
**Purpose:** Interactive menu for common test operations
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Interactive mode
|
||||
bash scripts/test.sh
|
||||
|
||||
# Non-interactive mode
|
||||
bash scripts/test.sh -t unit -f
|
||||
```
|
||||
|
||||
**Menu Options:**
|
||||
1. Quick test (unit only, no coverage)
|
||||
2. Full test suite (with coverage)
|
||||
3. Coverage report
|
||||
4. Unit tests only
|
||||
5. Integration tests only
|
||||
6. E2E tests only
|
||||
7. Run with custom markers
|
||||
8. Parallel execution
|
||||
9. CI mode
|
||||
|
||||
---
|
||||
|
||||
### 2. `quick_test.sh` - Fast Feedback Loop
|
||||
|
||||
**Purpose:** Rapid test execution during development
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
bash scripts/quick_test.sh
|
||||
```
|
||||
|
||||
**When to use:**
|
||||
- During active development
|
||||
- Before committing code
|
||||
- Quick verification of changes
|
||||
- TDD workflow
|
||||
|
||||
---
|
||||
|
||||
### 3. `run_tests.sh` - Main Test Runner
|
||||
|
||||
**Purpose:** Comprehensive test execution with full configuration options
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Run all tests with coverage (default)
|
||||
bash scripts/run_tests.sh
|
||||
|
||||
# Run only unit tests
|
||||
bash scripts/run_tests.sh -t unit
|
||||
|
||||
# Run without coverage
|
||||
bash scripts/run_tests.sh -n
|
||||
|
||||
# Run with custom markers
|
||||
bash scripts/run_tests.sh -m "unit and not slow"
|
||||
|
||||
# Fail on first error
|
||||
bash scripts/run_tests.sh -f
|
||||
|
||||
# Run tests in parallel
|
||||
bash scripts/run_tests.sh -p
|
||||
```
|
||||
|
||||
**Options:**
|
||||
```
|
||||
-t, --type TYPE Test type: all, unit, integration, e2e (default: all)
|
||||
-m, --markers MARKERS Run tests matching markers
|
||||
-f, --fail-fast Stop on first failure
|
||||
-n, --no-coverage Skip coverage reporting
|
||||
-v, --verbose Verbose output
|
||||
-p, --parallel Run tests in parallel
|
||||
--no-html Skip HTML coverage report
|
||||
-h, --help Show help message
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. `coverage_report.sh` - Coverage Analysis
|
||||
|
||||
**Purpose:** Generate detailed coverage reports
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Generate coverage report (default: 85% threshold)
|
||||
bash scripts/coverage_report.sh
|
||||
|
||||
# Set custom coverage threshold
|
||||
bash scripts/coverage_report.sh -m 90
|
||||
|
||||
# Generate and open HTML report
|
||||
bash scripts/coverage_report.sh -o
|
||||
```
|
||||
|
||||
**Options:**
|
||||
```
|
||||
-m, --min-coverage NUM Minimum coverage percentage (default: 85)
|
||||
-o, --open Open HTML report in browser
|
||||
-i, --include-integration Include integration and e2e tests
|
||||
-h, --help Show help message
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. `ci_test.sh` - CI/CD Optimized Runner
|
||||
|
||||
**Purpose:** Test execution optimized for CI/CD environments
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Basic CI run
|
||||
bash scripts/ci_test.sh
|
||||
|
||||
# Fail fast with custom coverage
|
||||
bash scripts/ci_test.sh -f -m 90
|
||||
|
||||
# Using environment variables
|
||||
CI_FAIL_FAST=true CI_COVERAGE_MIN=90 bash scripts/ci_test.sh
|
||||
```
|
||||
|
||||
**Environment Variables:**
|
||||
```bash
|
||||
CI_FAIL_FAST=true # Enable fail-fast mode
|
||||
CI_COVERAGE_MIN=90 # Set coverage threshold
|
||||
CI_PARALLEL=true # Enable parallel execution
|
||||
CI_VERBOSE=true # Enable verbose output
|
||||
```
|
||||
|
||||
**Output artifacts:**
|
||||
- `junit.xml` - Test results for CI reporting
|
||||
- `coverage.xml` - Coverage data for CI tools
|
||||
- `htmlcov/` - HTML coverage report
|
||||
|
||||
---
|
||||
|
||||
## Test Structure
|
||||
|
||||
```
|
||||
tests/
|
||||
├── conftest.py # Shared pytest fixtures
|
||||
├── unit/ # Fast, isolated tests
|
||||
├── integration/ # Tests with dependencies
|
||||
├── e2e/ # End-to-end tests
|
||||
├── performance/ # Performance benchmarks
|
||||
└── security/ # Security tests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test Markers
|
||||
|
||||
Tests are organized using pytest markers:
|
||||
|
||||
| Marker | Description | Usage |
|
||||
|--------|-------------|-------|
|
||||
| `unit` | Fast, isolated unit tests | `-m unit` |
|
||||
| `integration` | Tests with real dependencies | `-m integration` |
|
||||
| `e2e` | End-to-end tests (requires Docker) | `-m e2e` |
|
||||
| `slow` | Tests taking >10 seconds | `-m slow` |
|
||||
| `performance` | Performance benchmarks | `-m performance` |
|
||||
| `security` | Security tests | `-m security` |
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Run only unit tests
|
||||
bash scripts/run_tests.sh -m unit
|
||||
|
||||
# Run all except slow tests
|
||||
bash scripts/run_tests.sh -m "not slow"
|
||||
|
||||
# Combine markers
|
||||
bash scripts/run_tests.sh -m "unit and not slow"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Workflows
|
||||
|
||||
### During Development
|
||||
|
||||
```bash
|
||||
# Quick check before each commit
|
||||
bash scripts/quick_test.sh
|
||||
|
||||
# Run relevant test type
|
||||
bash scripts/run_tests.sh -t unit -f
|
||||
|
||||
# Full test before push
|
||||
bash scripts/run_tests.sh
|
||||
```
|
||||
|
||||
### Before Pull Request
|
||||
|
||||
```bash
|
||||
# Run full test suite
|
||||
bash scripts/run_tests.sh
|
||||
|
||||
# Generate coverage report
|
||||
bash scripts/coverage_report.sh -o
|
||||
|
||||
# Ensure coverage meets 85% threshold
|
||||
```
|
||||
|
||||
### CI/CD Pipeline
|
||||
|
||||
```bash
|
||||
# Run CI-optimized tests
|
||||
bash scripts/ci_test.sh -f -m 85
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Debugging Test Failures
|
||||
|
||||
```bash
|
||||
# Run with verbose output
|
||||
bash scripts/run_tests.sh -v -f
|
||||
|
||||
# Run specific test file
|
||||
./venv/bin/python -m pytest tests/unit/test_database.py -v
|
||||
|
||||
# Run specific test function
|
||||
./venv/bin/python -m pytest tests/unit/test_database.py::test_function -v
|
||||
|
||||
# Run with debugger on failure
|
||||
./venv/bin/python -m pytest --pdb tests/
|
||||
|
||||
# Show print statements
|
||||
./venv/bin/python -m pytest -s tests/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Coverage Configuration
|
||||
|
||||
Configured in `pytest.ini`:
|
||||
- Minimum coverage: 85%
|
||||
- Target coverage: 90%
|
||||
- Coverage reports: HTML, JSON, terminal
|
||||
|
||||
---
|
||||
|
||||
## Writing New Tests
|
||||
|
||||
### Unit Test Example
|
||||
|
||||
```python
|
||||
import pytest
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_function_returns_expected_value():
|
||||
# Arrange
|
||||
input_data = {"key": "value"}
|
||||
|
||||
# Act
|
||||
result = my_function(input_data)
|
||||
|
||||
# Assert
|
||||
assert result == expected_output
|
||||
```
|
||||
|
||||
### Integration Test Example
|
||||
|
||||
```python
|
||||
@pytest.mark.integration
|
||||
def test_database_integration(clean_db):
|
||||
conn = get_db_connection(clean_db)
|
||||
insert_data(conn, test_data)
|
||||
result = query_data(conn)
|
||||
assert len(result) == 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Docker Testing
|
||||
|
||||
### Docker Build Validation
|
||||
|
||||
```bash
|
||||
chmod +x scripts/*.sh
|
||||
bash scripts/validate_docker_build.sh
|
||||
```
|
||||
|
||||
@@ -30,35 +325,16 @@ Tests all API endpoints with real simulations.
|
||||
|
||||
---
|
||||
|
||||
## Unit Tests
|
||||
## Summary
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Run tests
|
||||
pytest tests/ -v
|
||||
|
||||
# With coverage
|
||||
pytest tests/ -v --cov=api --cov-report=term-missing
|
||||
|
||||
# Specific test file
|
||||
pytest tests/unit/test_job_manager.py -v
|
||||
```
|
||||
| Script | Purpose | Speed | Coverage | Use Case |
|
||||
|--------|---------|-------|----------|----------|
|
||||
| `test.sh` | Interactive menu | Varies | Optional | Local development |
|
||||
| `quick_test.sh` | Fast feedback | ⚡⚡⚡ | No | Active development |
|
||||
| `run_tests.sh` | Full test suite | ⚡⚡ | Yes | Pre-commit, pre-PR |
|
||||
| `coverage_report.sh` | Coverage analysis | ⚡ | Yes | Coverage review |
|
||||
| `ci_test.sh` | CI/CD pipeline | ⚡⚡ | Yes | Automation |
|
||||
|
||||
---
|
||||
|
||||
## Integration Tests
|
||||
|
||||
```bash
|
||||
# Run integration tests only
|
||||
pytest tests/integration/ -v
|
||||
|
||||
# Test with real API server
|
||||
docker-compose up -d
|
||||
pytest tests/integration/test_api_endpoints.py -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
For detailed testing procedures, see root [TESTING_GUIDE.md](../../TESTING_GUIDE.md).
|
||||
For detailed testing procedures and troubleshooting, see [TESTING_GUIDE.md](../../TESTING_GUIDE.md).
|
||||
|
||||
1291
docs/plans/2025-11-03-fix-position-tracking-bugs.md
Normal file
1291
docs/plans/2025-11-03-fix-position-tracking-bugs.md
Normal file
File diff suppressed because it is too large
Load Diff
278
docs/plans/2025-11-03-position-tracking-fixes-summary.md
Normal file
278
docs/plans/2025-11-03-position-tracking-fixes-summary.md
Normal file
@@ -0,0 +1,278 @@
|
||||
# Position Tracking Bug Fixes - Implementation Summary
|
||||
|
||||
**Date:** 2025-11-03
|
||||
**Implemented by:** Claude Code
|
||||
**Plan:** docs/plans/2025-11-03-fix-position-tracking-bugs.md
|
||||
|
||||
## Overview
|
||||
|
||||
Successfully implemented all fixes for three critical bugs in the position tracking system:
|
||||
1. Cash reset to initial value each trading day
|
||||
2. Positions lost over non-continuous trading days (weekends)
|
||||
3. Profit calculations showing trades as losses
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Tasks Completed
|
||||
|
||||
✅ **Task 1:** Write failing tests for current bugs
|
||||
✅ **Task 2:** Remove redundant `_write_results_to_db()` method
|
||||
✅ **Task 3:** Fix unit tests that mock non-existent methods
|
||||
✅ **Task 4:** Fix profit calculation logic (Bug #3)
|
||||
✅ **Task 5:** Verify all bug tests pass
|
||||
✅ **Task 6:** Integration test with real simulation (skipped - not needed)
|
||||
✅ **Task 7:** Update documentation
|
||||
✅ **Task 8:** Manual testing (skipped - automated tests sufficient)
|
||||
✅ **Task 9:** Final verification and cleanup
|
||||
|
||||
### Root Causes Identified
|
||||
|
||||
1. **Bugs #1 & #2 (Cash reset + positions lost):**
|
||||
- `ModelDayExecutor._write_results_to_db()` called non-existent methods on BaseAgent:
|
||||
- `get_positions()` → returned empty dict
|
||||
- `get_last_trade()` → returned None
|
||||
- `get_current_prices()` → returned empty dict
|
||||
- This created corrupt position records with `cash=0` and `holdings=[]`
|
||||
- `get_current_position_from_db()` then retrieved these corrupt records as "latest position"
|
||||
- Result: Cash reset to $0 or initial value, all holdings lost
|
||||
|
||||
2. **Bug #3 (Incorrect profit calculations):**
|
||||
- Profit calculation compared portfolio value to **previous day's final value**
|
||||
- When buying stocks: cash ↓ $927.50, stock value ↑ $927.50 → portfolio unchanged
|
||||
- Comparing to previous day showed profit=$0 (misleading) or rounding errors
|
||||
- Should compare to **start-of-day value** (same day, action_id=0) to show actual trading gains
|
||||
|
||||
### Solution Implemented
|
||||
|
||||
1. **Removed redundant method (Tasks 2-3):**
|
||||
- Deleted `ModelDayExecutor._write_results_to_db()` method entirely (lines 435-558)
|
||||
- Deleted helper method `_calculate_portfolio_value()` (lines 533-558)
|
||||
- Removed call to `_write_results_to_db()` from `execute_async()` (line 161-167)
|
||||
- Updated test mocks in `test_model_day_executor.py` to remove references
|
||||
- Updated test mocks in `test_model_day_executor_reasoning.py`
|
||||
|
||||
2. **Fixed profit calculation (Task 4):**
|
||||
- Changed `agent_tools/tool_trade.py`:
|
||||
- `_buy_impl()`: Compare to start-of-day value (action_id=0) instead of previous day
|
||||
- `_sell_impl()`: Same fix
|
||||
- Changed `tools/price_tools.py`:
|
||||
- `add_no_trade_record_to_db()`: Same fix
|
||||
- All profit calculations now use:
|
||||
```python
|
||||
SELECT portfolio_value FROM positions
|
||||
WHERE job_id = ? AND model = ? AND date = ? AND action_id = 0
|
||||
```
|
||||
Instead of:
|
||||
```python
|
||||
SELECT portfolio_value FROM positions
|
||||
WHERE job_id = ? AND model = ? AND date < ?
|
||||
ORDER BY date DESC, action_id DESC LIMIT 1
|
||||
```
|
||||
|
||||
### Files Modified
|
||||
|
||||
**Production Code:**
|
||||
- `api/model_day_executor.py`: Removed redundant methods
|
||||
- `agent_tools/tool_trade.py`: Fixed profit calculation in buy/sell
|
||||
- `tools/price_tools.py`: Fixed profit calculation in no_trade
|
||||
|
||||
**Tests:**
|
||||
- `tests/unit/test_position_tracking_bugs.py`: New regression tests (98 lines)
|
||||
- `tests/unit/test_model_day_executor.py`: Updated mocks and tests
|
||||
- `tests/unit/test_model_day_executor_reasoning.py`: Skipped obsolete test
|
||||
- `tests/unit/test_simulation_worker.py`: Fixed mock return values (3 values instead of 2)
|
||||
- `tests/integration/test_async_download.py`: Fixed mock return values
|
||||
- `tests/e2e/test_async_download_flow.py`: Fixed _execute_date mock signature
|
||||
|
||||
**Documentation:**
|
||||
- `CHANGELOG.md`: Added fix notes
|
||||
- `docs/developer/database-schema.md`: Updated profit calculation documentation
|
||||
- `docs/developer/testing.md`: Enhanced with comprehensive testing guide
|
||||
- `CLAUDE.md`: Added testing section with examples
|
||||
|
||||
**New Features (Task 7 bonus):**
|
||||
- `scripts/test.sh`: Interactive testing menu
|
||||
- `scripts/quick_test.sh`: Fast unit test runner
|
||||
- `scripts/run_tests.sh`: Full test suite with options
|
||||
- `scripts/coverage_report.sh`: Coverage analysis tool
|
||||
- `scripts/ci_test.sh`: CI/CD optimized testing
|
||||
- `scripts/README.md`: Quick reference guide
|
||||
|
||||
## Test Results
|
||||
|
||||
### Final Test Suite Status
|
||||
|
||||
```
|
||||
Platform: linux
|
||||
Python: 3.12.8
|
||||
Pytest: 8.4.2
|
||||
|
||||
Results:
|
||||
✅ 289 tests passed
|
||||
⏭️ 8 tests skipped (require MCP services or manual data setup)
|
||||
⚠️ 3326 warnings (mostly deprecation warnings in dependencies)
|
||||
|
||||
Coverage: 89.86% (exceeds 85% threshold)
|
||||
Time: 27.90 seconds
|
||||
```
|
||||
|
||||
### Critical Tests Verified
|
||||
|
||||
✅ `test_cash_not_reset_between_days` - Cash carries over correctly
|
||||
✅ `test_positions_persist_over_weekend` - Holdings persist across non-trading days
|
||||
✅ `test_profit_calculation_accuracy` - Profit shows $0 for trades without price changes
|
||||
✅ All model_day_executor tests pass
|
||||
✅ All simulation_worker tests pass
|
||||
✅ All async_download tests pass
|
||||
|
||||
### Cleanup Performed
|
||||
|
||||
✅ No debug print statements found
|
||||
✅ No references to deleted methods in production code
|
||||
✅ All test mocks updated to match new signatures
|
||||
✅ Documentation reflects current architecture
|
||||
|
||||
## Commits Created
|
||||
|
||||
1. `179cbda` - test: add tests for position tracking bugs (Task 1)
|
||||
2. `c47798d` - fix: remove redundant _write_results_to_db() creating corrupt position records (Task 2)
|
||||
3. `6cb56f8` - test: update tests after removing _write_results_to_db() (Task 3)
|
||||
4. `9be14a1` - fix: correct profit calculation to compare against start-of-day value (Task 4)
|
||||
5. `84320ab` - docs: update changelog and schema docs for position tracking fixes (Task 7)
|
||||
6. `923cdec` - feat: add standardized testing scripts and documentation (Task 7 + Task 9)
|
||||
|
||||
## Impact Assessment
|
||||
|
||||
### Before Fixes
|
||||
|
||||
**Cash Tracking:**
|
||||
- Day 1: Start with $10,000, buy $927.50 of stock → Cash = $9,072.50 ✅
|
||||
- Day 2: Cash reset to $10,000 or $0 ❌
|
||||
|
||||
**Position Persistence:**
|
||||
- Friday: Buy 5 NVDA shares ✅
|
||||
- Monday: NVDA position lost, holdings = [] ❌
|
||||
|
||||
**Profit Calculation:**
|
||||
- Buy 5 NVDA @ $185.50 (portfolio value unchanged)
|
||||
- Profit shown: $0 or small rounding error ❌ (misleading)
|
||||
|
||||
### After Fixes
|
||||
|
||||
**Cash Tracking:**
|
||||
- Day 1: Start with $10,000, buy $927.50 of stock → Cash = $9,072.50 ✅
|
||||
- Day 2: Cash = $9,072.50 (correct carry-over) ✅
|
||||
|
||||
**Position Persistence:**
|
||||
- Friday: Buy 5 NVDA shares ✅
|
||||
- Monday: Still have 5 NVDA shares ✅
|
||||
|
||||
**Profit Calculation:**
|
||||
- Buy 5 NVDA @ $185.50 (portfolio value unchanged)
|
||||
- Profit = $0.00 ✅ (accurate - no price movement, just traded)
|
||||
- If price rises to $190: Profit = $22.50 ✅ (5 shares × $4.50 gain)
|
||||
|
||||
## Architecture Changes
|
||||
|
||||
### Position Tracking Flow (New)
|
||||
|
||||
```
|
||||
ModelDayExecutor.execute()
|
||||
↓
|
||||
1. Create initial position (action_id=0) via _initialize_starting_position()
|
||||
↓
|
||||
2. Run AI agent trading session
|
||||
↓
|
||||
3. AI calls trade tools:
|
||||
- buy() → writes position record (action_id++)
|
||||
- sell() → writes position record (action_id++)
|
||||
- finish → add_no_trade_record_to_db() if no trades
|
||||
↓
|
||||
4. Each position record includes:
|
||||
- cash: Current cash balance
|
||||
- holdings: Stock quantities
|
||||
- portfolio_value: cash + sum(holdings × prices)
|
||||
- daily_profit: portfolio_value - start_of_day_value (action_id=0)
|
||||
↓
|
||||
5. Next day retrieves latest position from previous day
|
||||
```
|
||||
|
||||
### Key Principles
|
||||
|
||||
**Single Source of Truth:**
|
||||
- Trade tools (`buy()`, `sell()`) write position records
|
||||
- `add_no_trade_record_to_db()` writes position if no trades made
|
||||
- ModelDayExecutor DOES NOT write positions directly
|
||||
|
||||
**Profit Calculation:**
|
||||
- Always compare to start-of-day value (action_id=0, same date)
|
||||
- Never compare to previous day's final value
|
||||
- Ensures trades don't create false profit/loss signals
|
||||
|
||||
**Action ID Sequence:**
|
||||
- `action_id=0`: Start-of-day baseline (created once per day)
|
||||
- `action_id=1+`: Incremented for each trade or no-trade action
|
||||
|
||||
## Success Criteria Met
|
||||
|
||||
✅ All tests in `test_position_tracking_bugs.py` PASS
|
||||
✅ All existing unit tests continue to PASS
|
||||
✅ Code coverage: 89.86% (exceeds 85% threshold)
|
||||
✅ No references to deleted methods in production code
|
||||
✅ Documentation updated (CHANGELOG, database-schema)
|
||||
✅ Test suite enhanced with comprehensive testing scripts
|
||||
✅ All test mocks updated to match new signatures
|
||||
✅ Clean git history with clear commit messages
|
||||
|
||||
## Verification Steps Performed
|
||||
|
||||
1. ✅ Ran complete test suite: 289 passed, 8 skipped
|
||||
2. ✅ Checked for deleted method references: None found in production code
|
||||
3. ✅ Reviewed all modified files for debug prints: None found
|
||||
4. ✅ Verified test mocks match actual signatures: All updated
|
||||
5. ✅ Ran coverage report: 89.86% (exceeds threshold)
|
||||
6. ✅ Checked commit history: 6 commits with clear messages
|
||||
|
||||
## Future Maintenance Notes
|
||||
|
||||
**If modifying position tracking:**
|
||||
|
||||
1. **Run regression tests first:**
|
||||
```bash
|
||||
pytest tests/unit/test_position_tracking_bugs.py -v
|
||||
```
|
||||
|
||||
2. **Remember the architecture:**
|
||||
- Trade tools write positions (NOT ModelDayExecutor)
|
||||
- Profit compares to start-of-day (action_id=0)
|
||||
- Action IDs increment for each trade
|
||||
|
||||
3. **Key invariants to maintain:**
|
||||
- Cash must carry over between days
|
||||
- Holdings must persist until sold
|
||||
- Profit should be $0 for trades without price changes
|
||||
|
||||
4. **Test coverage:**
|
||||
- Unit tests: `test_position_tracking_bugs.py`
|
||||
- Integration tests: Available via test scripts
|
||||
- Manual verification: Use DEV mode to avoid API costs
|
||||
|
||||
## Lessons Learned
|
||||
|
||||
1. **Redundant code is dangerous:** The `_write_results_to_db()` method was creating corrupt data but silently failing because it called non-existent methods that returned empty defaults.
|
||||
|
||||
2. **Profit calculation matters:** Comparing to the wrong baseline (previous day vs start-of-day) completely changed the interpretation of trading results.
|
||||
|
||||
3. **Test coverage is essential:** The bugs existed because there were no specific tests for multi-day position continuity and profit accuracy.
|
||||
|
||||
4. **Documentation prevents regressions:** Clear documentation of profit calculation logic helps future developers understand why code is written a certain way.
|
||||
|
||||
## Conclusion
|
||||
|
||||
All three critical bugs have been successfully fixed:
|
||||
|
||||
✅ **Bug #1 (Cash reset):** Fixed by removing `_write_results_to_db()` that created corrupt records
|
||||
✅ **Bug #2 (Positions lost):** Fixed by same change - positions now persist correctly
|
||||
✅ **Bug #3 (Wrong profits):** Fixed by comparing to start-of-day value instead of previous day
|
||||
|
||||
The implementation is complete, tested, documented, and ready for production use. All 289 automated tests pass with 89.86% code coverage.
|
||||
109
scripts/README.md
Normal file
109
scripts/README.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# AI-Trader Scripts
|
||||
|
||||
This directory contains standardized scripts for testing, validation, and operations.
|
||||
|
||||
## Testing Scripts
|
||||
|
||||
### Interactive Testing
|
||||
|
||||
**`test.sh`** - Interactive test menu
|
||||
```bash
|
||||
bash scripts/test.sh
|
||||
```
|
||||
User-friendly menu for all testing operations. Best for local development.
|
||||
|
||||
### Development Testing
|
||||
|
||||
**`quick_test.sh`** - Fast unit test feedback
|
||||
```bash
|
||||
bash scripts/quick_test.sh
|
||||
```
|
||||
- Runs unit tests only
|
||||
- No coverage
|
||||
- Fails fast
|
||||
- ~10-30 seconds
|
||||
|
||||
**`run_tests.sh`** - Full test suite
|
||||
```bash
|
||||
bash scripts/run_tests.sh [OPTIONS]
|
||||
```
|
||||
- All test types (unit, integration, e2e)
|
||||
- Coverage reporting
|
||||
- Parallel execution support
|
||||
- Highly configurable
|
||||
|
||||
**`coverage_report.sh`** - Coverage analysis
|
||||
```bash
|
||||
bash scripts/coverage_report.sh [OPTIONS]
|
||||
```
|
||||
- Generate HTML/JSON/terminal reports
|
||||
- Check coverage thresholds
|
||||
- Open reports in browser
|
||||
|
||||
### CI/CD Testing
|
||||
|
||||
**`ci_test.sh`** - CI-optimized testing
|
||||
```bash
|
||||
bash scripts/ci_test.sh [OPTIONS]
|
||||
```
|
||||
- JUnit XML output
|
||||
- Coverage XML for CI tools
|
||||
- Environment variable configuration
|
||||
- Excludes Docker tests
|
||||
|
||||
## Validation Scripts
|
||||
|
||||
**`validate_docker_build.sh`** - Docker build validation
|
||||
```bash
|
||||
bash scripts/validate_docker_build.sh
|
||||
```
|
||||
Validates Docker setup, build, and container startup.
|
||||
|
||||
**`test_api_endpoints.sh`** - API endpoint testing
|
||||
```bash
|
||||
bash scripts/test_api_endpoints.sh
|
||||
```
|
||||
Tests all REST API endpoints with real simulations.
|
||||
|
||||
## Other Scripts
|
||||
|
||||
**`migrate_price_data.py`** - Data migration utility
|
||||
```bash
|
||||
python scripts/migrate_price_data.py
|
||||
```
|
||||
Migrates price data between formats.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | Script | Command |
|
||||
|------|--------|---------|
|
||||
| Quick test | `quick_test.sh` | `bash scripts/quick_test.sh` |
|
||||
| Full test | `run_tests.sh` | `bash scripts/run_tests.sh` |
|
||||
| Coverage | `coverage_report.sh` | `bash scripts/coverage_report.sh -o` |
|
||||
| CI test | `ci_test.sh` | `bash scripts/ci_test.sh -f` |
|
||||
| Interactive | `test.sh` | `bash scripts/test.sh` |
|
||||
| Docker validation | `validate_docker_build.sh` | `bash scripts/validate_docker_build.sh` |
|
||||
| API testing | `test_api_endpoints.sh` | `bash scripts/test_api_endpoints.sh` |
|
||||
|
||||
## Common Options
|
||||
|
||||
Most test scripts support:
|
||||
- `-h, --help` - Show help
|
||||
- `-v, --verbose` - Verbose output
|
||||
- `-f, --fail-fast` - Stop on first failure
|
||||
- `-t, --type TYPE` - Test type (unit, integration, e2e, all)
|
||||
- `-m, --markers MARKERS` - Pytest markers
|
||||
- `-p, --parallel` - Parallel execution
|
||||
|
||||
## Documentation
|
||||
|
||||
For detailed usage, see:
|
||||
- [Testing Guide](../docs/developer/testing.md)
|
||||
- [Testing & Validation Guide](../TESTING_GUIDE.md)
|
||||
|
||||
## Making Scripts Executable
|
||||
|
||||
If scripts are not executable:
|
||||
```bash
|
||||
chmod +x scripts/*.sh
|
||||
```
|
||||
243
scripts/ci_test.sh
Executable file
243
scripts/ci_test.sh
Executable file
@@ -0,0 +1,243 @@
|
||||
#!/bin/bash
|
||||
# AI-Trader CI Test Script
|
||||
# Optimized for CI/CD environments (GitHub Actions, Jenkins, etc.)
|
||||
|
||||
set -e
|
||||
|
||||
# Colors for output (disabled in CI if not supported)
|
||||
if [ -t 1 ]; then
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
else
|
||||
RED=''
|
||||
GREEN=''
|
||||
YELLOW=''
|
||||
BLUE=''
|
||||
NC=''
|
||||
fi
|
||||
|
||||
# Script directory
|
||||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
# CI-specific defaults
|
||||
FAIL_FAST=false
|
||||
JUNIT_XML=true
|
||||
COVERAGE_MIN=85
|
||||
PARALLEL=false
|
||||
VERBOSE=false
|
||||
|
||||
# Parse environment variables (common in CI)
|
||||
if [ -n "$CI_FAIL_FAST" ]; then
|
||||
FAIL_FAST="$CI_FAIL_FAST"
|
||||
fi
|
||||
|
||||
if [ -n "$CI_COVERAGE_MIN" ]; then
|
||||
COVERAGE_MIN="$CI_COVERAGE_MIN"
|
||||
fi
|
||||
|
||||
if [ -n "$CI_PARALLEL" ]; then
|
||||
PARALLEL="$CI_PARALLEL"
|
||||
fi
|
||||
|
||||
if [ -n "$CI_VERBOSE" ]; then
|
||||
VERBOSE="$CI_VERBOSE"
|
||||
fi
|
||||
|
||||
# Parse command line arguments (override env vars)
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
-f|--fail-fast)
|
||||
FAIL_FAST=true
|
||||
shift
|
||||
;;
|
||||
-m|--min-coverage)
|
||||
COVERAGE_MIN="$2"
|
||||
shift 2
|
||||
;;
|
||||
-p|--parallel)
|
||||
PARALLEL=true
|
||||
shift
|
||||
;;
|
||||
-v|--verbose)
|
||||
VERBOSE=true
|
||||
shift
|
||||
;;
|
||||
--no-junit)
|
||||
JUNIT_XML=false
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
cat << EOF
|
||||
Usage: $0 [OPTIONS]
|
||||
|
||||
CI-optimized test runner for AI-Trader.
|
||||
|
||||
OPTIONS:
|
||||
-f, --fail-fast Stop on first failure
|
||||
-m, --min-coverage NUM Minimum coverage percentage (default: 85)
|
||||
-p, --parallel Run tests in parallel
|
||||
-v, --verbose Verbose output
|
||||
--no-junit Skip JUnit XML generation
|
||||
-h, --help Show this help message
|
||||
|
||||
ENVIRONMENT VARIABLES:
|
||||
CI_FAIL_FAST Set to 'true' to enable fail-fast
|
||||
CI_COVERAGE_MIN Minimum coverage threshold
|
||||
CI_PARALLEL Set to 'true' to enable parallel execution
|
||||
CI_VERBOSE Set to 'true' for verbose output
|
||||
|
||||
EXAMPLES:
|
||||
# Basic CI run
|
||||
$0
|
||||
|
||||
# Fail fast with custom coverage threshold
|
||||
$0 -f -m 90
|
||||
|
||||
# Parallel execution
|
||||
$0 -p
|
||||
|
||||
# GitHub Actions
|
||||
CI_FAIL_FAST=true CI_COVERAGE_MIN=90 $0
|
||||
|
||||
EOF
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}Unknown option: $1${NC}"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo -e "${BLUE}AI-Trader CI Test Runner${NC}"
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo ""
|
||||
echo -e "${YELLOW}CI Configuration:${NC}"
|
||||
echo " Fail Fast: $FAIL_FAST"
|
||||
echo " Min Coverage: ${COVERAGE_MIN}%"
|
||||
echo " Parallel: $PARALLEL"
|
||||
echo " Verbose: $VERBOSE"
|
||||
echo " JUnit XML: $JUNIT_XML"
|
||||
echo " Environment: ${CI:-local}"
|
||||
echo ""
|
||||
|
||||
# Change to project root
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Check Python version
|
||||
echo -e "${YELLOW}Checking Python version...${NC}"
|
||||
PYTHON_VERSION=$(./venv/bin/python --version 2>&1)
|
||||
echo " $PYTHON_VERSION"
|
||||
echo ""
|
||||
|
||||
# Install/verify dependencies
|
||||
echo -e "${YELLOW}Verifying test dependencies...${NC}"
|
||||
./venv/bin/python -m pip install --quiet pytest pytest-cov pytest-xdist 2>&1 | grep -v "already satisfied" || true
|
||||
echo " ✓ Dependencies verified"
|
||||
echo ""
|
||||
|
||||
# Build pytest command
|
||||
PYTEST_CMD="./venv/bin/python -m pytest"
|
||||
PYTEST_ARGS="-v --tb=short --strict-markers"
|
||||
|
||||
# Coverage
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov=api --cov=agent --cov=tools"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-report=term-missing:skip-covered"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-report=html:htmlcov"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-report=xml:coverage.xml"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-fail-under=$COVERAGE_MIN"
|
||||
|
||||
# JUnit XML for CI integrations
|
||||
if [ "$JUNIT_XML" = true ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS --junit-xml=junit.xml"
|
||||
fi
|
||||
|
||||
# Fail fast
|
||||
if [ "$FAIL_FAST" = true ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS -x"
|
||||
fi
|
||||
|
||||
# Parallel execution
|
||||
if [ "$PARALLEL" = true ]; then
|
||||
# Check if pytest-xdist is available
|
||||
if ./venv/bin/python -c "import xdist" 2>/dev/null; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS -n auto"
|
||||
echo -e "${YELLOW}Parallel execution enabled${NC}"
|
||||
else
|
||||
echo -e "${YELLOW}Warning: pytest-xdist not available, running sequentially${NC}"
|
||||
fi
|
||||
echo ""
|
||||
fi
|
||||
|
||||
# Verbose
|
||||
if [ "$VERBOSE" = true ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS -vv"
|
||||
fi
|
||||
|
||||
# Exclude e2e tests in CI (require Docker)
|
||||
PYTEST_ARGS="$PYTEST_ARGS -m 'not e2e'"
|
||||
|
||||
# Test path
|
||||
PYTEST_ARGS="$PYTEST_ARGS tests/"
|
||||
|
||||
# Run tests
|
||||
echo -e "${BLUE}Running test suite...${NC}"
|
||||
echo ""
|
||||
echo "Command: $PYTEST_CMD $PYTEST_ARGS"
|
||||
echo ""
|
||||
|
||||
# Execute tests
|
||||
set +e # Don't exit on test failure, we want to process results
|
||||
$PYTEST_CMD $PYTEST_ARGS
|
||||
TEST_EXIT_CODE=$?
|
||||
set -e
|
||||
|
||||
echo ""
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo -e "${BLUE}Test Results${NC}"
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo ""
|
||||
|
||||
# Process results
|
||||
if [ $TEST_EXIT_CODE -eq 0 ]; then
|
||||
echo -e "${GREEN}✓ All tests passed!${NC}"
|
||||
echo ""
|
||||
|
||||
# Show artifacts
|
||||
echo -e "${YELLOW}Artifacts generated:${NC}"
|
||||
if [ -f "coverage.xml" ]; then
|
||||
echo " ✓ coverage.xml (for CI coverage tools)"
|
||||
fi
|
||||
if [ -f "junit.xml" ]; then
|
||||
echo " ✓ junit.xml (for CI test reporting)"
|
||||
fi
|
||||
if [ -d "htmlcov" ]; then
|
||||
echo " ✓ htmlcov/ (HTML coverage report)"
|
||||
fi
|
||||
else
|
||||
echo -e "${RED}✗ Tests failed (exit code: $TEST_EXIT_CODE)${NC}"
|
||||
echo ""
|
||||
|
||||
if [ $TEST_EXIT_CODE -eq 1 ]; then
|
||||
echo " Reason: Test failures"
|
||||
elif [ $TEST_EXIT_CODE -eq 2 ]; then
|
||||
echo " Reason: Test execution interrupted"
|
||||
elif [ $TEST_EXIT_CODE -eq 3 ]; then
|
||||
echo " Reason: Internal pytest error"
|
||||
elif [ $TEST_EXIT_CODE -eq 4 ]; then
|
||||
echo " Reason: pytest usage error"
|
||||
elif [ $TEST_EXIT_CODE -eq 5 ]; then
|
||||
echo " Reason: No tests collected"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
|
||||
# Exit with test result code
|
||||
exit $TEST_EXIT_CODE
|
||||
170
scripts/coverage_report.sh
Executable file
170
scripts/coverage_report.sh
Executable file
@@ -0,0 +1,170 @@
|
||||
#!/bin/bash
|
||||
# AI-Trader Coverage Report Generator
|
||||
# Generate detailed coverage reports and check coverage thresholds
|
||||
|
||||
set -e
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Script directory
|
||||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
# Default values
|
||||
MIN_COVERAGE=85
|
||||
OPEN_HTML=false
|
||||
INCLUDE_INTEGRATION=false
|
||||
|
||||
# Usage information
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 [OPTIONS]
|
||||
|
||||
Generate coverage reports for AI-Trader test suite.
|
||||
|
||||
OPTIONS:
|
||||
-m, --min-coverage NUM Minimum coverage percentage (default: 85)
|
||||
-o, --open Open HTML report in browser after generation
|
||||
-i, --include-integration Include integration and e2e tests
|
||||
-h, --help Show this help message
|
||||
|
||||
EXAMPLES:
|
||||
# Generate coverage report with default threshold (85%)
|
||||
$0
|
||||
|
||||
# Set custom coverage threshold
|
||||
$0 -m 90
|
||||
|
||||
# Generate and open HTML report
|
||||
$0 -o
|
||||
|
||||
# Include integration tests in coverage
|
||||
$0 -i
|
||||
|
||||
EOF
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Parse command line arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
-m|--min-coverage)
|
||||
MIN_COVERAGE="$2"
|
||||
shift 2
|
||||
;;
|
||||
-o|--open)
|
||||
OPEN_HTML=true
|
||||
shift
|
||||
;;
|
||||
-i|--include-integration)
|
||||
INCLUDE_INTEGRATION=true
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}Unknown option: $1${NC}"
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo -e "${BLUE}AI-Trader Coverage Report${NC}"
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo ""
|
||||
echo -e "${YELLOW}Configuration:${NC}"
|
||||
echo " Minimum Coverage: ${MIN_COVERAGE}%"
|
||||
echo " Include Integration: $INCLUDE_INTEGRATION"
|
||||
echo ""
|
||||
|
||||
# Check if virtual environment exists
|
||||
if [ ! -d "$PROJECT_ROOT/venv" ]; then
|
||||
echo -e "${RED}Error: Virtual environment not found${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Change to project root
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Build pytest command
|
||||
PYTEST_CMD="./venv/bin/python -m pytest tests/"
|
||||
PYTEST_ARGS="-v --tb=short"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov=api --cov=agent --cov=tools"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-report=term-missing"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-report=html:htmlcov"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-report=json:coverage.json"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-fail-under=$MIN_COVERAGE"
|
||||
|
||||
# Filter tests if not including integration
|
||||
if [ "$INCLUDE_INTEGRATION" = false ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS -m 'not e2e'"
|
||||
echo -e "${YELLOW}Running tests (excluding e2e)...${NC}"
|
||||
else
|
||||
echo -e "${YELLOW}Running all tests...${NC}"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
|
||||
# Run tests with coverage
|
||||
$PYTEST_CMD $PYTEST_ARGS
|
||||
TEST_EXIT_CODE=$?
|
||||
|
||||
echo ""
|
||||
|
||||
# Parse coverage from JSON report
|
||||
if [ -f "coverage.json" ]; then
|
||||
TOTAL_COVERAGE=$(./venv/bin/python -c "import json; data=json.load(open('coverage.json')); print(f\"{data['totals']['percent_covered']:.2f}\")")
|
||||
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo -e "${BLUE}Coverage Summary${NC}"
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo ""
|
||||
echo -e " Total Coverage: ${GREEN}${TOTAL_COVERAGE}%${NC}"
|
||||
echo -e " Minimum Required: ${MIN_COVERAGE}%"
|
||||
echo ""
|
||||
|
||||
if [ $TEST_EXIT_CODE -eq 0 ]; then
|
||||
echo -e "${GREEN}✓ Coverage threshold met!${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ Coverage below threshold${NC}"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo -e "${YELLOW}Reports Generated:${NC}"
|
||||
echo " HTML: file://$PROJECT_ROOT/htmlcov/index.html"
|
||||
echo " JSON: $PROJECT_ROOT/coverage.json"
|
||||
echo " Terminal: (shown above)"
|
||||
|
||||
# Open HTML report if requested
|
||||
if [ "$OPEN_HTML" = true ]; then
|
||||
echo ""
|
||||
echo -e "${BLUE}Opening HTML report...${NC}"
|
||||
|
||||
# Try different browsers/commands
|
||||
if command -v xdg-open &> /dev/null; then
|
||||
xdg-open "htmlcov/index.html"
|
||||
elif command -v open &> /dev/null; then
|
||||
open "htmlcov/index.html"
|
||||
elif command -v start &> /dev/null; then
|
||||
start "htmlcov/index.html"
|
||||
else
|
||||
echo -e "${YELLOW}Could not open browser automatically${NC}"
|
||||
echo "Please open: file://$PROJECT_ROOT/htmlcov/index.html"
|
||||
fi
|
||||
fi
|
||||
else
|
||||
echo -e "${RED}Error: coverage.json not generated${NC}"
|
||||
TEST_EXIT_CODE=1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
|
||||
exit $TEST_EXIT_CODE
|
||||
59
scripts/quick_test.sh
Executable file
59
scripts/quick_test.sh
Executable file
@@ -0,0 +1,59 @@
|
||||
#!/bin/bash
|
||||
# AI-Trader Quick Test Script
|
||||
# Fast test run for rapid feedback during development
|
||||
|
||||
set -e
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Script directory
|
||||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo -e "${BLUE}AI-Trader Quick Test${NC}"
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo ""
|
||||
echo -e "${YELLOW}Running unit tests (no coverage, fail-fast)${NC}"
|
||||
echo ""
|
||||
|
||||
# Change to project root
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Check if virtual environment exists
|
||||
if [ ! -d "./venv" ]; then
|
||||
echo -e "${RED}Error: Virtual environment not found${NC}"
|
||||
echo -e "${YELLOW}Please run: python3 -m venv venv && ./venv/bin/pip install -r requirements.txt${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Run unit tests only, no coverage, fail on first error
|
||||
./venv/bin/python -m pytest tests/ \
|
||||
-v \
|
||||
-m "unit and not slow" \
|
||||
-x \
|
||||
--tb=short \
|
||||
--no-cov
|
||||
|
||||
TEST_EXIT_CODE=$?
|
||||
|
||||
echo ""
|
||||
if [ $TEST_EXIT_CODE -eq 0 ]; then
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}✓ Quick tests passed!${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo ""
|
||||
echo -e "${YELLOW}For full test suite with coverage, run:${NC}"
|
||||
echo " bash scripts/run_tests.sh"
|
||||
else
|
||||
echo -e "${RED}========================================${NC}"
|
||||
echo -e "${RED}✗ Quick tests failed${NC}"
|
||||
echo -e "${RED}========================================${NC}"
|
||||
fi
|
||||
|
||||
exit $TEST_EXIT_CODE
|
||||
221
scripts/run_tests.sh
Executable file
221
scripts/run_tests.sh
Executable file
@@ -0,0 +1,221 @@
|
||||
#!/bin/bash
|
||||
# AI-Trader Test Runner
|
||||
# Standardized script for running tests with various options
|
||||
|
||||
set -e
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Script directory
|
||||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
# Default values
|
||||
TEST_TYPE="all"
|
||||
COVERAGE=true
|
||||
VERBOSE=false
|
||||
FAIL_FAST=false
|
||||
MARKERS=""
|
||||
PARALLEL=false
|
||||
HTML_REPORT=true
|
||||
|
||||
# Usage information
|
||||
usage() {
|
||||
cat << EOF
|
||||
Usage: $0 [OPTIONS]
|
||||
|
||||
Run AI-Trader test suite with standardized configuration.
|
||||
|
||||
OPTIONS:
|
||||
-t, --type TYPE Test type: all, unit, integration, e2e (default: all)
|
||||
-m, --markers MARKERS Run tests matching markers (e.g., "unit and not slow")
|
||||
-f, --fail-fast Stop on first failure
|
||||
-n, --no-coverage Skip coverage reporting
|
||||
-v, --verbose Verbose output
|
||||
-p, --parallel Run tests in parallel (requires pytest-xdist)
|
||||
--no-html Skip HTML coverage report
|
||||
-h, --help Show this help message
|
||||
|
||||
EXAMPLES:
|
||||
# Run all tests with coverage
|
||||
$0
|
||||
|
||||
# Run only unit tests
|
||||
$0 -t unit
|
||||
|
||||
# Run integration tests without coverage
|
||||
$0 -t integration -n
|
||||
|
||||
# Run specific markers with fail-fast
|
||||
$0 -m "unit and not slow" -f
|
||||
|
||||
# Run tests in parallel
|
||||
$0 -p
|
||||
|
||||
# Quick test run (unit only, no coverage, fail-fast)
|
||||
$0 -t unit -n -f
|
||||
|
||||
MARKERS:
|
||||
unit - Fast, isolated unit tests
|
||||
integration - Tests with real dependencies
|
||||
e2e - End-to-end tests (requires Docker)
|
||||
slow - Tests taking >10 seconds
|
||||
performance - Performance benchmarks
|
||||
security - Security tests
|
||||
|
||||
EOF
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Parse command line arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
-t|--type)
|
||||
TEST_TYPE="$2"
|
||||
shift 2
|
||||
;;
|
||||
-m|--markers)
|
||||
MARKERS="$2"
|
||||
shift 2
|
||||
;;
|
||||
-f|--fail-fast)
|
||||
FAIL_FAST=true
|
||||
shift
|
||||
;;
|
||||
-n|--no-coverage)
|
||||
COVERAGE=false
|
||||
shift
|
||||
;;
|
||||
-v|--verbose)
|
||||
VERBOSE=true
|
||||
shift
|
||||
;;
|
||||
-p|--parallel)
|
||||
PARALLEL=true
|
||||
shift
|
||||
;;
|
||||
--no-html)
|
||||
HTML_REPORT=false
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}Unknown option: $1${NC}"
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Build pytest command
|
||||
PYTEST_CMD="./venv/bin/python -m pytest"
|
||||
PYTEST_ARGS="-v --tb=short"
|
||||
|
||||
# Add test type markers
|
||||
if [ "$TEST_TYPE" != "all" ]; then
|
||||
if [ -n "$MARKERS" ]; then
|
||||
MARKERS="$TEST_TYPE and ($MARKERS)"
|
||||
else
|
||||
MARKERS="$TEST_TYPE"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Add custom markers
|
||||
if [ -n "$MARKERS" ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS -m \"$MARKERS\""
|
||||
fi
|
||||
|
||||
# Add coverage options
|
||||
if [ "$COVERAGE" = true ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov=api --cov=agent --cov=tools"
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-report=term-missing"
|
||||
|
||||
if [ "$HTML_REPORT" = true ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS --cov-report=html:htmlcov"
|
||||
fi
|
||||
else
|
||||
PYTEST_ARGS="$PYTEST_ARGS --no-cov"
|
||||
fi
|
||||
|
||||
# Add fail-fast
|
||||
if [ "$FAIL_FAST" = true ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS -x"
|
||||
fi
|
||||
|
||||
# Add parallel execution
|
||||
if [ "$PARALLEL" = true ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS -n auto"
|
||||
fi
|
||||
|
||||
# Add verbosity
|
||||
if [ "$VERBOSE" = true ]; then
|
||||
PYTEST_ARGS="$PYTEST_ARGS -vv"
|
||||
fi
|
||||
|
||||
# Add test path
|
||||
PYTEST_ARGS="$PYTEST_ARGS tests/"
|
||||
|
||||
# Print configuration
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo -e "${BLUE}AI-Trader Test Runner${NC}"
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo ""
|
||||
echo -e "${YELLOW}Configuration:${NC}"
|
||||
echo " Test Type: $TEST_TYPE"
|
||||
echo " Markers: ${MARKERS:-none}"
|
||||
echo " Coverage: $COVERAGE"
|
||||
echo " Fail Fast: $FAIL_FAST"
|
||||
echo " Parallel: $PARALLEL"
|
||||
echo " Verbose: $VERBOSE"
|
||||
echo ""
|
||||
|
||||
# Check if virtual environment exists
|
||||
if [ ! -d "$PROJECT_ROOT/venv" ]; then
|
||||
echo -e "${RED}Error: Virtual environment not found at $PROJECT_ROOT/venv${NC}"
|
||||
echo -e "${YELLOW}Please run: python3 -m venv venv && ./venv/bin/pip install -r requirements.txt${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if pytest is installed
|
||||
if ! ./venv/bin/python -c "import pytest" 2>/dev/null; then
|
||||
echo -e "${RED}Error: pytest not installed${NC}"
|
||||
echo -e "${YELLOW}Please run: ./venv/bin/pip install -r requirements.txt${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Change to project root
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Run tests
|
||||
echo -e "${BLUE}Running tests...${NC}"
|
||||
echo ""
|
||||
|
||||
# Execute pytest with eval to handle quotes properly
|
||||
eval "$PYTEST_CMD $PYTEST_ARGS"
|
||||
TEST_EXIT_CODE=$?
|
||||
|
||||
# Print results
|
||||
echo ""
|
||||
if [ $TEST_EXIT_CODE -eq 0 ]; then
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}✓ All tests passed!${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
|
||||
if [ "$COVERAGE" = true ] && [ "$HTML_REPORT" = true ]; then
|
||||
echo ""
|
||||
echo -e "${YELLOW}Coverage report generated:${NC}"
|
||||
echo " HTML: file://$PROJECT_ROOT/htmlcov/index.html"
|
||||
fi
|
||||
else
|
||||
echo -e "${RED}========================================${NC}"
|
||||
echo -e "${RED}✗ Tests failed${NC}"
|
||||
echo -e "${RED}========================================${NC}"
|
||||
fi
|
||||
|
||||
exit $TEST_EXIT_CODE
|
||||
249
scripts/test.sh
Executable file
249
scripts/test.sh
Executable file
@@ -0,0 +1,249 @@
|
||||
#!/bin/bash
|
||||
# AI-Trader Test Helper
|
||||
# Interactive menu for common test operations
|
||||
|
||||
set -e
|
||||
|
||||
# Colors
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
NC='\033[0m'
|
||||
|
||||
# Script directory
|
||||
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
|
||||
|
||||
show_menu() {
|
||||
clear
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo -e "${BLUE} AI-Trader Test Helper${NC}"
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo ""
|
||||
echo -e "${CYAN}Quick Actions:${NC}"
|
||||
echo " 1) Quick test (unit only, no coverage)"
|
||||
echo " 2) Full test suite (with coverage)"
|
||||
echo " 3) Coverage report"
|
||||
echo ""
|
||||
echo -e "${CYAN}Specific Test Types:${NC}"
|
||||
echo " 4) Unit tests only"
|
||||
echo " 5) Integration tests only"
|
||||
echo " 6) E2E tests only (requires Docker)"
|
||||
echo ""
|
||||
echo -e "${CYAN}Advanced Options:${NC}"
|
||||
echo " 7) Run with custom markers"
|
||||
echo " 8) Parallel execution"
|
||||
echo " 9) CI mode (for automation)"
|
||||
echo ""
|
||||
echo -e "${CYAN}Other:${NC}"
|
||||
echo " h) Show help"
|
||||
echo " q) Quit"
|
||||
echo ""
|
||||
echo -ne "${YELLOW}Select an option: ${NC}"
|
||||
}
|
||||
|
||||
run_quick_test() {
|
||||
echo -e "${BLUE}Running quick test...${NC}"
|
||||
bash "$SCRIPT_DIR/quick_test.sh"
|
||||
}
|
||||
|
||||
run_full_test() {
|
||||
echo -e "${BLUE}Running full test suite...${NC}"
|
||||
bash "$SCRIPT_DIR/run_tests.sh"
|
||||
}
|
||||
|
||||
run_coverage() {
|
||||
echo -e "${BLUE}Generating coverage report...${NC}"
|
||||
bash "$SCRIPT_DIR/coverage_report.sh" -o
|
||||
}
|
||||
|
||||
run_unit() {
|
||||
echo -e "${BLUE}Running unit tests...${NC}"
|
||||
bash "$SCRIPT_DIR/run_tests.sh" -t unit
|
||||
}
|
||||
|
||||
run_integration() {
|
||||
echo -e "${BLUE}Running integration tests...${NC}"
|
||||
bash "$SCRIPT_DIR/run_tests.sh" -t integration
|
||||
}
|
||||
|
||||
run_e2e() {
|
||||
echo -e "${BLUE}Running E2E tests...${NC}"
|
||||
echo -e "${YELLOW}Note: This requires Docker to be running${NC}"
|
||||
read -p "Continue? (y/n) " -n 1 -r
|
||||
echo
|
||||
if [[ $REPLY =~ ^[Yy]$ ]]; then
|
||||
bash "$SCRIPT_DIR/run_tests.sh" -t e2e
|
||||
fi
|
||||
}
|
||||
|
||||
run_custom_markers() {
|
||||
echo ""
|
||||
echo -e "${YELLOW}Available markers:${NC}"
|
||||
echo " - unit"
|
||||
echo " - integration"
|
||||
echo " - e2e"
|
||||
echo " - slow"
|
||||
echo " - performance"
|
||||
echo " - security"
|
||||
echo ""
|
||||
echo -e "${YELLOW}Examples:${NC}"
|
||||
echo " unit and not slow"
|
||||
echo " integration or performance"
|
||||
echo " not e2e"
|
||||
echo ""
|
||||
read -p "Enter markers expression: " markers
|
||||
|
||||
if [ -n "$markers" ]; then
|
||||
echo -e "${BLUE}Running tests with markers: $markers${NC}"
|
||||
bash "$SCRIPT_DIR/run_tests.sh" -m "$markers"
|
||||
else
|
||||
echo -e "${RED}No markers provided, skipping${NC}"
|
||||
sleep 2
|
||||
fi
|
||||
}
|
||||
|
||||
run_parallel() {
|
||||
echo -e "${BLUE}Running tests in parallel...${NC}"
|
||||
bash "$SCRIPT_DIR/run_tests.sh" -p
|
||||
}
|
||||
|
||||
run_ci() {
|
||||
echo -e "${BLUE}Running in CI mode...${NC}"
|
||||
bash "$SCRIPT_DIR/ci_test.sh"
|
||||
}
|
||||
|
||||
show_help() {
|
||||
clear
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo -e "${BLUE}AI-Trader Test Scripts Help${NC}"
|
||||
echo -e "${BLUE}========================================${NC}"
|
||||
echo ""
|
||||
echo -e "${CYAN}Available Scripts:${NC}"
|
||||
echo ""
|
||||
echo -e "${GREEN}1. quick_test.sh${NC}"
|
||||
echo " Fast feedback loop for development"
|
||||
echo " - Runs unit tests only"
|
||||
echo " - No coverage reporting"
|
||||
echo " - Fails fast on first error"
|
||||
echo " Usage: bash scripts/quick_test.sh"
|
||||
echo ""
|
||||
echo -e "${GREEN}2. run_tests.sh${NC}"
|
||||
echo " Main test runner with full options"
|
||||
echo " - Supports all test types (unit, integration, e2e)"
|
||||
echo " - Coverage reporting"
|
||||
echo " - Custom marker filtering"
|
||||
echo " - Parallel execution"
|
||||
echo " Usage: bash scripts/run_tests.sh [OPTIONS]"
|
||||
echo " Examples:"
|
||||
echo " bash scripts/run_tests.sh -t unit"
|
||||
echo " bash scripts/run_tests.sh -m 'not slow' -f"
|
||||
echo " bash scripts/run_tests.sh -p"
|
||||
echo ""
|
||||
echo -e "${GREEN}3. coverage_report.sh${NC}"
|
||||
echo " Generate detailed coverage reports"
|
||||
echo " - HTML, JSON, and terminal reports"
|
||||
echo " - Configurable coverage thresholds"
|
||||
echo " - Can open HTML report in browser"
|
||||
echo " Usage: bash scripts/coverage_report.sh [OPTIONS]"
|
||||
echo " Examples:"
|
||||
echo " bash scripts/coverage_report.sh -o"
|
||||
echo " bash scripts/coverage_report.sh -m 90"
|
||||
echo ""
|
||||
echo -e "${GREEN}4. ci_test.sh${NC}"
|
||||
echo " CI/CD optimized test runner"
|
||||
echo " - JUnit XML output"
|
||||
echo " - Coverage XML for CI tools"
|
||||
echo " - Environment variable configuration"
|
||||
echo " - Skips Docker-dependent tests"
|
||||
echo " Usage: bash scripts/ci_test.sh [OPTIONS]"
|
||||
echo " Examples:"
|
||||
echo " bash scripts/ci_test.sh -f -m 90"
|
||||
echo " CI_PARALLEL=true bash scripts/ci_test.sh"
|
||||
echo ""
|
||||
echo -e "${CYAN}Common Options:${NC}"
|
||||
echo " -t, --type Test type (unit, integration, e2e, all)"
|
||||
echo " -m, --markers Pytest markers expression"
|
||||
echo " -f, --fail-fast Stop on first failure"
|
||||
echo " -p, --parallel Run tests in parallel"
|
||||
echo " -n, --no-coverage Skip coverage reporting"
|
||||
echo " -v, --verbose Verbose output"
|
||||
echo " -h, --help Show help"
|
||||
echo ""
|
||||
echo -e "${CYAN}Test Markers:${NC}"
|
||||
echo " unit - Fast, isolated unit tests"
|
||||
echo " integration - Tests with real dependencies"
|
||||
echo " e2e - End-to-end tests (requires Docker)"
|
||||
echo " slow - Tests taking >10 seconds"
|
||||
echo " performance - Performance benchmarks"
|
||||
echo " security - Security tests"
|
||||
echo ""
|
||||
echo -e "Press any key to return to menu..."
|
||||
read -n 1 -s
|
||||
}
|
||||
|
||||
# Main menu loop
|
||||
if [ $# -eq 0 ]; then
|
||||
# Interactive mode
|
||||
while true; do
|
||||
show_menu
|
||||
read -n 1 choice
|
||||
echo ""
|
||||
|
||||
case $choice in
|
||||
1)
|
||||
run_quick_test
|
||||
;;
|
||||
2)
|
||||
run_full_test
|
||||
;;
|
||||
3)
|
||||
run_coverage
|
||||
;;
|
||||
4)
|
||||
run_unit
|
||||
;;
|
||||
5)
|
||||
run_integration
|
||||
;;
|
||||
6)
|
||||
run_e2e
|
||||
;;
|
||||
7)
|
||||
run_custom_markers
|
||||
;;
|
||||
8)
|
||||
run_parallel
|
||||
;;
|
||||
9)
|
||||
run_ci
|
||||
;;
|
||||
h|H)
|
||||
show_help
|
||||
;;
|
||||
q|Q)
|
||||
echo -e "${GREEN}Goodbye!${NC}"
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}Invalid option${NC}"
|
||||
sleep 1
|
||||
;;
|
||||
esac
|
||||
|
||||
if [ $? -eq 0 ]; then
|
||||
echo ""
|
||||
echo -e "${GREEN}Operation completed successfully!${NC}"
|
||||
else
|
||||
echo ""
|
||||
echo -e "${RED}Operation failed!${NC}"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
read -p "Press Enter to continue..."
|
||||
done
|
||||
else
|
||||
# Non-interactive: forward to run_tests.sh
|
||||
bash "$SCRIPT_DIR/run_tests.sh" "$@"
|
||||
fi
|
||||
@@ -55,7 +55,7 @@ def test_complete_async_download_flow(test_client, monkeypatch):
|
||||
monkeypatch.setattr("api.price_data_manager.PriceDataManager", MockPriceManager)
|
||||
|
||||
# Mock execution to avoid actual trading
|
||||
def mock_execute_date(self, date, models, config_path):
|
||||
def mock_execute_date(self, date, models, config_path, completion_skips=None):
|
||||
# Update job details to simulate successful execution
|
||||
from api.job_manager import JobManager
|
||||
job_manager = JobManager(db_path=test_client.app.state.db_path)
|
||||
@@ -155,7 +155,7 @@ def test_flow_with_partial_data(test_client, monkeypatch):
|
||||
|
||||
monkeypatch.setattr("api.price_data_manager.PriceDataManager", MockPriceManagerPartial)
|
||||
|
||||
def mock_execute_date(self, date, models, config_path):
|
||||
def mock_execute_date(self, date, models, config_path, completion_skips=None):
|
||||
# Update job details to simulate successful execution
|
||||
from api.job_manager import JobManager
|
||||
job_manager = JobManager(db_path=test_client.app.state.db_path)
|
||||
|
||||
@@ -26,7 +26,7 @@ def test_worker_prepares_data_before_execution(tmp_path):
|
||||
|
||||
def mock_prepare(*args, **kwargs):
|
||||
prepare_called.append(True)
|
||||
return (["2025-10-01"], []) # Return available dates, no warnings
|
||||
return (["2025-10-01"], [], {}) # Return available dates, no warnings, no completion skips
|
||||
|
||||
worker._prepare_data = mock_prepare
|
||||
|
||||
@@ -55,7 +55,7 @@ def test_worker_handles_no_available_dates(tmp_path):
|
||||
worker = SimulationWorker(job_id=job_id, db_path=db_path)
|
||||
|
||||
# Mock _prepare_data to return empty dates
|
||||
worker._prepare_data = Mock(return_value=([], []))
|
||||
worker._prepare_data = Mock(return_value=([], [], {}))
|
||||
|
||||
# Run worker
|
||||
result = worker.run()
|
||||
@@ -84,7 +84,7 @@ def test_worker_stores_warnings(tmp_path):
|
||||
|
||||
# Mock _prepare_data to return warnings
|
||||
warnings = ["Rate limited", "Skipped 1 date"]
|
||||
worker._prepare_data = Mock(return_value=(["2025-10-01"], warnings))
|
||||
worker._prepare_data = Mock(return_value=(["2025-10-01"], warnings, {}))
|
||||
worker._execute_date = Mock()
|
||||
|
||||
# Run worker
|
||||
|
||||
@@ -18,21 +18,21 @@ from unittest.mock import Mock, patch, MagicMock, AsyncMock
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def create_mock_agent(positions=None, last_trade=None, current_prices=None,
|
||||
reasoning_steps=None, tool_usage=None, session_result=None,
|
||||
def create_mock_agent(reasoning_steps=None, tool_usage=None, session_result=None,
|
||||
conversation_history=None):
|
||||
"""Helper to create properly mocked agent."""
|
||||
mock_agent = Mock()
|
||||
|
||||
# Default values
|
||||
mock_agent.get_positions.return_value = positions or {"CASH": 10000.0}
|
||||
mock_agent.get_last_trade.return_value = last_trade
|
||||
mock_agent.get_current_prices.return_value = current_prices or {}
|
||||
# Note: Removed get_positions, get_last_trade, get_current_prices
|
||||
# These methods don't exist in BaseAgent and were only used by
|
||||
# the now-deleted _write_results_to_db() method
|
||||
|
||||
mock_agent.get_reasoning_steps.return_value = reasoning_steps or []
|
||||
mock_agent.get_tool_usage.return_value = tool_usage or {}
|
||||
mock_agent.get_conversation_history.return_value = conversation_history or []
|
||||
|
||||
# Async methods - use AsyncMock
|
||||
mock_agent.set_context = AsyncMock()
|
||||
mock_agent.run_trading_session = AsyncMock(return_value=session_result or {"success": True})
|
||||
mock_agent.generate_summary = AsyncMock(return_value="Mock summary")
|
||||
mock_agent.summarize_message = AsyncMock(return_value="Mock message summary")
|
||||
@@ -93,23 +93,33 @@ class TestModelDayExecutorInitialization:
|
||||
class TestModelDayExecutorExecution:
|
||||
"""Test trading session execution."""
|
||||
|
||||
def test_execute_success(self, clean_db, sample_job_data):
|
||||
def test_execute_success(self, clean_db, sample_job_data, tmp_path):
|
||||
"""Should execute trading session and write results to DB."""
|
||||
from api.model_day_executor import ModelDayExecutor
|
||||
from api.job_manager import JobManager
|
||||
import json
|
||||
|
||||
# Create a temporary config file
|
||||
config_path = tmp_path / "test_config.json"
|
||||
config_data = {
|
||||
"agent_type": "BaseAgent",
|
||||
"models": [],
|
||||
"agent_config": {
|
||||
"initial_cash": 10000.0
|
||||
}
|
||||
}
|
||||
config_path.write_text(json.dumps(config_data))
|
||||
|
||||
# Create job and job_detail
|
||||
manager = JobManager(db_path=clean_db)
|
||||
job_id = manager.create_job(
|
||||
config_path="configs/test.json",
|
||||
config_path=str(config_path),
|
||||
date_range=["2025-01-16"],
|
||||
models=["gpt-5"]
|
||||
)
|
||||
|
||||
# Mock agent execution
|
||||
mock_agent = create_mock_agent(
|
||||
positions={"AAPL": 10, "CASH": 7500.0},
|
||||
current_prices={"AAPL": 250.0},
|
||||
session_result={"success": True, "total_steps": 15, "stop_signal_received": True}
|
||||
)
|
||||
|
||||
@@ -122,7 +132,7 @@ class TestModelDayExecutorExecution:
|
||||
job_id=job_id,
|
||||
date="2025-01-16",
|
||||
model_sig="gpt-5",
|
||||
config_path="configs/test.json",
|
||||
config_path=str(config_path),
|
||||
db_path=clean_db
|
||||
)
|
||||
|
||||
@@ -182,25 +192,34 @@ class TestModelDayExecutorExecution:
|
||||
class TestModelDayExecutorDataPersistence:
|
||||
"""Test result persistence to SQLite."""
|
||||
|
||||
def test_writes_position_to_database(self, clean_db):
|
||||
"""Should write position record to SQLite."""
|
||||
def test_creates_initial_position(self, clean_db, tmp_path):
|
||||
"""Should create initial position record (action_id=0) on first day."""
|
||||
from api.model_day_executor import ModelDayExecutor
|
||||
from api.job_manager import JobManager
|
||||
from api.database import get_db_connection
|
||||
import json
|
||||
|
||||
# Create a temporary config file
|
||||
config_path = tmp_path / "test_config.json"
|
||||
config_data = {
|
||||
"agent_type": "BaseAgent",
|
||||
"models": [],
|
||||
"agent_config": {
|
||||
"initial_cash": 10000.0
|
||||
}
|
||||
}
|
||||
config_path.write_text(json.dumps(config_data))
|
||||
|
||||
# Create job
|
||||
manager = JobManager(db_path=clean_db)
|
||||
job_id = manager.create_job(
|
||||
config_path="configs/test.json",
|
||||
config_path=str(config_path),
|
||||
date_range=["2025-01-16"],
|
||||
models=["gpt-5"]
|
||||
)
|
||||
|
||||
# Mock successful execution
|
||||
# Mock successful execution (no trades)
|
||||
mock_agent = create_mock_agent(
|
||||
positions={"AAPL": 10, "CASH": 7500.0},
|
||||
last_trade={"action": "buy", "symbol": "AAPL", "amount": 10, "price": 250.0},
|
||||
current_prices={"AAPL": 250.0},
|
||||
session_result={"success": True, "total_steps": 10}
|
||||
)
|
||||
|
||||
@@ -213,84 +232,32 @@ class TestModelDayExecutorDataPersistence:
|
||||
job_id=job_id,
|
||||
date="2025-01-16",
|
||||
model_sig="gpt-5",
|
||||
config_path="configs/test.json",
|
||||
config_path=str(config_path),
|
||||
db_path=clean_db
|
||||
)
|
||||
|
||||
with patch.object(executor, '_initialize_agent', return_value=mock_agent):
|
||||
executor.execute()
|
||||
|
||||
# Verify position written to database
|
||||
# Verify initial position created (action_id=0)
|
||||
conn = get_db_connection(clean_db)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
SELECT job_id, date, model, action_id, action_type
|
||||
SELECT job_id, date, model, action_id, action_type, cash, portfolio_value
|
||||
FROM positions
|
||||
WHERE job_id = ? AND date = ? AND model = ?
|
||||
""", (job_id, "2025-01-16", "gpt-5"))
|
||||
|
||||
row = cursor.fetchone()
|
||||
assert row is not None
|
||||
assert row is not None, "Should create initial position record"
|
||||
assert row[0] == job_id
|
||||
assert row[1] == "2025-01-16"
|
||||
assert row[2] == "gpt-5"
|
||||
|
||||
conn.close()
|
||||
|
||||
def test_writes_holdings_to_database(self, clean_db):
|
||||
"""Should write holdings records to SQLite."""
|
||||
from api.model_day_executor import ModelDayExecutor
|
||||
from api.job_manager import JobManager
|
||||
from api.database import get_db_connection
|
||||
|
||||
# Create job
|
||||
manager = JobManager(db_path=clean_db)
|
||||
job_id = manager.create_job(
|
||||
config_path="configs/test.json",
|
||||
date_range=["2025-01-16"],
|
||||
models=["gpt-5"]
|
||||
)
|
||||
|
||||
# Mock successful execution
|
||||
mock_agent = create_mock_agent(
|
||||
positions={"AAPL": 10, "MSFT": 5, "CASH": 7500.0},
|
||||
current_prices={"AAPL": 250.0, "MSFT": 300.0},
|
||||
session_result={"success": True}
|
||||
)
|
||||
|
||||
with patch("api.model_day_executor.RuntimeConfigManager") as mock_runtime:
|
||||
mock_instance = Mock()
|
||||
mock_instance.create_runtime_config.return_value = "/tmp/runtime_test.json"
|
||||
mock_runtime.return_value = mock_instance
|
||||
|
||||
executor = ModelDayExecutor(
|
||||
job_id=job_id,
|
||||
date="2025-01-16",
|
||||
model_sig="gpt-5",
|
||||
config_path="configs/test.json",
|
||||
db_path=clean_db
|
||||
)
|
||||
|
||||
with patch.object(executor, '_initialize_agent', return_value=mock_agent):
|
||||
executor.execute()
|
||||
|
||||
# Verify holdings written
|
||||
conn = get_db_connection(clean_db)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
SELECT h.symbol, h.quantity
|
||||
FROM holdings h
|
||||
JOIN positions p ON h.position_id = p.id
|
||||
WHERE p.job_id = ? AND p.date = ? AND p.model = ?
|
||||
ORDER BY h.symbol
|
||||
""", (job_id, "2025-01-16", "gpt-5"))
|
||||
|
||||
holdings = cursor.fetchall()
|
||||
assert len(holdings) == 3
|
||||
assert holdings[0][0] == "AAPL"
|
||||
assert holdings[0][1] == 10.0
|
||||
assert row[3] == 0, "Initial position should have action_id=0"
|
||||
assert row[4] == "no_trade"
|
||||
assert row[5] == 10000.0, "Initial cash should be $10,000"
|
||||
assert row[6] == 10000.0, "Initial portfolio value should be $10,000"
|
||||
|
||||
conn.close()
|
||||
|
||||
@@ -310,7 +277,6 @@ class TestModelDayExecutorDataPersistence:
|
||||
|
||||
# Mock execution with reasoning
|
||||
mock_agent = create_mock_agent(
|
||||
positions={"CASH": 10000.0},
|
||||
reasoning_steps=[
|
||||
{"step": 1, "reasoning": "Analyzing market data"},
|
||||
{"step": 2, "reasoning": "Evaluating risk"}
|
||||
@@ -361,7 +327,6 @@ class TestModelDayExecutorCleanup:
|
||||
)
|
||||
|
||||
mock_agent = create_mock_agent(
|
||||
positions={"CASH": 10000.0},
|
||||
session_result={"success": True}
|
||||
)
|
||||
|
||||
@@ -421,57 +386,10 @@ class TestModelDayExecutorCleanup:
|
||||
class TestModelDayExecutorPositionCalculations:
|
||||
"""Test position and P&L calculations."""
|
||||
|
||||
@pytest.mark.skip(reason="Method _calculate_portfolio_value() removed - portfolio value calculated by trade tools")
|
||||
def test_calculates_portfolio_value(self, clean_db):
|
||||
"""Should calculate total portfolio value."""
|
||||
from api.model_day_executor import ModelDayExecutor
|
||||
from api.job_manager import JobManager
|
||||
from api.database import get_db_connection
|
||||
|
||||
manager = JobManager(db_path=clean_db)
|
||||
job_id = manager.create_job(
|
||||
config_path="configs/test.json",
|
||||
date_range=["2025-01-16"],
|
||||
models=["gpt-5"]
|
||||
)
|
||||
|
||||
mock_agent = create_mock_agent(
|
||||
positions={"AAPL": 10, "CASH": 7500.0}, # 10 shares @ $250 = $2500
|
||||
current_prices={"AAPL": 250.0},
|
||||
session_result={"success": True}
|
||||
)
|
||||
|
||||
with patch("api.model_day_executor.RuntimeConfigManager") as mock_runtime:
|
||||
mock_instance = Mock()
|
||||
mock_instance.create_runtime_config.return_value = "/tmp/runtime_test.json"
|
||||
mock_runtime.return_value = mock_instance
|
||||
|
||||
executor = ModelDayExecutor(
|
||||
job_id=job_id,
|
||||
date="2025-01-16",
|
||||
model_sig="gpt-5",
|
||||
config_path="configs/test.json",
|
||||
db_path=clean_db
|
||||
)
|
||||
|
||||
with patch.object(executor, '_initialize_agent', return_value=mock_agent):
|
||||
executor.execute()
|
||||
|
||||
# Verify portfolio value calculated correctly
|
||||
conn = get_db_connection(clean_db)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
SELECT portfolio_value
|
||||
FROM positions
|
||||
WHERE job_id = ? AND date = ? AND model = ?
|
||||
""", (job_id, "2025-01-16", "gpt-5"))
|
||||
|
||||
row = cursor.fetchone()
|
||||
assert row is not None
|
||||
# Portfolio value should be 2500 (stocks) + 7500 (cash) = 10000
|
||||
assert row[0] == 10000.0
|
||||
|
||||
conn.close()
|
||||
"""DEPRECATED: Portfolio value is now calculated by trade tools, not ModelDayExecutor."""
|
||||
pass
|
||||
|
||||
|
||||
# Coverage target: 90%+ for api/model_day_executor.py
|
||||
|
||||
@@ -211,56 +211,7 @@ async def test_store_reasoning_logs_with_tool_messages(test_db):
|
||||
conn.close()
|
||||
|
||||
|
||||
@pytest.mark.skip(reason="Method _write_results_to_db() removed - positions written by trade tools")
|
||||
def test_write_results_includes_session_id(test_db):
|
||||
"""Should include session_id when writing positions."""
|
||||
from agent.mock_provider.mock_langchain_model import MockChatModel
|
||||
from agent.base_agent.base_agent import BaseAgent
|
||||
|
||||
executor = ModelDayExecutor(
|
||||
job_id="test-job",
|
||||
date="2025-01-01",
|
||||
model_sig="test-model",
|
||||
config_path="configs/default_config.json",
|
||||
db_path=test_db
|
||||
)
|
||||
|
||||
# Create mock agent with positions
|
||||
agent = BaseAgent(
|
||||
signature="test-model",
|
||||
basemodel="mock",
|
||||
stock_symbols=["AAPL"],
|
||||
init_date="2025-01-01"
|
||||
)
|
||||
agent.model = MockChatModel(model="test", signature="test")
|
||||
|
||||
# Mock positions data
|
||||
agent.positions = {"AAPL": 10, "CASH": 8500.0}
|
||||
agent.last_trade = {"action": "buy", "symbol": "AAPL", "amount": 10, "price": 150.0}
|
||||
agent.current_prices = {"AAPL": 150.0}
|
||||
|
||||
# Add required methods
|
||||
agent.get_positions = lambda: agent.positions
|
||||
agent.get_last_trade = lambda: agent.last_trade
|
||||
agent.get_current_prices = lambda: agent.current_prices
|
||||
|
||||
conn = get_db_connection(test_db)
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Create session
|
||||
session_id = executor._create_trading_session(cursor)
|
||||
conn.commit()
|
||||
|
||||
# Write results
|
||||
executor._write_results_to_db(agent, session_id)
|
||||
|
||||
# Verify position has session_id
|
||||
cursor.execute("SELECT * FROM positions WHERE job_id = ? AND model = ?",
|
||||
("test-job", "test-model"))
|
||||
position = cursor.fetchone()
|
||||
|
||||
assert position is not None
|
||||
assert position['session_id'] == session_id
|
||||
assert position['action_type'] == 'buy'
|
||||
assert position['symbol'] == 'AAPL'
|
||||
|
||||
conn.close()
|
||||
"""DEPRECATED: This test verified _write_results_to_db() which has been removed."""
|
||||
pass
|
||||
|
||||
309
tests/unit/test_position_tracking_bugs.py
Normal file
309
tests/unit/test_position_tracking_bugs.py
Normal file
@@ -0,0 +1,309 @@
|
||||
"""
|
||||
Tests demonstrating position tracking bugs before fix.
|
||||
|
||||
These tests should FAIL before implementing fixes, and PASS after.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from datetime import datetime
|
||||
from api.database import get_db_connection, initialize_database
|
||||
from api.job_manager import JobManager
|
||||
from agent_tools.tool_trade import _buy_impl
|
||||
from tools.price_tools import add_no_trade_record_to_db
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@pytest.fixture(scope="function")
|
||||
def test_db_with_prices():
|
||||
"""
|
||||
Create test database with price data using production database path.
|
||||
|
||||
Note: Since agent_tools hardcode db_path="data/jobs.db", we must use
|
||||
the production database path for integration testing.
|
||||
"""
|
||||
# Use production database path
|
||||
db_path = "data/jobs.db"
|
||||
|
||||
# Ensure directory exists
|
||||
Path(db_path).parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Initialize database
|
||||
initialize_database(db_path)
|
||||
|
||||
# Clear existing test data if any
|
||||
conn = get_db_connection(db_path)
|
||||
cursor = conn.cursor()
|
||||
|
||||
# Clean up any existing test data (in correct order for foreign keys)
|
||||
cursor.execute("DELETE FROM holdings WHERE position_id IN (SELECT id FROM positions WHERE model = 'claude-sonnet-4.5')")
|
||||
cursor.execute("DELETE FROM positions WHERE model = 'claude-sonnet-4.5'")
|
||||
cursor.execute("DELETE FROM trading_sessions WHERE model = 'claude-sonnet-4.5'")
|
||||
cursor.execute("DELETE FROM job_details WHERE model = 'claude-sonnet-4.5'")
|
||||
cursor.execute("DELETE FROM price_data WHERE symbol = 'NVDA' AND date IN ('2025-10-06', '2025-10-07')")
|
||||
|
||||
# Mark any pending/running jobs as completed to allow new test jobs
|
||||
cursor.execute("UPDATE jobs SET status = 'completed' WHERE status IN ('pending', 'running')")
|
||||
|
||||
# Insert price data for testing
|
||||
# 2025-10-06 prices
|
||||
cursor.execute("""
|
||||
INSERT INTO price_data (symbol, date, open, high, low, close, volume, created_at)
|
||||
VALUES ('NVDA', '2025-10-06', 185.5, 190.0, 185.0, 188.0, 1000000, ?)
|
||||
""", (datetime.utcnow().isoformat() + "Z",))
|
||||
|
||||
# 2025-10-07 prices (Monday after weekend)
|
||||
cursor.execute("""
|
||||
INSERT INTO price_data (symbol, date, open, high, low, close, volume, created_at)
|
||||
VALUES ('NVDA', '2025-10-07', 186.23, 190.0, 186.0, 189.0, 1000000, ?)
|
||||
""", (datetime.utcnow().isoformat() + "Z",))
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
yield db_path
|
||||
|
||||
# Cleanup after test
|
||||
conn = get_db_connection(db_path)
|
||||
cursor = conn.cursor()
|
||||
cursor.execute("DELETE FROM holdings WHERE position_id IN (SELECT id FROM positions WHERE model = 'claude-sonnet-4.5')")
|
||||
cursor.execute("DELETE FROM positions WHERE model = 'claude-sonnet-4.5'")
|
||||
cursor.execute("DELETE FROM trading_sessions WHERE model = 'claude-sonnet-4.5'")
|
||||
cursor.execute("DELETE FROM job_details WHERE model = 'claude-sonnet-4.5'")
|
||||
cursor.execute("DELETE FROM price_data WHERE symbol = 'NVDA' AND date IN ('2025-10-06', '2025-10-07')")
|
||||
|
||||
# Mark any pending/running jobs as completed
|
||||
cursor.execute("UPDATE jobs SET status = 'completed' WHERE status IN ('pending', 'running')")
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
|
||||
@pytest.mark.unit
|
||||
class TestPositionTrackingBugs:
|
||||
"""Tests demonstrating the three critical bugs."""
|
||||
|
||||
def test_cash_not_reset_between_days(self, test_db_with_prices):
|
||||
"""
|
||||
Bug #1: Cash should carry over from previous day, not reset to initial value.
|
||||
|
||||
Scenario:
|
||||
- Day 1: Start with $10,000, buy 5 NVDA @ $185.50 = $927.50, cash left = $9,072.50
|
||||
- Day 2: Should start with $9,072.50 cash, not $10,000
|
||||
"""
|
||||
# Create job
|
||||
manager = JobManager(db_path=test_db_with_prices)
|
||||
job_id = manager.create_job(
|
||||
config_path="configs/test.json",
|
||||
date_range=["2025-10-06", "2025-10-07"],
|
||||
models=["claude-sonnet-4.5"]
|
||||
)
|
||||
|
||||
# Day 1: Initial position (action_id=0)
|
||||
conn = get_db_connection(test_db_with_prices)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
INSERT INTO trading_sessions (job_id, date, model, started_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
""", (job_id, "2025-10-06", "claude-sonnet-4.5", datetime.utcnow().isoformat() + "Z"))
|
||||
session_id_day1 = cursor.lastrowid
|
||||
|
||||
cursor.execute("""
|
||||
INSERT INTO positions (
|
||||
job_id, date, model, action_id, action_type,
|
||||
cash, portfolio_value, session_id, created_at
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""", (
|
||||
job_id, "2025-10-06", "claude-sonnet-4.5", 0, "no_trade",
|
||||
10000.0, 10000.0, session_id_day1, datetime.utcnow().isoformat() + "Z"
|
||||
))
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
# Day 1: Buy 5 NVDA @ $185.50
|
||||
result = _buy_impl(
|
||||
symbol="NVDA",
|
||||
amount=5,
|
||||
signature="claude-sonnet-4.5",
|
||||
today_date="2025-10-06",
|
||||
job_id=job_id,
|
||||
session_id=session_id_day1
|
||||
)
|
||||
|
||||
assert "error" not in result
|
||||
assert result["CASH"] == 9072.5 # 10000 - (5 * 185.5)
|
||||
|
||||
# Day 2: Create new session
|
||||
conn = get_db_connection(test_db_with_prices)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
INSERT INTO trading_sessions (job_id, date, model, started_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
""", (job_id, "2025-10-07", "claude-sonnet-4.5", datetime.utcnow().isoformat() + "Z"))
|
||||
session_id_day2 = cursor.lastrowid
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
# Day 2: Check starting cash (should be $9,072.50, not $10,000)
|
||||
from agent_tools.tool_trade import get_current_position_from_db
|
||||
|
||||
position, next_action_id = get_current_position_from_db(
|
||||
job_id=job_id,
|
||||
model="claude-sonnet-4.5",
|
||||
date="2025-10-07"
|
||||
)
|
||||
|
||||
# BUG: This will fail before fix - cash resets to $10,000 or $0
|
||||
assert position["CASH"] == 9072.5, f"Expected cash $9,072.50 but got ${position['CASH']}"
|
||||
assert position["NVDA"] == 5, f"Expected 5 NVDA shares but got {position.get('NVDA', 0)}"
|
||||
|
||||
def test_positions_persist_over_weekend(self, test_db_with_prices):
|
||||
"""
|
||||
Bug #2: Positions should persist over non-trading days (weekends).
|
||||
|
||||
Scenario:
|
||||
- Friday 2025-10-06: Buy 5 NVDA
|
||||
- Monday 2025-10-07: Should still have 5 NVDA
|
||||
"""
|
||||
# Create job
|
||||
manager = JobManager(db_path=test_db_with_prices)
|
||||
job_id = manager.create_job(
|
||||
config_path="configs/test.json",
|
||||
date_range=["2025-10-06", "2025-10-07"],
|
||||
models=["claude-sonnet-4.5"]
|
||||
)
|
||||
|
||||
# Friday: Initial position + buy
|
||||
conn = get_db_connection(test_db_with_prices)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
INSERT INTO trading_sessions (job_id, date, model, started_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
""", (job_id, "2025-10-06", "claude-sonnet-4.5", datetime.utcnow().isoformat() + "Z"))
|
||||
session_id = cursor.lastrowid
|
||||
|
||||
cursor.execute("""
|
||||
INSERT INTO positions (
|
||||
job_id, date, model, action_id, action_type,
|
||||
cash, portfolio_value, session_id, created_at
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""", (
|
||||
job_id, "2025-10-06", "claude-sonnet-4.5", 0, "no_trade",
|
||||
10000.0, 10000.0, session_id, datetime.utcnow().isoformat() + "Z"
|
||||
))
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
_buy_impl(
|
||||
symbol="NVDA",
|
||||
amount=5,
|
||||
signature="claude-sonnet-4.5",
|
||||
today_date="2025-10-06",
|
||||
job_id=job_id,
|
||||
session_id=session_id
|
||||
)
|
||||
|
||||
# Monday: Check positions persist
|
||||
from agent_tools.tool_trade import get_current_position_from_db
|
||||
|
||||
position, _ = get_current_position_from_db(
|
||||
job_id=job_id,
|
||||
model="claude-sonnet-4.5",
|
||||
date="2025-10-07"
|
||||
)
|
||||
|
||||
# BUG: This will fail before fix - positions lost, holdings=[]
|
||||
assert "NVDA" in position, "NVDA position should persist over weekend"
|
||||
assert position["NVDA"] == 5, f"Expected 5 NVDA shares but got {position.get('NVDA', 0)}"
|
||||
|
||||
def test_profit_calculation_accuracy(self, test_db_with_prices):
|
||||
"""
|
||||
Bug #3: Profit should reflect actual gains/losses, not show trades as losses.
|
||||
|
||||
Scenario:
|
||||
- Start with $10,000 cash, portfolio value = $10,000
|
||||
- Buy 5 NVDA @ $185.50 = $927.50
|
||||
- New position: cash = $9,072.50, 5 NVDA worth $927.50
|
||||
- Portfolio value = $9,072.50 + $927.50 = $10,000 (unchanged)
|
||||
- Expected profit = $0 (no price change yet, just traded)
|
||||
|
||||
Current bug: Shows profit = -$927.50 or similar (treating trade as loss)
|
||||
"""
|
||||
# Create job
|
||||
manager = JobManager(db_path=test_db_with_prices)
|
||||
job_id = manager.create_job(
|
||||
config_path="configs/test.json",
|
||||
date_range=["2025-10-06"],
|
||||
models=["claude-sonnet-4.5"]
|
||||
)
|
||||
|
||||
# Create session and initial position
|
||||
conn = get_db_connection(test_db_with_prices)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
INSERT INTO trading_sessions (job_id, date, model, started_at)
|
||||
VALUES (?, ?, ?, ?)
|
||||
""", (job_id, "2025-10-06", "claude-sonnet-4.5", datetime.utcnow().isoformat() + "Z"))
|
||||
session_id = cursor.lastrowid
|
||||
|
||||
cursor.execute("""
|
||||
INSERT INTO positions (
|
||||
job_id, date, model, action_id, action_type,
|
||||
cash, portfolio_value, daily_profit, daily_return_pct,
|
||||
session_id, created_at
|
||||
)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""", (
|
||||
job_id, "2025-10-06", "claude-sonnet-4.5", 0, "no_trade",
|
||||
10000.0, 10000.0, None, None,
|
||||
session_id, datetime.utcnow().isoformat() + "Z"
|
||||
))
|
||||
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
# Buy 5 NVDA @ $185.50
|
||||
_buy_impl(
|
||||
symbol="NVDA",
|
||||
amount=5,
|
||||
signature="claude-sonnet-4.5",
|
||||
today_date="2025-10-06",
|
||||
job_id=job_id,
|
||||
session_id=session_id
|
||||
)
|
||||
|
||||
# Check profit calculation
|
||||
conn = get_db_connection(test_db_with_prices)
|
||||
cursor = conn.cursor()
|
||||
|
||||
cursor.execute("""
|
||||
SELECT portfolio_value, daily_profit, daily_return_pct
|
||||
FROM positions
|
||||
WHERE job_id = ? AND model = ? AND date = ? AND action_id = 1
|
||||
""", (job_id, "claude-sonnet-4.5", "2025-10-06"))
|
||||
|
||||
row = cursor.fetchone()
|
||||
conn.close()
|
||||
|
||||
portfolio_value = row[0]
|
||||
daily_profit = row[1]
|
||||
daily_return_pct = row[2]
|
||||
|
||||
# Portfolio value should be $10,000 (cash $9,072.50 + 5 NVDA @ $185.50)
|
||||
assert abs(portfolio_value - 10000.0) < 0.01, \
|
||||
f"Expected portfolio value $10,000 but got ${portfolio_value}"
|
||||
|
||||
# BUG: This will fail before fix - shows profit as negative or zero when should be zero
|
||||
# Profit should be $0 (no price movement, just traded)
|
||||
assert abs(daily_profit) < 0.01, \
|
||||
f"Expected profit $0 (no price change) but got ${daily_profit}"
|
||||
assert abs(daily_return_pct) < 0.01, \
|
||||
f"Expected return 0% but got {daily_return_pct}%"
|
||||
@@ -50,7 +50,7 @@ class TestSimulationWorkerExecution:
|
||||
worker = SimulationWorker(job_id=job_id, db_path=clean_db)
|
||||
|
||||
# Mock _prepare_data to return both dates
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16", "2025-01-17"], []))
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16", "2025-01-17"], [], {}))
|
||||
|
||||
# Mock ModelDayExecutor
|
||||
with patch("api.simulation_worker.ModelDayExecutor") as mock_executor_class:
|
||||
@@ -82,7 +82,7 @@ class TestSimulationWorkerExecution:
|
||||
worker = SimulationWorker(job_id=job_id, db_path=clean_db)
|
||||
|
||||
# Mock _prepare_data to return both dates
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16", "2025-01-17"], []))
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16", "2025-01-17"], [], {}))
|
||||
|
||||
execution_order = []
|
||||
|
||||
@@ -127,7 +127,7 @@ class TestSimulationWorkerExecution:
|
||||
worker = SimulationWorker(job_id=job_id, db_path=clean_db)
|
||||
|
||||
# Mock _prepare_data to return the date
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16"], []))
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16"], [], {}))
|
||||
|
||||
def create_mock_executor(job_id, date, model_sig, config_path, db_path):
|
||||
"""Create mock executor that simulates job detail status updates."""
|
||||
@@ -168,7 +168,7 @@ class TestSimulationWorkerExecution:
|
||||
worker = SimulationWorker(job_id=job_id, db_path=clean_db)
|
||||
|
||||
# Mock _prepare_data to return the date
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16"], []))
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16"], [], {}))
|
||||
|
||||
call_count = 0
|
||||
|
||||
@@ -223,7 +223,7 @@ class TestSimulationWorkerErrorHandling:
|
||||
worker = SimulationWorker(job_id=job_id, db_path=clean_db)
|
||||
|
||||
# Mock _prepare_data to return the date
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16"], []))
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16"], [], {}))
|
||||
|
||||
execution_count = 0
|
||||
|
||||
@@ -298,7 +298,7 @@ class TestSimulationWorkerConcurrency:
|
||||
worker = SimulationWorker(job_id=job_id, db_path=clean_db)
|
||||
|
||||
# Mock _prepare_data to return the date
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16"], []))
|
||||
worker._prepare_data = Mock(return_value=(["2025-01-16"], [], {}))
|
||||
|
||||
with patch("api.simulation_worker.ModelDayExecutor") as mock_executor_class:
|
||||
mock_executor = Mock()
|
||||
@@ -521,7 +521,7 @@ class TestSimulationWorkerHelperMethods:
|
||||
worker.job_manager.get_completed_model_dates = Mock(return_value={})
|
||||
|
||||
# Execute
|
||||
available_dates, warnings = worker._prepare_data(
|
||||
available_dates, warnings, completion_skips = worker._prepare_data(
|
||||
requested_dates=["2025-10-01"],
|
||||
models=["gpt-5"],
|
||||
config_path="config.json"
|
||||
@@ -570,7 +570,7 @@ class TestSimulationWorkerHelperMethods:
|
||||
worker.job_manager.get_completed_model_dates = Mock(return_value={})
|
||||
|
||||
# Execute
|
||||
available_dates, warnings = worker._prepare_data(
|
||||
available_dates, warnings, completion_skips = worker._prepare_data(
|
||||
requested_dates=["2025-10-01"],
|
||||
models=["gpt-5"],
|
||||
config_path="config.json"
|
||||
|
||||
@@ -414,20 +414,25 @@ def add_no_trade_record_to_db(
|
||||
logger.warning(f"Price not found for {symbol} on {today_date}")
|
||||
pass
|
||||
|
||||
# Get previous value for P&L
|
||||
# Get start-of-day portfolio value (action_id=0 for today) for P&L calculation
|
||||
cursor.execute("""
|
||||
SELECT portfolio_value
|
||||
FROM positions
|
||||
WHERE job_id = ? AND model = ? AND date < ?
|
||||
ORDER BY date DESC, action_id DESC
|
||||
WHERE job_id = ? AND model = ? AND date = ? AND action_id = 0
|
||||
LIMIT 1
|
||||
""", (job_id, modelname, today_date))
|
||||
|
||||
row = cursor.fetchone()
|
||||
previous_value = row[0] if row else 10000.0
|
||||
|
||||
daily_profit = portfolio_value - previous_value
|
||||
daily_return_pct = (daily_profit / previous_value * 100) if previous_value > 0 else 0
|
||||
if row:
|
||||
# Compare to start of day (action_id=0)
|
||||
start_of_day_value = row[0]
|
||||
daily_profit = portfolio_value - start_of_day_value
|
||||
daily_return_pct = (daily_profit / start_of_day_value * 100) if start_of_day_value > 0 else 0
|
||||
else:
|
||||
# First action of first day - no baseline yet
|
||||
daily_profit = 0.0
|
||||
daily_return_pct = 0.0
|
||||
|
||||
# Insert position record
|
||||
created_at = datetime.utcnow().isoformat() + "Z"
|
||||
|
||||
Reference in New Issue
Block a user