mirror of
https://github.com/Xe138/AI-Trader.git
synced 2026-04-02 01:27:24 -04:00
feat: add standardized testing scripts and documentation
Add comprehensive suite of testing scripts for different workflows: - test.sh: Interactive menu for all testing operations - quick_test.sh: Fast unit test feedback (~10-30s) - run_tests.sh: Main test runner with full configuration options - coverage_report.sh: Coverage analysis with HTML/JSON/terminal reports - ci_test.sh: CI/CD optimized testing with JUnit/coverage XML output Features: - Colored terminal output with clear error messages - Consistent option flags across all scripts - Support for test markers (unit, integration, e2e, slow, etc.) - Parallel execution support - Coverage thresholds (default: 85%) - Virtual environment and dependency checks Documentation: - Update CLAUDE.md with testing section and examples - Expand docs/developer/testing.md with comprehensive guide - Add scripts/README.md with quick reference All scripts are tested and executable. This standardizes the testing process for local development, CI/CD, and pull request workflows.
This commit is contained in:
@@ -1,15 +1,310 @@
|
||||
# Testing Guide
|
||||
|
||||
Guide for testing AI-Trader-Server during development.
|
||||
This guide covers running tests for the AI-Trader project, including unit tests, integration tests, and end-to-end tests.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Interactive test menu (recommended for local development)
|
||||
bash scripts/test.sh
|
||||
|
||||
# Quick unit tests (fast feedback)
|
||||
bash scripts/quick_test.sh
|
||||
|
||||
# Full test suite with coverage
|
||||
bash scripts/run_tests.sh
|
||||
|
||||
# Generate coverage report
|
||||
bash scripts/coverage_report.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Automated Testing
|
||||
## Test Scripts Overview
|
||||
|
||||
### 1. `test.sh` - Interactive Test Helper
|
||||
|
||||
**Purpose:** Interactive menu for common test operations
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Interactive mode
|
||||
bash scripts/test.sh
|
||||
|
||||
# Non-interactive mode
|
||||
bash scripts/test.sh -t unit -f
|
||||
```
|
||||
|
||||
**Menu Options:**
|
||||
1. Quick test (unit only, no coverage)
|
||||
2. Full test suite (with coverage)
|
||||
3. Coverage report
|
||||
4. Unit tests only
|
||||
5. Integration tests only
|
||||
6. E2E tests only
|
||||
7. Run with custom markers
|
||||
8. Parallel execution
|
||||
9. CI mode
|
||||
|
||||
---
|
||||
|
||||
### 2. `quick_test.sh` - Fast Feedback Loop
|
||||
|
||||
**Purpose:** Rapid test execution during development
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
bash scripts/quick_test.sh
|
||||
```
|
||||
|
||||
**When to use:**
|
||||
- During active development
|
||||
- Before committing code
|
||||
- Quick verification of changes
|
||||
- TDD workflow
|
||||
|
||||
---
|
||||
|
||||
### 3. `run_tests.sh` - Main Test Runner
|
||||
|
||||
**Purpose:** Comprehensive test execution with full configuration options
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Run all tests with coverage (default)
|
||||
bash scripts/run_tests.sh
|
||||
|
||||
# Run only unit tests
|
||||
bash scripts/run_tests.sh -t unit
|
||||
|
||||
# Run without coverage
|
||||
bash scripts/run_tests.sh -n
|
||||
|
||||
# Run with custom markers
|
||||
bash scripts/run_tests.sh -m "unit and not slow"
|
||||
|
||||
# Fail on first error
|
||||
bash scripts/run_tests.sh -f
|
||||
|
||||
# Run tests in parallel
|
||||
bash scripts/run_tests.sh -p
|
||||
```
|
||||
|
||||
**Options:**
|
||||
```
|
||||
-t, --type TYPE Test type: all, unit, integration, e2e (default: all)
|
||||
-m, --markers MARKERS Run tests matching markers
|
||||
-f, --fail-fast Stop on first failure
|
||||
-n, --no-coverage Skip coverage reporting
|
||||
-v, --verbose Verbose output
|
||||
-p, --parallel Run tests in parallel
|
||||
--no-html Skip HTML coverage report
|
||||
-h, --help Show help message
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 4. `coverage_report.sh` - Coverage Analysis
|
||||
|
||||
**Purpose:** Generate detailed coverage reports
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Generate coverage report (default: 85% threshold)
|
||||
bash scripts/coverage_report.sh
|
||||
|
||||
# Set custom coverage threshold
|
||||
bash scripts/coverage_report.sh -m 90
|
||||
|
||||
# Generate and open HTML report
|
||||
bash scripts/coverage_report.sh -o
|
||||
```
|
||||
|
||||
**Options:**
|
||||
```
|
||||
-m, --min-coverage NUM Minimum coverage percentage (default: 85)
|
||||
-o, --open Open HTML report in browser
|
||||
-i, --include-integration Include integration and e2e tests
|
||||
-h, --help Show help message
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. `ci_test.sh` - CI/CD Optimized Runner
|
||||
|
||||
**Purpose:** Test execution optimized for CI/CD environments
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Basic CI run
|
||||
bash scripts/ci_test.sh
|
||||
|
||||
# Fail fast with custom coverage
|
||||
bash scripts/ci_test.sh -f -m 90
|
||||
|
||||
# Using environment variables
|
||||
CI_FAIL_FAST=true CI_COVERAGE_MIN=90 bash scripts/ci_test.sh
|
||||
```
|
||||
|
||||
**Environment Variables:**
|
||||
```bash
|
||||
CI_FAIL_FAST=true # Enable fail-fast mode
|
||||
CI_COVERAGE_MIN=90 # Set coverage threshold
|
||||
CI_PARALLEL=true # Enable parallel execution
|
||||
CI_VERBOSE=true # Enable verbose output
|
||||
```
|
||||
|
||||
**Output artifacts:**
|
||||
- `junit.xml` - Test results for CI reporting
|
||||
- `coverage.xml` - Coverage data for CI tools
|
||||
- `htmlcov/` - HTML coverage report
|
||||
|
||||
---
|
||||
|
||||
## Test Structure
|
||||
|
||||
```
|
||||
tests/
|
||||
├── conftest.py # Shared pytest fixtures
|
||||
├── unit/ # Fast, isolated tests
|
||||
├── integration/ # Tests with dependencies
|
||||
├── e2e/ # End-to-end tests
|
||||
├── performance/ # Performance benchmarks
|
||||
└── security/ # Security tests
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Test Markers
|
||||
|
||||
Tests are organized using pytest markers:
|
||||
|
||||
| Marker | Description | Usage |
|
||||
|--------|-------------|-------|
|
||||
| `unit` | Fast, isolated unit tests | `-m unit` |
|
||||
| `integration` | Tests with real dependencies | `-m integration` |
|
||||
| `e2e` | End-to-end tests (requires Docker) | `-m e2e` |
|
||||
| `slow` | Tests taking >10 seconds | `-m slow` |
|
||||
| `performance` | Performance benchmarks | `-m performance` |
|
||||
| `security` | Security tests | `-m security` |
|
||||
|
||||
**Examples:**
|
||||
```bash
|
||||
# Run only unit tests
|
||||
bash scripts/run_tests.sh -m unit
|
||||
|
||||
# Run all except slow tests
|
||||
bash scripts/run_tests.sh -m "not slow"
|
||||
|
||||
# Combine markers
|
||||
bash scripts/run_tests.sh -m "unit and not slow"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Common Workflows
|
||||
|
||||
### During Development
|
||||
|
||||
```bash
|
||||
# Quick check before each commit
|
||||
bash scripts/quick_test.sh
|
||||
|
||||
# Run relevant test type
|
||||
bash scripts/run_tests.sh -t unit -f
|
||||
|
||||
# Full test before push
|
||||
bash scripts/run_tests.sh
|
||||
```
|
||||
|
||||
### Before Pull Request
|
||||
|
||||
```bash
|
||||
# Run full test suite
|
||||
bash scripts/run_tests.sh
|
||||
|
||||
# Generate coverage report
|
||||
bash scripts/coverage_report.sh -o
|
||||
|
||||
# Ensure coverage meets 85% threshold
|
||||
```
|
||||
|
||||
### CI/CD Pipeline
|
||||
|
||||
```bash
|
||||
# Run CI-optimized tests
|
||||
bash scripts/ci_test.sh -f -m 85
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Debugging Test Failures
|
||||
|
||||
```bash
|
||||
# Run with verbose output
|
||||
bash scripts/run_tests.sh -v -f
|
||||
|
||||
# Run specific test file
|
||||
./venv/bin/python -m pytest tests/unit/test_database.py -v
|
||||
|
||||
# Run specific test function
|
||||
./venv/bin/python -m pytest tests/unit/test_database.py::test_function -v
|
||||
|
||||
# Run with debugger on failure
|
||||
./venv/bin/python -m pytest --pdb tests/
|
||||
|
||||
# Show print statements
|
||||
./venv/bin/python -m pytest -s tests/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Coverage Configuration
|
||||
|
||||
Configured in `pytest.ini`:
|
||||
- Minimum coverage: 85%
|
||||
- Target coverage: 90%
|
||||
- Coverage reports: HTML, JSON, terminal
|
||||
|
||||
---
|
||||
|
||||
## Writing New Tests
|
||||
|
||||
### Unit Test Example
|
||||
|
||||
```python
|
||||
import pytest
|
||||
|
||||
@pytest.mark.unit
|
||||
def test_function_returns_expected_value():
|
||||
# Arrange
|
||||
input_data = {"key": "value"}
|
||||
|
||||
# Act
|
||||
result = my_function(input_data)
|
||||
|
||||
# Assert
|
||||
assert result == expected_output
|
||||
```
|
||||
|
||||
### Integration Test Example
|
||||
|
||||
```python
|
||||
@pytest.mark.integration
|
||||
def test_database_integration(clean_db):
|
||||
conn = get_db_connection(clean_db)
|
||||
insert_data(conn, test_data)
|
||||
result = query_data(conn)
|
||||
assert len(result) == 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Docker Testing
|
||||
|
||||
### Docker Build Validation
|
||||
|
||||
```bash
|
||||
chmod +x scripts/*.sh
|
||||
bash scripts/validate_docker_build.sh
|
||||
```
|
||||
|
||||
@@ -30,35 +325,16 @@ Tests all API endpoints with real simulations.
|
||||
|
||||
---
|
||||
|
||||
## Unit Tests
|
||||
## Summary
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Run tests
|
||||
pytest tests/ -v
|
||||
|
||||
# With coverage
|
||||
pytest tests/ -v --cov=api --cov-report=term-missing
|
||||
|
||||
# Specific test file
|
||||
pytest tests/unit/test_job_manager.py -v
|
||||
```
|
||||
| Script | Purpose | Speed | Coverage | Use Case |
|
||||
|--------|---------|-------|----------|----------|
|
||||
| `test.sh` | Interactive menu | Varies | Optional | Local development |
|
||||
| `quick_test.sh` | Fast feedback | ⚡⚡⚡ | No | Active development |
|
||||
| `run_tests.sh` | Full test suite | ⚡⚡ | Yes | Pre-commit, pre-PR |
|
||||
| `coverage_report.sh` | Coverage analysis | ⚡ | Yes | Coverage review |
|
||||
| `ci_test.sh` | CI/CD pipeline | ⚡⚡ | Yes | Automation |
|
||||
|
||||
---
|
||||
|
||||
## Integration Tests
|
||||
|
||||
```bash
|
||||
# Run integration tests only
|
||||
pytest tests/integration/ -v
|
||||
|
||||
# Test with real API server
|
||||
docker-compose up -d
|
||||
pytest tests/integration/test_api_endpoints.py -v
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
For detailed testing procedures, see root [TESTING_GUIDE.md](../../TESTING_GUIDE.md).
|
||||
For detailed testing procedures and troubleshooting, see [TESTING_GUIDE.md](../../TESTING_GUIDE.md).
|
||||
|
||||
Reference in New Issue
Block a user