docs: restructure documentation for improved clarity and navigation

Reorganize documentation into user-focused, developer-focused, and deployment-focused sections.

**New structure:**
- Root: README.md (streamlined), QUICK_START.md, API_REFERENCE.md
- docs/user-guide/: configuration, API usage, integrations, troubleshooting
- docs/developer/: contributing, development setup, testing, architecture
- docs/deployment/: Docker deployment, production checklist, monitoring
- docs/reference/: environment variables, MCP tools, data formats

**Changes:**
- Streamline README.md from 831 to 469 lines
- Create QUICK_START.md for 5-minute onboarding
- Create API_REFERENCE.md as single source of truth for API
- Remove 9 outdated specification docs (v0.2.0 API design)
- Remove DOCKER_API.md (content consolidated into new structure)
- Remove docs/plans/ directory with old design documents
- Update CLAUDE.md with documentation structure guide
- Remove orchestration-specific references

**Benefits:**
- Clear entry points for different audiences
- No content duplication
- Better discoverability through logical hierarchy
- All content reflects current v0.3.0 API

docs/deployment/docker-deployment.md

@@ -0,0 +1,95 @@
+# Docker Deployment
+
+Production Docker deployment guide.
+
+---
+
+## Quick Deployment
+
+```bash
+git clone https://github.com/Xe138/AI-Trader.git
+cd AI-Trader
+cp .env.example .env
+# Edit .env with API keys
+docker-compose up -d
+```
+
+---
+
+## Production Configuration
+
+### Use Pre-built Image
+
+```yaml
+# docker-compose.yml
+services:
+  ai-trader:
+    image: ghcr.io/xe138/ai-trader:latest
+    # ... rest of config
+```
+
+### Build Locally
+
+```yaml
+# docker-compose.yml
+services:
+  ai-trader:
+    build: .
+    # ... rest of config
+```
+
+---
+
+## Volume Persistence
+
+Ensure data persists across restarts:
+
+```yaml
+volumes:
+  - ./data:/app/data          # Required: database and cache
+  - ./logs:/app/logs          # Recommended: application logs
+  - ./configs:/app/configs    # Required: model configurations
+```
+
+---
+
+## Environment Security
+
+- Never commit `.env` to version control
+- Use secrets management (Docker secrets, Kubernetes secrets)
+- Rotate API keys regularly
+- Restrict network access to API port
+
+---
+
+## Health Checks
+
+Docker automatically restarts unhealthy containers:
+
+```yaml
+healthcheck:
+  test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
+  interval: 30s
+  timeout: 10s
+  retries: 3
+  start_period: 40s
+```
+
+---
+
+## Monitoring
+
+```bash
+# Container status
+docker ps
+
+# Resource usage
+docker stats ai-trader
+
+# Logs
+docker logs -f ai-trader
+```
+
+---
+
+See [DOCKER_API.md](../../DOCKER_API.md) for detailed Docker documentation.

docs/deployment/monitoring.md

@@ -0,0 +1,49 @@
+# Monitoring
+
+Health checks, logging, and metrics.
+
+---
+
+## Health Checks
+
+```bash
+# Manual check
+curl http://localhost:8080/health
+
+# Automated monitoring (cron)
+*/5 * * * * curl -f http://localhost:8080/health || echo "API down" | mail -s "Alert" admin@example.com
+```
+
+---
+
+## Logging
+
+```bash
+# View logs
+docker logs -f ai-trader
+
+# Filter errors
+docker logs ai-trader 2>&1 | grep -i error
+
+# Export logs
+docker logs ai-trader > ai-trader.log 2>&1
+```
+
+---
+
+## Database Monitoring
+
+```bash
+# Database size
+docker exec ai-trader du -h /app/data/jobs.db
+
+# Job statistics
+docker exec ai-trader sqlite3 /app/data/jobs.db \
+  "SELECT status, COUNT(*) FROM jobs GROUP BY status;"
+```
+
+---
+
+## Metrics (Future)
+
+Prometheus metrics planned for v0.4.0.

docs/deployment/production-checklist.md

@@ -0,0 +1,50 @@
+# Production Deployment Checklist
+
+Pre-deployment verification.
+
+---
+
+## Pre-Deployment
+
+- [ ] API keys configured in `.env`
+- [ ] Environment variables reviewed
+- [ ] Model configuration validated
+- [ ] Port availability confirmed
+- [ ] Volume mounts configured
+- [ ] Health checks enabled
+- [ ] Restart policy set
+
+---
+
+## Testing
+
+- [ ] `bash scripts/validate_docker_build.sh` passes
+- [ ] `bash scripts/test_api_endpoints.sh` passes
+- [ ] Health endpoint responds correctly
+- [ ] Sample simulation completes successfully
+
+---
+
+## Monitoring
+
+- [ ] Log aggregation configured
+- [ ] Health check monitoring enabled
+- [ ] Alerting configured for failures
+- [ ] Database backup strategy defined
+
+---
+
+## Security
+
+- [ ] API keys stored securely (not in code)
+- [ ] `.env` excluded from version control
+- [ ] Network access restricted
+- [ ] SSL/TLS configured (if exposing publicly)
+
+---
+
+## Documentation
+
+- [ ] Runbook created for operations team
+- [ ] Escalation procedures documented
+- [ ] Recovery procedures tested

docs/deployment/scaling.md

@@ -0,0 +1,46 @@
+# Scaling
+
+Running multiple instances and load balancing.
+
+---
+
+## Current Limitations
+
+- Maximum 1 concurrent job per instance
+- No built-in load balancing
+- Single SQLite database per instance
+
+---
+
+## Multi-Instance Deployment
+
+For parallel simulations, deploy multiple instances:
+
+```yaml
+# docker-compose.yml
+services:
+  ai-trader-1:
+    image: ghcr.io/xe138/ai-trader:latest
+    ports:
+      - "8081:8080"
+    volumes:
+      - ./data1:/app/data
+
+  ai-trader-2:
+    image: ghcr.io/xe138/ai-trader:latest
+    ports:
+      - "8082:8080"
+    volumes:
+      - ./data2:/app/data
+```
+
+**Note:** Each instance needs separate database and data volumes.
+
+---
+
+## Load Balancing (Future)
+
+Planned for v0.4.0:
+- Shared PostgreSQL database
+- Job queue with multiple workers
+- Horizontal scaling support