fix: set PYTHONPATH for MCP services to resolve import errors

Root cause: Python adds script directory (not working directory)
to sys.path. Services in /app/agent_tools/ couldn't import from
/app/tools/ because sys.path[0] was /app/agent_tools.

Solution: Set PYTHONPATH=/app when starting services so /app is
always in the Python import path.

Fixes: ModuleNotFoundError: No module named 'tools' in price.log

.env.example

@@ -5,21 +5,28 @@
 # Docker Compose automatically reads .env from project root
 
 # AI Model API Configuration
-OPENAI_API_BASE=https://your-openai-proxy.com/v1
-OPENAI_API_KEY=your_openai_key_here
+# OPENAI_API_BASE: Leave empty to use default OpenAI endpoint, or set to custom proxy URL
+OPENAI_API_BASE=
+OPENAI_API_KEY=your_openai_key_here # https://platform.openai.com/api-keys
 
 # Data Source Configuration
-ALPHAADVANTAGE_API_KEY=your_alphavantage_key_here
-JINA_API_KEY=your_jina_key_here
+ALPHAADVANTAGE_API_KEY=your_alphavantage_key_here  # https://www.alphavantage.co/support/#api-key
+JINA_API_KEY=your_jina_key_here # https://jina.ai/
 
 # System Configuration (Docker default paths)
 RUNTIME_ENV_PATH=/app/data/runtime_env.json
 
-# MCP Service Ports (defaults shown)
+# MCP Service Host Ports (exposed on host machine)
+# Container always uses 8000-8003 internally
+# Change these if you need different ports on your host
 MATH_HTTP_PORT=8000
 SEARCH_HTTP_PORT=8001
 TRADE_HTTP_PORT=8002
 GETPRICE_HTTP_PORT=8003
 
+# Web Interface Host Port (exposed on host machine)
+# Container always uses 8888 internally
+WEB_HTTP_PORT=8888
+
 # Agent Configuration
 AGENT_MAX_STEP=30

Dockerfile

@@ -15,6 +15,12 @@ WORKDIR /app
 # Copy application code
 COPY . .
 
+# Copy data scripts to separate directory (volume mount won't overlay these)
+RUN mkdir -p /app/scripts && \
+    cp data/get_daily_price.py /app/scripts/ && \
+    cp data/get_interdaily_price.py /app/scripts/ && \
+    cp data/merge_jsonl.py /app/scripts/
+
 # Create necessary directories
 RUN mkdir -p data logs data/agent_data
 

README.md

@@ -218,7 +218,7 @@ AI-Trader Bench/
 
 ```bash
 # 1. Clone project
-git clone https://github.com/HKUDS/AI-Trader.git
+git clone https://github.com/Xe138/AI-Trader.git
 cd AI-Trader
 
 # 2. Install dependencies
@@ -331,7 +331,7 @@ The easiest way to run AI-Trader is with Docker Compose:
 
 ```bash
 # 1. Clone and setup
-git clone https://github.com/HKUDS/AI-Trader.git
+git clone https://github.com/Xe138/AI-Trader.git
 cd AI-Trader
 
 # 2. Configure environment
@@ -590,8 +590,8 @@ We welcome contributions of all kinds! Especially AI trading strategies and agen
 
 ## 📞 Support & Community
 
-- **💬 Discussions**: [GitHub Discussions](https://github.com/HKUDS/AI-Trader/discussions)
-- **🐛 Issues**: [GitHub Issues](https://github.com/HKUDS/AI-Trader/issues)
+- **💬 Discussions**: [GitHub Discussions](https://github.com/Xe138/AI-Trader/discussions)
+- **🐛 Issues**: [GitHub Issues](https://github.com/Xe138/AI-Trader/issues)
 
 ## 📄 License
 
@@ -615,8 +615,8 @@ The materials provided by the AI-Trader project are for research purposes only a
 
 **🌟 If this project helps you, please give us a Star!**
 
-[![GitHub stars](https://img.shields.io/github/stars/HKUDS/AI-Trader?style=social)](https://github.com/HKUDS/AI-Trader)
-[![GitHub forks](https://img.shields.io/github/forks/HKUDS/AI-Trader?style=social)](https://github.com/HKUDS/AI-Trader)
+[![GitHub stars](https://img.shields.io/github/stars/Xe138/AI-Trader?style=social)](https://github.com/Xe138/AI-Trader)
+[![GitHub forks](https://img.shields.io/github/forks/Xe138/AI-Trader?style=social)](https://github.com/Xe138/AI-Trader)
 
 **🤖 Experience AI's full potential in financial markets through complete autonomous decision-making!**  
 **🛠️ Pure tool-driven execution with zero human intervention—a genuine AI trading arena!** 🚀

agent_tools/start_mcp_services.py

@@ -27,32 +27,33 @@ class MCPServiceManager:
             'price': int(os.getenv('GETPRICE_HTTP_PORT', '8003'))
         }
         
-        # Service configurations
+        # Service configurations (paths relative to /app)
+        self.agent_tools_dir = Path(__file__).parent.resolve()
         self.service_configs = {
             'math': {
-                'script': 'tool_math.py',
+                'script': self.agent_tools_dir / 'tool_math.py',
                 'name': 'Math',
                 'port': self.ports['math']
             },
             'search': {
-                'script': 'tool_jina_search.py',
+                'script': self.agent_tools_dir / 'tool_jina_search.py',
                 'name': 'Search',
                 'port': self.ports['search']
             },
             'trade': {
-                'script': 'tool_trade.py',
+                'script': self.agent_tools_dir / 'tool_trade.py',
                 'name': 'TradeTools',
                 'port': self.ports['trade']
             },
             'price': {
-                'script': 'tool_get_price_local.py',
+                'script': self.agent_tools_dir / 'tool_get_price_local.py',
                 'name': 'LocalPrices',
                 'port': self.ports['price']
             }
         }
-        
+
         # Create logs directory
-        self.log_dir = Path('../logs')
+        self.log_dir = Path('logs')
         self.log_dir.mkdir(exist_ok=True)
         
         # Set signal handlers
@@ -70,20 +71,26 @@ class MCPServiceManager:
         script_path = config['script']
         service_name = config['name']
         port = config['port']
-        
-        if not Path(script_path).exists():
+
+        if not script_path.exists():
             print(f"❌ Script file not found: {script_path}")
             return False
-        
+
         try:
             # Start service process
             log_file = self.log_dir / f"{service_id}.log"
+
+            # Set PYTHONPATH to /app so services can import from tools module
+            env = os.environ.copy()
+            env['PYTHONPATH'] = str(Path.cwd())
+
             with open(log_file, 'w') as f:
                 process = subprocess.Popen(
-                    [sys.executable, script_path],
+                    [sys.executable, str(script_path)],
                     stdout=f,
                     stderr=subprocess.STDOUT,
-                    cwd=os.getcwd()
+                    cwd=Path.cwd(),  # Use current working directory (/app)
+                    env=env  # Pass environment with PYTHONPATH
                 )
             
             self.services[service_id] = {

data/get_daily_price.py

@@ -26,10 +26,10 @@ def get_daily_price(SYMBOL: str):
     url = f'https://www.alphavantage.co/query?function={FUNCTION}&symbol={SYMBOL}&outputsize={OUTPUTSIZE}&apikey={APIKEY}'
     r = requests.get(url)
     data = r.json()
-    print(data)
     if data.get('Note') is not None or data.get('Information') is not None:
-        print(f"Error")
+        print(f"⚠️  {SYMBOL}: API rate limit or error - {data.get('Note') or data.get('Information')}")
         return
+    print(f"✓ Fetched {SYMBOL}")
     with open(f'./daily_prices_{SYMBOL}.json', 'w', encoding='utf-8') as f:
         json.dump(data, f, ensure_ascii=False, indent=4)
     if SYMBOL == "QQQ":

docker-compose.yml

@@ -1,6 +1,8 @@
 services:
   ai-trader:
-    build: .
+    image: ghcr.io/xe138/ai-trader:latest
+    # Uncomment to build locally instead of pulling:
+    # build: .
     container_name: ai-trader-app
     volumes:
       - ./data:/app/data
@@ -17,18 +19,19 @@ services:
       # System Configuration
       - RUNTIME_ENV_PATH=/app/data/runtime_env.json
 
-      # MCP Service Ports
-      - MATH_HTTP_PORT=${MATH_HTTP_PORT:-8000}
-      - SEARCH_HTTP_PORT=${SEARCH_HTTP_PORT:-8001}
-      - TRADE_HTTP_PORT=${TRADE_HTTP_PORT:-8002}
-      - GETPRICE_HTTP_PORT=${GETPRICE_HTTP_PORT:-8003}
+      # MCP Service Ports (fixed internally)
+      - MATH_HTTP_PORT=8000
+      - SEARCH_HTTP_PORT=8001
+      - TRADE_HTTP_PORT=8002
+      - GETPRICE_HTTP_PORT=8003
 
       # Agent Configuration
       - AGENT_MAX_STEP=${AGENT_MAX_STEP:-30}
     ports:
-      - "8000:8000"
-      - "8001:8001"
-      - "8002:8002"
-      - "8003:8003"
-      - "8888:8888"
-    restart: unless-stopped
+      # Format: "HOST:CONTAINER" - container ports are fixed, host ports configurable via .env
+      - "${MATH_HTTP_PORT:-8000}:8000"
+      - "${SEARCH_HTTP_PORT:-8001}:8001"
+      - "${TRADE_HTTP_PORT:-8002}:8002"
+      - "${GETPRICE_HTTP_PORT:-8003}:8003"
+      - "${WEB_HTTP_PORT:-8888}:8888"
+    restart: on-failure:3  # Restart max 3 times on failure, prevents endless loops

docs/plans/2025-10-30-data-cache-reuse-design.md

@@ -0,0 +1,197 @@
+# Data Cache Reuse Design
+
+**Date:** 2025-10-30
+**Status:** Approved
+
+## Problem Statement
+
+Docker containers currently fetch all 103 NASDAQ 100 tickers from Alpha Vantage on every startup, even when price data is volume-mounted and already cached in `./data`. This causes:
+- Slow startup times (103 API calls)
+- Unnecessary API quota consumption
+- Rate limit risks during frequent development iterations
+
+## Solution Overview
+
+Implement staleness-based data refresh with configurable age threshold. Container checks all `daily_prices_*.json` files and only refetches if any file is missing or older than `MAX_DATA_AGE_DAYS`.
+
+## Design Decisions
+
+### Architecture Choice
+**Selected:** Check all `daily_prices_*.json` files individually
+**Rationale:** Ensures data integrity by detecting partial/missing files, not just stale merged data
+
+### Implementation Location
+**Selected:** Bash wrapper logic in `entrypoint.sh`
+**Rationale:** Keeps data fetching scripts unchanged, adds orchestration at container startup layer
+
+### Staleness Threshold
+**Selected:** Configurable via `MAX_DATA_AGE_DAYS` environment variable (default: 7 days)
+**Rationale:** Balances freshness with API usage; flexible for different use cases (development vs production)
+
+## Technical Design
+
+### Components
+
+#### 1. Staleness Check Function
+Location: `entrypoint.sh` (after environment validation, before data fetch)
+
+```bash
+should_refresh_data() {
+    MAX_AGE=${MAX_DATA_AGE_DAYS:-7}
+
+    # Check if at least one price file exists
+    if ! ls /app/data/daily_prices_*.json >/dev/null 2>&1; then
+        echo "📭 No price data found"
+        return 0  # Need refresh
+    fi
+
+    # Find any files older than MAX_AGE days
+    STALE_COUNT=$(find /app/data -name "daily_prices_*.json" -mtime +$MAX_AGE | wc -l)
+    TOTAL_COUNT=$(ls /app/data/daily_prices_*.json 2>/dev/null | wc -l)
+
+    if [ $STALE_COUNT -gt 0 ]; then
+        echo "📅 Found $STALE_COUNT stale files (>$MAX_AGE days old)"
+        return 0  # Need refresh
+    fi
+
+    echo "✅ All $TOTAL_COUNT price files are fresh (<$MAX_AGE days old)"
+    return 1  # Skip refresh
+}
+```
+
+**Logic:**
+- Uses `find -mtime +N` to detect files modified more than N days ago
+- Returns shell exit codes: 0 (refresh needed), 1 (skip refresh)
+- Logs informative messages for debugging
+
+#### 2. Conditional Data Fetch
+Location: `entrypoint.sh` lines 40-46 (replace existing unconditional fetch)
+
+```bash
+# Step 1: Data preparation (conditional)
+echo "📊 Checking price data freshness..."
+
+if should_refresh_data; then
+    echo "🔄 Fetching and merging price data..."
+    cd /app/data
+    python /app/scripts/get_daily_price.py
+    python /app/scripts/merge_jsonl.py
+    cd /app
+else
+    echo "⏭️  Skipping data fetch (using cached data)"
+fi
+```
+
+#### 3. Environment Configuration
+**docker-compose.yml:**
+```yaml
+environment:
+  - MAX_DATA_AGE_DAYS=${MAX_DATA_AGE_DAYS:-7}
+```
+
+**.env.example:**
+```bash
+# Data Refresh Configuration
+MAX_DATA_AGE_DAYS=7  # Refresh price data older than N days (0=always refresh)
+```
+
+### Data Flow
+
+1. **Container Startup** → entrypoint.sh begins execution
+2. **Environment Validation** → Check required API keys (existing logic)
+3. **Staleness Check** → `should_refresh_data()` scans `/app/data/daily_prices_*.json`
+   - No files found → Return 0 (refresh)
+   - Any file older than `MAX_DATA_AGE_DAYS` → Return 0 (refresh)
+   - All files fresh → Return 1 (skip)
+4. **Conditional Fetch** → Run get_daily_price.py only if refresh needed
+5. **Merge Data** → Always run merge_jsonl.py (handles missing merged.jsonl)
+6. **MCP Services** → Start services (existing logic)
+7. **Trading Agent** → Begin trading (existing logic)
+
+### Edge Cases
+
+| Scenario | Behavior |
+|----------|----------|
+| **First run (no data)** | Detects no files → triggers full fetch |
+| **Restart within 7 days** | All files fresh → skips fetch (fast startup) |
+| **Restart after 7 days** | Files stale → refreshes all data |
+| **Partial data (some files missing)** | Missing files treated as infinitely old → triggers refresh |
+| **Corrupt merged.jsonl but fresh price files** | Skips fetch, re-runs merge to rebuild merged.jsonl |
+| **MAX_DATA_AGE_DAYS=0** | Always refresh (useful for testing/production) |
+| **MAX_DATA_AGE_DAYS unset** | Defaults to 7 days |
+| **Alpha Vantage rate limit** | get_daily_price.py handles with warning (existing behavior) |
+
+## Configuration Options
+
+| Variable | Default | Purpose |
+|----------|---------|---------|
+| `MAX_DATA_AGE_DAYS` | 7 | Days before price data considered stale |
+
+**Special Values:**
+- `0` → Always refresh (force fresh data)
+- `999` → Never refresh (use cached data indefinitely)
+
+## User Experience
+
+### Scenario 1: Fresh Container
+```
+🚀 Starting AI-Trader...
+🔍 Validating environment variables...
+✅ Environment variables validated
+📊 Checking price data freshness...
+📭 No price data found
+🔄 Fetching and merging price data...
+✓ Fetched NVDA
+✓ Fetched MSFT
+...
+```
+
+### Scenario 2: Restart Within 7 Days
+```
+🚀 Starting AI-Trader...
+🔍 Validating environment variables...
+✅ Environment variables validated
+📊 Checking price data freshness...
+✅ All 103 price files are fresh (<7 days old)
+⏭️  Skipping data fetch (using cached data)
+🔧 Starting MCP services...
+```
+
+### Scenario 3: Restart After 7 Days
+```
+🚀 Starting AI-Trader...
+🔍 Validating environment variables...
+✅ Environment variables validated
+📊 Checking price data freshness...
+📅 Found 103 stale files (>7 days old)
+🔄 Fetching and merging price data...
+✓ Fetched NVDA
+✓ Fetched MSFT
+...
+```
+
+## Testing Plan
+
+1. **Test fresh container:** Delete `./data/daily_prices_*.json`, start container → should fetch all
+2. **Test cached data:** Restart immediately → should skip fetch
+3. **Test staleness:** `touch -d "8 days ago" ./data/daily_prices_AAPL.json`, restart → should refresh
+4. **Test partial data:** Delete 10 random price files → should refresh all
+5. **Test MAX_DATA_AGE_DAYS=0:** Restart with env var set → should always fetch
+6. **Test MAX_DATA_AGE_DAYS=30:** Restart with 8-day-old data → should skip
+
+## Documentation Updates
+
+Files requiring updates:
+- `entrypoint.sh` → Add function and conditional logic
+- `docker-compose.yml` → Add MAX_DATA_AGE_DAYS environment variable
+- `.env.example` → Document MAX_DATA_AGE_DAYS with default value
+- `CLAUDE.md` → Update "Docker Deployment" section with new env var
+- `docs/DOCKER.md` (if exists) → Explain data caching behavior
+
+## Benefits
+
+- **Development:** Instant container restarts during iteration
+- **API Quota:** ~103 fewer API calls per restart
+- **Reliability:** No rate limit risks during frequent testing
+- **Flexibility:** Configurable threshold for different use cases
+- **Consistency:** Checks all files to ensure complete data

entrypoint.sh

@@ -3,19 +3,53 @@ set -e  # Exit on any error
 
 echo "🚀 Starting AI-Trader..."
 
+# Validate required environment variables
+echo "🔍 Validating environment variables..."
+MISSING_VARS=()
+
+if [ -z "$OPENAI_API_KEY" ]; then
+    MISSING_VARS+=("OPENAI_API_KEY")
+fi
+
+if [ -z "$ALPHAADVANTAGE_API_KEY" ]; then
+    MISSING_VARS+=("ALPHAADVANTAGE_API_KEY")
+fi
+
+if [ -z "$JINA_API_KEY" ]; then
+    MISSING_VARS+=("JINA_API_KEY")
+fi
+
+if [ ${#MISSING_VARS[@]} -gt 0 ]; then
+    echo ""
+    echo "❌ ERROR: Missing required environment variables:"
+    for var in "${MISSING_VARS[@]}"; do
+        echo "   - $var"
+    done
+    echo ""
+    echo "Please set these variables in your .env file:"
+    echo "   1. Copy .env.example to .env"
+    echo "   2. Edit .env and add your API keys"
+    echo "   3. Restart the container"
+    echo ""
+    echo "See docs/DOCKER.md for more information."
+    exit 1
+fi
+
+echo "✅ Environment variables validated"
+
 # Step 1: Data preparation
 echo "📊 Fetching and merging price data..."
+# Run scripts from /app/scripts but output to /app/data
 cd /app/data
-python get_daily_price.py
-python merge_jsonl.py
+python /app/scripts/get_daily_price.py
+python /app/scripts/merge_jsonl.py
 cd /app
 
 # Step 2: Start MCP services in background
 echo "🔧 Starting MCP services..."
-cd /app/agent_tools
-python start_mcp_services.py &
-MCP_PID=$!
 cd /app
+python agent_tools/start_mcp_services.py &
+MCP_PID=$!
 
 # Step 3: Wait for services to initialize
 echo "⏳ Waiting for MCP services to start..."