fix: merge script now writes to current directory for volume compatibility

Changed merge_jsonl.py to use os.getcwd() instead of os.path.dirname(__file__)
to ensure merged.jsonl is written to the working directory where data files exist,
not to the script's installation directory.

Root cause:
- Dockerfile copies scripts to /app/scripts/ for volume compatibility
- entrypoint.sh runs: cd /app/data && python /app/scripts/merge_jsonl.py
- Old logic used script directory (/app/scripts/), ignoring working directory
- This caused merged.jsonl to be created in /app/scripts/ instead of /app/data/
- Since /app/data/ is volume-mounted, merged file was not visible to host

Solution:
- Scripts now respect current working directory (Unix philosophy)
- Works correctly with volume mounts and script relocation
- Tested in both local and Docker directory structure scenarios

Fixes the issue where merged.jsonl was missing from mounted data volume.

.env.example

@@ -30,3 +30,9 @@ WEB_HTTP_PORT=8888
 
 # Agent Configuration
 AGENT_MAX_STEP=30
+
+# Data Volume Configuration
+# Base directory for all persistent data (will contain data/, logs/, configs/ subdirectories)
+# Use relative paths (./volumes) or absolute paths (/home/user/ai-trader-volumes)
+# Defaults to current directory (.) if not set
+VOLUME_PATH=.

data/get_daily_price.py

@@ -1,8 +1,12 @@
-import requests
-import os
-from dotenv import load_dotenv
-load_dotenv()
 import json
+import os
+import subprocess
+import sys
+
+import requests
+from dotenv import load_dotenv
+
+load_dotenv()
 
 
 all_nasdaq_100_symbols = [
@@ -42,4 +46,15 @@ if __name__ == "__main__":
     for symbol in all_nasdaq_100_symbols:
         get_daily_price(symbol)
 
-    get_daily_price("QQQ")
+    get_daily_price("QQQ")
+
+    # Automatically run merge after fetching
+    print("\n📦 Merging price data...")
+    try:
+        script_dir = os.path.dirname(os.path.abspath(__file__))
+        merge_script = os.path.join(script_dir, "merge_jsonl.py")
+        subprocess.run([sys.executable, merge_script], check=True)
+        print("✅ Price data merged successfully")
+    except Exception as e:
+        print(f"⚠️  Failed to merge data: {e}")
+        print("   Please run 'python merge_jsonl.py' manually")

data/merge_jsonl.py

@@ -18,7 +18,8 @@ all_nasdaq_100_symbols = [
 ]
 
 # 合并所有以 daily_price 开头的 json，逐文件一行写入 merged.jsonl
-current_dir = os.path.dirname(__file__)
+# Use current working directory instead of script directory for volume compatibility
+current_dir = os.getcwd()
 pattern = os.path.join(current_dir, 'daily_price*.json')
 files = sorted(glob.glob(pattern))
 

docker-compose.yml

@@ -5,9 +5,9 @@ services:
     # build: .
     container_name: ai-trader-app
     volumes:
-      - ./data:/app/data
-      - ./logs:/app/logs
-      - ./configs:/app/configs
+      - ${VOLUME_PATH:-.}/data:/app/data
+      - ${VOLUME_PATH:-.}/logs:/app/logs
+      - ${VOLUME_PATH:-.}/configs:/app/configs
     environment:
       # AI Model API Configuration
       - OPENAI_API_BASE=${OPENAI_API_BASE}

docs/DOCKER.md

@@ -112,17 +112,43 @@ docker-compose up
 
 ### Volume Mounts
 
-Docker Compose mounts three volumes:
+Docker Compose mounts three volumes for persistent data. By default, these are stored in the project directory:
 
 - `./data:/app/data` - Price data and trading records
 - `./logs:/app/logs` - MCP service logs
 - `./configs:/app/configs` - Configuration files (allows editing configs without rebuilding)
 
-Data persists across container restarts. To reset:
+### Custom Volume Location
+
+You can change where data is stored by setting `VOLUME_PATH` in your `.env` file:
+
+```bash
+# Store data in a different location
+VOLUME_PATH=/home/user/trading-data
+
+# Or use a relative path
+VOLUME_PATH=./volumes
+```
+
+This will store data in:
+- `/home/user/trading-data/data/`
+- `/home/user/trading-data/logs/`
+- `/home/user/trading-data/configs/`
+
+**Note:** The directory structure is automatically created. You'll need to copy your existing configs:
+```bash
+# After changing VOLUME_PATH
+mkdir -p /home/user/trading-data/configs
+cp configs/custom_config.json /home/user/trading-data/configs/
+```
+
+### Reset Data
+
+To reset all trading data:
 
 ```bash
 docker-compose down
-rm -rf data/agent_data/* logs/*
+rm -rf ${VOLUME_PATH:-.}/data/agent_data/* ${VOLUME_PATH:-.}/logs/*
 docker-compose up
 ```
 

entrypoint.sh

@@ -38,12 +38,18 @@ fi
 echo "✅ Environment variables validated"
 
 # Step 1: Data preparation
-echo "📊 Fetching and merging price data..."
-# Run scripts from /app/scripts but output to /app/data
-cd /app/data
-python /app/scripts/get_daily_price.py
-python /app/scripts/merge_jsonl.py
-cd /app
+echo "📊 Checking price data..."
+if [ -f "/app/data/merged.jsonl" ] && [ -s "/app/data/merged.jsonl" ]; then
+    echo "✅ Using existing price data ($(wc -l < /app/data/merged.jsonl) stocks)"
+    echo "   To refresh data, delete /app/data/merged.jsonl and restart"
+else
+    echo "📊 Fetching and merging price data..."
+    # Run script from /app/scripts but output to /app/data
+    # Note: get_daily_price.py now automatically calls merge_jsonl.py after fetching
+    cd /app/data
+    python /app/scripts/get_daily_price.py
+    cd /app
+fi
 
 # Step 2: Start MCP services in background
 echo "🔧 Starting MCP services..."

main.sh

@@ -10,8 +10,8 @@ echo "🚀 Launching AI Trader Environment..."
 
 echo "📊 Now getting and merging price data..."
 cd ./data
+# Note: get_daily_price.py now automatically calls merge_jsonl.py after fetching
 python get_daily_price.py
-python merge_jsonl.py
 cd ../
 
 echo "🔧 Now starting MCP services..."