feat: automate merged.jsonl creation during price fetching

Streamline the data preparation workflow by having get_daily_price.py
automatically invoke merge_jsonl.py after fetching all stock prices.

Changes:
- Modified get_daily_price.py to call merge_jsonl.py automatically
- Updated entrypoint.sh to remove redundant merge_jsonl.py call
- Updated main.sh to remove redundant merge_jsonl.py call
- Fixed import order for linting compliance

Benefits:
- Single command now handles both fetching and merging
- Ensures merged.jsonl is always created after price updates
- Simplifies Docker container startup process
- Prevents missing merged.jsonl errors in production

.env.example

@@ -30,3 +30,9 @@ WEB_HTTP_PORT=8888
 
 # Agent Configuration
 AGENT_MAX_STEP=30
+
+# Data Volume Configuration
+# Base directory for all persistent data (will contain data/, logs/, configs/ subdirectories)
+# Use relative paths (./volumes) or absolute paths (/home/user/ai-trader-volumes)
+# Defaults to current directory (.) if not set
+VOLUME_PATH=.

data/get_daily_price.py

@@ -1,8 +1,12 @@
-import requests
-import os
-from dotenv import load_dotenv
-load_dotenv()
 import json
+import os
+import subprocess
+import sys
+
+import requests
+from dotenv import load_dotenv
+
+load_dotenv()
 
 
 all_nasdaq_100_symbols = [
@@ -42,4 +46,15 @@ if __name__ == "__main__":
     for symbol in all_nasdaq_100_symbols:
         get_daily_price(symbol)
 
-    get_daily_price("QQQ")
+    get_daily_price("QQQ")
+
+    # Automatically run merge after fetching
+    print("\n📦 Merging price data...")
+    try:
+        script_dir = os.path.dirname(os.path.abspath(__file__))
+        merge_script = os.path.join(script_dir, "merge_jsonl.py")
+        subprocess.run([sys.executable, merge_script], check=True)
+        print("✅ Price data merged successfully")
+    except Exception as e:
+        print(f"⚠️  Failed to merge data: {e}")
+        print("   Please run 'python merge_jsonl.py' manually")

docker-compose.yml

@@ -5,9 +5,9 @@ services:
     # build: .
     container_name: ai-trader-app
     volumes:
-      - ./data:/app/data
-      - ./logs:/app/logs
-      - ./configs:/app/configs
+      - ${VOLUME_PATH:-.}/data:/app/data
+      - ${VOLUME_PATH:-.}/logs:/app/logs
+      - ${VOLUME_PATH:-.}/configs:/app/configs
     environment:
       # AI Model API Configuration
       - OPENAI_API_BASE=${OPENAI_API_BASE}

docs/DOCKER.md

@@ -112,17 +112,43 @@ docker-compose up
 
 ### Volume Mounts
 
-Docker Compose mounts three volumes:
+Docker Compose mounts three volumes for persistent data. By default, these are stored in the project directory:
 
 - `./data:/app/data` - Price data and trading records
 - `./logs:/app/logs` - MCP service logs
 - `./configs:/app/configs` - Configuration files (allows editing configs without rebuilding)
 
-Data persists across container restarts. To reset:
+### Custom Volume Location
+
+You can change where data is stored by setting `VOLUME_PATH` in your `.env` file:
+
+```bash
+# Store data in a different location
+VOLUME_PATH=/home/user/trading-data
+
+# Or use a relative path
+VOLUME_PATH=./volumes
+```
+
+This will store data in:
+- `/home/user/trading-data/data/`
+- `/home/user/trading-data/logs/`
+- `/home/user/trading-data/configs/`
+
+**Note:** The directory structure is automatically created. You'll need to copy your existing configs:
+```bash
+# After changing VOLUME_PATH
+mkdir -p /home/user/trading-data/configs
+cp configs/custom_config.json /home/user/trading-data/configs/
+```
+
+### Reset Data
+
+To reset all trading data:
 
 ```bash
 docker-compose down
-rm -rf data/agent_data/* logs/*
+rm -rf ${VOLUME_PATH:-.}/data/agent_data/* ${VOLUME_PATH:-.}/logs/*
 docker-compose up
 ```
 

entrypoint.sh

@@ -44,10 +44,10 @@ if [ -f "/app/data/merged.jsonl" ] && [ -s "/app/data/merged.jsonl" ]; then
     echo "   To refresh data, delete /app/data/merged.jsonl and restart"
 else
     echo "📊 Fetching and merging price data..."
-    # Run scripts from /app/scripts but output to /app/data
+    # Run script from /app/scripts but output to /app/data
+    # Note: get_daily_price.py now automatically calls merge_jsonl.py after fetching
     cd /app/data
     python /app/scripts/get_daily_price.py
-    python /app/scripts/merge_jsonl.py
     cd /app
 fi
 

main.sh

@@ -10,8 +10,8 @@ echo "🚀 Launching AI Trader Environment..."
 
 echo "📊 Now getting and merging price data..."
 cd ./data
+# Note: get_daily_price.py now automatically calls merge_jsonl.py after fetching
 python get_daily_price.py
-python merge_jsonl.py
 cd ../
 
 echo "🔧 Now starting MCP services..."