Compare commits

..

5 Commits

Author SHA1 Message Date
31e346ecbb debug: add logging to verify conversation history capture
Added debug output to confirm:
- How many messages are in conversation_history
- How many assistant vs tool messages
- Preview of first assistant message content
- What the summarizer receives

This will verify that the full detailed reasoning (like portfolio
analysis, trade execution details) is being captured and passed
to the summarizer.

Output will show:
[DEBUG] Generating summary from N messages
[DEBUG] Assistant messages: X, Tool messages: Y
[DEBUG] First assistant message preview: ...
[DEBUG ReasoningSummarizer] Formatting N messages
[DEBUG ReasoningSummarizer] Breakdown: X assistant, Y tool
2025-11-05 00:46:30 -05:00
abb9cd0726 fix: capture tool messages in conversation history for summarizer
**Root Cause:**
The summarizer was not receiving tool execution results (buy/sell trades)
because they were never captured to conversation_history.

**What was captured:**
- User: 'Please analyze positions'
- Assistant: 'I will buy/sell...'
- Assistant: 'Done <FINISH_SIGNAL>'

**What was MISSING:**
- Tool: buy 14 NVDA at $185.24
- Tool: sell 1 GOOGL at $245.15

**Changes:**
- Added tool message capture in trading loop (line 649)
- Extract tool_name and tool_content from each tool message
- Capture to conversation_history before processing
- Changed message['tool_name'] to message['name'] for consistency

**Impact:**
Now the summarizer sees the actual tool results, not just the AI's
intentions. Combined with alpha.8's prompt improvements, summaries
will accurately reflect executed trades.

Fixes reasoning summaries that contradicted actual trades.
2025-11-05 00:44:24 -05:00
6d126db03c fix: improve reasoning summary to explicitly mention trades
The reasoning summary was not accurately reflecting actual trades.
For example, 2 sell trades were summarized as 'maintain core holdings'.

Changes:
- Updated prompt to require explicit mention of trades executed
- Added emphasis on buy/sell tool calls in formatted log
- Trades now highlighted at top of log with TRADES EXECUTED section
- Prompt instructs: state specific trades (symbols, quantities, action)

Example before: 'chose to maintain core holdings'
Example after: 'sold 1 GOOGL and 1 AMZN to reduce exposure'

This ensures reasoning field accurately describes what the AI actually did.
2025-11-05 00:41:59 -05:00
1e7bdb509b chore: remove debug logging from ContextInjector
Removed noisy debug print statements that were added during
troubleshooting. The context injection is now working correctly
and no longer needs diagnostic output.

Cleaned up:
- Entry point logging
- Before/after injection logging
- Tool name and args logging
2025-11-05 00:31:16 -05:00
a8d912bb4b fix: calculate final holdings from actions instead of querying database
**Problem:**
Final positions showed empty holdings despite executing 15+ trades.
The issue persisted even after fixing the get_current_position_from_db query.

**Root Cause:**
At end of trading day, base_agent.py line 672 called
_get_current_portfolio_state() which queried the database for current
position. On the FIRST trading day, this query returns empty holdings
because there's no previous day's record.

**Why the Previous Fix Wasn't Enough:**
The previous fix (date < instead of date <=) correctly retrieves
STARTING position for subsequent days, but didn't address END-OF-DAY
position calculation, which needs to account for trades executed
during the current session.

**Solution:**
Added new method _calculate_final_position_from_actions() that:
1. Gets starting holdings from previous day (via get_starting_holdings)
2. Gets all actions from actions table for current trading day
3. Applies each buy/sell to calculate final state:
   - Buy: holdings[symbol] += qty, cash -= qty * price
   - Sell: holdings[symbol] -= qty, cash += qty * price
4. Returns accurate final holdings and cash

**Impact:**
- First trading day: Correctly saves all executed trades as final holdings
- Subsequent days: Final position reflects all trades from that day
- Holdings now persist correctly across all trading days

**Tests:**
- test_calculate_final_position_first_day_with_trades: 15 trades on first day
- test_calculate_final_position_with_previous_holdings: Multi-day scenario
- test_calculate_final_position_no_trades: No-trade edge case

All tests pass 
2025-11-04 23:51:54 -05:00
4 changed files with 323 additions and 17 deletions

View File

@@ -319,6 +319,60 @@ class BaseAgent:
print(f"⚠️ Could not get position from database: {e}")
return {}, self.initial_cash
def _calculate_final_position_from_actions(
self,
trading_day_id: int,
starting_cash: float
) -> tuple[Dict[str, int], float]:
"""
Calculate final holdings and cash from starting position + actions.
This is the correct way to get end-of-day position: start with the
starting position and apply all trades from the actions table.
Args:
trading_day_id: The trading day ID
starting_cash: Cash at start of day
Returns:
(holdings_dict, final_cash) where holdings_dict maps symbol -> quantity
"""
from api.database import Database
db = Database()
# 1. Get starting holdings (from previous day's ending)
starting_holdings_list = db.get_starting_holdings(trading_day_id)
holdings = {h["symbol"]: h["quantity"] for h in starting_holdings_list}
# 2. Initialize cash
cash = starting_cash
# 3. Get all actions for this trading day
actions = db.get_actions(trading_day_id)
# 4. Apply each action to calculate final state
for action in actions:
symbol = action["symbol"]
quantity = action["quantity"]
price = action["price"]
action_type = action["action_type"]
if action_type == "buy":
# Add to holdings
holdings[symbol] = holdings.get(symbol, 0) + quantity
# Deduct from cash
cash -= quantity * price
elif action_type == "sell":
# Remove from holdings
holdings[symbol] = holdings.get(symbol, 0) - quantity
# Add to cash
cash += quantity * price
# 5. Return final state
return holdings, cash
def _calculate_portfolio_value(
self,
holdings: Dict[str, int],
@@ -365,7 +419,7 @@ class BaseAgent:
}
if tool_name:
message["tool_name"] = tool_name
message["name"] = tool_name # Use "name" not "tool_name" for consistency with summarizer
if tool_input:
message["tool_input"] = tool_input
@@ -589,6 +643,11 @@ Summary:"""
tool_msgs = extract_tool_messages(response)
for tool_msg in tool_msgs:
tool_name = getattr(tool_msg, 'name', None) or tool_msg.get('name') if isinstance(tool_msg, dict) else None
tool_content = getattr(tool_msg, 'content', '') or tool_msg.get('content', '') if isinstance(tool_msg, dict) else str(tool_msg)
# Capture tool message to conversation history
self._capture_message("tool", tool_content, tool_name=tool_name)
if tool_name in ['buy', 'sell']:
action_count += 1
@@ -611,11 +670,26 @@ Summary:"""
session_duration = time.time() - session_start
# 7. Generate reasoning summary
# Debug: Log conversation history size
print(f"\n[DEBUG] Generating summary from {len(self.conversation_history)} messages")
assistant_msgs = [m for m in self.conversation_history if m.get('role') == 'assistant']
tool_msgs = [m for m in self.conversation_history if m.get('role') == 'tool']
print(f"[DEBUG] Assistant messages: {len(assistant_msgs)}, Tool messages: {len(tool_msgs)}")
if assistant_msgs:
first_assistant = assistant_msgs[0]
print(f"[DEBUG] First assistant message preview: {first_assistant.get('content', '')[:200]}...")
summarizer = ReasoningSummarizer(model=self.model)
summary = await summarizer.generate_summary(self.conversation_history)
# 8. Get current portfolio state from database
current_holdings, current_cash = self._get_current_portfolio_state(today_date, job_id)
# 8. Calculate final portfolio state from starting position + actions
# NOTE: We must calculate from actions, not query database, because:
# - On first day, database query returns empty (no previous day)
# - This method applies all trades to get accurate final state
current_holdings, current_cash = self._calculate_final_position_from_actions(
trading_day_id=trading_day_id,
starting_cash=starting_cash
)
# 9. Save final holdings to database
for symbol, quantity in current_holdings.items():

View File

@@ -52,10 +52,6 @@ class ContextInjector:
"""
# Inject context parameters for trade tools
if request.name in ["buy", "sell"]:
# Debug: Log self attributes BEFORE injection
print(f"[ContextInjector.__call__] ENTRY: id={id(self)}, self.signature={self.signature}, self.today_date={self.today_date}, self.job_id={self.job_id}, self.session_id={self.session_id}, self.trading_day_id={self.trading_day_id}")
print(f"[ContextInjector.__call__] Args BEFORE injection: {request.args}")
# ALWAYS inject/override context parameters (don't trust AI-provided values)
request.args["signature"] = self.signature
request.args["today_date"] = self.today_date
@@ -66,8 +62,5 @@ class ContextInjector:
if self.trading_day_id:
request.args["trading_day_id"] = self.trading_day_id
# Debug logging
print(f"[ContextInjector] Tool: {request.name}, Args after injection: {request.args}")
# Call the actual tool handler
return await handler(request)

View File

@@ -36,15 +36,17 @@ class ReasoningSummarizer:
summary_prompt = f"""You are reviewing your own trading decisions for the day.
Summarize your trading strategy and key decisions in 2-3 sentences.
IMPORTANT: Explicitly state what trades you executed (e.g., "sold 2 GOOGL shares" or "bought 10 NVDA shares"). If you made no trades, state that clearly.
Focus on:
- What you analyzed
- Why you made the trades you did
- What specific trades you executed (buy/sell, symbols, quantities)
- Why you made those trades
- Your overall strategy for the day
Trading session log:
{log_text}
Provide a concise summary:"""
Provide a concise summary that includes the actual trades executed:"""
response = await self.model.ainvoke([
{"role": "user", "content": summary_prompt}
@@ -67,21 +69,39 @@ Provide a concise summary:"""
reasoning_log: List of message dicts
Returns:
Formatted text representation
Formatted text representation with emphasis on trades
"""
# Debug: Log what we're formatting
print(f"[DEBUG ReasoningSummarizer] Formatting {len(reasoning_log)} messages")
assistant_count = sum(1 for m in reasoning_log if m.get('role') == 'assistant')
tool_count = sum(1 for m in reasoning_log if m.get('role') == 'tool')
print(f"[DEBUG ReasoningSummarizer] Breakdown: {assistant_count} assistant, {tool_count} tool")
formatted_parts = []
trades_executed = []
for msg in reasoning_log:
role = msg.get("role", "")
content = msg.get("content", "")
tool_name = msg.get("name", "")
if role == "assistant":
# AI's thoughts
formatted_parts.append(f"AI: {content[:200]}")
elif role == "tool":
# Tool results
tool_name = msg.get("name", "tool")
formatted_parts.append(f"{tool_name}: {content[:100]}")
# Highlight trade tool calls
if tool_name in ["buy", "sell"]:
trades_executed.append(f"{tool_name.upper()}: {content[:150]}")
formatted_parts.append(f"TRADE - {tool_name.upper()}: {content[:150]}")
else:
# Other tool results (search, price, etc.)
formatted_parts.append(f"{tool_name}: {content[:100]}")
# Add summary of trades at the top
if trades_executed:
trade_summary = f"TRADES EXECUTED ({len(trades_executed)}):\n" + "\n".join(trades_executed)
formatted_parts.insert(0, trade_summary)
formatted_parts.insert(1, "\n--- FULL LOG ---")
return "\n".join(formatted_parts)

View File

@@ -0,0 +1,219 @@
"""Test _calculate_final_position_from_actions method."""
import pytest
from unittest.mock import patch
from agent.base_agent.base_agent import BaseAgent
from api.database import Database
@pytest.fixture
def test_db():
"""Create test database with schema."""
db = Database(":memory:")
# Create jobs record
db.connection.execute("""
INSERT INTO jobs (job_id, config_path, status, date_range, models, created_at)
VALUES ('test-job', 'test.json', 'running', '2025-10-07 to 2025-10-07', 'gpt-5', '2025-10-07T00:00:00Z')
""")
db.connection.commit()
return db
def test_calculate_final_position_first_day_with_trades(test_db):
"""Test calculating final position on first trading day with multiple trades."""
# Create trading_day for first day
trading_day_id = test_db.create_trading_day(
job_id='test-job',
model='gpt-5',
date='2025-10-07',
starting_cash=10000.0,
starting_portfolio_value=10000.0,
daily_profit=0.0,
daily_return_pct=0.0,
ending_cash=10000.0, # Not yet calculated
ending_portfolio_value=10000.0, # Not yet calculated
days_since_last_trading=1
)
# Add 15 buy actions (matching your real data)
actions_data = [
("MSFT", 3, 528.285, "buy"),
("GOOGL", 6, 248.27, "buy"),
("NVDA", 10, 186.23, "buy"),
("LRCX", 6, 149.23, "buy"),
("AVGO", 2, 337.025, "buy"),
("AMZN", 5, 220.88, "buy"),
("MSFT", 2, 528.285, "buy"), # Additional MSFT
("AMD", 4, 214.85, "buy"),
("CRWD", 1, 497.0, "buy"),
("QCOM", 4, 169.9, "buy"),
("META", 1, 717.72, "buy"),
("NVDA", 20, 186.23, "buy"), # Additional NVDA
("NVDA", 13, 186.23, "buy"), # Additional NVDA
("NVDA", 20, 186.23, "buy"), # Additional NVDA
("NVDA", 53, 186.23, "buy"), # Additional NVDA
]
for symbol, quantity, price, action_type in actions_data:
test_db.create_action(
trading_day_id=trading_day_id,
action_type=action_type,
symbol=symbol,
quantity=quantity,
price=price
)
test_db.connection.commit()
# Create BaseAgent instance
agent = BaseAgent(signature="gpt-5", basemodel="anthropic/claude-sonnet-4", stock_symbols=[])
# Mock Database() to return our test_db
with patch('api.database.Database', return_value=test_db):
# Calculate final position
holdings, cash = agent._calculate_final_position_from_actions(
trading_day_id=trading_day_id,
starting_cash=10000.0
)
# Verify holdings
assert holdings["MSFT"] == 5, f"Expected 5 MSFT (3+2) but got {holdings.get('MSFT', 0)}"
assert holdings["GOOGL"] == 6, f"Expected 6 GOOGL but got {holdings.get('GOOGL', 0)}"
assert holdings["NVDA"] == 116, f"Expected 116 NVDA (10+20+13+20+53) but got {holdings.get('NVDA', 0)}"
assert holdings["LRCX"] == 6, f"Expected 6 LRCX but got {holdings.get('LRCX', 0)}"
assert holdings["AVGO"] == 2, f"Expected 2 AVGO but got {holdings.get('AVGO', 0)}"
assert holdings["AMZN"] == 5, f"Expected 5 AMZN but got {holdings.get('AMZN', 0)}"
assert holdings["AMD"] == 4, f"Expected 4 AMD but got {holdings.get('AMD', 0)}"
assert holdings["CRWD"] == 1, f"Expected 1 CRWD but got {holdings.get('CRWD', 0)}"
assert holdings["QCOM"] == 4, f"Expected 4 QCOM but got {holdings.get('QCOM', 0)}"
assert holdings["META"] == 1, f"Expected 1 META but got {holdings.get('META', 0)}"
# Verify cash (should be less than starting)
assert cash < 10000.0, f"Cash should be less than $10,000 but got ${cash}"
# Calculate expected cash
total_spent = sum(qty * price for _, qty, price, _ in actions_data)
expected_cash = 10000.0 - total_spent
assert abs(cash - expected_cash) < 0.01, f"Expected cash ${expected_cash} but got ${cash}"
def test_calculate_final_position_with_previous_holdings(test_db):
"""Test calculating final position when starting with existing holdings."""
# Create day 1 with ending holdings
day1_id = test_db.create_trading_day(
job_id='test-job',
model='gpt-5',
date='2025-10-06',
starting_cash=10000.0,
starting_portfolio_value=10000.0,
daily_profit=0.0,
daily_return_pct=0.0,
ending_cash=8000.0,
ending_portfolio_value=9500.0,
days_since_last_trading=1
)
# Add day 1 ending holdings
test_db.create_holding(day1_id, "AAPL", 10)
test_db.create_holding(day1_id, "MSFT", 5)
# Create day 2
day2_id = test_db.create_trading_day(
job_id='test-job',
model='gpt-5',
date='2025-10-07',
starting_cash=8000.0,
starting_portfolio_value=9500.0,
daily_profit=0.0,
daily_return_pct=0.0,
ending_cash=8000.0,
ending_portfolio_value=9500.0,
days_since_last_trading=1
)
# Add day 2 actions (buy more AAPL, sell some MSFT)
test_db.create_action(day2_id, "buy", "AAPL", 5, 150.0)
test_db.create_action(day2_id, "sell", "MSFT", 2, 500.0)
test_db.connection.commit()
# Create BaseAgent instance
agent = BaseAgent(signature="gpt-5", basemodel="anthropic/claude-sonnet-4", stock_symbols=[])
# Mock Database() to return our test_db
with patch('api.database.Database', return_value=test_db):
# Calculate final position for day 2
holdings, cash = agent._calculate_final_position_from_actions(
trading_day_id=day2_id,
starting_cash=8000.0
)
# Verify holdings
assert holdings["AAPL"] == 15, f"Expected 15 AAPL (10+5) but got {holdings.get('AAPL', 0)}"
assert holdings["MSFT"] == 3, f"Expected 3 MSFT (5-2) but got {holdings.get('MSFT', 0)}"
# Verify cash
# Started: 8000
# Buy 5 AAPL @ 150 = -750
# Sell 2 MSFT @ 500 = +1000
# Final: 8000 - 750 + 1000 = 8250
expected_cash = 8000.0 - (5 * 150.0) + (2 * 500.0)
assert abs(cash - expected_cash) < 0.01, f"Expected cash ${expected_cash} but got ${cash}"
def test_calculate_final_position_no_trades(test_db):
"""Test calculating final position when no trades were executed."""
# Create day 1 with ending holdings
day1_id = test_db.create_trading_day(
job_id='test-job',
model='gpt-5',
date='2025-10-06',
starting_cash=10000.0,
starting_portfolio_value=10000.0,
daily_profit=0.0,
daily_return_pct=0.0,
ending_cash=9000.0,
ending_portfolio_value=10000.0,
days_since_last_trading=1
)
test_db.create_holding(day1_id, "AAPL", 10)
# Create day 2 with NO actions
day2_id = test_db.create_trading_day(
job_id='test-job',
model='gpt-5',
date='2025-10-07',
starting_cash=9000.0,
starting_portfolio_value=10000.0,
daily_profit=0.0,
daily_return_pct=0.0,
ending_cash=9000.0,
ending_portfolio_value=10000.0,
days_since_last_trading=1
)
# No actions added
test_db.connection.commit()
# Create BaseAgent instance
agent = BaseAgent(signature="gpt-5", basemodel="anthropic/claude-sonnet-4", stock_symbols=[])
# Mock Database() to return our test_db
with patch('api.database.Database', return_value=test_db):
# Calculate final position
holdings, cash = agent._calculate_final_position_from_actions(
trading_day_id=day2_id,
starting_cash=9000.0
)
# Verify holdings unchanged
assert holdings["AAPL"] == 10, f"Expected 10 AAPL but got {holdings.get('AAPL', 0)}"
# Verify cash unchanged
assert abs(cash - 9000.0) < 0.01, f"Expected cash $9000 but got ${cash}"