feat: add resume mode and idempotent behavior to /simulate/trigger endpoint

BREAKING CHANGE: end_date is now required and cannot be null/empty

New Features:
- Resume mode: Set start_date to null to continue from last completed date per model
- Idempotent by default: Skip already-completed dates with replace_existing=false
- Per-model independence: Each model resumes from its own last completed date
- Cold start handling: If no data exists in resume mode, runs only end_date as single day

API Changes:
- start_date: Now optional (null enables resume mode)
- end_date: Now REQUIRED (cannot be null or empty string)
- replace_existing: New optional field (default: false for idempotent behavior)

Implementation:
- Added JobManager.get_last_completed_date_for_model() method
- Added JobManager.get_completed_model_dates() method
- Updated create_job() to support model_day_filter for selective task creation
- Fixed bug with start_date=None in price data checks

Documentation:
- Updated API_REFERENCE.md with complete examples and behavior matrix
- Updated QUICK_START.md with resume mode examples
- Updated docs/user-guide/using-the-api.md
- Added CHANGELOG_NEW_API.md with migration guide
- Updated all integration tests for new schema
- Updated client library examples (Python, TypeScript)

Migration:
- Old: {"start_date": "2025-01-16"}
- New: {"start_date": "2025-01-16", "end_date": "2025-01-16"}
- Resume: {"start_date": null, "end_date": "2025-01-31"}

See CHANGELOG_NEW_API.md for complete details.
This commit is contained in:
2025-11-01 13:34:20 -04:00
parent b9353e34e5
commit 7aa93af6db
7 changed files with 747 additions and 94 deletions

View File

@@ -54,7 +54,8 @@ class JobManager:
self,
config_path: str,
date_range: List[str],
models: List[str]
models: List[str],
model_day_filter: Optional[List[tuple]] = None
) -> str:
"""
Create new simulation job.
@@ -63,6 +64,8 @@ class JobManager:
config_path: Path to configuration file
date_range: List of dates to simulate (YYYY-MM-DD)
models: List of model signatures to execute
model_day_filter: Optional list of (model, date) tuples to limit job_details.
If None, creates job_details for all model-date combinations.
Returns:
job_id: UUID of created job
@@ -95,9 +98,10 @@ class JobManager:
created_at
))
# Create job_details for each model-day combination
for date in date_range:
for model in models:
# Create job_details based on filter
if model_day_filter is not None:
# Only create job_details for specified model-day pairs
for model, date in model_day_filter:
cursor.execute("""
INSERT INTO job_details (
job_id, date, model, status
@@ -105,8 +109,21 @@ class JobManager:
VALUES (?, ?, ?, ?)
""", (job_id, date, model, "pending"))
logger.info(f"Created job {job_id} with {len(model_day_filter)} model-day tasks (filtered)")
else:
# Create job_details for all model-day combinations
for date in date_range:
for model in models:
cursor.execute("""
INSERT INTO job_details (
job_id, date, model, status
)
VALUES (?, ?, ?, ?)
""", (job_id, date, model, "pending"))
logger.info(f"Created job {job_id} with {len(date_range)} dates and {len(models)} models")
conn.commit()
logger.info(f"Created job {job_id} with {len(date_range)} dates and {len(models)} models")
return job_id
@@ -585,6 +602,67 @@ class JobManager:
finally:
conn.close()
def get_last_completed_date_for_model(self, model: str) -> Optional[str]:
"""
Get last completed simulation date for a specific model.
Args:
model: Model signature
Returns:
Last completed date (YYYY-MM-DD) or None if no data exists
"""
conn = get_db_connection(self.db_path)
cursor = conn.cursor()
try:
cursor.execute("""
SELECT date
FROM job_details
WHERE model = ? AND status = 'completed'
ORDER BY date DESC
LIMIT 1
""", (model,))
row = cursor.fetchone()
return row[0] if row else None
finally:
conn.close()
def get_completed_model_dates(self, models: List[str], start_date: str, end_date: str) -> Dict[str, List[str]]:
"""
Get all completed dates for each model within a date range.
Args:
models: List of model signatures
start_date: Start date (YYYY-MM-DD)
end_date: End date (YYYY-MM-DD)
Returns:
Dict mapping model signature to list of completed dates
"""
conn = get_db_connection(self.db_path)
cursor = conn.cursor()
try:
result = {model: [] for model in models}
for model in models:
cursor.execute("""
SELECT DISTINCT date
FROM job_details
WHERE model = ? AND status = 'completed' AND date >= ? AND date <= ?
ORDER BY date
""", (model, start_date, end_date))
result[model] = [row[0] for row in cursor.fetchall()]
return result
finally:
conn.close()
def cleanup_old_jobs(self, days: int = 30) -> Dict[str, int]:
"""
Delete jobs older than threshold.