Bill Ballou
3e7ad39eb8
This feature (002-direct-db-access) enables scripts to query schedule data directly via SQL after loading XER files through MCP. Key additions: - spec.md: Feature specification with 3 user stories - plan.md: Implementation plan with constitution check - research.md: Technology decisions (SQLite file, WAL mode, atomic writes) - data-model.md: DatabaseInfo, SchemaInfo, TableInfo entities - contracts/mcp-tools.json: Extended load_xer schema, new get_database_info tool - quickstart.md: Usage examples for direct database access - tasks.md: 16 implementation tasks across 6 phases
111 lines
4.3 KiB
Markdown
111 lines
4.3 KiB
Markdown
# Research: Direct Database Access for Scripts
|
|
|
|
**Date**: 2026-01-08
|
|
**Branch**: `002-direct-db-access`
|
|
|
|
## Research Topics
|
|
|
|
### 1. SQLite File-Based vs In-Memory Database
|
|
|
|
**Decision**: Support both file-based and in-memory databases through the same DatabaseManager interface.
|
|
|
|
**Rationale**:
|
|
- File-based SQLite allows external scripts to query data directly without MCP overhead
|
|
- In-memory remains the default for backward compatibility with existing tools
|
|
- SQLite's file format is universally readable (Python sqlite3, DBeaver, sqlitebrowser, etc.)
|
|
- Single connection string change (`:memory:` vs file path) switches modes
|
|
|
|
**Alternatives Considered**:
|
|
- **Separate database manager for files**: Rejected - unnecessary duplication; SQLite handles both modes identically
|
|
- **Export to CSV/JSON**: Rejected - loses relational structure; no query capability
|
|
- **Network database (PostgreSQL)**: Rejected - overkill for single-user local access; requires external server
|
|
|
|
### 2. Database File Location Strategy
|
|
|
|
**Decision**: Accept user-specified path, or default to a predictable location derived from the XER file path.
|
|
|
|
**Rationale**:
|
|
- User-specified path gives maximum flexibility for script integration
|
|
- Default path (`{xer_file_directory}/{xer_basename}.sqlite`) is predictable and colocated with source
|
|
- Absolute paths in responses eliminate ambiguity for scripts
|
|
|
|
**Default Path Algorithm**:
|
|
```
|
|
Input: /path/to/schedule.xer
|
|
Output: /path/to/schedule.sqlite
|
|
```
|
|
|
|
**Alternatives Considered**:
|
|
- **Temp directory only**: Rejected - less predictable; may be cleaned up unexpectedly
|
|
- **Fixed location (e.g., ~/.xer-mcp/db/)**: Rejected - less convenient; separates DB from source file
|
|
- **Always require user to specify**: Rejected - worse developer experience for common case
|
|
|
|
### 3. Atomic Write Strategy
|
|
|
|
**Decision**: Use SQLite's built-in transaction support; write to temp file and rename for atomicity.
|
|
|
|
**Rationale**:
|
|
- SQLite transactions ensure data integrity during writes
|
|
- Write-then-rename pattern prevents partial/corrupted files if process interrupted
|
|
- External scripts won't see incomplete data
|
|
|
|
**Implementation**:
|
|
1. Create database at `{target_path}.tmp`
|
|
2. Load all data within a transaction
|
|
3. Commit transaction
|
|
4. Rename `{target_path}.tmp` to `{target_path}` (atomic on POSIX)
|
|
|
|
**Alternatives Considered**:
|
|
- **Direct write to final path**: Rejected - risk of corruption on interruption
|
|
- **Lock file mechanism**: Rejected - SQLite already handles locking; adds complexity
|
|
|
|
### 4. Schema Introspection Approach
|
|
|
|
**Decision**: Query SQLite's `sqlite_master` table and `PRAGMA table_info()` for schema information.
|
|
|
|
**Rationale**:
|
|
- Standard SQLite introspection APIs - no custom metadata needed
|
|
- Returns actual schema, not just documentation
|
|
- Works with any SQLite database, even if created by different tools
|
|
|
|
**Schema Information Returned**:
|
|
- Table names
|
|
- Column names and types for each table
|
|
- Primary key information
|
|
- Foreign key relationships (via `PRAGMA foreign_key_list()`)
|
|
|
|
**Alternatives Considered**:
|
|
- **Hardcoded schema in response**: Rejected - may drift from actual schema; not dynamic
|
|
- **Separate metadata table**: Rejected - adds complexity; standard introspection sufficient
|
|
|
|
### 5. Concurrent Access Handling
|
|
|
|
**Decision**: Use SQLite WAL (Write-Ahead Logging) mode for concurrent read access.
|
|
|
|
**Rationale**:
|
|
- WAL mode allows multiple readers while one writer operates
|
|
- Scripts can query while MCP server operates without blocking
|
|
- Minimal configuration change: `PRAGMA journal_mode=WAL`
|
|
|
|
**Implementation**:
|
|
- Enable WAL mode when creating file-based database
|
|
- In-memory databases don't need WAL (single connection)
|
|
|
|
**Alternatives Considered**:
|
|
- **Reader/writer locks in application**: Rejected - SQLite handles this natively
|
|
- **Copy-on-read**: Rejected - unnecessary with WAL mode
|
|
|
|
## Technology Decisions Summary
|
|
|
|
| Decision | Choice | Key Reason |
|
|
|----------|--------|------------|
|
|
| Database format | SQLite file | Universal compatibility, no server needed |
|
|
| Default location | Same directory as XER file | Predictable, colocated |
|
|
| Atomicity | Temp file + rename | Prevents corruption |
|
|
| Schema info | SQLite introspection APIs | Dynamic, accurate |
|
|
| Concurrency | WAL mode | Multiple readers supported |
|
|
|
|
## Open Questions Resolved
|
|
|
|
All technical questions have been resolved through research. No clarifications needed.
|