docs: add specification and implementation plan for direct database access feature
This feature (002-direct-db-access) enables scripts to query schedule data directly via SQL after loading XER files through MCP. Key additions: - spec.md: Feature specification with 3 user stories - plan.md: Implementation plan with constitution check - research.md: Technology decisions (SQLite file, WAL mode, atomic writes) - data-model.md: DatabaseInfo, SchemaInfo, TableInfo entities - contracts/mcp-tools.json: Extended load_xer schema, new get_database_info tool - quickstart.md: Usage examples for direct database access - tasks.md: 16 implementation tasks across 6 phases
This commit is contained in:
110
specs/002-direct-db-access/research.md
Normal file
110
specs/002-direct-db-access/research.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# Research: Direct Database Access for Scripts
|
||||
|
||||
**Date**: 2026-01-08
|
||||
**Branch**: `002-direct-db-access`
|
||||
|
||||
## Research Topics
|
||||
|
||||
### 1. SQLite File-Based vs In-Memory Database
|
||||
|
||||
**Decision**: Support both file-based and in-memory databases through the same DatabaseManager interface.
|
||||
|
||||
**Rationale**:
|
||||
- File-based SQLite allows external scripts to query data directly without MCP overhead
|
||||
- In-memory remains the default for backward compatibility with existing tools
|
||||
- SQLite's file format is universally readable (Python sqlite3, DBeaver, sqlitebrowser, etc.)
|
||||
- Single connection string change (`:memory:` vs file path) switches modes
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Separate database manager for files**: Rejected - unnecessary duplication; SQLite handles both modes identically
|
||||
- **Export to CSV/JSON**: Rejected - loses relational structure; no query capability
|
||||
- **Network database (PostgreSQL)**: Rejected - overkill for single-user local access; requires external server
|
||||
|
||||
### 2. Database File Location Strategy
|
||||
|
||||
**Decision**: Accept user-specified path, or default to a predictable location derived from the XER file path.
|
||||
|
||||
**Rationale**:
|
||||
- User-specified path gives maximum flexibility for script integration
|
||||
- Default path (`{xer_file_directory}/{xer_basename}.sqlite`) is predictable and colocated with source
|
||||
- Absolute paths in responses eliminate ambiguity for scripts
|
||||
|
||||
**Default Path Algorithm**:
|
||||
```
|
||||
Input: /path/to/schedule.xer
|
||||
Output: /path/to/schedule.sqlite
|
||||
```
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Temp directory only**: Rejected - less predictable; may be cleaned up unexpectedly
|
||||
- **Fixed location (e.g., ~/.xer-mcp/db/)**: Rejected - less convenient; separates DB from source file
|
||||
- **Always require user to specify**: Rejected - worse developer experience for common case
|
||||
|
||||
### 3. Atomic Write Strategy
|
||||
|
||||
**Decision**: Use SQLite's built-in transaction support; write to temp file and rename for atomicity.
|
||||
|
||||
**Rationale**:
|
||||
- SQLite transactions ensure data integrity during writes
|
||||
- Write-then-rename pattern prevents partial/corrupted files if process interrupted
|
||||
- External scripts won't see incomplete data
|
||||
|
||||
**Implementation**:
|
||||
1. Create database at `{target_path}.tmp`
|
||||
2. Load all data within a transaction
|
||||
3. Commit transaction
|
||||
4. Rename `{target_path}.tmp` to `{target_path}` (atomic on POSIX)
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Direct write to final path**: Rejected - risk of corruption on interruption
|
||||
- **Lock file mechanism**: Rejected - SQLite already handles locking; adds complexity
|
||||
|
||||
### 4. Schema Introspection Approach
|
||||
|
||||
**Decision**: Query SQLite's `sqlite_master` table and `PRAGMA table_info()` for schema information.
|
||||
|
||||
**Rationale**:
|
||||
- Standard SQLite introspection APIs - no custom metadata needed
|
||||
- Returns actual schema, not just documentation
|
||||
- Works with any SQLite database, even if created by different tools
|
||||
|
||||
**Schema Information Returned**:
|
||||
- Table names
|
||||
- Column names and types for each table
|
||||
- Primary key information
|
||||
- Foreign key relationships (via `PRAGMA foreign_key_list()`)
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Hardcoded schema in response**: Rejected - may drift from actual schema; not dynamic
|
||||
- **Separate metadata table**: Rejected - adds complexity; standard introspection sufficient
|
||||
|
||||
### 5. Concurrent Access Handling
|
||||
|
||||
**Decision**: Use SQLite WAL (Write-Ahead Logging) mode for concurrent read access.
|
||||
|
||||
**Rationale**:
|
||||
- WAL mode allows multiple readers while one writer operates
|
||||
- Scripts can query while MCP server operates without blocking
|
||||
- Minimal configuration change: `PRAGMA journal_mode=WAL`
|
||||
|
||||
**Implementation**:
|
||||
- Enable WAL mode when creating file-based database
|
||||
- In-memory databases don't need WAL (single connection)
|
||||
|
||||
**Alternatives Considered**:
|
||||
- **Reader/writer locks in application**: Rejected - SQLite handles this natively
|
||||
- **Copy-on-read**: Rejected - unnecessary with WAL mode
|
||||
|
||||
## Technology Decisions Summary
|
||||
|
||||
| Decision | Choice | Key Reason |
|
||||
|----------|--------|------------|
|
||||
| Database format | SQLite file | Universal compatibility, no server needed |
|
||||
| Default location | Same directory as XER file | Predictable, colocated |
|
||||
| Atomicity | Temp file + rename | Prevents corruption |
|
||||
| Schema info | SQLite introspection APIs | Dynamic, accurate |
|
||||
| Concurrency | WAL mode | Multiple readers supported |
|
||||
|
||||
## Open Questions Resolved
|
||||
|
||||
All technical questions have been resolved through research. No clarifications needed.
|
||||
Reference in New Issue
Block a user