# Research: Direct Database Access for Scripts

**Date**: 2026-01-08
**Branch**: `002-direct-db-access`

## Research Topics

### 1. SQLite File-Based vs In-Memory Database

**Decision**: Support both file-based and in-memory databases through the same DatabaseManager interface.

**Rationale**:
- File-based SQLite allows external scripts to query data directly without MCP overhead
- In-memory remains the default for backward compatibility with existing tools
- SQLite's file format is universally readable (Python sqlite3, DBeaver, sqlitebrowser, etc.)
- Single connection string change (`:memory:` vs file path) switches modes

**Alternatives Considered**:
- **Separate database manager for files**: Rejected - unnecessary duplication; SQLite handles both modes identically
- **Export to CSV/JSON**: Rejected - loses relational structure; no query capability
- **Network database (PostgreSQL)**: Rejected - overkill for single-user local access; requires external server

### 2. Database File Location Strategy

**Decision**: Accept user-specified path, or default to a predictable location derived from the XER file path.

**Rationale**:
- User-specified path gives maximum flexibility for script integration
- Default path (`{xer_file_directory}/{xer_basename}.sqlite`) is predictable and colocated with source
- Absolute paths in responses eliminate ambiguity for scripts

**Default Path Algorithm**:
```
Input:  /path/to/schedule.xer
Output: /path/to/schedule.sqlite
```

**Alternatives Considered**:
- **Temp directory only**: Rejected - less predictable; may be cleaned up unexpectedly
- **Fixed location (e.g., ~/.xer-mcp/db/)**: Rejected - less convenient; separates DB from source file
- **Always require user to specify**: Rejected - worse developer experience for common case

### 3. Atomic Write Strategy

**Decision**: Use SQLite's built-in transaction support; write to temp file and rename for atomicity.

**Rationale**:
- SQLite transactions ensure data integrity during writes
- Write-then-rename pattern prevents partial/corrupted files if process interrupted
- External scripts won't see incomplete data

**Implementation**:
1. Create database at `{target_path}.tmp`
2. Load all data within a transaction
3. Commit transaction
4. Rename `{target_path}.tmp` to `{target_path}` (atomic on POSIX)

**Alternatives Considered**:
- **Direct write to final path**: Rejected - risk of corruption on interruption
- **Lock file mechanism**: Rejected - SQLite already handles locking; adds complexity

### 4. Schema Introspection Approach

**Decision**: Query SQLite's `sqlite_master` table and `PRAGMA table_info()` for schema information.

**Rationale**:
- Standard SQLite introspection APIs - no custom metadata needed
- Returns actual schema, not just documentation
- Works with any SQLite database, even if created by different tools

**Schema Information Returned**:
- Table names
- Column names and types for each table
- Primary key information
- Foreign key relationships (via `PRAGMA foreign_key_list()`)

**Alternatives Considered**:
- **Hardcoded schema in response**: Rejected - may drift from actual schema; not dynamic
- **Separate metadata table**: Rejected - adds complexity; standard introspection sufficient

### 5. Concurrent Access Handling

**Decision**: Use SQLite WAL (Write-Ahead Logging) mode for concurrent read access.

**Rationale**:
- WAL mode allows multiple readers while one writer operates
- Scripts can query while MCP server operates without blocking
- Minimal configuration change: `PRAGMA journal_mode=WAL`

**Implementation**:
- Enable WAL mode when creating file-based database
- In-memory databases don't need WAL (single connection)

**Alternatives Considered**:
- **Reader/writer locks in application**: Rejected - SQLite handles this natively
- **Copy-on-read**: Rejected - unnecessary with WAL mode

## Technology Decisions Summary

| Decision | Choice | Key Reason |
|----------|--------|------------|
| Database format | SQLite file | Universal compatibility, no server needed |
| Default location | Same directory as XER file | Predictable, colocated |
| Atomicity | Temp file + rename | Prevents corruption |
| Schema info | SQLite introspection APIs | Dynamic, accurate |
| Concurrency | WAL mode | Multiple readers supported |

## Open Questions Resolved

All technical questions have been resolved through research. No clarifications needed.