Add audit subagent and restructure skill for progressive disclosure

- Add audit subagent that runs automatically after bill entries
- Create download-attachment.sh for retrieving invoice/receipt PDFs
- Create verify-pdf.py for PDF extraction with OCR fallback
- Restructure SKILL.md from 856 to 177 lines using reference files
- Move detailed content to references/:
  - schema.md: table schemas
  - workflows.md: code examples
  - queries.md: SQL queries and financial reports
  - audit.md: audit queries and remediation steps
This commit is contained in:
2026-01-12 12:45:32 -05:00
parent 70ac6be681
commit 4ebc19408c
7 changed files with 1215 additions and 568 deletions

161
references/audit.md Normal file
View File

@@ -0,0 +1,161 @@
# Audit Reference
Detailed audit queries, workflows, and remediation steps.
## Contents
- [Audit SQL Queries](#audit-sql-queries)
- [PDF Verification Workflow](#pdf-verification-workflow)
- [Post-Entry Audit Checklist](#post-entry-audit-checklist)
- [Output Formats](#output-formats)
- [Remediation Steps](#remediation-steps)
## Audit SQL Queries
### Check 1: Unbalanced Transactions
```sql
-- Simple check using formula column
SELECT id, Date, Description, IsBalanced FROM Transactions WHERE IsBalanced = 0
-- Detailed check with actual sums (for debugging)
SELECT t.id, t.Description,
(SELECT SUM(Debit) FROM TransactionLines WHERE Transaction = t.id) as TotalDebit,
(SELECT SUM(Credit) FROM TransactionLines WHERE Transaction = t.id) as TotalCredit
FROM Transactions t
WHERE t.IsBalanced = 0
```
### Check 2: Invalid Account Usage
Detect debits to non-debit-normal accounts or credits to non-credit-normal accounts:
```sql
SELECT t.id, t.Description, tl.id as LineId, a.Name, a.Type,
tl.Debit, tl.Credit
FROM TransactionLines tl
JOIN Transactions t ON tl.Transaction = t.id
JOIN Accounts a ON tl.Account = a.id
WHERE (tl.Debit > 0 AND a.Type NOT IN ('Asset', 'Expense'))
OR (tl.Credit > 0 AND a.Type NOT IN ('Liability', 'Equity', 'Income'))
```
Note: This query flags unusual patterns. Some are valid (e.g., paying down AP debits a Liability). Review flagged items manually.
### Check 3: Bills Missing Required Links
```sql
SELECT b.id, b.BillNumber, b.Status, b.EntryTransaction, b.Vendor
FROM Bills b
WHERE b.EntryTransaction IS NULL
OR b.Vendor IS NULL
```
### Check 4: Bill/Transaction Amount Mismatch
```sql
SELECT b.id, b.BillNumber, b.Amount as BillAmount,
(SELECT SUM(tl.Debit) FROM TransactionLines tl
JOIN Accounts a ON tl.Account = a.id
WHERE tl.Transaction = b.EntryTransaction AND a.Type = 'Expense') as TxnExpense
FROM Bills b
WHERE b.EntryTransaction IS NOT NULL
AND b.Amount != (SELECT SUM(tl.Debit) FROM TransactionLines tl
JOIN Accounts a ON tl.Account = a.id
WHERE tl.Transaction = b.EntryTransaction AND a.Type = 'Expense')
```
### Check 5: Paid Bills Without BillPayments
```sql
SELECT b.id, b.BillNumber, b.Amount, b.AmountPaid, b.Status
FROM Bills b
WHERE b.Status = 'Paid' AND (b.AmountPaid IS NULL OR b.AmountPaid = 0)
```
### Check 6: Bills Missing Attachments
```sql
SELECT b.id, b.BillNumber, b.Invoice, b.Receipt, b.Status
FROM Bills b
WHERE b.Invoice IS NULL
OR (b.Status = 'Paid' AND b.Receipt IS NULL)
```
## PDF Verification Workflow
When an invoice attachment exists, verify its contents match the bill record:
1. **Download attachment**
```bash
bash scripts/download-attachment.sh <attachment_id> /tmp/invoice.pdf $TOKEN
```
2. **Extract invoice data**
```bash
python scripts/verify-pdf.py /tmp/invoice.pdf --json
```
3. **Compare extracted values**
- Invoice number vs Bill.BillNumber
- Date vs Bill.BillDate (allow 1 day tolerance)
- Amount vs Bill.Amount (must match within $0.01)
- Vendor name vs Vendors.Name (fuzzy match)
4. **Report discrepancies** with severity levels
## Post-Entry Audit Checklist
After completing a bill entry, run these checks on the newly created records:
**Step 1: Transaction Balance**
```sql
SELECT IsBalanced FROM Transactions WHERE id = {txn_id}
```
Expected: `true`
**Step 2: Account Usage**
Verify the transaction lines use correct accounts:
- Expense account (Type = 'Expense') for the debit
- AP account (id=4, code 2000) for the credit
**Step 3: Bill Integrity**
```sql
SELECT id, Vendor, EntryTransaction, Amount
FROM Bills WHERE id = {bill_id}
```
Expected: All fields populated, Amount > 0
**Step 4: PDF Verification** (if Invoice attachment exists)
Run the PDF verification workflow above.
## Output Formats
### Single Entry Audit
| Check | Status | Details |
|-------|--------|---------|
| Transaction Balanced | PASS | Debits = Credits = $X.XX |
| Account Usage | PASS | Expense: 5080, AP: 2000 |
| Bill Integrity | PASS | All required fields set |
| PDF Verification | WARN | Date mismatch: PDF shows 10/5, Bill has 10/6 |
### Full Audit Report
| Category | Severity | Count | Details |
|----------|----------|-------|---------|
| Unbalanced Transactions | Critical | 0 | None |
| Account Misuse | Warning | 2 | Txn #5, #12 |
| Missing Bill Links | Error | 1 | Bill #123 |
| Amount Mismatches | Error | 0 | None |
| PDF Discrepancies | Warning | 3 | Bills #1, #2, #5 |
| Missing Attachments | Warning | 5 | Bills #3, #4, #6, #7, #8 |
### Severity Levels
- **PASS**: Check passed
- **WARN**: Minor discrepancy, review recommended
- **ERROR**: Significant issue, correction required
- **CRITICAL**: Data integrity problem, must fix immediately
## Remediation Steps
| Issue | Remediation |
|-------|-------------|
| Unbalanced transaction | Review TransactionLines, add/adjust lines until SUM(Debit) = SUM(Credit) |
| Wrong account type | Update TransactionLine.Account to correct account |
| Missing EntryTransaction | Link bill to transaction: `update_records("Bills", [{"id": X, "fields": {"EntryTransaction": Y}}])` |
| Missing Vendor | Set Bill.Vendor to appropriate vendor ID |
| Amount mismatch | Review bill lines and transaction lines, correct the discrepancy |
| PDF mismatch | Verify source document, update bill fields if database is wrong |
| Missing attachment | Upload invoice/receipt using `scripts/upload-attachment.sh` |

181
references/queries.md Normal file
View File

@@ -0,0 +1,181 @@
# SQL Queries and Reports
Common queries and financial report templates.
## Contents
- [Common Queries](#common-queries)
- [Financial Reports](#financial-reports)
- [Balance Sheet](#balance-sheet)
- [Income Statement](#income-statement)
- [Trial Balance](#trial-balance)
- [AP Aging](#accounts-payable-aging)
## Common Queries
### Unpaid Bills by Vendor
```sql
SELECT v.Name, b.BillNumber, b.BillDate, b.Amount, b.AmountDue
FROM Bills b
JOIN Vendors v ON b.Vendor = v.id
WHERE b.Status IN ('Open', 'Partial')
ORDER BY b.DueDate
```
### Bills Summary by Vendor
```sql
SELECT v.Name as Vendor, COUNT(b.id) as Bills, SUM(b.Amount) as Total, SUM(b.AmountDue) as Due
FROM Bills b
JOIN Vendors v ON b.Vendor = v.id
GROUP BY v.Name
ORDER BY Total DESC
```
### Account Balances (Non-Zero)
```sql
SELECT Code, Name, Type, Balance
FROM Accounts
WHERE Balance != 0
ORDER BY Code
```
### Owner Reimbursement Balance
```sql
SELECT Balance FROM Accounts WHERE Code = '2203'
```
### Expense Summary by Account
```sql
SELECT a.Code, a.Name, a.Balance
FROM Accounts a
WHERE a.Type = 'Expense' AND a.Balance != 0
ORDER BY a.Balance DESC
```
### Transaction History for Account
```sql
SELECT t.Date, t.Description, t.Reference, tl.Debit, tl.Credit
FROM TransactionLines tl
JOIN Transactions t ON tl.Transaction = t.id
WHERE tl.Account = {account_id}
ORDER BY t.Date DESC
```
### Verify All Transactions Balance
```sql
SELECT id, Description, Total, IsBalanced
FROM Transactions
WHERE IsBalanced = false
```
## Financial Reports
### Balance Sheet
Shows Assets = Liabilities + Equity at a point in time.
**Important:** Parent accounts roll up child balances. Query only top-level parents (Parent = 0) to avoid double-counting.
```sql
-- Assets, Liabilities, Equity (top-level only)
SELECT Code, Name, Type, Balance
FROM Accounts
WHERE Type IN ('Asset', 'Liability', 'Equity')
AND Parent = 0
ORDER BY Type, Code
```
```sql
-- Net Income (for Equity section)
SELECT
COALESCE(SUM(CASE WHEN Type = 'Income' THEN Balance ELSE 0 END), 0) -
COALESCE(SUM(CASE WHEN Type = 'Expense' THEN Balance ELSE 0 END), 0) as NetIncome
FROM Accounts
WHERE Type IN ('Income', 'Expense')
AND id NOT IN (SELECT DISTINCT Parent FROM Accounts WHERE Parent != 0)
```
**Presentation:**
| **Assets** | |
|---|---:|
| Cash & Bank Accounts | $X.XX |
| Accounts Receivable | $X.XX |
| **Total Assets** | **$X.XX** |
| **Liabilities** | |
|---|---:|
| Accounts Payable | $X.XX |
| Due to Owner | $X.XX |
| **Total Liabilities** | **$X.XX** |
| **Equity** | |
|---|---:|
| Retained Earnings | $X.XX |
| Net Income (Loss) | $X.XX |
| **Total Equity** | **$X.XX** |
| **Total Liabilities + Equity** | **$X.XX** |
### Income Statement
Shows Revenue - Expenses = Net Income for a period.
```sql
SELECT Code, Name, Type, Balance
FROM Accounts
WHERE Type IN ('Income', 'Expense')
AND Balance != 0
AND id NOT IN (SELECT DISTINCT Parent FROM Accounts WHERE Parent != 0)
ORDER BY Type DESC, Code
```
**Presentation:**
| **Income** | |
|---|---:|
| Service Revenue | $X.XX |
| **Total Income** | **$X.XX** |
| **Expenses** | |
|---|---:|
| Software & Subscriptions | $X.XX |
| Professional Services | $X.XX |
| **Total Expenses** | **$X.XX** |
| **Net Income (Loss)** | **$X.XX** |
### Trial Balance
Lists all accounts with non-zero balances. Debits should equal Credits.
```sql
SELECT
Code,
Name,
Type,
CASE WHEN Type IN ('Asset', 'Expense') THEN Balance ELSE 0 END as Debit,
CASE WHEN Type IN ('Liability', 'Equity', 'Income') THEN Balance ELSE 0 END as Credit
FROM Accounts
WHERE Balance != 0
AND id NOT IN (SELECT DISTINCT Parent FROM Accounts WHERE Parent != 0)
ORDER BY Code
```
### Accounts Payable Aging
```sql
SELECT
v.Name as Vendor,
b.BillNumber,
b.BillDate,
b.DueDate,
b.AmountDue,
CASE
WHEN b.DueDate >= strftime('%s', 'now') THEN 'Current'
WHEN b.DueDate >= strftime('%s', 'now') - 2592000 THEN '1-30 Days'
WHEN b.DueDate >= strftime('%s', 'now') - 5184000 THEN '31-60 Days'
ELSE '60+ Days'
END as Aging
FROM Bills b
JOIN Vendors v ON b.Vendor = v.id
WHERE b.Status IN ('Open', 'Partial')
ORDER BY b.DueDate
```

97
references/schema.md Normal file
View File

@@ -0,0 +1,97 @@
# Database Schema
Complete table schemas for the Grist accounting system.
## Accounts
| Column | Type | Notes |
|--------|------|-------|
| Code | Text | Account number (e.g., "5080") |
| Name | Text | Account name |
| Type | Choice | "Asset", "Liability", "Equity", "Income", "Expense" |
| Parent | Ref:Accounts | Parent account for hierarchy (0 = top-level) |
| Description | Text | |
| IsActive | Bool | |
| Balance | Formula | Calculated from transactions |
## Vendors
| Column | Type | Notes |
|--------|------|-------|
| Name | Text | Vendor name |
| DefaultExpenseAccount | Ref:Accounts | Auto-fills on bill lines |
| PaymentTerms | Choice | "Due on Receipt", "Net 15", "Net 30" |
| Notes | Text | |
| IsActive | Bool | |
| Balance | Formula | Sum of unpaid bills |
## Items
| Column | Type | Notes |
|--------|------|-------|
| Name | Text | Item name (e.g., "Software Subscription") |
| DefaultAccount | Ref:Accounts | Expense account for this item |
| DefaultDescription | Text | Auto-fills on bill lines |
| IsActive | Bool | |
## Bills
| Column | Type | Notes |
|--------|------|-------|
| Vendor | Ref:Vendors | Required |
| BillNumber | Text | Invoice number from vendor |
| BillDate | Date | Unix timestamp |
| DueDate | Date | Unix timestamp |
| Status | Choice | "Open", "Partial", "Paid" |
| Memo | Text | |
| EntryTransaction | Ref:Transactions | Link to journal entry |
| Invoice | Attachments | Vendor invoice document (use `["L", id]` format) |
| Receipt | Attachments | Payment receipt/confirmation (use `["L", id]` format) |
| Amount | Formula | Sum of BillLines.Amount |
| AmountPaid | Formula | Sum of BillPayments.Amount |
| AmountDue | Formula | Amount - AmountPaid |
## BillLines
| Column | Type | Notes |
|--------|------|-------|
| Bill | Ref:Bills | Required |
| Item | Ref:Items | Optional - auto-fills Account/Description |
| Account | Ref:Accounts | Expense account |
| Description | Text | |
| Amount | Numeric | Line item amount |
## Transactions
| Column | Type | Notes |
|--------|------|-------|
| Date | Date | Unix timestamp |
| Description | Text | Transaction description |
| Reference | Text | Check number, invoice reference, etc. |
| Status | Choice | "Draft", "Posted", "Cleared" |
| Memo | Text | |
| Total | Formula | Sum of debits |
| IsBalanced | Formula | True if debits = credits |
## TransactionLines
| Column | Type | Notes |
|--------|------|-------|
| Transaction | Ref:Transactions | Required |
| Account | Ref:Accounts | Required |
| Debit | Numeric | Debit amount (or 0) |
| Credit | Numeric | Credit amount (or 0) |
| Memo | Text | |
## BillPayments
| Column | Type | Notes |
|--------|------|-------|
| Bill | Ref:Bills | Required |
| Transaction | Ref:Transactions | Payment journal entry |
| Amount | Numeric | Payment amount |
| PaymentDate | Date | Unix timestamp |
## Formula Columns (Auto-Calculated)
| Table.Column | Description |
|--------------|-------------|
| Accounts.Balance | OwnBalance + ChildrenBalance |
| Transactions.IsBalanced | True if sum of debits = sum of credits |
| Transactions.Total | Sum of debit amounts |
| Bills.Amount | Sum of BillLines.Amount |
| Bills.AmountPaid | Sum of BillPayments.Amount |
| Bills.AmountDue | Amount - AmountPaid |
| Vendors.Balance | Sum of AmountDue for unpaid bills |

229
references/workflows.md Normal file
View File

@@ -0,0 +1,229 @@
# Workflow Examples
Detailed code examples for common accounting operations.
## Contents
- [Create a Vendor](#create-a-vendor)
- [Create Items](#create-items-for-common-purchases)
- [Complete Bill Entry](#complete-bill-entry-6-steps)
- [Pay Bill from Checking](#pay-bill-from-checking-account)
- [Pay Bill via Owner](#pay-bill-via-owner-reimbursement)
- [Reimburse Owner](#reimburse-owner)
- [Batch Operations](#batch-operations)
## Create a Vendor
```python
add_records("Vendors", [{
"Name": "Acme Corp",
"DefaultExpenseAccount": 36, # Software & Subscriptions
"PaymentTerms": "Due on Receipt",
"Notes": "Software vendor",
"IsActive": True
}])
# Returns: {"inserted_ids": [vendor_id]}
```
## Create Items for Common Purchases
```python
add_records("Items", [{
"Name": "Monthly Software",
"DefaultAccount": 36,
"DefaultDescription": "Monthly SaaS subscription",
"IsActive": True
}])
```
## Complete Bill Entry (6 Steps)
**Step 1: Create Bill Header**
```python
add_records("Bills", [{
"Vendor": 1, # vendor_id
"BillNumber": "INV-001",
"BillDate": 1759708800, # Unix timestamp
"DueDate": 1759708800,
"Status": "Open",
"Memo": "October services"
}])
# Returns: {"inserted_ids": [bill_id]}
```
**Step 2: Create Bill Line(s)**
```python
add_records("BillLines", [{
"Bill": 1, # bill_id from step 1
"Item": 1, # optional - auto-fills Account/Description
"Account": 36, # expense account
"Description": "Monthly subscription",
"Amount": 100.00
}])
```
**Step 3: Create Journal Entry**
```python
# Transaction header
add_records("Transactions", [{
"Date": 1759708800,
"Description": "Acme Corp - October services",
"Reference": "INV-001",
"Status": "Posted"
}])
# Returns: {"inserted_ids": [txn_id]}
# Transaction lines: Debit expense, Credit AP
add_records("TransactionLines", [
{"Transaction": 1, "Account": 36, "Debit": 100.00, "Credit": 0, "Memo": "Monthly subscription"},
{"Transaction": 1, "Account": 4, "Debit": 0, "Credit": 100.00, "Memo": "Monthly subscription"}
])
```
**Step 4: Link Bill to Transaction**
```python
update_records("Bills", [{"id": 1, "fields": {"EntryTransaction": 1}}])
```
**Step 5: Upload Invoice (if available)**
```bash
bash scripts/upload-attachment.sh invoice.pdf Bills 1 $TOKEN Invoice
```
**Step 6: Post-Entry Audit (REQUIRED)**
Run audit checks before concluding. See [Audit Reference](audit.md) for details.
```sql
-- Check 1: Transaction balanced
SELECT IsBalanced FROM Transactions WHERE id = {txn_id}
-- Expected: true
-- Check 2: Bill integrity
SELECT id, Vendor, EntryTransaction, Amount FROM Bills WHERE id = {bill_id}
-- Expected: All fields populated, Amount > 0
```
## Pay Bill from Checking Account
```python
# Step 1: Create payment transaction
add_records("Transactions", [{
"Date": 1760832000,
"Description": "Payment - Acme Corp INV-001",
"Reference": "Check #1001",
"Status": "Cleared"
}])
# Returns: {"inserted_ids": [txn_id]}
# Step 2: Debit AP, Credit Checking
add_records("TransactionLines", [
{"Transaction": 2, "Account": 4, "Debit": 100.00, "Credit": 0, "Memo": "Pay INV-001"},
{"Transaction": 2, "Account": 14, "Debit": 0, "Credit": 100.00, "Memo": "Pay INV-001"}
])
# Step 3: Create BillPayment record
add_records("BillPayments", [{
"Bill": 1,
"Transaction": 2,
"Amount": 100.00,
"PaymentDate": 1760832000
}])
# Step 4: Update bill status
update_records("Bills", [{"id": 1, "fields": {"Status": "Paid"}}])
# Step 5: Upload receipt (if available)
# bash scripts/upload-attachment.sh receipt.pdf Bills 1 $TOKEN Receipt
# Step 6: Post-Payment Audit (REQUIRED)
# Verify payment transaction balances and bill status updated correctly
```
## Pay Bill via Owner Reimbursement
When the owner pays a business expense personally:
```python
# Step 1: Create payment transaction
add_records("Transactions", [{
"Date": 1760832000,
"Description": "Owner payment - Acme Corp INV-001",
"Reference": "Owner Reimb",
"Status": "Posted"
}])
# Step 2: Debit AP, Credit Due to Owner (not Checking)
add_records("TransactionLines", [
{"Transaction": 2, "Account": 4, "Debit": 100.00, "Credit": 0, "Memo": "Pay INV-001"},
{"Transaction": 2, "Account": 22, "Debit": 0, "Credit": 100.00, "Memo": "Owner paid"}
])
# Step 3: Create BillPayment record
add_records("BillPayments", [{
"Bill": 1,
"Transaction": 2,
"Amount": 100.00,
"PaymentDate": 1760832000
}])
# Step 4: Update bill status
update_records("Bills", [{"id": 1, "fields": {"Status": "Paid"}}])
# Step 5: Upload receipt (if available)
# bash scripts/upload-attachment.sh receipt.pdf Bills 1 $TOKEN Receipt
# Step 6: Post-Payment Audit (REQUIRED)
```
## Reimburse Owner
When business pays back the owner:
```python
add_records("Transactions", [{
"Date": 1762041600,
"Description": "Owner reimbursement",
"Reference": "Transfer",
"Status": "Cleared"
}])
add_records("TransactionLines", [
{"Transaction": 3, "Account": 22, "Debit": 500.00, "Credit": 0, "Memo": "Reimburse owner"},
{"Transaction": 3, "Account": 14, "Debit": 0, "Credit": 500.00, "Memo": "Reimburse owner"}
])
```
## Batch Operations
When entering multiple bills efficiently:
1. **Create all Bills first** → collect inserted IDs
2. **Create all BillLines** referencing bill IDs
3. **Create all Transactions** → collect inserted IDs
4. **Create all TransactionLines** referencing transaction IDs
5. **Update all Bills** with EntryTransaction links in one call
6. (If paying) Create payment transactions, lines, and BillPayments
7. **Upload invoice attachments** if files are available
### Batch Attachment Uploads
```bash
# Example batch upload pattern
TOKEN=$(request_session_token with write permission)
for each (bill_id, invoice_path):
curl -X POST -H "Authorization: Bearer $TOKEN" \
-F "file=@$invoice_path" \
https://grist-mcp.bballou.com/api/v1/attachments
# Returns attachment_id
update_records("Bills", [{"id": bill_id, "fields": {"Invoice": ["L", attachment_id]}}])
```
### Batch Update Example
```python
update_records("Bills", [
{"id": 1, "fields": {"EntryTransaction": 1}},
{"id": 2, "fields": {"EntryTransaction": 2}},
{"id": 3, "fields": {"EntryTransaction": 3}}
])
```