Compare commits
25 Commits
v1.0.0-alp
...
v1.1.0-alp
| Author | SHA1 | Date | |
|---|---|---|---|
| d80eac4a0d | |||
| 58807ddbd0 | |||
| 09b6be14df | |||
| 4e75709c4e | |||
| 0cdf06546c | |||
| c6fbadecfc | |||
| 3cf2400232 | |||
| 1eb64803be | |||
| 38ccaa9cb8 | |||
| 51e90abd2d | |||
| d6fb3f4ef0 | |||
| 163b48f1f4 | |||
| a668baa4d0 | |||
| 69a65a68a6 | |||
| ff7dff7571 | |||
| 77027d762e | |||
| 210cfabb52 | |||
| a31b2652bb | |||
| 16691e1d21 | |||
| 204d00caf4 | |||
| ca03d22b97 | |||
| 107db82c52 | |||
| 4b89837b43 | |||
| 5aaa943010 | |||
| c8cea249bc |
@@ -13,19 +13,6 @@ jobs:
|
||||
build:
|
||||
runs-on: ubuntu-docker
|
||||
steps:
|
||||
- name: Debug environment
|
||||
run: |
|
||||
echo "=== Environment ==="
|
||||
echo "PATH: $PATH"
|
||||
echo "=== Looking for docker ==="
|
||||
which docker || echo "docker not in PATH"
|
||||
ls -la /usr/local/bin/ || echo "no /usr/local/bin"
|
||||
ls -la /usr/bin/docker || echo "no /usr/bin/docker"
|
||||
echo "=== Docker socket ==="
|
||||
ls -la /var/run/docker.sock || echo "no docker socket"
|
||||
echo "=== All binaries ==="
|
||||
ls /usr/local/bin/ 2>/dev/null || true
|
||||
|
||||
- name: Checkout repository
|
||||
run: |
|
||||
git clone --depth 1 --branch ${GITHUB_REF_NAME} ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git .
|
||||
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -6,3 +6,4 @@ __pycache__/
|
||||
*.egg-info/
|
||||
dist/
|
||||
.pytest_cache/
|
||||
.worktrees/
|
||||
|
||||
20
CHANGELOG.md
20
CHANGELOG.md
@@ -5,6 +5,26 @@ All notable changes to this project will be documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [1.1.0] - 2026-01-02
|
||||
|
||||
### Added
|
||||
|
||||
#### Logging
|
||||
- **Tool Call Logging**: Human-readable logs for every MCP tool call with agent identity, document, stats, and duration
|
||||
- **Token Truncation**: Secure token display in logs (first/last 3 chars only)
|
||||
- **Stats Extraction**: Meaningful operation stats per tool (e.g., "42 records", "3 tables")
|
||||
- **LOG_LEVEL Support**: Configure logging verbosity via environment variable (DEBUG, INFO, WARNING, ERROR)
|
||||
- **Health Check Suppression**: `/health` requests logged at DEBUG level to reduce noise
|
||||
|
||||
#### Log Format
|
||||
```
|
||||
2026-01-02 10:15:23 | agent-name (abc...xyz) | get_records | sales | 42 records | success | 125ms
|
||||
```
|
||||
|
||||
- Pipe-delimited format for easy parsing
|
||||
- Multi-line error details with indentation
|
||||
- Duration tracking in milliseconds
|
||||
|
||||
## [1.0.0] - 2026-01-01
|
||||
|
||||
Initial release of grist-mcp, an MCP server for AI agents to interact with Grist spreadsheets.
|
||||
|
||||
268
README.md
268
README.md
@@ -15,50 +15,33 @@ grist-mcp is a [Model Context Protocol (MCP)](https://modelcontextprotocol.io/)
|
||||
- **Security**: Token-based authentication with per-document permission scopes (read, write, schema)
|
||||
- **Multi-tenant**: Support multiple Grist instances and documents
|
||||
|
||||
## Requirements
|
||||
## Quick Start (Docker)
|
||||
|
||||
- Python 3.14+
|
||||
### Prerequisites
|
||||
|
||||
- Docker and Docker Compose
|
||||
- Access to one or more Grist documents with API keys
|
||||
|
||||
## Installation
|
||||
### 1. Create configuration directory
|
||||
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone https://github.com/your-org/grist-mcp.git
|
||||
cd grist-mcp
|
||||
|
||||
# Install with uv
|
||||
uv sync --dev
|
||||
mkdir grist-mcp && cd grist-mcp
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Create a `config.yaml` file based on the example:
|
||||
### 2. Download configuration files
|
||||
|
||||
```bash
|
||||
# Download docker-compose.yml
|
||||
curl -O https://raw.githubusercontent.com/Xe138/grist-mcp-server/master/deploy/prod/docker-compose.yml
|
||||
|
||||
# Download example config
|
||||
curl -O https://raw.githubusercontent.com/Xe138/grist-mcp-server/master/config.yaml.example
|
||||
cp config.yaml.example config.yaml
|
||||
```
|
||||
|
||||
### Configuration Structure
|
||||
### 3. Generate tokens
|
||||
|
||||
```yaml
|
||||
# Document definitions
|
||||
documents:
|
||||
my-document:
|
||||
url: https://docs.getgrist.com # Grist instance URL
|
||||
doc_id: abcd1234 # Document ID from URL
|
||||
api_key: ${GRIST_API_KEY} # API key (supports env vars)
|
||||
|
||||
# Agent tokens with access scopes
|
||||
tokens:
|
||||
- token: your-secret-token # Unique token for this agent
|
||||
name: my-agent # Human-readable name
|
||||
scope:
|
||||
- document: my-document
|
||||
permissions: [read, write] # Allowed: read, write, schema
|
||||
```
|
||||
|
||||
### Generating Tokens
|
||||
Generate a secure token for your agent:
|
||||
|
||||
```bash
|
||||
python -c "import secrets; print(secrets.token_urlsafe(32))"
|
||||
@@ -66,34 +49,53 @@ python -c "import secrets; print(secrets.token_urlsafe(32))"
|
||||
openssl rand -base64 32
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
### 4. Configure config.yaml
|
||||
|
||||
- `CONFIG_PATH`: Path to config file (default: `/app/config.yaml`)
|
||||
- `GRIST_MCP_TOKEN`: Agent token for authentication
|
||||
- Config file supports `${VAR}` syntax for API keys
|
||||
Edit `config.yaml` to define your Grist documents and agent tokens:
|
||||
|
||||
## Usage
|
||||
```yaml
|
||||
# Document definitions
|
||||
documents:
|
||||
my-document: # Friendly name (used in token scopes)
|
||||
url: https://docs.getgrist.com # Your Grist instance URL
|
||||
doc_id: abcd1234efgh5678 # Document ID from the URL
|
||||
api_key: your-grist-api-key # Grist API key (or use ${ENV_VAR} syntax)
|
||||
|
||||
### Running the Server
|
||||
|
||||
The server uses SSE (Server-Sent Events) transport over HTTP:
|
||||
|
||||
```bash
|
||||
# Set your agent token
|
||||
export GRIST_MCP_TOKEN="your-agent-token"
|
||||
|
||||
# Run with custom config path (defaults to port 3000)
|
||||
CONFIG_PATH=./config.yaml uv run python -m grist_mcp.main
|
||||
|
||||
# Or specify a custom port
|
||||
PORT=8080 CONFIG_PATH=./config.yaml uv run python -m grist_mcp.main
|
||||
# Agent tokens with access scopes
|
||||
tokens:
|
||||
- token: your-generated-token-here # The token you generated in step 3
|
||||
name: my-agent # Human-readable name
|
||||
scope:
|
||||
- document: my-document # Must match a document name above
|
||||
permissions: [read, write] # Allowed: read, write, schema
|
||||
```
|
||||
|
||||
The server exposes two endpoints:
|
||||
- `http://localhost:3000/sse` - SSE connection endpoint
|
||||
- `http://localhost:3000/messages` - Message posting endpoint
|
||||
**Finding your Grist document ID**: Open your Grist document in a browser. The URL will look like:
|
||||
`https://docs.getgrist.com/abcd1234efgh5678/My-Document` - the document ID is `abcd1234efgh5678`.
|
||||
|
||||
### MCP Client Configuration
|
||||
**Getting a Grist API key**: In Grist, go to Profile Settings → API → Create API Key.
|
||||
|
||||
### 5. Create .env file
|
||||
|
||||
Create a `.env` file with your agent token:
|
||||
|
||||
```bash
|
||||
# .env
|
||||
GRIST_MCP_TOKEN=your-generated-token-here
|
||||
PORT=3000
|
||||
```
|
||||
|
||||
The `GRIST_MCP_TOKEN` must match one of the tokens defined in `config.yaml`.
|
||||
|
||||
### 6. Start the server
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
The server will be available at `http://localhost:3000`.
|
||||
|
||||
### 7. Configure your MCP client
|
||||
|
||||
Add to your MCP client configuration (e.g., Claude Desktop):
|
||||
|
||||
@@ -101,24 +103,13 @@ Add to your MCP client configuration (e.g., Claude Desktop):
|
||||
{
|
||||
"mcpServers": {
|
||||
"grist": {
|
||||
"type": "sse",
|
||||
"url": "http://localhost:3000/sse"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For remote deployments, use the server's public URL:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"grist": {
|
||||
"url": "https://your-server.example.com/sse"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Available Tools
|
||||
|
||||
### Discovery
|
||||
@@ -149,6 +140,54 @@ For remote deployments, use the server's public URL:
|
||||
| `modify_column` | Change a column's type or formula |
|
||||
| `delete_column` | Remove a column from a table |
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `PORT` | Server port | `3000` |
|
||||
| `GRIST_MCP_TOKEN` | Agent authentication token (required) | - |
|
||||
| `CONFIG_PATH` | Path to config file inside container | `/app/config.yaml` |
|
||||
|
||||
### config.yaml Structure
|
||||
|
||||
```yaml
|
||||
# Document definitions (each is self-contained)
|
||||
documents:
|
||||
budget-2024:
|
||||
url: https://work.getgrist.com
|
||||
doc_id: mK7xB2pQ9mN4v
|
||||
api_key: ${GRIST_WORK_API_KEY} # Supports environment variable substitution
|
||||
|
||||
personal-tracker:
|
||||
url: https://docs.getgrist.com
|
||||
doc_id: pN0zE5sT2qP7x
|
||||
api_key: ${GRIST_PERSONAL_API_KEY}
|
||||
|
||||
# Agent tokens with access scopes
|
||||
tokens:
|
||||
- token: your-secure-token-here
|
||||
name: finance-agent
|
||||
scope:
|
||||
- document: budget-2024
|
||||
permissions: [read, write] # Can read and write
|
||||
|
||||
- token: another-token-here
|
||||
name: readonly-agent
|
||||
scope:
|
||||
- document: budget-2024
|
||||
permissions: [read] # Read only
|
||||
- document: personal-tracker
|
||||
permissions: [read, write, schema] # Full access
|
||||
```
|
||||
|
||||
### Permission Levels
|
||||
|
||||
- `read`: Query tables and records, run SQL queries
|
||||
- `write`: Add, update, delete records
|
||||
- `schema`: Create tables, add/modify/delete columns
|
||||
|
||||
## Security
|
||||
|
||||
- **Token-based auth**: Each agent has a unique token with specific document access
|
||||
@@ -159,10 +198,30 @@ For remote deployments, use the server's public URL:
|
||||
|
||||
## Development
|
||||
|
||||
### Running Tests
|
||||
### Requirements
|
||||
|
||||
- Python 3.14+
|
||||
- uv package manager
|
||||
|
||||
### Local Setup
|
||||
|
||||
```bash
|
||||
uv run pytest -v
|
||||
# Clone the repository
|
||||
git clone https://github.com/Xe138/grist-mcp-server.git
|
||||
cd grist-mcp-server
|
||||
|
||||
# Install dependencies
|
||||
uv sync --dev
|
||||
|
||||
# Run tests
|
||||
make test-unit
|
||||
```
|
||||
|
||||
### Running Locally
|
||||
|
||||
```bash
|
||||
export GRIST_MCP_TOKEN="your-agent-token"
|
||||
CONFIG_PATH=./config.yaml uv run python -m grist_mcp.main
|
||||
```
|
||||
|
||||
### Project Structure
|
||||
@@ -170,7 +229,6 @@ uv run pytest -v
|
||||
```
|
||||
grist-mcp/
|
||||
├── src/grist_mcp/
|
||||
│ ├── __init__.py
|
||||
│ ├── main.py # Entry point
|
||||
│ ├── server.py # MCP server setup and tool registration
|
||||
│ ├── config.py # Configuration loading
|
||||
@@ -182,73 +240,13 @@ grist-mcp/
|
||||
│ ├── write.py # Write operations
|
||||
│ └── schema.py # Schema operations
|
||||
├── tests/
|
||||
├── config.yaml.example
|
||||
└── pyproject.toml
|
||||
```
|
||||
|
||||
## Docker Deployment
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Docker and Docker Compose
|
||||
|
||||
### Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Copy example files
|
||||
cp .env.example .env
|
||||
cp config.yaml.example config.yaml
|
||||
|
||||
# 2. Edit .env with your tokens and API keys
|
||||
# - Set GRIST_MCP_TOKEN to a secure agent token
|
||||
# - Set your Grist API keys
|
||||
|
||||
# 3. Edit config.yaml with your document settings
|
||||
# - Configure your Grist documents
|
||||
# - Set up token scopes and permissions
|
||||
|
||||
# 4. Start the server
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
| Variable | Description | Default |
|
||||
|----------|-------------|---------|
|
||||
| `PORT` | Server port | `3000` |
|
||||
| `GRIST_MCP_TOKEN` | Agent authentication token (required) | - |
|
||||
| `CONFIG_PATH` | Path to config file inside container | `/app/config.yaml` |
|
||||
| `GRIST_*_API_KEY` | Grist API keys referenced in config.yaml | - |
|
||||
|
||||
### Using Prebuilt Images
|
||||
|
||||
To use a prebuilt image from a container registry:
|
||||
|
||||
```yaml
|
||||
# docker-compose.yaml
|
||||
services:
|
||||
grist-mcp:
|
||||
image: your-registry/grist-mcp:latest
|
||||
ports:
|
||||
- "${PORT:-3000}:3000"
|
||||
volumes:
|
||||
- ./config.yaml:/app/config.yaml:ro
|
||||
env_file:
|
||||
- .env
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
### Building Locally
|
||||
|
||||
```bash
|
||||
# Build the image
|
||||
docker build -t grist-mcp .
|
||||
|
||||
# Run directly
|
||||
docker run -p 3000:3000 \
|
||||
-v $(pwd)/config.yaml:/app/config.yaml:ro \
|
||||
--env-file .env \
|
||||
grist-mcp
|
||||
│ ├── unit/ # Unit tests
|
||||
│ └── integration/ # Integration tests
|
||||
├── deploy/
|
||||
│ ├── dev/ # Development docker-compose
|
||||
│ ├── test/ # Test docker-compose
|
||||
│ └── prod/ # Production docker-compose
|
||||
└── config.yaml.example
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
@@ -23,6 +23,14 @@ documents:
|
||||
doc_id: pN0zE5sT2qP7x
|
||||
api_key: ${GRIST_PERSONAL_API_KEY}
|
||||
|
||||
# Docker networking example: connect via internal hostname,
|
||||
# but send the external domain in the Host header
|
||||
docker-grist:
|
||||
url: http://grist:8080
|
||||
doc_id: abc123
|
||||
api_key: ${GRIST_API_KEY}
|
||||
host_header: grist.example.com # Required when Grist validates Host header
|
||||
|
||||
# Agent tokens with access scopes
|
||||
tokens:
|
||||
- token: REPLACE_WITH_GENERATED_TOKEN
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
# Production environment - resource limits, logging, restart policy
|
||||
# Production environment
|
||||
services:
|
||||
grist-mcp:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: Dockerfile
|
||||
image: ghcr.io/xe138/grist-mcp-server:latest
|
||||
ports:
|
||||
- "${PORT:-3000}:3000"
|
||||
volumes:
|
||||
@@ -12,18 +10,6 @@ services:
|
||||
- CONFIG_PATH=/app/config.yaml
|
||||
- EXTERNAL_PORT=${PORT:-3000}
|
||||
restart: unless-stopped
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 512M
|
||||
cpus: "1"
|
||||
reservations:
|
||||
memory: 128M
|
||||
logging:
|
||||
driver: "json-file"
|
||||
options:
|
||||
max-size: "50m"
|
||||
max-file: "5"
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:3000/health')"]
|
||||
interval: 30s
|
||||
|
||||
96
docs/plans/2026-01-02-logging-improvements-design.md
Normal file
96
docs/plans/2026-01-02-logging-improvements-design.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Logging Improvements Design
|
||||
|
||||
## Overview
|
||||
|
||||
Improve MCP server logging to provide meaningful operational visibility. Replace generic HTTP request logs with application-level context including agent identity, tool usage, document access, and operation stats.
|
||||
|
||||
## Current State
|
||||
|
||||
Logs show only uvicorn HTTP requests with no application context:
|
||||
```
|
||||
INFO: 172.20.0.2:43254 - "POST /messages?session_id=... HTTP/1.1" 202 Accepted
|
||||
INFO: 127.0.0.1:41508 - "GET /health HTTP/1.1" 200 OK
|
||||
```
|
||||
|
||||
## Desired State
|
||||
|
||||
Human-readable single-line format with full context:
|
||||
```
|
||||
2025-01-02 10:15:23 | dev-agent (abc...xyz) | get_records | sales | 42 records | success | 125ms
|
||||
2025-01-02 10:15:24 | dev-agent (abc...xyz) | update_records | sales | 3 records | success | 89ms
|
||||
2025-01-02 10:15:25 | dev-agent (abc...xyz) | add_records | inventory | 5 records | error | 89ms
|
||||
Grist API error: Invalid column 'foo'
|
||||
```
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| Decision | Choice |
|
||||
|----------|--------|
|
||||
| Log format | Human-readable single-line (pipe-delimited) |
|
||||
| Configuration | Environment variable only (`LOG_LEVEL`) |
|
||||
| Log levels | Standard (DEBUG/INFO/WARNING/ERROR) |
|
||||
| Health checks | DEBUG level only (suppressed at INFO) |
|
||||
| Error details | Multi-line (indented on second line) |
|
||||
|
||||
## Log Format
|
||||
|
||||
```
|
||||
YYYY-MM-DD HH:MM:SS | <agent_name> (<token_truncated>) | <tool> | <document> | <stats> | <status> | <duration>
|
||||
```
|
||||
|
||||
**Token truncation:** First 3 and last 3 characters (e.g., `abc...xyz`). Tokens <=8 chars show `***`.
|
||||
|
||||
**Document field:** Shows `-` for tools without a document (e.g., `list_documents`).
|
||||
|
||||
## Log Levels
|
||||
|
||||
| Level | Events |
|
||||
|-------|--------|
|
||||
| ERROR | Unhandled exceptions, Grist API failures |
|
||||
| WARNING | Auth errors (invalid token, permission denied) |
|
||||
| INFO | Tool calls (one line per call with stats) |
|
||||
| DEBUG | Health checks, detailed arguments, full results |
|
||||
|
||||
**Environment variable:** `LOG_LEVEL` (default: `INFO`)
|
||||
|
||||
## Stats Per Tool
|
||||
|
||||
| Tool | Stats |
|
||||
|------|-------|
|
||||
| `list_documents` | `N docs` |
|
||||
| `list_tables` | `N tables` |
|
||||
| `describe_table` | `N columns` |
|
||||
| `get_records` | `N records` |
|
||||
| `sql_query` | `N rows` |
|
||||
| `add_records` | `N records` |
|
||||
| `update_records` | `N records` |
|
||||
| `delete_records` | `N records` |
|
||||
| `create_table` | `N columns` |
|
||||
| `add_column` | `1 column` |
|
||||
| `modify_column` | `1 column` |
|
||||
| `delete_column` | `1 column` |
|
||||
|
||||
## Files Changed
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `src/grist_mcp/logging.py` | New - logging setup, formatters, stats extraction |
|
||||
| `src/grist_mcp/main.py` | Call `setup_logging()`, configure uvicorn logger |
|
||||
| `src/grist_mcp/server.py` | Wrap `call_tool` with logging |
|
||||
| `tests/unit/test_logging.py` | New - unit tests for logging module |
|
||||
|
||||
Tool implementations in `tools/` remain unchanged - logging is handled at the server layer.
|
||||
|
||||
## Testing
|
||||
|
||||
**Unit tests:**
|
||||
- `test_setup_logging_default_level`
|
||||
- `test_setup_logging_from_env`
|
||||
- `test_token_truncation`
|
||||
- `test_extract_stats`
|
||||
- `test_format_log_line`
|
||||
- `test_error_multiline_format`
|
||||
|
||||
**Manual verification:**
|
||||
- Run `make dev-up`, make tool calls, verify log format
|
||||
- Test with `LOG_LEVEL=DEBUG` for verbose output
|
||||
821
docs/plans/2026-01-02-logging-improvements-impl.md
Normal file
821
docs/plans/2026-01-02-logging-improvements-impl.md
Normal file
@@ -0,0 +1,821 @@
|
||||
# Logging Improvements Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Add informative application-level logging that shows agent identity, tool usage, document access, and operation stats.
|
||||
|
||||
**Architecture:** New `logging.py` module provides setup and formatting. `server.py` wraps tool calls with timing and stats extraction. `main.py` initializes logging and configures uvicorn to suppress health check noise.
|
||||
|
||||
**Tech Stack:** Python `logging` stdlib, custom `Formatter`, uvicorn log config
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Token Truncation Helper
|
||||
|
||||
**Files:**
|
||||
- Create: `src/grist_mcp/logging.py`
|
||||
- Test: `tests/unit/test_logging.py`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Create `tests/unit/test_logging.py`:
|
||||
|
||||
```python
|
||||
"""Unit tests for logging module."""
|
||||
|
||||
import pytest
|
||||
|
||||
from grist_mcp.logging import truncate_token
|
||||
|
||||
|
||||
class TestTruncateToken:
|
||||
def test_normal_token_shows_prefix_suffix(self):
|
||||
token = "abcdefghijklmnop"
|
||||
assert truncate_token(token) == "abc...nop"
|
||||
|
||||
def test_short_token_shows_asterisks(self):
|
||||
token = "abcdefgh" # 8 chars
|
||||
assert truncate_token(token) == "***"
|
||||
|
||||
def test_very_short_token_shows_asterisks(self):
|
||||
token = "abc"
|
||||
assert truncate_token(token) == "***"
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py -v`
|
||||
Expected: FAIL with "No module named 'grist_mcp.logging'"
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Create `src/grist_mcp/logging.py`:
|
||||
|
||||
```python
|
||||
"""Logging configuration and utilities."""
|
||||
|
||||
|
||||
def truncate_token(token: str) -> str:
|
||||
"""Truncate token to show first 3 and last 3 chars.
|
||||
|
||||
Tokens 8 chars or shorter show *** for security.
|
||||
"""
|
||||
if len(token) <= 8:
|
||||
return "***"
|
||||
return f"{token[:3]}...{token[-3:]}"
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py -v`
|
||||
Expected: PASS (3 tests)
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/grist_mcp/logging.py tests/unit/test_logging.py
|
||||
git commit -m "feat(logging): add token truncation helper"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Stats Extraction Function
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/grist_mcp/logging.py`
|
||||
- Modify: `tests/unit/test_logging.py`
|
||||
|
||||
**Step 1: Write the failing tests**
|
||||
|
||||
Add to `tests/unit/test_logging.py`:
|
||||
|
||||
```python
|
||||
from grist_mcp.logging import truncate_token, extract_stats
|
||||
|
||||
|
||||
class TestExtractStats:
|
||||
def test_list_documents(self):
|
||||
result = {"documents": [{"name": "a"}, {"name": "b"}, {"name": "c"}]}
|
||||
assert extract_stats("list_documents", {}, result) == "3 docs"
|
||||
|
||||
def test_list_tables(self):
|
||||
result = {"tables": ["Orders", "Products"]}
|
||||
assert extract_stats("list_tables", {}, result) == "2 tables"
|
||||
|
||||
def test_describe_table(self):
|
||||
result = {"columns": [{"id": "A"}, {"id": "B"}]}
|
||||
assert extract_stats("describe_table", {}, result) == "2 columns"
|
||||
|
||||
def test_get_records(self):
|
||||
result = {"records": [{"id": 1}, {"id": 2}]}
|
||||
assert extract_stats("get_records", {}, result) == "2 records"
|
||||
|
||||
def test_sql_query(self):
|
||||
result = {"records": [{"a": 1}, {"a": 2}, {"a": 3}]}
|
||||
assert extract_stats("sql_query", {}, result) == "3 rows"
|
||||
|
||||
def test_add_records_from_args(self):
|
||||
args = {"records": [{"a": 1}, {"a": 2}]}
|
||||
assert extract_stats("add_records", args, {"ids": [1, 2]}) == "2 records"
|
||||
|
||||
def test_update_records_from_args(self):
|
||||
args = {"records": [{"id": 1, "fields": {}}, {"id": 2, "fields": {}}]}
|
||||
assert extract_stats("update_records", args, {}) == "2 records"
|
||||
|
||||
def test_delete_records_from_args(self):
|
||||
args = {"record_ids": [1, 2, 3]}
|
||||
assert extract_stats("delete_records", args, {}) == "3 records"
|
||||
|
||||
def test_create_table(self):
|
||||
args = {"columns": [{"id": "A"}, {"id": "B"}]}
|
||||
assert extract_stats("create_table", args, {}) == "2 columns"
|
||||
|
||||
def test_single_column_ops(self):
|
||||
assert extract_stats("add_column", {}, {}) == "1 column"
|
||||
assert extract_stats("modify_column", {}, {}) == "1 column"
|
||||
assert extract_stats("delete_column", {}, {}) == "1 column"
|
||||
|
||||
def test_unknown_tool(self):
|
||||
assert extract_stats("unknown_tool", {}, {}) == "-"
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py::TestExtractStats -v`
|
||||
Expected: FAIL with "cannot import name 'extract_stats'"
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Add to `src/grist_mcp/logging.py`:
|
||||
|
||||
```python
|
||||
def extract_stats(tool_name: str, arguments: dict, result: dict) -> str:
|
||||
"""Extract meaningful stats from tool call based on tool type."""
|
||||
if tool_name == "list_documents":
|
||||
count = len(result.get("documents", []))
|
||||
return f"{count} docs"
|
||||
|
||||
if tool_name == "list_tables":
|
||||
count = len(result.get("tables", []))
|
||||
return f"{count} tables"
|
||||
|
||||
if tool_name == "describe_table":
|
||||
count = len(result.get("columns", []))
|
||||
return f"{count} columns"
|
||||
|
||||
if tool_name == "get_records":
|
||||
count = len(result.get("records", []))
|
||||
return f"{count} records"
|
||||
|
||||
if tool_name == "sql_query":
|
||||
count = len(result.get("records", []))
|
||||
return f"{count} rows"
|
||||
|
||||
if tool_name == "add_records":
|
||||
count = len(arguments.get("records", []))
|
||||
return f"{count} records"
|
||||
|
||||
if tool_name == "update_records":
|
||||
count = len(arguments.get("records", []))
|
||||
return f"{count} records"
|
||||
|
||||
if tool_name == "delete_records":
|
||||
count = len(arguments.get("record_ids", []))
|
||||
return f"{count} records"
|
||||
|
||||
if tool_name == "create_table":
|
||||
count = len(arguments.get("columns", []))
|
||||
return f"{count} columns"
|
||||
|
||||
if tool_name in ("add_column", "modify_column", "delete_column"):
|
||||
return "1 column"
|
||||
|
||||
return "-"
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py -v`
|
||||
Expected: PASS (all tests)
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/grist_mcp/logging.py tests/unit/test_logging.py
|
||||
git commit -m "feat(logging): add stats extraction for all tools"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Log Line Formatter
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/grist_mcp/logging.py`
|
||||
- Modify: `tests/unit/test_logging.py`
|
||||
|
||||
**Step 1: Write the failing tests**
|
||||
|
||||
Add to `tests/unit/test_logging.py`:
|
||||
|
||||
```python
|
||||
from grist_mcp.logging import truncate_token, extract_stats, format_tool_log
|
||||
|
||||
|
||||
class TestFormatToolLog:
|
||||
def test_success_format(self):
|
||||
line = format_tool_log(
|
||||
agent_name="dev-agent",
|
||||
token="abcdefghijklmnop",
|
||||
tool="get_records",
|
||||
document="sales",
|
||||
stats="42 records",
|
||||
status="success",
|
||||
duration_ms=125,
|
||||
)
|
||||
assert "dev-agent" in line
|
||||
assert "abc...nop" in line
|
||||
assert "get_records" in line
|
||||
assert "sales" in line
|
||||
assert "42 records" in line
|
||||
assert "success" in line
|
||||
assert "125ms" in line
|
||||
# Check pipe-delimited format
|
||||
assert line.count("|") == 6
|
||||
|
||||
def test_no_document(self):
|
||||
line = format_tool_log(
|
||||
agent_name="dev-agent",
|
||||
token="abcdefghijklmnop",
|
||||
tool="list_documents",
|
||||
document=None,
|
||||
stats="3 docs",
|
||||
status="success",
|
||||
duration_ms=45,
|
||||
)
|
||||
assert "| - |" in line
|
||||
|
||||
def test_error_format(self):
|
||||
line = format_tool_log(
|
||||
agent_name="dev-agent",
|
||||
token="abcdefghijklmnop",
|
||||
tool="add_records",
|
||||
document="inventory",
|
||||
stats="5 records",
|
||||
status="error",
|
||||
duration_ms=89,
|
||||
error_message="Grist API error: Invalid column 'foo'",
|
||||
)
|
||||
assert "error" in line
|
||||
assert "\n Grist API error: Invalid column 'foo'" in line
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py::TestFormatToolLog -v`
|
||||
Expected: FAIL with "cannot import name 'format_tool_log'"
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Add to `src/grist_mcp/logging.py`:
|
||||
|
||||
```python
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
def format_tool_log(
|
||||
agent_name: str,
|
||||
token: str,
|
||||
tool: str,
|
||||
document: str | None,
|
||||
stats: str,
|
||||
status: str,
|
||||
duration_ms: int,
|
||||
error_message: str | None = None,
|
||||
) -> str:
|
||||
"""Format a tool call log line.
|
||||
|
||||
Format: YYYY-MM-DD HH:MM:SS | agent (token) | tool | doc | stats | status | duration
|
||||
"""
|
||||
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
truncated = truncate_token(token)
|
||||
doc = document if document else "-"
|
||||
|
||||
line = f"{timestamp} | {agent_name} ({truncated}) | {tool} | {doc} | {stats} | {status} | {duration_ms}ms"
|
||||
|
||||
if error_message:
|
||||
line += f"\n {error_message}"
|
||||
|
||||
return line
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py -v`
|
||||
Expected: PASS (all tests)
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/grist_mcp/logging.py tests/unit/test_logging.py
|
||||
git commit -m "feat(logging): add log line formatter"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Setup Logging Function
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/grist_mcp/logging.py`
|
||||
- Modify: `tests/unit/test_logging.py`
|
||||
|
||||
**Step 1: Write the failing tests**
|
||||
|
||||
Add to `tests/unit/test_logging.py`:
|
||||
|
||||
```python
|
||||
import logging
|
||||
import os
|
||||
|
||||
|
||||
class TestSetupLogging:
|
||||
def test_default_level_is_info(self, monkeypatch):
|
||||
monkeypatch.delenv("LOG_LEVEL", raising=False)
|
||||
|
||||
from grist_mcp.logging import setup_logging
|
||||
setup_logging()
|
||||
|
||||
logger = logging.getLogger("grist_mcp")
|
||||
assert logger.level == logging.INFO
|
||||
|
||||
def test_respects_log_level_env(self, monkeypatch):
|
||||
monkeypatch.setenv("LOG_LEVEL", "DEBUG")
|
||||
|
||||
from grist_mcp.logging import setup_logging
|
||||
setup_logging()
|
||||
|
||||
logger = logging.getLogger("grist_mcp")
|
||||
assert logger.level == logging.DEBUG
|
||||
|
||||
def test_invalid_level_defaults_to_info(self, monkeypatch):
|
||||
monkeypatch.setenv("LOG_LEVEL", "INVALID")
|
||||
|
||||
from grist_mcp.logging import setup_logging
|
||||
setup_logging()
|
||||
|
||||
logger = logging.getLogger("grist_mcp")
|
||||
assert logger.level == logging.INFO
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py::TestSetupLogging -v`
|
||||
Expected: FAIL with "cannot import name 'setup_logging'"
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Add to `src/grist_mcp/logging.py`:
|
||||
|
||||
```python
|
||||
import logging
|
||||
import os
|
||||
|
||||
|
||||
def setup_logging() -> None:
|
||||
"""Configure logging based on LOG_LEVEL environment variable.
|
||||
|
||||
Valid levels: DEBUG, INFO, WARNING, ERROR (default: INFO)
|
||||
"""
|
||||
level_name = os.environ.get("LOG_LEVEL", "INFO").upper()
|
||||
level = getattr(logging, level_name, None)
|
||||
|
||||
if not isinstance(level, int):
|
||||
level = logging.INFO
|
||||
|
||||
logger = logging.getLogger("grist_mcp")
|
||||
logger.setLevel(level)
|
||||
|
||||
# Only add handler if not already configured
|
||||
if not logger.handlers:
|
||||
handler = logging.StreamHandler()
|
||||
handler.setFormatter(logging.Formatter("%(message)s"))
|
||||
logger.addHandler(handler)
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py -v`
|
||||
Expected: PASS (all tests)
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/grist_mcp/logging.py tests/unit/test_logging.py
|
||||
git commit -m "feat(logging): add setup_logging with LOG_LEVEL support"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Get Logger Helper
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/grist_mcp/logging.py`
|
||||
- Modify: `tests/unit/test_logging.py`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Add to `tests/unit/test_logging.py`:
|
||||
|
||||
```python
|
||||
class TestGetLogger:
|
||||
def test_returns_child_logger(self):
|
||||
from grist_mcp.logging import get_logger
|
||||
|
||||
logger = get_logger("server")
|
||||
assert logger.name == "grist_mcp.server"
|
||||
|
||||
def test_inherits_parent_level(self, monkeypatch):
|
||||
monkeypatch.setenv("LOG_LEVEL", "WARNING")
|
||||
|
||||
from grist_mcp.logging import setup_logging, get_logger
|
||||
setup_logging()
|
||||
|
||||
logger = get_logger("test")
|
||||
# Child inherits from parent when level is NOTSET
|
||||
assert logger.getEffectiveLevel() == logging.WARNING
|
||||
```
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py::TestGetLogger -v`
|
||||
Expected: FAIL with "cannot import name 'get_logger'"
|
||||
|
||||
**Step 3: Write minimal implementation**
|
||||
|
||||
Add to `src/grist_mcp/logging.py`:
|
||||
|
||||
```python
|
||||
def get_logger(name: str) -> logging.Logger:
|
||||
"""Get a child logger under the grist_mcp namespace."""
|
||||
return logging.getLogger(f"grist_mcp.{name}")
|
||||
```
|
||||
|
||||
**Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `uv run pytest tests/unit/test_logging.py -v`
|
||||
Expected: PASS (all tests)
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/grist_mcp/logging.py tests/unit/test_logging.py
|
||||
git commit -m "feat(logging): add get_logger helper"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Integrate Logging into Server
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/grist_mcp/server.py`
|
||||
|
||||
**Step 1: Add logging imports and logger**
|
||||
|
||||
At the top of `src/grist_mcp/server.py`, add imports:
|
||||
|
||||
```python
|
||||
import time
|
||||
from grist_mcp.logging import get_logger, extract_stats, format_tool_log
|
||||
|
||||
logger = get_logger("server")
|
||||
```
|
||||
|
||||
**Step 2: Wrap call_tool with logging**
|
||||
|
||||
Replace the `call_tool` function body (lines 209-276) with this logged version:
|
||||
|
||||
```python
|
||||
@server.call_tool()
|
||||
async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
start_time = time.time()
|
||||
document = arguments.get("document")
|
||||
|
||||
# Log arguments at DEBUG level
|
||||
logger.debug(
|
||||
format_tool_log(
|
||||
agent_name=_current_agent.name,
|
||||
token=_current_agent.token,
|
||||
tool=name,
|
||||
document=document,
|
||||
stats=f"args: {json.dumps(arguments)}",
|
||||
status="started",
|
||||
duration_ms=0,
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
if name == "list_documents":
|
||||
result = await _list_documents(_current_agent)
|
||||
elif name == "list_tables":
|
||||
result = await _list_tables(_current_agent, auth, arguments["document"])
|
||||
elif name == "describe_table":
|
||||
result = await _describe_table(
|
||||
_current_agent, auth, arguments["document"], arguments["table"]
|
||||
)
|
||||
elif name == "get_records":
|
||||
result = await _get_records(
|
||||
_current_agent, auth, arguments["document"], arguments["table"],
|
||||
filter=arguments.get("filter"),
|
||||
sort=arguments.get("sort"),
|
||||
limit=arguments.get("limit"),
|
||||
)
|
||||
elif name == "sql_query":
|
||||
result = await _sql_query(
|
||||
_current_agent, auth, arguments["document"], arguments["query"]
|
||||
)
|
||||
elif name == "add_records":
|
||||
result = await _add_records(
|
||||
_current_agent, auth, arguments["document"], arguments["table"],
|
||||
arguments["records"],
|
||||
)
|
||||
elif name == "update_records":
|
||||
result = await _update_records(
|
||||
_current_agent, auth, arguments["document"], arguments["table"],
|
||||
arguments["records"],
|
||||
)
|
||||
elif name == "delete_records":
|
||||
result = await _delete_records(
|
||||
_current_agent, auth, arguments["document"], arguments["table"],
|
||||
arguments["record_ids"],
|
||||
)
|
||||
elif name == "create_table":
|
||||
result = await _create_table(
|
||||
_current_agent, auth, arguments["document"], arguments["table_id"],
|
||||
arguments["columns"],
|
||||
)
|
||||
elif name == "add_column":
|
||||
result = await _add_column(
|
||||
_current_agent, auth, arguments["document"], arguments["table"],
|
||||
arguments["column_id"], arguments["column_type"],
|
||||
formula=arguments.get("formula"),
|
||||
)
|
||||
elif name == "modify_column":
|
||||
result = await _modify_column(
|
||||
_current_agent, auth, arguments["document"], arguments["table"],
|
||||
arguments["column_id"],
|
||||
type=arguments.get("type"),
|
||||
formula=arguments.get("formula"),
|
||||
)
|
||||
elif name == "delete_column":
|
||||
result = await _delete_column(
|
||||
_current_agent, auth, arguments["document"], arguments["table"],
|
||||
arguments["column_id"],
|
||||
)
|
||||
else:
|
||||
return [TextContent(type="text", text=f"Unknown tool: {name}")]
|
||||
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
stats = extract_stats(name, arguments, result)
|
||||
|
||||
logger.info(
|
||||
format_tool_log(
|
||||
agent_name=_current_agent.name,
|
||||
token=_current_agent.token,
|
||||
tool=name,
|
||||
document=document,
|
||||
stats=stats,
|
||||
status="success",
|
||||
duration_ms=duration_ms,
|
||||
)
|
||||
)
|
||||
|
||||
return [TextContent(type="text", text=json.dumps(result))]
|
||||
|
||||
except AuthError as e:
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.warning(
|
||||
format_tool_log(
|
||||
agent_name=_current_agent.name,
|
||||
token=_current_agent.token,
|
||||
tool=name,
|
||||
document=document,
|
||||
stats="-",
|
||||
status="auth_error",
|
||||
duration_ms=duration_ms,
|
||||
error_message=str(e),
|
||||
)
|
||||
)
|
||||
return [TextContent(type="text", text=f"Authorization error: {e}")]
|
||||
|
||||
except Exception as e:
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.error(
|
||||
format_tool_log(
|
||||
agent_name=_current_agent.name,
|
||||
token=_current_agent.token,
|
||||
tool=name,
|
||||
document=document,
|
||||
stats="-",
|
||||
status="error",
|
||||
duration_ms=duration_ms,
|
||||
error_message=str(e),
|
||||
)
|
||||
)
|
||||
return [TextContent(type="text", text=f"Error: {e}")]
|
||||
```
|
||||
|
||||
**Step 3: Run tests to verify nothing broke**
|
||||
|
||||
Run: `uv run pytest tests/unit/ -v`
|
||||
Expected: PASS (all tests)
|
||||
|
||||
**Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add src/grist_mcp/server.py
|
||||
git commit -m "feat(logging): add tool call logging to server"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Initialize Logging in Main
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/grist_mcp/main.py`
|
||||
|
||||
**Step 1: Add logging setup to main()**
|
||||
|
||||
Add import at top of `src/grist_mcp/main.py`:
|
||||
|
||||
```python
|
||||
from grist_mcp.logging import setup_logging
|
||||
```
|
||||
|
||||
**Step 2: Call setup_logging at start of main()**
|
||||
|
||||
In the `main()` function, add as the first line after the port/config variables:
|
||||
|
||||
```python
|
||||
def main():
|
||||
"""Run the SSE server."""
|
||||
port = int(os.environ.get("PORT", "3000"))
|
||||
external_port = int(os.environ.get("EXTERNAL_PORT", str(port)))
|
||||
config_path = os.environ.get("CONFIG_PATH", "/app/config.yaml")
|
||||
|
||||
setup_logging() # <-- Add this line
|
||||
|
||||
if not _ensure_config(config_path):
|
||||
```
|
||||
|
||||
**Step 3: Run tests to verify nothing broke**
|
||||
|
||||
Run: `uv run pytest tests/unit/ -v`
|
||||
Expected: PASS (all tests)
|
||||
|
||||
**Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add src/grist_mcp/main.py
|
||||
git commit -m "feat(logging): initialize logging on server startup"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Suppress Health Check Noise
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/grist_mcp/main.py`
|
||||
|
||||
**Step 1: Configure uvicorn to use custom log config**
|
||||
|
||||
Replace the `uvicorn.run` call in `main()` with:
|
||||
|
||||
```python
|
||||
# Configure uvicorn logging to reduce health check noise
|
||||
log_config = uvicorn.config.LOGGING_CONFIG
|
||||
log_config["formatters"]["default"]["fmt"] = "%(message)s"
|
||||
log_config["formatters"]["access"]["fmt"] = "%(message)s"
|
||||
|
||||
uvicorn.run(app, host="0.0.0.0", port=port, log_config=log_config)
|
||||
```
|
||||
|
||||
**Step 2: Add health check filter**
|
||||
|
||||
Create a filter class and apply it. Add before the `main()` function:
|
||||
|
||||
```python
|
||||
class HealthCheckFilter(logging.Filter):
|
||||
"""Filter out health check requests at INFO level."""
|
||||
|
||||
def filter(self, record: logging.LogRecord) -> bool:
|
||||
message = record.getMessage()
|
||||
if "/health" in message:
|
||||
# Downgrade to DEBUG by changing the level
|
||||
record.levelno = logging.DEBUG
|
||||
record.levelname = "DEBUG"
|
||||
return True
|
||||
```
|
||||
|
||||
Add import at top:
|
||||
|
||||
```python
|
||||
import logging
|
||||
```
|
||||
|
||||
**Step 3: Apply filter in main()**
|
||||
|
||||
After `setup_logging()` call, add:
|
||||
|
||||
```python
|
||||
setup_logging()
|
||||
|
||||
# Add health check filter to uvicorn access logger
|
||||
logging.getLogger("uvicorn.access").addFilter(HealthCheckFilter())
|
||||
```
|
||||
|
||||
**Step 4: Run tests to verify nothing broke**
|
||||
|
||||
Run: `uv run pytest tests/unit/ -v`
|
||||
Expected: PASS (all tests)
|
||||
|
||||
**Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/grist_mcp/main.py
|
||||
git commit -m "feat(logging): suppress health checks at INFO level"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Manual Verification
|
||||
|
||||
**Step 1: Start development environment**
|
||||
|
||||
Run: `make dev-up`
|
||||
|
||||
**Step 2: Make some tool calls**
|
||||
|
||||
Use Claude Code or another MCP client to call some tools (list_documents, get_records, etc.)
|
||||
|
||||
**Step 3: Verify log format**
|
||||
|
||||
Check docker logs show the expected format:
|
||||
```
|
||||
2026-01-02 10:15:23 | dev-agent (abc...xyz) | get_records | sales | 42 records | success | 125ms
|
||||
```
|
||||
|
||||
**Step 4: Test DEBUG level**
|
||||
|
||||
Restart with `LOG_LEVEL=DEBUG` and verify:
|
||||
- Health checks appear
|
||||
- Detailed args appear for each call
|
||||
|
||||
**Step 5: Clean up**
|
||||
|
||||
Run: `make dev-down`
|
||||
|
||||
---
|
||||
|
||||
### Task 10: Update Module Exports
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/grist_mcp/logging.py`
|
||||
|
||||
**Step 1: Add __all__ export list**
|
||||
|
||||
At the top of `src/grist_mcp/logging.py` (after imports), add:
|
||||
|
||||
```python
|
||||
__all__ = [
|
||||
"setup_logging",
|
||||
"get_logger",
|
||||
"truncate_token",
|
||||
"extract_stats",
|
||||
"format_tool_log",
|
||||
]
|
||||
```
|
||||
|
||||
**Step 2: Run all tests**
|
||||
|
||||
Run: `uv run pytest tests/unit/ -v`
|
||||
Expected: PASS (all tests)
|
||||
|
||||
**Step 3: Final commit**
|
||||
|
||||
```bash
|
||||
git add src/grist_mcp/logging.py
|
||||
git commit -m "chore(logging): add module exports"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
After completing all tasks, the logging module provides:
|
||||
- `LOG_LEVEL` environment variable support (DEBUG/INFO/WARNING/ERROR)
|
||||
- Human-readable pipe-delimited log format
|
||||
- Token truncation for security
|
||||
- Stats extraction per tool type
|
||||
- Health check suppression at INFO level
|
||||
- Multi-line error details
|
||||
|
||||
The implementation follows TDD with frequent commits, keeping each change small and verifiable.
|
||||
310
docs/plans/2026-01-02-session-proxy-design.md
Normal file
310
docs/plans/2026-01-02-session-proxy-design.md
Normal file
@@ -0,0 +1,310 @@
|
||||
# Session Token Proxy Design
|
||||
|
||||
## Problem
|
||||
|
||||
When an agent needs to insert, update, or query thousands of records, the LLM must generate all that JSON in its response. This is slow regardless of how fast the actual API call is. The LLM generation time is the bottleneck.
|
||||
|
||||
## Solution
|
||||
|
||||
Add a "session token" mechanism that lets agents delegate bulk data operations to scripts that call grist-mcp directly over HTTP, bypassing LLM generation entirely.
|
||||
|
||||
## Flow
|
||||
|
||||
```
|
||||
1. Agent calls MCP tool:
|
||||
request_session_token(document="sales", permissions=["write"], ttl_seconds=300)
|
||||
|
||||
2. Server generates token, stores in memory:
|
||||
{"sess_abc123...": {document: "sales", permissions: ["write"], expires: <timestamp>}}
|
||||
|
||||
3. Server returns token to agent:
|
||||
{"token": "sess_abc123...", "expires_in": 300, "proxy_url": "/api/v1/proxy"}
|
||||
|
||||
4. Agent spawns script with token:
|
||||
python bulk_insert.py --token sess_abc123... --file data.csv
|
||||
|
||||
5. Script calls grist-mcp HTTP endpoint:
|
||||
POST /api/v1/proxy
|
||||
Authorization: Bearer sess_abc123...
|
||||
{"table": "Orders", "method": "add_records", "records": [...]}
|
||||
|
||||
6. Server validates token, executes against Grist, returns result
|
||||
```
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| Decision | Choice | Rationale |
|
||||
|----------|--------|-----------|
|
||||
| Token scope | Single document + permission level | Simpler than multi-doc; matches existing permission model |
|
||||
| Token storage | In-memory dict | Appropriate for short-lived tokens; restart invalidates (acceptable) |
|
||||
| HTTP interface | Wrapped endpoint `/api/v1/proxy` | Simpler than mirroring Grist API paths |
|
||||
| Request format | Discrete fields (table, method, etc.) | Scripts don't need to know Grist internals or doc IDs |
|
||||
| Document in request | Implicit from token | Token is scoped to one document; no need to specify |
|
||||
| Server architecture | Single process, add routes | Already running HTTP for SSE; just add routes |
|
||||
|
||||
## MCP Tool: get_proxy_documentation
|
||||
|
||||
Returns complete documentation for the HTTP proxy API. Agents call this when writing scripts that will use the proxy.
|
||||
|
||||
**Input schema**:
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {},
|
||||
"required": []
|
||||
}
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"description": "HTTP proxy API for bulk data operations. Use request_session_token to get a short-lived token, then call the proxy endpoint directly from scripts.",
|
||||
"endpoint": "POST /api/v1/proxy",
|
||||
"authentication": "Bearer token in Authorization header",
|
||||
"request_format": {
|
||||
"method": "Operation name (required)",
|
||||
"table": "Table name (required for most operations)",
|
||||
"...": "Additional fields vary by method"
|
||||
},
|
||||
"methods": {
|
||||
"get_records": {
|
||||
"description": "Fetch records from a table",
|
||||
"fields": {"table": "string", "filter": "object (optional)", "sort": "string (optional)", "limit": "integer (optional)"}
|
||||
},
|
||||
"sql_query": {
|
||||
"description": "Run a read-only SQL query",
|
||||
"fields": {"query": "string"}
|
||||
},
|
||||
"list_tables": {
|
||||
"description": "List all tables in the document",
|
||||
"fields": {}
|
||||
},
|
||||
"describe_table": {
|
||||
"description": "Get column information for a table",
|
||||
"fields": {"table": "string"}
|
||||
},
|
||||
"add_records": {
|
||||
"description": "Add records to a table",
|
||||
"fields": {"table": "string", "records": "array of objects"}
|
||||
},
|
||||
"update_records": {
|
||||
"description": "Update existing records",
|
||||
"fields": {"table": "string", "records": "array of {id, fields}"}
|
||||
},
|
||||
"delete_records": {
|
||||
"description": "Delete records by ID",
|
||||
"fields": {"table": "string", "record_ids": "array of integers"}
|
||||
},
|
||||
"create_table": {
|
||||
"description": "Create a new table",
|
||||
"fields": {"table_id": "string", "columns": "array of {id, type}"}
|
||||
},
|
||||
"add_column": {
|
||||
"description": "Add a column to a table",
|
||||
"fields": {"table": "string", "column_id": "string", "column_type": "string", "formula": "string (optional)"}
|
||||
},
|
||||
"modify_column": {
|
||||
"description": "Modify a column's type or formula",
|
||||
"fields": {"table": "string", "column_id": "string", "type": "string (optional)", "formula": "string (optional)"}
|
||||
},
|
||||
"delete_column": {
|
||||
"description": "Delete a column",
|
||||
"fields": {"table": "string", "column_id": "string"}
|
||||
}
|
||||
},
|
||||
"response_format": {
|
||||
"success": {"success": true, "data": "..."},
|
||||
"error": {"success": false, "error": "message", "code": "ERROR_CODE"}
|
||||
},
|
||||
"error_codes": ["UNAUTHORIZED", "INVALID_TOKEN", "TOKEN_EXPIRED", "INVALID_REQUEST", "GRIST_ERROR"],
|
||||
"example_script": "#!/usr/bin/env python3\nimport requests\nimport sys\n\ntoken = sys.argv[1]\nhost = sys.argv[2]\n\nresponse = requests.post(\n f'{host}/api/v1/proxy',\n headers={'Authorization': f'Bearer {token}'},\n json={'method': 'add_records', 'table': 'Orders', 'records': [{'item': 'Widget', 'qty': 100}]}\n)\nprint(response.json())"
|
||||
}
|
||||
```
|
||||
|
||||
## MCP Tool: request_session_token
|
||||
|
||||
**Input schema**:
|
||||
```json
|
||||
{
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"document": {
|
||||
"type": "string",
|
||||
"description": "Document name to grant access to"
|
||||
},
|
||||
"permissions": {
|
||||
"type": "array",
|
||||
"items": {"type": "string", "enum": ["read", "write", "schema"]},
|
||||
"description": "Permission levels to grant (cannot exceed agent's permissions)"
|
||||
},
|
||||
"ttl_seconds": {
|
||||
"type": "integer",
|
||||
"description": "Token lifetime in seconds (max 3600, default 300)"
|
||||
}
|
||||
},
|
||||
"required": ["document", "permissions"]
|
||||
}
|
||||
```
|
||||
|
||||
**Response**:
|
||||
```json
|
||||
{
|
||||
"token": "sess_a1b2c3d4...",
|
||||
"document": "sales",
|
||||
"permissions": ["write"],
|
||||
"expires_at": "2025-01-02T15:30:00Z",
|
||||
"proxy_url": "/api/v1/proxy"
|
||||
}
|
||||
```
|
||||
|
||||
**Validation**:
|
||||
- Agent must have access to the requested document
|
||||
- Requested permissions cannot exceed agent's permissions for that document
|
||||
- TTL capped at 3600 seconds (1 hour), default 300 seconds (5 minutes)
|
||||
|
||||
## Proxy Endpoint
|
||||
|
||||
**Endpoint**: `POST /api/v1/proxy`
|
||||
|
||||
**Authentication**: `Authorization: Bearer <session_token>`
|
||||
|
||||
**Request body** - method determines required fields:
|
||||
|
||||
```python
|
||||
# Read operations
|
||||
{"method": "get_records", "table": "Orders", "filter": {...}, "sort": "date", "limit": 1000}
|
||||
{"method": "sql_query", "query": "SELECT * FROM Orders WHERE amount > 100"}
|
||||
{"method": "list_tables"}
|
||||
{"method": "describe_table", "table": "Orders"}
|
||||
|
||||
# Write operations
|
||||
{"method": "add_records", "table": "Orders", "records": [{...}, {...}]}
|
||||
{"method": "update_records", "table": "Orders", "records": [{"id": 1, "fields": {...}}]}
|
||||
{"method": "delete_records", "table": "Orders", "record_ids": [1, 2, 3]}
|
||||
|
||||
# Schema operations
|
||||
{"method": "create_table", "table_id": "NewTable", "columns": [{...}]}
|
||||
{"method": "add_column", "table": "Orders", "column_id": "status", "column_type": "Text"}
|
||||
{"method": "modify_column", "table": "Orders", "column_id": "status", "type": "Choice"}
|
||||
{"method": "delete_column", "table": "Orders", "column_id": "old_field"}
|
||||
```
|
||||
|
||||
**Response format**:
|
||||
```json
|
||||
{"success": true, "data": {...}}
|
||||
{"success": false, "error": "Permission denied", "code": "UNAUTHORIZED"}
|
||||
```
|
||||
|
||||
**Error codes**:
|
||||
- `UNAUTHORIZED` - Permission denied for this operation
|
||||
- `INVALID_TOKEN` - Token format invalid or not found
|
||||
- `TOKEN_EXPIRED` - Token has expired
|
||||
- `INVALID_REQUEST` - Malformed request body
|
||||
- `GRIST_ERROR` - Error from Grist API
|
||||
|
||||
## Implementation Architecture
|
||||
|
||||
### New Files
|
||||
|
||||
**`src/grist_mcp/session.py`** - Session token management:
|
||||
```python
|
||||
@dataclass
|
||||
class SessionToken:
|
||||
token: str
|
||||
document: str
|
||||
permissions: list[str]
|
||||
agent_name: str
|
||||
created_at: datetime
|
||||
expires_at: datetime
|
||||
|
||||
class SessionTokenManager:
|
||||
def __init__(self):
|
||||
self._tokens: dict[str, SessionToken] = {}
|
||||
|
||||
def create_token(self, agent: Agent, document: str,
|
||||
permissions: list[str], ttl_seconds: int) -> SessionToken:
|
||||
"""Create a new session token. Validates permissions against agent's scope."""
|
||||
...
|
||||
|
||||
def validate_token(self, token: str) -> SessionToken | None:
|
||||
"""Validate token and return session info. Returns None if invalid/expired."""
|
||||
# Also cleans up this token if expired
|
||||
...
|
||||
|
||||
def cleanup_expired(self) -> int:
|
||||
"""Remove all expired tokens. Returns count removed."""
|
||||
...
|
||||
```
|
||||
|
||||
**`src/grist_mcp/proxy.py`** - HTTP proxy handler:
|
||||
```python
|
||||
async def handle_proxy(
|
||||
scope: Scope,
|
||||
receive: Receive,
|
||||
send: Send,
|
||||
token_manager: SessionTokenManager,
|
||||
auth: Authenticator
|
||||
) -> None:
|
||||
"""Handle POST /api/v1/proxy requests."""
|
||||
# 1. Extract Bearer token from Authorization header
|
||||
# 2. Validate session token
|
||||
# 3. Parse request body (method, table, etc.)
|
||||
# 4. Check permissions for requested method
|
||||
# 5. Build GristClient for the token's document
|
||||
# 6. Dispatch to appropriate tool function
|
||||
# 7. Return JSON response
|
||||
```
|
||||
|
||||
### Modified Files
|
||||
|
||||
**`src/grist_mcp/main.py`**:
|
||||
- Import `SessionTokenManager` and `handle_proxy`
|
||||
- Instantiate `SessionTokenManager` in `create_app()`
|
||||
- Add route: `elif path == "/api/v1/proxy" and method == "POST"`
|
||||
- Pass `token_manager` to `create_server()`
|
||||
|
||||
**`src/grist_mcp/server.py`**:
|
||||
- Accept `token_manager` parameter in `create_server()`
|
||||
- Add `get_proxy_documentation` tool to `list_tools()` (no parameters, returns static docs)
|
||||
- Add `request_session_token` tool to `list_tools()`
|
||||
- Add handlers in `call_tool()` for both tools
|
||||
|
||||
## Security
|
||||
|
||||
1. **No privilege escalation** - Session token can only grant permissions the agent already has for the document. Validated at token creation.
|
||||
|
||||
2. **Short-lived by default** - 5 minute default TTL, 1 hour maximum cap.
|
||||
|
||||
3. **Token format** - Prefixed with `sess_` to distinguish from agent tokens. Generated with `secrets.token_urlsafe(32)`.
|
||||
|
||||
4. **Lazy cleanup** - Expired tokens removed during validation. No background task needed.
|
||||
|
||||
5. **Audit logging** - Token creation and proxy requests logged with agent name, document, method.
|
||||
|
||||
## Testing
|
||||
|
||||
### Unit Tests
|
||||
|
||||
**`tests/unit/test_session.py`**:
|
||||
- Token creation with valid permissions
|
||||
- Token creation fails when exceeding agent permissions
|
||||
- Token validation succeeds for valid token
|
||||
- Token validation fails for expired token
|
||||
- Token validation fails for unknown token
|
||||
- TTL capping at maximum
|
||||
- Cleanup removes expired tokens
|
||||
|
||||
**`tests/unit/test_proxy.py`**:
|
||||
- Request parsing for each method type
|
||||
- Error response for invalid token
|
||||
- Error response for expired token
|
||||
- Error response for permission denied
|
||||
- Error response for malformed request
|
||||
- Successful dispatch to each tool function (mocked)
|
||||
|
||||
### Integration Tests
|
||||
|
||||
**`tests/integration/test_session_proxy.py`**:
|
||||
- Full flow: MCP token request → HTTP proxy call → Grist operation
|
||||
- Verify data actually written to Grist
|
||||
- Verify token expiry prevents access
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "grist-mcp"
|
||||
version = "1.0.0"
|
||||
version = "1.1.0"
|
||||
description = "MCP server for AI agents to interact with Grist documents"
|
||||
requires-python = ">=3.14"
|
||||
dependencies = [
|
||||
|
||||
3
renovate.json
Normal file
3
renovate.json
Normal file
@@ -0,0 +1,3 @@
|
||||
{
|
||||
"$schema": "https://docs.renovatebot.com/renovate-schema.json"
|
||||
}
|
||||
@@ -14,6 +14,7 @@ class Document:
|
||||
url: str
|
||||
doc_id: str
|
||||
api_key: str
|
||||
host_header: str | None = None # Override Host header for Docker networking
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -78,6 +79,7 @@ def load_config(config_path: str) -> Config:
|
||||
url=doc_data["url"],
|
||||
doc_id=doc_data["doc_id"],
|
||||
api_key=doc_data["api_key"],
|
||||
host_header=doc_data.get("host_header"),
|
||||
)
|
||||
|
||||
# Parse tokens
|
||||
|
||||
@@ -17,6 +17,8 @@ class GristClient:
|
||||
self._doc = document
|
||||
self._base_url = f"{document.url.rstrip('/')}/api/docs/{document.doc_id}"
|
||||
self._headers = {"Authorization": f"Bearer {document.api_key}"}
|
||||
if document.host_header:
|
||||
self._headers["Host"] = document.host_header
|
||||
self._timeout = timeout
|
||||
|
||||
async def _request(self, method: str, path: str, **kwargs) -> dict:
|
||||
|
||||
119
src/grist_mcp/logging.py
Normal file
119
src/grist_mcp/logging.py
Normal file
@@ -0,0 +1,119 @@
|
||||
"""Logging configuration and utilities."""
|
||||
|
||||
import logging
|
||||
import os
|
||||
from datetime import datetime
|
||||
|
||||
__all__ = [
|
||||
"setup_logging",
|
||||
"get_logger",
|
||||
"truncate_token",
|
||||
"extract_stats",
|
||||
"format_tool_log",
|
||||
]
|
||||
|
||||
|
||||
def setup_logging() -> None:
|
||||
"""Configure logging based on LOG_LEVEL environment variable.
|
||||
|
||||
Valid levels: DEBUG, INFO, WARNING, ERROR (default: INFO)
|
||||
"""
|
||||
level_name = os.environ.get("LOG_LEVEL", "INFO").upper()
|
||||
level = getattr(logging, level_name, None)
|
||||
|
||||
if not isinstance(level, int):
|
||||
level = logging.INFO
|
||||
|
||||
logger = logging.getLogger("grist_mcp")
|
||||
logger.setLevel(level)
|
||||
|
||||
# Only add handler if not already configured
|
||||
if not logger.handlers:
|
||||
handler = logging.StreamHandler()
|
||||
handler.setFormatter(logging.Formatter("%(message)s"))
|
||||
logger.addHandler(handler)
|
||||
|
||||
|
||||
def extract_stats(tool_name: str, arguments: dict, result: dict) -> str:
|
||||
"""Extract meaningful stats from tool call based on tool type."""
|
||||
if tool_name == "list_documents":
|
||||
count = len(result.get("documents", []))
|
||||
return f"{count} docs"
|
||||
|
||||
if tool_name == "list_tables":
|
||||
count = len(result.get("tables", []))
|
||||
return f"{count} tables"
|
||||
|
||||
if tool_name == "describe_table":
|
||||
count = len(result.get("columns", []))
|
||||
return f"{count} columns"
|
||||
|
||||
if tool_name == "get_records":
|
||||
count = len(result.get("records", []))
|
||||
return f"{count} records"
|
||||
|
||||
if tool_name == "sql_query":
|
||||
count = len(result.get("records", []))
|
||||
return f"{count} rows"
|
||||
|
||||
if tool_name == "add_records":
|
||||
count = len(arguments.get("records", []))
|
||||
return f"{count} records"
|
||||
|
||||
if tool_name == "update_records":
|
||||
count = len(arguments.get("records", []))
|
||||
return f"{count} records"
|
||||
|
||||
if tool_name == "delete_records":
|
||||
count = len(arguments.get("record_ids", []))
|
||||
return f"{count} records"
|
||||
|
||||
if tool_name == "create_table":
|
||||
count = len(arguments.get("columns", []))
|
||||
return f"{count} columns"
|
||||
|
||||
if tool_name in ("add_column", "modify_column", "delete_column"):
|
||||
return "1 column"
|
||||
|
||||
return "-"
|
||||
|
||||
|
||||
def truncate_token(token: str) -> str:
|
||||
"""Truncate token to show first 3 and last 3 chars.
|
||||
|
||||
Tokens 8 chars or shorter show *** for security.
|
||||
"""
|
||||
if len(token) <= 8:
|
||||
return "***"
|
||||
return f"{token[:3]}...{token[-3:]}"
|
||||
|
||||
|
||||
def format_tool_log(
|
||||
agent_name: str,
|
||||
token: str,
|
||||
tool: str,
|
||||
document: str | None,
|
||||
stats: str,
|
||||
status: str,
|
||||
duration_ms: int,
|
||||
error_message: str | None = None,
|
||||
) -> str:
|
||||
"""Format a tool call log line.
|
||||
|
||||
Format: YYYY-MM-DD HH:MM:SS | agent (token) | tool | doc | stats | status | duration
|
||||
"""
|
||||
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
|
||||
truncated = truncate_token(token)
|
||||
doc = document if document else "-"
|
||||
|
||||
line = f"{timestamp} | {agent_name} ({truncated}) | {tool} | {doc} | {stats} | {status} | {duration_ms}ms"
|
||||
|
||||
if error_message:
|
||||
line += f"\n {error_message}"
|
||||
|
||||
return line
|
||||
|
||||
|
||||
def get_logger(name: str) -> logging.Logger:
|
||||
"""Get a child logger under the grist_mcp namespace."""
|
||||
return logging.getLogger(f"grist_mcp.{name}")
|
||||
@@ -1,6 +1,7 @@
|
||||
"""Main entry point for the MCP server with SSE transport."""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from typing import Any
|
||||
@@ -11,6 +12,7 @@ from mcp.server.sse import SseServerTransport
|
||||
from grist_mcp.server import create_server
|
||||
from grist_mcp.config import Config, load_config
|
||||
from grist_mcp.auth import Authenticator, AuthError
|
||||
from grist_mcp.logging import setup_logging
|
||||
|
||||
|
||||
Scope = dict[str, Any]
|
||||
@@ -74,19 +76,34 @@ def _ensure_config(config_path: str) -> bool:
|
||||
|
||||
# Check if path is a directory (Docker creates this when mounting missing file)
|
||||
if os.path.isdir(path):
|
||||
os.rmdir(path)
|
||||
print(f"ERROR: Config path is a directory: {path}")
|
||||
print()
|
||||
print("This usually means the config file doesn't exist on the host.")
|
||||
print("Please create the config file before starting the container:")
|
||||
print()
|
||||
print(f" mkdir -p $(dirname {config_path})")
|
||||
print(f" cat > {config_path} << 'EOF'")
|
||||
print(CONFIG_TEMPLATE)
|
||||
print("EOF")
|
||||
print()
|
||||
return False
|
||||
|
||||
if os.path.exists(path):
|
||||
return True
|
||||
|
||||
# Create template config
|
||||
with open(path, "w") as f:
|
||||
f.write(CONFIG_TEMPLATE)
|
||||
|
||||
print(f"Created template configuration at: {path}")
|
||||
print()
|
||||
print("Please edit this file to configure your Grist documents and agent tokens,")
|
||||
print("then restart the server.")
|
||||
try:
|
||||
with open(path, "w") as f:
|
||||
f.write(CONFIG_TEMPLATE)
|
||||
print(f"Created template configuration at: {path}")
|
||||
print()
|
||||
print("Please edit this file to configure your Grist documents and agent tokens,")
|
||||
print("then restart the server.")
|
||||
except PermissionError:
|
||||
print(f"ERROR: Cannot create config file at: {path}")
|
||||
print()
|
||||
print("Please create the config file manually before starting the container.")
|
||||
print()
|
||||
return False
|
||||
|
||||
|
||||
@@ -174,12 +191,27 @@ def _print_mcp_config(external_port: int, tokens: list) -> None:
|
||||
print()
|
||||
|
||||
|
||||
class HealthCheckFilter(logging.Filter):
|
||||
"""Suppress health check requests unless LOG_LEVEL is DEBUG."""
|
||||
|
||||
def filter(self, record: logging.LogRecord) -> bool:
|
||||
if "/health" in record.getMessage():
|
||||
# Only show health checks at DEBUG level
|
||||
return os.environ.get("LOG_LEVEL", "INFO").upper() == "DEBUG"
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
"""Run the SSE server."""
|
||||
port = int(os.environ.get("PORT", "3000"))
|
||||
external_port = int(os.environ.get("EXTERNAL_PORT", str(port)))
|
||||
config_path = os.environ.get("CONFIG_PATH", "/app/config.yaml")
|
||||
|
||||
setup_logging()
|
||||
|
||||
# Add health check filter to uvicorn access logger
|
||||
logging.getLogger("uvicorn.access").addFilter(HealthCheckFilter())
|
||||
|
||||
if not _ensure_config(config_path):
|
||||
return
|
||||
|
||||
@@ -192,7 +224,13 @@ def main():
|
||||
_print_mcp_config(external_port, config.tokens)
|
||||
|
||||
app = create_app(config)
|
||||
uvicorn.run(app, host="0.0.0.0", port=port)
|
||||
|
||||
# Configure uvicorn logging to reduce health check noise
|
||||
log_config = uvicorn.config.LOGGING_CONFIG
|
||||
log_config["formatters"]["default"]["fmt"] = "%(message)s"
|
||||
log_config["formatters"]["access"]["fmt"] = "%(message)s"
|
||||
|
||||
uvicorn.run(app, host="0.0.0.0", port=port, log_config=log_config)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -1,11 +1,15 @@
|
||||
"""MCP server setup and tool registration."""
|
||||
|
||||
import json
|
||||
import time
|
||||
|
||||
from mcp.server import Server
|
||||
from mcp.types import Tool, TextContent
|
||||
|
||||
from grist_mcp.auth import Authenticator, Agent, AuthError
|
||||
from grist_mcp.logging import get_logger, extract_stats, format_tool_log
|
||||
|
||||
logger = get_logger("server")
|
||||
|
||||
from grist_mcp.tools.discovery import list_documents as _list_documents
|
||||
from grist_mcp.tools.read import list_tables as _list_tables
|
||||
@@ -207,6 +211,22 @@ def create_server(auth: Authenticator, agent: Agent) -> Server:
|
||||
|
||||
@server.call_tool()
|
||||
async def call_tool(name: str, arguments: dict) -> list[TextContent]:
|
||||
start_time = time.time()
|
||||
document = arguments.get("document")
|
||||
|
||||
# Log arguments at DEBUG level
|
||||
logger.debug(
|
||||
format_tool_log(
|
||||
agent_name=_current_agent.name,
|
||||
token=_current_agent.token,
|
||||
tool=name,
|
||||
document=document,
|
||||
stats=f"args: {json.dumps(arguments)}",
|
||||
status="started",
|
||||
duration_ms=0,
|
||||
)
|
||||
)
|
||||
|
||||
try:
|
||||
if name == "list_documents":
|
||||
result = await _list_documents(_current_agent)
|
||||
@@ -268,11 +288,53 @@ def create_server(auth: Authenticator, agent: Agent) -> Server:
|
||||
else:
|
||||
return [TextContent(type="text", text=f"Unknown tool: {name}")]
|
||||
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
stats = extract_stats(name, arguments, result)
|
||||
|
||||
logger.info(
|
||||
format_tool_log(
|
||||
agent_name=_current_agent.name,
|
||||
token=_current_agent.token,
|
||||
tool=name,
|
||||
document=document,
|
||||
stats=stats,
|
||||
status="success",
|
||||
duration_ms=duration_ms,
|
||||
)
|
||||
)
|
||||
|
||||
return [TextContent(type="text", text=json.dumps(result))]
|
||||
|
||||
except AuthError as e:
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.warning(
|
||||
format_tool_log(
|
||||
agent_name=_current_agent.name,
|
||||
token=_current_agent.token,
|
||||
tool=name,
|
||||
document=document,
|
||||
stats="-",
|
||||
status="auth_error",
|
||||
duration_ms=duration_ms,
|
||||
error_message=str(e),
|
||||
)
|
||||
)
|
||||
return [TextContent(type="text", text=f"Authorization error: {e}")]
|
||||
|
||||
except Exception as e:
|
||||
duration_ms = int((time.time() - start_time) * 1000)
|
||||
logger.error(
|
||||
format_tool_log(
|
||||
agent_name=_current_agent.name,
|
||||
token=_current_agent.token,
|
||||
tool=name,
|
||||
document=document,
|
||||
stats="-",
|
||||
status="error",
|
||||
duration_ms=duration_ms,
|
||||
error_message=str(e),
|
||||
)
|
||||
)
|
||||
return [TextContent(type="text", text=f"Error: {e}")]
|
||||
|
||||
return server
|
||||
|
||||
170
tests/unit/test_logging.py
Normal file
170
tests/unit/test_logging.py
Normal file
@@ -0,0 +1,170 @@
|
||||
"""Unit tests for logging module."""
|
||||
|
||||
import logging
|
||||
|
||||
from grist_mcp.logging import truncate_token, extract_stats, format_tool_log
|
||||
|
||||
|
||||
class TestTruncateToken:
|
||||
def test_normal_token_shows_prefix_suffix(self):
|
||||
token = "abcdefghijklmnop"
|
||||
assert truncate_token(token) == "abc...nop"
|
||||
|
||||
def test_short_token_shows_asterisks(self):
|
||||
token = "abcdefgh" # 8 chars
|
||||
assert truncate_token(token) == "***"
|
||||
|
||||
def test_very_short_token_shows_asterisks(self):
|
||||
token = "abc"
|
||||
assert truncate_token(token) == "***"
|
||||
|
||||
def test_empty_token_shows_asterisks(self):
|
||||
assert truncate_token("") == "***"
|
||||
|
||||
def test_boundary_token_shows_prefix_suffix(self):
|
||||
token = "abcdefghi" # 9 chars - first to show truncation
|
||||
assert truncate_token(token) == "abc...ghi"
|
||||
|
||||
|
||||
class TestExtractStats:
|
||||
def test_list_documents(self):
|
||||
result = {"documents": [{"name": "a"}, {"name": "b"}, {"name": "c"}]}
|
||||
assert extract_stats("list_documents", {}, result) == "3 docs"
|
||||
|
||||
def test_list_tables(self):
|
||||
result = {"tables": ["Orders", "Products"]}
|
||||
assert extract_stats("list_tables", {}, result) == "2 tables"
|
||||
|
||||
def test_describe_table(self):
|
||||
result = {"columns": [{"id": "A"}, {"id": "B"}]}
|
||||
assert extract_stats("describe_table", {}, result) == "2 columns"
|
||||
|
||||
def test_get_records(self):
|
||||
result = {"records": [{"id": 1}, {"id": 2}]}
|
||||
assert extract_stats("get_records", {}, result) == "2 records"
|
||||
|
||||
def test_sql_query(self):
|
||||
result = {"records": [{"a": 1}, {"a": 2}, {"a": 3}]}
|
||||
assert extract_stats("sql_query", {}, result) == "3 rows"
|
||||
|
||||
def test_add_records_from_args(self):
|
||||
args = {"records": [{"a": 1}, {"a": 2}]}
|
||||
assert extract_stats("add_records", args, {"ids": [1, 2]}) == "2 records"
|
||||
|
||||
def test_update_records_from_args(self):
|
||||
args = {"records": [{"id": 1, "fields": {}}, {"id": 2, "fields": {}}]}
|
||||
assert extract_stats("update_records", args, {}) == "2 records"
|
||||
|
||||
def test_delete_records_from_args(self):
|
||||
args = {"record_ids": [1, 2, 3]}
|
||||
assert extract_stats("delete_records", args, {}) == "3 records"
|
||||
|
||||
def test_create_table(self):
|
||||
args = {"columns": [{"id": "A"}, {"id": "B"}]}
|
||||
assert extract_stats("create_table", args, {}) == "2 columns"
|
||||
|
||||
def test_single_column_ops(self):
|
||||
assert extract_stats("add_column", {}, {}) == "1 column"
|
||||
assert extract_stats("modify_column", {}, {}) == "1 column"
|
||||
assert extract_stats("delete_column", {}, {}) == "1 column"
|
||||
|
||||
def test_empty_result_returns_zero(self):
|
||||
assert extract_stats("list_documents", {}, {"documents": []}) == "0 docs"
|
||||
|
||||
def test_unknown_tool(self):
|
||||
assert extract_stats("unknown_tool", {}, {}) == "-"
|
||||
|
||||
|
||||
class TestFormatToolLog:
|
||||
def test_success_format(self):
|
||||
line = format_tool_log(
|
||||
agent_name="dev-agent",
|
||||
token="abcdefghijklmnop",
|
||||
tool="get_records",
|
||||
document="sales",
|
||||
stats="42 records",
|
||||
status="success",
|
||||
duration_ms=125,
|
||||
)
|
||||
assert "dev-agent" in line
|
||||
assert "abc...nop" in line
|
||||
assert "get_records" in line
|
||||
assert "sales" in line
|
||||
assert "42 records" in line
|
||||
assert "success" in line
|
||||
assert "125ms" in line
|
||||
# Check pipe-delimited format
|
||||
assert line.count("|") == 6
|
||||
|
||||
def test_no_document(self):
|
||||
line = format_tool_log(
|
||||
agent_name="dev-agent",
|
||||
token="abcdefghijklmnop",
|
||||
tool="list_documents",
|
||||
document=None,
|
||||
stats="3 docs",
|
||||
status="success",
|
||||
duration_ms=45,
|
||||
)
|
||||
assert "| - |" in line
|
||||
|
||||
def test_error_format(self):
|
||||
line = format_tool_log(
|
||||
agent_name="dev-agent",
|
||||
token="abcdefghijklmnop",
|
||||
tool="add_records",
|
||||
document="inventory",
|
||||
stats="5 records",
|
||||
status="error",
|
||||
duration_ms=89,
|
||||
error_message="Grist API error: Invalid column 'foo'",
|
||||
)
|
||||
assert "error" in line
|
||||
assert "\n Grist API error: Invalid column 'foo'" in line
|
||||
|
||||
|
||||
class TestSetupLogging:
|
||||
def test_default_level_is_info(self, monkeypatch):
|
||||
monkeypatch.delenv("LOG_LEVEL", raising=False)
|
||||
|
||||
from grist_mcp.logging import setup_logging
|
||||
setup_logging()
|
||||
|
||||
logger = logging.getLogger("grist_mcp")
|
||||
assert logger.level == logging.INFO
|
||||
|
||||
def test_respects_log_level_env(self, monkeypatch):
|
||||
monkeypatch.setenv("LOG_LEVEL", "DEBUG")
|
||||
|
||||
from grist_mcp.logging import setup_logging
|
||||
setup_logging()
|
||||
|
||||
logger = logging.getLogger("grist_mcp")
|
||||
assert logger.level == logging.DEBUG
|
||||
|
||||
def test_invalid_level_defaults_to_info(self, monkeypatch):
|
||||
monkeypatch.setenv("LOG_LEVEL", "INVALID")
|
||||
|
||||
from grist_mcp.logging import setup_logging
|
||||
setup_logging()
|
||||
|
||||
logger = logging.getLogger("grist_mcp")
|
||||
assert logger.level == logging.INFO
|
||||
|
||||
|
||||
class TestGetLogger:
|
||||
def test_returns_child_logger(self):
|
||||
from grist_mcp.logging import get_logger
|
||||
|
||||
logger = get_logger("server")
|
||||
assert logger.name == "grist_mcp.server"
|
||||
|
||||
def test_inherits_parent_level(self, monkeypatch):
|
||||
monkeypatch.setenv("LOG_LEVEL", "WARNING")
|
||||
|
||||
from grist_mcp.logging import setup_logging, get_logger
|
||||
setup_logging()
|
||||
|
||||
logger = get_logger("test")
|
||||
# Child inherits from parent when level is NOTSET
|
||||
assert logger.getEffectiveLevel() == logging.WARNING
|
||||
Reference in New Issue
Block a user