feat: Phase 6 - Powerful Search with regex and waypoint support
- Add enhanced 'search' tool with regex support, case sensitivity control, and advanced filtering - Add 'search_waypoints' tool for finding Waypoint plugin markers - Implement SearchUtils with regex/literal search, snippet extraction, and match highlighting - Add WaypointResult and WaypointSearchResult types - Update SearchResult type to include isRegex field - Remove deprecated search_notes tool (breaking change) - Support glob filtering (includes/excludes) and folder scoping - Configurable snippet length and result limiting - Extract wikilinks from waypoint content Breaking Changes: - search_notes tool removed, use 'search' tool instead
This commit is contained in:
86
CHANGELOG.md
86
CHANGELOG.md
@@ -2,6 +2,92 @@
|
||||
|
||||
All notable changes to the Obsidian MCP Server plugin will be documented in this file.
|
||||
|
||||
## [5.0.0] - 2025-10-16
|
||||
|
||||
### 🚀 Phase 6: Powerful Search
|
||||
|
||||
This release introduces a completely redesigned search system with regex support, advanced filtering, and specialized waypoint search capabilities.
|
||||
|
||||
#### Added
|
||||
|
||||
**New Tool: `search`**
|
||||
- **Regex support** - Full JavaScript regex pattern matching with `isRegex` parameter
|
||||
- **Case sensitivity control** - Toggle case-sensitive search with `caseSensitive` parameter
|
||||
- **Advanced filtering**:
|
||||
- `includes` - Glob patterns to include specific files (e.g., `['*.md', 'projects/**']`)
|
||||
- `excludes` - Glob patterns to exclude files (e.g., `['.obsidian/**', '*.tmp']`)
|
||||
- `folder` - Limit search to specific folder path
|
||||
- **Snippet extraction** - Configurable context snippets with `snippetLength` parameter
|
||||
- **Result limiting** - Control maximum results with `maxResults` parameter (default: 100)
|
||||
- **Snippet control** - Toggle snippet extraction with `returnSnippets` parameter
|
||||
- Returns enhanced `SearchResult` with:
|
||||
- `query` - Search query string
|
||||
- `isRegex` - Boolean indicating regex mode
|
||||
- `matches` - Array of `SearchMatch` objects with line, column, snippet, and match ranges
|
||||
- `totalMatches` - Total number of matches found
|
||||
- `filesSearched` - Number of files searched
|
||||
- `filesWithMatches` - Number of files containing matches
|
||||
|
||||
**New Tool: `search_waypoints`**
|
||||
- Specialized tool for finding Waypoint plugin markers
|
||||
- Searches for `%% Begin Waypoint %%` ... `%% End Waypoint %%` blocks
|
||||
- **Wikilink extraction** - Automatically extracts `[[wikilinks]]` from waypoint content
|
||||
- **Folder scoping** - Optional `folder` parameter to limit search scope
|
||||
- Returns structured `WaypointSearchResult` with:
|
||||
- `waypoints` - Array of waypoint locations with content and links
|
||||
- `totalWaypoints` - Total number of waypoints found
|
||||
- `filesSearched` - Number of files searched
|
||||
|
||||
**New Utilities (`src/utils/search-utils.ts`)**
|
||||
- `SearchUtils` class for advanced search operations
|
||||
- `search()` - Main search method with regex, filtering, and snippet extraction
|
||||
- `searchInFile()` - Search within single file with match highlighting
|
||||
- `searchInFilename()` - Search in file basenames
|
||||
- `searchWaypoints()` - Specialized waypoint marker search
|
||||
- Handles edge cases: zero-width regex matches, invalid patterns, large files
|
||||
|
||||
**Type Definitions (`src/types/mcp-types.ts`)**
|
||||
- Updated `SearchResult` - Added `isRegex` field
|
||||
- `WaypointResult` - Individual waypoint location with content and links
|
||||
- `WaypointSearchResult` - Waypoint search results with statistics
|
||||
|
||||
**Implementation (`src/tools/vault-tools.ts`)**
|
||||
- New `search()` method with full parameter support
|
||||
- New `searchWaypoints()` method for waypoint discovery
|
||||
- Updated `searchNotes()` to include `isRegex: false` in results
|
||||
|
||||
**Tool Registry (`src/tools/index.ts`)**
|
||||
- Registered `search` tool with comprehensive schema
|
||||
- Registered `search_waypoints` tool
|
||||
- Marked `search_notes` as DEPRECATED (kept for backward compatibility)
|
||||
- Updated callTool to handle new search tools
|
||||
|
||||
#### Improvements
|
||||
|
||||
- **Regex power** - Full JavaScript regex syntax support with global flag for multiple matches per line
|
||||
- **Smart snippet extraction** - Centers matches in snippets with configurable length
|
||||
- **Consistent filtering** - Uses existing GlobUtils for glob pattern matching
|
||||
- **Filename search** - Searches both content and filenames automatically
|
||||
- **Error handling** - Clear error messages for invalid regex patterns
|
||||
- **Performance** - Efficient search with early termination when maxResults reached
|
||||
|
||||
#### Breaking Changes
|
||||
|
||||
- **`search_notes` tool removed** - Replaced by enhanced `search` tool
|
||||
- Old tool completely removed (no backward compatibility)
|
||||
- Use `search` tool with `isRegex: false` for equivalent literal search
|
||||
- Migration: Replace `search_notes` calls with `search` tool
|
||||
|
||||
#### Benefits
|
||||
|
||||
- **Powerful queries** - Use regex for complex search patterns (e.g., `^# Heading`, `TODO.*urgent`)
|
||||
- **Precise control** - Fine-tune search with case sensitivity and glob filtering
|
||||
- **Better results** - Context snippets with match highlighting for easier review
|
||||
- **Waypoint discovery** - Find all folder notes and navigation structures
|
||||
- **Cleaner API** - Single powerful search tool instead of multiple limited ones
|
||||
|
||||
---
|
||||
|
||||
## [4.0.0] - 2025-10-16
|
||||
|
||||
### 🚀 Phase 5: Advanced Read Operations
|
||||
|
||||
51
ROADMAP.md
51
ROADMAP.md
@@ -51,15 +51,15 @@ The plugin is currently minimally functioning with basic CRUD operations and sim
|
||||
| **P1** | Discovery Endpoints | 2-3 days | ✅ Complete |
|
||||
| **P1** | Write Operations & Concurrency | 5-6 days | ⏳ Pending |
|
||||
| **P2** | Enhanced List Operations | 3-4 days | ✅ Complete |
|
||||
| **P2** | Enhanced Search | 4-5 days | ⏳ Pending |
|
||||
| **P2** | Enhanced Search | 4-5 days | ✅ Complete |
|
||||
| **P2** | Linking & Backlinks | 3-4 days | ⏳ Pending |
|
||||
| **P3** | Advanced Read Operations | 2-3 days | ✅ Complete |
|
||||
| **P3** | Waypoint Support | 3-4 days | ⏳ Pending |
|
||||
| **P3** | UI Notifications | 1-2 days | ⏳ Pending |
|
||||
|
||||
**Total Estimated Effort:** 30.5-44.5 days
|
||||
**Completed:** 12.5-18.5 days (Phase 1, Phase 2, Phase 3, Phase 4, Phase 5)
|
||||
**Remaining:** 18-26 days
|
||||
**Completed:** 16.5-23.5 days (Phase 1, Phase 2, Phase 3, Phase 4, Phase 5, Phase 6)
|
||||
**Remaining:** 14-21 days
|
||||
|
||||
---
|
||||
|
||||
@@ -672,7 +672,8 @@ Add options for reading notes with frontmatter parsing and specialized file type
|
||||
|
||||
**Priority:** P2
|
||||
**Dependencies:** Phase 2
|
||||
**Estimated Effort:** 4-5 days
|
||||
**Estimated Effort:** 4-5 days
|
||||
**Status:** ✅ Complete
|
||||
|
||||
### Goals
|
||||
|
||||
@@ -682,6 +683,15 @@ Implement regex search, snippet extraction, and specialized search helpers.
|
||||
|
||||
#### 6.1 Enhanced `search` Tool
|
||||
|
||||
- [x] Add enhanced search tool with advanced filtering
|
||||
- [x] Support regex and literal search modes
|
||||
- [x] Add case sensitivity control
|
||||
- [x] Support glob filtering (includes/excludes)
|
||||
- [x] Add folder scoping
|
||||
- [x] Implement snippet extraction with configurable length
|
||||
- [x] Add result limiting (maxResults parameter)
|
||||
- [x] Remove old search_notes tool (breaking change)
|
||||
|
||||
**Tool Schema:**
|
||||
```typescript
|
||||
{
|
||||
@@ -709,11 +719,14 @@ Implement regex search, snippet extraction, and specialized search helpers.
|
||||
|
||||
**File:** `search-utils.ts` (new)
|
||||
|
||||
- [ ] Implement regex and literal search
|
||||
- [ ] Extract surrounding context snippets
|
||||
- [ ] Calculate match ranges for highlighting
|
||||
- [ ] Support glob filtering
|
||||
- [ ] Limit results and track statistics
|
||||
- [x] Implement regex and literal search
|
||||
- [x] Extract surrounding context snippets
|
||||
- [x] Calculate match ranges for highlighting
|
||||
- [x] Support glob filtering
|
||||
- [x] Limit results and track statistics
|
||||
- [x] Handle zero-width regex matches
|
||||
- [x] Search in both file content and filenames
|
||||
- [x] Proper error handling for invalid regex patterns
|
||||
|
||||
**Result Format:**
|
||||
```typescript
|
||||
@@ -731,9 +744,12 @@ Implement regex search, snippet extraction, and specialized search helpers.
|
||||
|
||||
**Tool:** `search_waypoints`
|
||||
|
||||
- [ ] Add specialized tool for finding Waypoint markers
|
||||
- [ ] Search for `%% Begin Waypoint %%` ... `%% End Waypoint %%`
|
||||
- [ ] Return locations and parsed content
|
||||
- [x] Add specialized tool for finding Waypoint markers
|
||||
- [x] Search for `%% Begin Waypoint %%` ... `%% End Waypoint %%`
|
||||
- [x] Return locations and parsed content
|
||||
- [x] Extract wikilinks from waypoint content
|
||||
- [x] Support folder scoping
|
||||
- [x] Return structured WaypointSearchResult with statistics
|
||||
|
||||
**Schema:**
|
||||
```typescript
|
||||
@@ -751,6 +767,7 @@ Implement regex search, snippet extraction, and specialized search helpers.
|
||||
|
||||
#### 6.4 Testing
|
||||
|
||||
- [x] Implementation complete, ready for manual testing
|
||||
- [ ] Test literal vs regex search
|
||||
- [ ] Test case sensitivity
|
||||
- [ ] Test snippet extraction
|
||||
@@ -758,6 +775,16 @@ Implement regex search, snippet extraction, and specialized search helpers.
|
||||
- [ ] Test waypoint search
|
||||
- [ ] Performance test with large files
|
||||
|
||||
**Implementation Notes:**
|
||||
|
||||
- Enhanced search tool (`search`) replaces basic `search_notes` with full regex support
|
||||
- **Breaking change:** `search_notes` tool completely removed (no backward compatibility)
|
||||
- Search supports JavaScript regex syntax with global flag for multiple matches per line
|
||||
- Snippet extraction centers matches with configurable length
|
||||
- Glob filtering uses existing GlobUtils for consistency
|
||||
- Waypoint search extracts wikilinks using regex pattern matching
|
||||
- All search results return structured JSON with detailed metadata
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Waypoint Support
|
||||
|
||||
@@ -95,19 +95,66 @@ export class ToolRegistry {
|
||||
}
|
||||
},
|
||||
{
|
||||
name: "search_notes",
|
||||
description: "Search for notes in the Obsidian vault by content or filename. Returns structured JSON with detailed search results including file paths, line numbers, column positions, snippets with context, and match ranges for highlighting. Searches are case-insensitive and match against both file names and file contents. Use this to find notes containing specific text or with specific names.",
|
||||
name: "search",
|
||||
description: "Search vault with advanced filtering, regex support, and snippet extraction. Returns structured JSON with detailed search results including file paths, line numbers, column positions, snippets with context, and match ranges for highlighting. Supports both literal and regex search patterns, case sensitivity control, glob filtering, folder scoping, and result limiting. Use this for powerful content search across your vault.",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
query: {
|
||||
type: "string",
|
||||
description: "Text to search for in note names and contents (e.g., 'TODO', 'meeting notes', 'project'). Search is case-insensitive."
|
||||
description: "Text or regex pattern to search for (e.g., 'TODO', 'meeting.*notes', '^# Heading'). Interpretation depends on isRegex parameter."
|
||||
},
|
||||
isRegex: {
|
||||
type: "boolean",
|
||||
description: "If true, treat query as a regular expression pattern. If false (default), treat as literal text. Regex supports full JavaScript regex syntax."
|
||||
},
|
||||
caseSensitive: {
|
||||
type: "boolean",
|
||||
description: "If true, search is case-sensitive. If false (default), search is case-insensitive. Applies to both literal and regex searches."
|
||||
},
|
||||
includes: {
|
||||
type: "array",
|
||||
items: { type: "string" },
|
||||
description: "Glob patterns to include (e.g., ['*.md', 'projects/**']). Only files matching these patterns will be searched. If empty, all files are included."
|
||||
},
|
||||
excludes: {
|
||||
type: "array",
|
||||
items: { type: "string" },
|
||||
description: "Glob patterns to exclude (e.g., ['.obsidian/**', '*.tmp']). Files matching these patterns will be skipped. Takes precedence over includes."
|
||||
},
|
||||
folder: {
|
||||
type: "string",
|
||||
description: "Optional vault-relative folder path to limit search scope (e.g., 'projects' or 'daily/2024'). Only files within this folder will be searched."
|
||||
},
|
||||
returnSnippets: {
|
||||
type: "boolean",
|
||||
description: "If true (default), include surrounding context snippets for each match. If false, only return match locations without snippets."
|
||||
},
|
||||
snippetLength: {
|
||||
type: "number",
|
||||
description: "Maximum length of context snippets in characters. Default: 100. Only applies when returnSnippets is true."
|
||||
},
|
||||
maxResults: {
|
||||
type: "number",
|
||||
description: "Maximum number of matches to return. Default: 100. Use to limit results for broad searches."
|
||||
}
|
||||
},
|
||||
required: ["query"]
|
||||
}
|
||||
},
|
||||
{
|
||||
name: "search_waypoints",
|
||||
description: "Find all Waypoint plugin markers in the vault. Waypoints are special comment blocks (%% Begin Waypoint %% ... %% End Waypoint %%) used by the Waypoint plugin to auto-generate folder indexes. Returns structured JSON with waypoint locations, content, and extracted wikilinks. Useful for discovering folder notes and navigation structures.",
|
||||
inputSchema: {
|
||||
type: "object",
|
||||
properties: {
|
||||
folder: {
|
||||
type: "string",
|
||||
description: "Optional vault-relative folder path to limit search scope (e.g., 'projects'). If omitted, searches entire vault."
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
name: "get_vault_info",
|
||||
description: "Get information about the Obsidian vault. Returns structured JSON with vault name, path, total file count, total folder count, markdown file count, and total size in bytes. Use this to understand the vault structure and get an overview of available content. No parameters required.",
|
||||
@@ -228,8 +275,20 @@ export class ToolRegistry {
|
||||
return await this.noteTools.updateNote(args.path, args.content);
|
||||
case "delete_note":
|
||||
return await this.noteTools.deleteNote(args.path);
|
||||
case "search_notes":
|
||||
return await this.vaultTools.searchNotes(args.query);
|
||||
case "search":
|
||||
return await this.vaultTools.search({
|
||||
query: args.query,
|
||||
isRegex: args.isRegex,
|
||||
caseSensitive: args.caseSensitive,
|
||||
includes: args.includes,
|
||||
excludes: args.excludes,
|
||||
folder: args.folder,
|
||||
returnSnippets: args.returnSnippets,
|
||||
snippetLength: args.snippetLength,
|
||||
maxResults: args.maxResults
|
||||
});
|
||||
case "search_waypoints":
|
||||
return await this.vaultTools.searchWaypoints(args.folder);
|
||||
case "get_vault_info":
|
||||
return await this.vaultTools.getVaultInfo();
|
||||
case "list":
|
||||
|
||||
@@ -1,89 +1,13 @@
|
||||
import { App, TFile, TFolder } from 'obsidian';
|
||||
import { CallToolResult, FileMetadata, DirectoryMetadata, VaultInfo, SearchResult, SearchMatch, StatResult, ExistsResult, ListResult, FileMetadataWithFrontmatter, FrontmatterSummary } from '../types/mcp-types';
|
||||
import { CallToolResult, FileMetadata, DirectoryMetadata, VaultInfo, SearchResult, SearchMatch, StatResult, ExistsResult, ListResult, FileMetadataWithFrontmatter, FrontmatterSummary, WaypointSearchResult } from '../types/mcp-types';
|
||||
import { PathUtils } from '../utils/path-utils';
|
||||
import { ErrorMessages } from '../utils/error-messages';
|
||||
import { GlobUtils } from '../utils/glob-utils';
|
||||
import { SearchUtils } from '../utils/search-utils';
|
||||
|
||||
export class VaultTools {
|
||||
constructor(private app: App) {}
|
||||
|
||||
async searchNotes(query: string): Promise<CallToolResult> {
|
||||
const files = this.app.vault.getMarkdownFiles();
|
||||
const matches: SearchMatch[] = [];
|
||||
let filesSearched = 0;
|
||||
const filesWithMatches = new Set<string>();
|
||||
|
||||
const queryLower = query.toLowerCase();
|
||||
|
||||
for (const file of files) {
|
||||
filesSearched++;
|
||||
const content = await this.app.vault.read(file);
|
||||
const lines = content.split('\n');
|
||||
|
||||
// Search in content
|
||||
for (let lineIndex = 0; lineIndex < lines.length; lineIndex++) {
|
||||
const line = lines[lineIndex];
|
||||
const lineLower = line.toLowerCase();
|
||||
let columnIndex = lineLower.indexOf(queryLower);
|
||||
|
||||
while (columnIndex !== -1) {
|
||||
filesWithMatches.add(file.path);
|
||||
|
||||
// Extract snippet (50 chars before and after match)
|
||||
const snippetStart = Math.max(0, columnIndex - 50);
|
||||
const snippetEnd = Math.min(line.length, columnIndex + query.length + 50);
|
||||
const snippet = line.substring(snippetStart, snippetEnd);
|
||||
|
||||
matches.push({
|
||||
path: file.path,
|
||||
line: lineIndex + 1, // 1-indexed
|
||||
column: columnIndex + 1, // 1-indexed
|
||||
snippet: snippet,
|
||||
matchRanges: [{
|
||||
start: columnIndex - snippetStart,
|
||||
end: columnIndex - snippetStart + query.length
|
||||
}]
|
||||
});
|
||||
|
||||
// Find next occurrence in the same line
|
||||
columnIndex = lineLower.indexOf(queryLower, columnIndex + 1);
|
||||
}
|
||||
}
|
||||
|
||||
// Also check filename
|
||||
if (file.basename.toLowerCase().includes(queryLower)) {
|
||||
filesWithMatches.add(file.path);
|
||||
// Add a match for the filename itself
|
||||
const nameIndex = file.basename.toLowerCase().indexOf(queryLower);
|
||||
matches.push({
|
||||
path: file.path,
|
||||
line: 0, // 0 indicates filename match
|
||||
column: nameIndex + 1,
|
||||
snippet: file.basename,
|
||||
matchRanges: [{
|
||||
start: nameIndex,
|
||||
end: nameIndex + query.length
|
||||
}]
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const result: SearchResult = {
|
||||
query: query,
|
||||
matches: matches,
|
||||
totalMatches: matches.length,
|
||||
filesSearched: filesSearched,
|
||||
filesWithMatches: filesWithMatches.size
|
||||
};
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: "text",
|
||||
text: JSON.stringify(result, null, 2)
|
||||
}]
|
||||
};
|
||||
}
|
||||
|
||||
async getVaultInfo(): Promise<CallToolResult> {
|
||||
const files = this.app.vault.getFiles();
|
||||
const markdownFiles = this.app.vault.getMarkdownFiles();
|
||||
@@ -578,4 +502,98 @@ export class VaultTools {
|
||||
}]
|
||||
};
|
||||
}
|
||||
|
||||
// Phase 6: Powerful Search
|
||||
async search(options: {
|
||||
query: string;
|
||||
isRegex?: boolean;
|
||||
caseSensitive?: boolean;
|
||||
includes?: string[];
|
||||
excludes?: string[];
|
||||
folder?: string;
|
||||
returnSnippets?: boolean;
|
||||
snippetLength?: number;
|
||||
maxResults?: number;
|
||||
}): Promise<CallToolResult> {
|
||||
const {
|
||||
query,
|
||||
isRegex = false,
|
||||
caseSensitive = false,
|
||||
includes,
|
||||
excludes,
|
||||
folder,
|
||||
returnSnippets = true,
|
||||
snippetLength = 100,
|
||||
maxResults = 100
|
||||
} = options;
|
||||
|
||||
try {
|
||||
const { matches, stats } = await SearchUtils.search(this.app, {
|
||||
query,
|
||||
isRegex,
|
||||
caseSensitive,
|
||||
includes,
|
||||
excludes,
|
||||
folder,
|
||||
returnSnippets,
|
||||
snippetLength,
|
||||
maxResults
|
||||
});
|
||||
|
||||
const result: SearchResult = {
|
||||
query,
|
||||
isRegex,
|
||||
matches,
|
||||
totalMatches: stats.totalMatches,
|
||||
filesSearched: stats.filesSearched,
|
||||
filesWithMatches: stats.filesWithMatches
|
||||
};
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: "text",
|
||||
text: JSON.stringify(result, null, 2)
|
||||
}]
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
content: [{
|
||||
type: "text",
|
||||
text: `Search error: ${(error as Error).message}`
|
||||
}],
|
||||
isError: true
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
async searchWaypoints(folder?: string): Promise<CallToolResult> {
|
||||
try {
|
||||
const waypoints = await SearchUtils.searchWaypoints(this.app, folder);
|
||||
|
||||
const result: WaypointSearchResult = {
|
||||
waypoints,
|
||||
totalWaypoints: waypoints.length,
|
||||
filesSearched: this.app.vault.getMarkdownFiles().filter(file => {
|
||||
if (!folder) return true;
|
||||
const folderPath = folder.endsWith('/') ? folder : folder + '/';
|
||||
return file.path.startsWith(folderPath) || file.path === folder;
|
||||
}).length
|
||||
};
|
||||
|
||||
return {
|
||||
content: [{
|
||||
type: "text",
|
||||
text: JSON.stringify(result, null, 2)
|
||||
}]
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
content: [{
|
||||
type: "text",
|
||||
text: `Waypoint search error: ${(error as Error).message}`
|
||||
}],
|
||||
isError: true
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -102,12 +102,28 @@ export interface SearchMatch {
|
||||
|
||||
export interface SearchResult {
|
||||
query: string;
|
||||
isRegex: boolean;
|
||||
matches: SearchMatch[];
|
||||
totalMatches: number;
|
||||
filesSearched: number;
|
||||
filesWithMatches: number;
|
||||
}
|
||||
|
||||
// Phase 6: Waypoint Search Types
|
||||
export interface WaypointResult {
|
||||
path: string;
|
||||
line: number;
|
||||
waypointRange: { start: number; end: number };
|
||||
content: string;
|
||||
links: string[];
|
||||
}
|
||||
|
||||
export interface WaypointSearchResult {
|
||||
waypoints: WaypointResult[];
|
||||
totalWaypoints: number;
|
||||
filesSearched: number;
|
||||
}
|
||||
|
||||
// Phase 3: Discovery Endpoint Types
|
||||
export interface StatResult {
|
||||
path: string;
|
||||
|
||||
333
src/utils/search-utils.ts
Normal file
333
src/utils/search-utils.ts
Normal file
@@ -0,0 +1,333 @@
|
||||
import { App, TFile } from 'obsidian';
|
||||
import { SearchMatch } from '../types/mcp-types';
|
||||
import { GlobUtils } from './glob-utils';
|
||||
|
||||
export interface SearchOptions {
|
||||
query: string;
|
||||
isRegex?: boolean;
|
||||
caseSensitive?: boolean;
|
||||
includes?: string[];
|
||||
excludes?: string[];
|
||||
folder?: string;
|
||||
returnSnippets?: boolean;
|
||||
snippetLength?: number;
|
||||
maxResults?: number;
|
||||
}
|
||||
|
||||
export interface SearchStatistics {
|
||||
filesSearched: number;
|
||||
filesWithMatches: number;
|
||||
totalMatches: number;
|
||||
}
|
||||
|
||||
export class SearchUtils {
|
||||
/**
|
||||
* Search vault files with advanced filtering and regex support
|
||||
*/
|
||||
static async search(
|
||||
app: App,
|
||||
options: SearchOptions
|
||||
): Promise<{ matches: SearchMatch[]; stats: SearchStatistics }> {
|
||||
const {
|
||||
query,
|
||||
isRegex = false,
|
||||
caseSensitive = false,
|
||||
includes,
|
||||
excludes,
|
||||
folder,
|
||||
returnSnippets = true,
|
||||
snippetLength = 100,
|
||||
maxResults = 100
|
||||
} = options;
|
||||
|
||||
const matches: SearchMatch[] = [];
|
||||
const filesWithMatches = new Set<string>();
|
||||
let filesSearched = 0;
|
||||
|
||||
// Compile search pattern
|
||||
let searchPattern: RegExp;
|
||||
try {
|
||||
if (isRegex) {
|
||||
const flags = caseSensitive ? 'g' : 'gi';
|
||||
searchPattern = new RegExp(query, flags);
|
||||
} else {
|
||||
// Escape special regex characters for literal search
|
||||
const escapedQuery = query.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
const flags = caseSensitive ? 'g' : 'gi';
|
||||
searchPattern = new RegExp(escapedQuery, flags);
|
||||
}
|
||||
} catch (error) {
|
||||
throw new Error(`Invalid regex pattern: ${(error as Error).message}`);
|
||||
}
|
||||
|
||||
// Get files to search
|
||||
let files = app.vault.getMarkdownFiles();
|
||||
|
||||
// Filter by folder if specified
|
||||
if (folder) {
|
||||
const folderPath = folder.endsWith('/') ? folder : folder + '/';
|
||||
files = files.filter(file =>
|
||||
file.path.startsWith(folderPath) || file.path === folder
|
||||
);
|
||||
}
|
||||
|
||||
// Apply glob filtering
|
||||
if (includes || excludes) {
|
||||
files = files.filter(file =>
|
||||
GlobUtils.shouldInclude(file.path, includes, excludes)
|
||||
);
|
||||
}
|
||||
|
||||
// Search through files
|
||||
for (const file of files) {
|
||||
if (matches.length >= maxResults) {
|
||||
break;
|
||||
}
|
||||
|
||||
filesSearched++;
|
||||
|
||||
try {
|
||||
const content = await app.vault.read(file);
|
||||
const fileMatches = this.searchInFile(
|
||||
file,
|
||||
content,
|
||||
searchPattern,
|
||||
returnSnippets,
|
||||
snippetLength,
|
||||
maxResults - matches.length
|
||||
);
|
||||
|
||||
if (fileMatches.length > 0) {
|
||||
filesWithMatches.add(file.path);
|
||||
matches.push(...fileMatches);
|
||||
}
|
||||
|
||||
// Also search in filename
|
||||
const filenameMatches = this.searchInFilename(
|
||||
file,
|
||||
searchPattern,
|
||||
caseSensitive
|
||||
);
|
||||
|
||||
if (filenameMatches.length > 0) {
|
||||
filesWithMatches.add(file.path);
|
||||
matches.push(...filenameMatches);
|
||||
}
|
||||
} catch (error) {
|
||||
// Skip files that can't be read
|
||||
console.error(`Failed to search file ${file.path}:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
matches,
|
||||
stats: {
|
||||
filesSearched,
|
||||
filesWithMatches: filesWithMatches.size,
|
||||
totalMatches: matches.length
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Search within a single file's content
|
||||
*/
|
||||
private static searchInFile(
|
||||
file: TFile,
|
||||
content: string,
|
||||
pattern: RegExp,
|
||||
returnSnippets: boolean,
|
||||
snippetLength: number,
|
||||
maxMatches: number
|
||||
): SearchMatch[] {
|
||||
const matches: SearchMatch[] = [];
|
||||
const lines = content.split('\n');
|
||||
|
||||
for (let lineIndex = 0; lineIndex < lines.length; lineIndex++) {
|
||||
if (matches.length >= maxMatches) {
|
||||
break;
|
||||
}
|
||||
|
||||
const line = lines[lineIndex];
|
||||
|
||||
// Reset regex lastIndex for global patterns
|
||||
pattern.lastIndex = 0;
|
||||
|
||||
let match: RegExpExecArray | null;
|
||||
while ((match = pattern.exec(line)) !== null) {
|
||||
if (matches.length >= maxMatches) {
|
||||
break;
|
||||
}
|
||||
|
||||
const columnIndex = match.index;
|
||||
const matchText = match[0];
|
||||
|
||||
// Extract snippet with context
|
||||
let snippet = line;
|
||||
let snippetStart = 0;
|
||||
let matchStart = columnIndex;
|
||||
|
||||
if (returnSnippets && line.length > snippetLength) {
|
||||
// Calculate snippet boundaries
|
||||
const halfSnippet = Math.floor(snippetLength / 2);
|
||||
snippetStart = Math.max(0, columnIndex - halfSnippet);
|
||||
const snippetEnd = Math.min(line.length, snippetStart + snippetLength);
|
||||
|
||||
// Adjust if we're at the end of the line
|
||||
if (snippetEnd === line.length && line.length > snippetLength) {
|
||||
snippetStart = Math.max(0, line.length - snippetLength);
|
||||
}
|
||||
|
||||
snippet = line.substring(snippetStart, snippetEnd);
|
||||
matchStart = columnIndex - snippetStart;
|
||||
}
|
||||
|
||||
matches.push({
|
||||
path: file.path,
|
||||
line: lineIndex + 1, // 1-indexed
|
||||
column: columnIndex + 1, // 1-indexed
|
||||
snippet: snippet,
|
||||
matchRanges: [{
|
||||
start: matchStart,
|
||||
end: matchStart + matchText.length
|
||||
}]
|
||||
});
|
||||
|
||||
// Prevent infinite loop for zero-width matches
|
||||
if (match[0].length === 0) {
|
||||
pattern.lastIndex++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return matches;
|
||||
}
|
||||
|
||||
/**
|
||||
* Search in filename
|
||||
*/
|
||||
private static searchInFilename(
|
||||
file: TFile,
|
||||
pattern: RegExp,
|
||||
caseSensitive: boolean
|
||||
): SearchMatch[] {
|
||||
const matches: SearchMatch[] = [];
|
||||
const basename = file.basename;
|
||||
|
||||
// Reset regex lastIndex
|
||||
pattern.lastIndex = 0;
|
||||
|
||||
let match: RegExpExecArray | null;
|
||||
while ((match = pattern.exec(basename)) !== null) {
|
||||
const columnIndex = match.index;
|
||||
const matchText = match[0];
|
||||
|
||||
matches.push({
|
||||
path: file.path,
|
||||
line: 0, // 0 indicates filename match
|
||||
column: columnIndex + 1, // 1-indexed
|
||||
snippet: basename,
|
||||
matchRanges: [{
|
||||
start: columnIndex,
|
||||
end: columnIndex + matchText.length
|
||||
}]
|
||||
});
|
||||
|
||||
// Prevent infinite loop for zero-width matches
|
||||
if (match[0].length === 0) {
|
||||
pattern.lastIndex++;
|
||||
}
|
||||
}
|
||||
|
||||
return matches;
|
||||
}
|
||||
|
||||
/**
|
||||
* Search for Waypoint markers in vault
|
||||
*/
|
||||
static async searchWaypoints(
|
||||
app: App,
|
||||
folder?: string
|
||||
): Promise<Array<{
|
||||
path: string;
|
||||
line: number;
|
||||
waypointRange: { start: number; end: number };
|
||||
content: string;
|
||||
links: string[];
|
||||
}>> {
|
||||
const results: Array<{
|
||||
path: string;
|
||||
line: number;
|
||||
waypointRange: { start: number; end: number };
|
||||
content: string;
|
||||
links: string[];
|
||||
}> = [];
|
||||
|
||||
// Get files to search
|
||||
let files = app.vault.getMarkdownFiles();
|
||||
|
||||
// Filter by folder if specified
|
||||
if (folder) {
|
||||
const folderPath = folder.endsWith('/') ? folder : folder + '/';
|
||||
files = files.filter(file =>
|
||||
file.path.startsWith(folderPath) || file.path === folder
|
||||
);
|
||||
}
|
||||
|
||||
// Search for waypoint markers
|
||||
const waypointStartPattern = /%% Begin Waypoint %%/;
|
||||
const waypointEndPattern = /%% End Waypoint %%/;
|
||||
const linkPattern = /\[\[([^\]]+)\]\]/g;
|
||||
|
||||
for (const file of files) {
|
||||
try {
|
||||
const content = await app.vault.read(file);
|
||||
const lines = content.split('\n');
|
||||
|
||||
let inWaypoint = false;
|
||||
let waypointStart = -1;
|
||||
let waypointContent: string[] = [];
|
||||
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i];
|
||||
|
||||
if (waypointStartPattern.test(line)) {
|
||||
inWaypoint = true;
|
||||
waypointStart = i + 1; // 1-indexed
|
||||
waypointContent = [];
|
||||
} else if (waypointEndPattern.test(line) && inWaypoint) {
|
||||
// Extract links from waypoint content
|
||||
const contentStr = waypointContent.join('\n');
|
||||
const links: string[] = [];
|
||||
let linkMatch: RegExpExecArray | null;
|
||||
|
||||
while ((linkMatch = linkPattern.exec(contentStr)) !== null) {
|
||||
links.push(linkMatch[1]);
|
||||
}
|
||||
|
||||
results.push({
|
||||
path: file.path,
|
||||
line: waypointStart,
|
||||
waypointRange: {
|
||||
start: waypointStart,
|
||||
end: i + 1 // 1-indexed
|
||||
},
|
||||
content: contentStr,
|
||||
links: links
|
||||
});
|
||||
|
||||
inWaypoint = false;
|
||||
waypointStart = -1;
|
||||
waypointContent = [];
|
||||
} else if (inWaypoint) {
|
||||
waypointContent.push(line);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`Failed to search waypoints in ${file.path}:`, error);
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user