feat: Make semantic search optional and disabled by default

- Add SemanticSearchConfig interface with enabled toggle to siteConfig.ts
- Default semantic search to disabled (enabled: false) to avoid blocking forks without OPENAI_API_KEY
- Update SearchModal.tsx to conditionally show mode toggle based on config
- Update sync-posts.ts to skip embedding generation when disabled
- Add semantic search toggle to Dashboard config generator
- Update FORK_CONFIG.md with Semantic Search Configuration section
- Update fork-config.json.example with semanticSearch option
- Update docs-semantic-search.md with enable/disable instructions
- Update changelog and documentation

When disabled (default):
- Search modal shows only keyword search (no mode toggle)
- Embedding generation skipped during sync
- No OpenAI API key required

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Wayne Sutton
2026-01-05 22:22:50 -08:00
parent 5a8df46681
commit 3c9feb071b
15 changed files with 239 additions and 49 deletions

View File

@@ -22,7 +22,7 @@ Your content is instantly available to browsers, LLMs, and AI agents.. Write mar
- **Total Posts**: 17 - **Total Posts**: 17
- **Total Pages**: 4 - **Total Pages**: 4
- **Latest Post**: 2025-12-29 - **Latest Post**: 2025-12-29
- **Last Updated**: 2026-01-05T18:54:36.240Z - **Last Updated**: 2026-01-06T02:32:19.578Z
## Tech stack ## Tech stack

View File

@@ -5,7 +5,7 @@ Project instructions for Claude Code.
## Project context ## Project context
<!-- Auto-updated by sync:discovery --> <!-- Auto-updated by sync:discovery -->
<!-- Site: markdown | Posts: 17 | Pages: 4 | Updated: 2026-01-05T18:54:36.241Z --> <!-- Site: markdown | Posts: 17 | Pages: 4 | Updated: 2026-01-06T02:32:19.579Z -->
Markdown sync framework. Write markdown in `content/`, run sync commands, content appears instantly via Convex real-time database. Built for developers and AI agents. Markdown sync framework. Write markdown in `content/`, run sync commands, content appears instantly via Convex real-time database. Built for developers and AI agents.

View File

@@ -831,6 +831,52 @@ imageLightbox: {
--- ---
## Semantic Search Configuration
Enable AI-powered semantic search using OpenAI embeddings. When disabled, only keyword search is available.
### In fork-config.json
```json
{
"semanticSearch": {
"enabled": false
}
}
```
### Manual Configuration
In `src/config/siteConfig.ts`:
```typescript
semanticSearch: {
enabled: true, // Enable semantic search (requires OPENAI_API_KEY)
},
```
**Requirements:**
When enabled, set the OpenAI API key in Convex:
```bash
npx convex env set OPENAI_API_KEY sk-your-key-here
```
**Features:**
- Toggle between Keyword and Semantic modes in search modal (Cmd+K)
- Keyword search: exact word matching (instant, free)
- Semantic search: finds content by meaning (~300ms, ~$0.0001/query)
- Similarity scores displayed as percentages
- Embeddings generated automatically during `npm run sync`
**Default:** `enabled: false` (keyword search only, no API key required)
See [Semantic Search](/docs-semantic-search) for detailed documentation.
---
## MCP Server Configuration ## MCP Server Configuration
HTTP-based Model Context Protocol server for AI tool integration (Cursor, Claude Desktop). HTTP-based Model Context Protocol server for AI tool integration (Cursor, Claude Desktop).

10
TASK.md
View File

@@ -4,10 +4,18 @@
## Current Status ## Current Status
v2.10.0 ready. Semantic search with vector embeddings added to complement keyword search. v2.10.1 ready. Semantic search now optional via siteConfig.semanticSearch.enabled toggle.
## Completed ## Completed
- [x] Optional semantic search configuration
- [x] Added `SemanticSearchConfig` interface to `siteConfig.ts`
- [x] Added `semanticSearch.enabled` toggle (default: true)
- [x] Updated `SearchModal.tsx` to conditionally show mode toggle
- [x] Updated `sync-posts.ts` to skip embedding generation when disabled
- [x] Updated `docs-semantic-search.md` with enable/disable section
- [x] Updated `docs.md` with semantic search configuration note
- [x] Semantic search with vector embeddings - [x] Semantic search with vector embeddings
- [x] Dual search modes: Keyword (exact match) and Semantic (meaning-based) - [x] Dual search modes: Keyword (exact match) and Semantic (meaning-based)
- [x] Toggle between modes in search modal (Cmd+K) with TextAa and Brain icons - [x] Toggle between modes in search modal (Cmd+K) with TextAa and Brain icons

View File

@@ -4,6 +4,28 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [2.10.1] - 2026-01-05
### Added
- Optional semantic search configuration via `siteConfig.semanticSearch`
- New `enabled` toggle (default: `false` to avoid blocking forks without API key)
- When disabled, search modal shows only keyword search (no mode toggle)
- Embedding generation skipped during sync when disabled (saves API costs)
- Existing embeddings preserved in database when disabled (no data loss)
- Tab key shortcut hints hidden when semantic search is disabled
- Dashboard config generator includes semantic search toggle
### Technical
- New `SemanticSearchConfig` interface in `src/config/siteConfig.ts`
- Updated `src/components/SearchModal.tsx` to conditionally render mode toggle
- Updated `scripts/sync-posts.ts` to check config before embedding generation
- Updated `src/pages/Dashboard.tsx` with semantic search config option
- Updated `FORK_CONFIG.md` with semantic search configuration section
- Updated `fork-config.json.example` with semanticSearch option
- Updated documentation: `docs-semantic-search.md`, `docs.md`
## [2.10.0] - 2026-01-05 ## [2.10.0] - 2026-01-05
### Added ### Added

View File

@@ -11,6 +11,29 @@ docsSectionOrder: 4
All notable changes to this project. All notable changes to this project.
## v2.10.1
Released January 5, 2026
**Optional semantic search configuration**
Semantic search can now be disabled via `siteConfig.semanticSearch.enabled`:
```typescript
semanticSearch: {
enabled: false, // Disable semantic search, use keyword only
},
```
When disabled:
- Search modal shows only keyword search (no mode toggle)
- Embedding generation skipped during sync (saves API costs)
- Existing embeddings preserved in database (no data loss)
Default is `enabled: false` (keyword search only, no API key required). Set to `true` and configure OPENAI_API_KEY to enable semantic search.
Updated files: `src/config/siteConfig.ts`, `src/components/SearchModal.tsx`, `scripts/sync-posts.ts`, `src/pages/Dashboard.tsx`, `FORK_CONFIG.md`, `fork-config.json.example`, `content/pages/docs-semantic-search.md`, `content/pages/docs.md`
## v2.10.0 ## v2.10.0
Released January 5, 2026 Released January 5, 2026

View File

@@ -87,6 +87,31 @@ If the key is not configured:
- Keyword search continues to work normally - Keyword search continues to work normally
- Sync script skips embedding generation - Sync script skips embedding generation
### Enable/Disable Semantic Search
Semantic search is **disabled by default** to avoid requiring API keys for forks. Enable it via `src/config/siteConfig.ts`:
```typescript
semanticSearch: {
enabled: true, // Enable semantic search (requires OPENAI_API_KEY)
},
```
When disabled (default):
- Search modal shows only keyword search (no mode toggle)
- Embedding generation skipped during sync (saves API costs)
- No OpenAI API key required
When enabled:
- Search modal shows both Keyword and Semantic modes
- Embeddings generated during `npm run sync`
- Requires OPENAI_API_KEY in Convex
To enable semantic search:
1. Set `semanticSearch.enabled: true` in siteConfig.ts
2. Set `OPENAI_API_KEY` in Convex: `npx convex env set OPENAI_API_KEY sk-xxx`
3. Run `npm run sync` to generate embeddings
### How embeddings are generated ### How embeddings are generated
When you run `npm run sync`: When you run `npm run sync`:

View File

@@ -98,8 +98,11 @@ Press `Command+K` (Mac) or `Ctrl+K` (Windows/Linux) to open the search modal. Cl
- Result snippets with context around matches - Result snippets with context around matches
- Distinguishes between posts and pages - Distinguishes between posts and pages
- Works with all four themes - Works with all four themes
- Two search modes: [Keyword](/docs-search) (exact match) and [Semantic](/docs-semantic-search) (meaning-based)
Search uses Convex full text search indexes. No configuration needed. Search uses Convex full text search indexes. No configuration needed for keyword search.
**Semantic search configuration:** Requires `OPENAI_API_KEY` in Convex. Can be disabled via `siteConfig.semanticSearch.enabled: false`. See [Semantic Search](/docs-semantic-search) for details.
## Copy Page dropdown ## Copy Page dropdown

View File

@@ -35,7 +35,7 @@ A brief description of each file in the codebase.
| File | Description | | File | Description |
| --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `siteConfig.ts` | Centralized site configuration (name, logo, blog page, posts display with homepage post limit and read more link, featured section with configurable title via featuredTitle, GitHub contributions, nav order, inner page logo settings, hardcoded navigation items for React routes, GitHub repository config for AI service raw URLs, font family configuration, right sidebar configuration, footer configuration with markdown support, social footer configuration, homepage configuration, AI chat configuration, aiDashboard configuration with multi-model support for text chat and image generation, newsletter configuration with admin and notifications, contact form configuration, weekly digest configuration, stats page configuration with public/private toggle, dashboard configuration with optional WorkOS authentication via requireAuth, image lightbox configuration with enabled toggle) | | `siteConfig.ts` | Centralized site configuration (name, logo, blog page, posts display with homepage post limit and read more link, featured section with configurable title via featuredTitle, GitHub contributions, nav order, inner page logo settings, hardcoded navigation items for React routes, GitHub repository config for AI service raw URLs, font family configuration, right sidebar configuration, footer configuration with markdown support, social footer configuration, homepage configuration, AI chat configuration, aiDashboard configuration with multi-model support for text chat and image generation, newsletter configuration with admin and notifications, contact form configuration, weekly digest configuration, stats page configuration with public/private toggle, dashboard configuration with optional WorkOS authentication via requireAuth, image lightbox configuration with enabled toggle, semantic search configuration with enabled toggle) |
### Pages (`src/pages/`) ### Pages (`src/pages/`)

View File

@@ -179,6 +179,9 @@
"dashboard": { "dashboard": {
"enabled": true, "enabled": true,
"requireAuth": false "requireAuth": false
},
"semanticSearch": {
"enabled": false
} }
} }

View File

@@ -1,6 +1,6 @@
# llms.txt - Information for AI assistants and LLMs # llms.txt - Information for AI assistants and LLMs
# Learn more: https://llmstxt.org/ # Learn more: https://llmstxt.org/
# Last updated: 2026-01-05T18:54:36.241Z # Last updated: 2026-01-06T02:32:19.579Z
> Your content is instantly available to browsers, LLMs, and AI agents. > Your content is instantly available to browsers, LLMs, and AI agents.

View File

@@ -4,6 +4,7 @@ import matter from "gray-matter";
import { ConvexHttpClient } from "convex/browser"; import { ConvexHttpClient } from "convex/browser";
import { api } from "../convex/_generated/api"; import { api } from "../convex/_generated/api";
import dotenv from "dotenv"; import dotenv from "dotenv";
import { siteConfig } from "../src/config/siteConfig";
// Load environment variables based on SYNC_ENV // Load environment variables based on SYNC_ENV
const isProduction = process.env.SYNC_ENV === "production"; const isProduction = process.env.SYNC_ENV === "production";
@@ -375,22 +376,26 @@ async function syncPosts() {
} }
} }
// Generate embeddings for semantic search (if OPENAI_API_KEY is configured) // Generate embeddings for semantic search (if enabled in siteConfig and OPENAI_API_KEY is configured)
console.log("\nGenerating embeddings for semantic search..."); if (siteConfig.semanticSearch?.enabled === false) {
try { console.log("\nSkipping embedding generation (semantic search disabled in siteConfig)");
const embeddingResult = await client.action( } else {
api.embeddings.generateMissingEmbeddings, console.log("\nGenerating embeddings for semantic search...");
{} try {
); const embeddingResult = await client.action(
if (embeddingResult.skipped) { api.embeddings.generateMissingEmbeddings,
console.log(" Skipped: OPENAI_API_KEY not configured"); {}
} else { );
console.log(` Posts: ${embeddingResult.postsProcessed} embeddings generated`); if (embeddingResult.skipped) {
console.log(` Pages: ${embeddingResult.pagesProcessed} embeddings generated`); console.log(" Skipped: OPENAI_API_KEY not configured");
} else {
console.log(` Posts: ${embeddingResult.postsProcessed} embeddings generated`);
console.log(` Pages: ${embeddingResult.pagesProcessed} embeddings generated`);
}
} catch (error) {
// Non-fatal - continue even if embedding generation fails
console.log(" Warning: Could not generate embeddings:", error);
} }
} catch (error) {
// Non-fatal - continue even if embedding generation fails
console.log(" Warning: Could not generate embeddings:", error);
} }
// Generate static raw markdown files in public/raw/ // Generate static raw markdown files in public/raw/

View File

@@ -11,6 +11,7 @@ import {
TextAa, TextAa,
Brain, Brain,
} from "@phosphor-icons/react"; } from "@phosphor-icons/react";
import { siteConfig } from "../config/siteConfig";
interface SearchModalProps { interface SearchModalProps {
isOpen: boolean; isOpen: boolean;
@@ -31,6 +32,9 @@ interface SearchResult {
} }
export default function SearchModal({ isOpen, onClose }: SearchModalProps) { export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
// Check if semantic search is enabled in siteConfig
const semanticEnabled = siteConfig.semanticSearch?.enabled !== false;
const [searchQuery, setSearchQuery] = useState(""); const [searchQuery, setSearchQuery] = useState("");
const [selectedIndex, setSelectedIndex] = useState(0); const [selectedIndex, setSelectedIndex] = useState(0);
const [searchMode, setSearchMode] = useState<SearchMode>("keyword"); const [searchMode, setSearchMode] = useState<SearchMode>("keyword");
@@ -100,8 +104,8 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
// Handle keyboard navigation // Handle keyboard navigation
const handleKeyDown = useCallback( const handleKeyDown = useCallback(
(e: React.KeyboardEvent) => { (e: React.KeyboardEvent) => {
// Tab toggles between search modes // Tab toggles between search modes (only if semantic search is enabled)
if (e.key === "Tab") { if (e.key === "Tab" && semanticEnabled) {
e.preventDefault(); e.preventDefault();
setSearchMode((prev) => (prev === "keyword" ? "semantic" : "keyword")); setSearchMode((prev) => (prev === "keyword" ? "semantic" : "keyword"));
return; return;
@@ -142,7 +146,7 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
break; break;
} }
}, },
[results, selectedIndex, navigate, onClose, searchMode, searchQuery] [results, selectedIndex, navigate, onClose, searchMode, searchQuery, semanticEnabled]
); );
// Handle clicking on a result // Handle clicking on a result
@@ -168,25 +172,27 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
return ( return (
<div className="search-modal-backdrop" onClick={handleBackdropClick}> <div className="search-modal-backdrop" onClick={handleBackdropClick}>
<div className="search-modal"> <div className="search-modal">
{/* Search mode toggle */} {/* Search mode toggle - only shown when semantic search is enabled */}
<div className="search-mode-toggle"> {semanticEnabled && (
<button <div className="search-mode-toggle">
className={`search-mode-btn ${searchMode === "keyword" ? "active" : ""}`} <button
onClick={() => setSearchMode("keyword")} className={`search-mode-btn ${searchMode === "keyword" ? "active" : ""}`}
title="Keyword search - matches exact words" onClick={() => setSearchMode("keyword")}
> title="Keyword search - matches exact words"
<TextAa size={16} weight="bold" /> >
<span>Keyword</span> <TextAa size={16} weight="bold" />
</button> <span>Keyword</span>
<button </button>
className={`search-mode-btn ${searchMode === "semantic" ? "active" : ""}`} <button
onClick={() => setSearchMode("semantic")} className={`search-mode-btn ${searchMode === "semantic" ? "active" : ""}`}
title="Semantic search - finds similar meaning" onClick={() => setSearchMode("semantic")}
> title="Semantic search - finds similar meaning"
<Brain size={16} weight="bold" /> >
<span>Semantic</span> <Brain size={16} weight="bold" />
</button> <span>Semantic</span>
</div> </button>
</div>
)}
{/* Search input */} {/* Search input */}
<div className="search-modal-input-wrapper"> <div className="search-modal-input-wrapper">
@@ -223,9 +229,11 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
: "Describe what you're looking for"} : "Describe what you're looking for"}
</p> </p>
<div className="search-modal-shortcuts"> <div className="search-modal-shortcuts">
<span className="search-shortcut"> {semanticEnabled && (
<kbd>Tab</kbd> Switch mode <span className="search-shortcut">
</span> <kbd>Tab</kbd> Switch mode
</span>
)}
<span className="search-shortcut"> <span className="search-shortcut">
<kbd></kbd> <kbd></kbd>
<kbd></kbd> Navigate <kbd></kbd> Navigate
@@ -292,9 +300,11 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
{/* Footer with keyboard hints */} {/* Footer with keyboard hints */}
{results && results.length > 0 && ( {results && results.length > 0 && (
<div className="search-modal-footer"> <div className="search-modal-footer">
<span className="search-footer-hint"> {semanticEnabled && (
<kbd>Tab</kbd> switch mode <span className="search-footer-hint">
</span> <kbd>Tab</kbd> switch mode
</span>
)}
<span className="search-footer-hint"> <span className="search-footer-hint">
<kbd></kbd> select <kbd></kbd> select
</span> </span>

View File

@@ -236,6 +236,13 @@ export interface ImageLightboxConfig {
enabled: boolean; // Global toggle for image lightbox feature enabled: boolean; // Global toggle for image lightbox feature
} }
// Semantic search configuration
// Enables AI-powered search using vector embeddings
// Requires OPENAI_API_KEY environment variable in Convex dashboard
export interface SemanticSearchConfig {
enabled: boolean; // Global toggle for semantic search feature
}
// Social link configuration for social footer // Social link configuration for social footer
export interface SocialLink { export interface SocialLink {
platform: platform:
@@ -365,6 +372,9 @@ export interface SiteConfig {
// AI Dashboard configuration (optional) // AI Dashboard configuration (optional)
aiDashboard?: AIDashboardConfig; aiDashboard?: AIDashboardConfig;
// Semantic search configuration (optional)
semanticSearch?: SemanticSearchConfig;
} }
// Default site configuration // Default site configuration
@@ -744,6 +754,13 @@ export const siteConfig: SiteConfig = {
}, },
], ],
}, },
// Semantic search configuration
// Set enabled: true to enable semantic search (requires OPENAI_API_KEY in Convex)
// When disabled, only keyword search is available (no API key needed)
semanticSearch: {
enabled: false, // Set to true to enable semantic search (requires OPENAI_API_KEY)
},
}; };
// Export the config as default for easy importing // Export the config as default for easy importing

View File

@@ -4597,6 +4597,8 @@ function ConfigSection({
mcpServerRequireAuth: siteConfig.mcpServer?.requireAuth || false, mcpServerRequireAuth: siteConfig.mcpServer?.requireAuth || false,
// Image lightbox // Image lightbox
imageLightboxEnabled: siteConfig.imageLightbox?.enabled !== false, imageLightboxEnabled: siteConfig.imageLightbox?.enabled !== false,
// Semantic search
semanticSearchEnabled: siteConfig.semanticSearch?.enabled || false,
}); });
const [copied, setCopied] = useState(false); const [copied, setCopied] = useState(false);
@@ -4762,6 +4764,12 @@ export const siteConfig: SiteConfig = {
imageLightbox: { imageLightbox: {
enabled: ${config.imageLightboxEnabled}, enabled: ${config.imageLightboxEnabled},
}, },
// Semantic search configuration
// Set enabled: true to enable AI-powered semantic search (requires OPENAI_API_KEY in Convex)
semanticSearch: {
enabled: ${config.semanticSearchEnabled},
},
}; };
export default siteConfig; export default siteConfig;
@@ -5573,6 +5581,26 @@ export default siteConfig;
</div> </div>
</div> </div>
{/* Semantic Search */}
<div className="dashboard-config-card">
<h3>Semantic Search</h3>
<div className="config-field checkbox">
<label>
<input
type="checkbox"
checked={config.semanticSearchEnabled}
onChange={(e) =>
handleChange("semanticSearchEnabled", e.target.checked)
}
/>
<span>Enable semantic search (requires OPENAI_API_KEY in Convex)</span>
</label>
</div>
<p className="config-hint">
When enabled, search modal shows both Keyword and Semantic modes. Requires OpenAI API key for embeddings.
</p>
</div>
{/* Links */} {/* Links */}
<div className="dashboard-config-card"> <div className="dashboard-config-card">
<h3>External Links</h3> <h3>External Links</h3>