feat: Make semantic search optional and disabled by default

- Add SemanticSearchConfig interface with enabled toggle to siteConfig.ts - Default semantic search to disabled (enabled: false) to avoid blocking forks without OPENAI_API_KEY - Update SearchModal.tsx to conditionally show mode toggle based on config - Update sync-posts.ts to skip embedding generation when disabled - Add semantic search toggle to Dashboard config generator - Update FORK_CONFIG.md with Semantic Search Configuration section - Update fork-config.json.example with semanticSearch option - Update docs-semantic-search.md with enable/disable instructions - Update changelog and documentation When disabled (default): - Search modal shows only keyword search (no mode toggle) - Embedding generation skipped during sync - No OpenAI API key required 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 20:08:57 +00:00 · 2026-01-05 22:22:50 -08:00
parent 5a8df46681
commit 3c9feb071b
15 changed files with 239 additions and 49 deletions
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -22,7 +22,7 @@ Your content is instantly available to browsers, LLMs, and AI agents.. Write mar
 - **Total Posts**: 17
 - **Total Pages**: 4
 - **Latest Post**: 2025-12-29
- **Last Updated**: 2026-01-05T18:54:36.240Z
+- **Last Updated**: 2026-01-06T02:32:19.578Z

 ## Tech stack

--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -5,7 +5,7 @@ Project instructions for Claude Code.
 ## Project context

 <!-- Auto-updated by sync:discovery -->
-<!-- Site: markdown  | Posts: 17 | Pages: 4 | Updated: 2026-01-05T18:54:36.241Z -->
+<!-- Site: markdown  | Posts: 17 | Pages: 4 | Updated: 2026-01-06T02:32:19.579Z -->

 Markdown sync framework. Write markdown in `content/`, run sync commands, content appears instantly via Convex real-time database. Built for developers and AI agents.

--- a/FORK_CONFIG.md
+++ b/FORK_CONFIG.md
@@ -831,6 +831,52 @@ imageLightbox: {

 ---

+## Semantic Search Configuration
+
+Enable AI-powered semantic search using OpenAI embeddings. When disabled, only keyword search is available.
+
+### In fork-config.json
+
+```json
+{
+  "semanticSearch": {
+    "enabled": false
+  }
+}
+```
+
+### Manual Configuration
+
+In `src/config/siteConfig.ts`:
+
+```typescript
+semanticSearch: {
+  enabled: true, // Enable semantic search (requires OPENAI_API_KEY)
+},
+```
+
+**Requirements:**
+
+When enabled, set the OpenAI API key in Convex:
+
+```bash
+npx convex env set OPENAI_API_KEY sk-your-key-here
+```
+
+**Features:**
+
+- Toggle between Keyword and Semantic modes in search modal (Cmd+K)
+- Keyword search: exact word matching (instant, free)
+- Semantic search: finds content by meaning (~300ms, ~$0.0001/query)
+- Similarity scores displayed as percentages
+- Embeddings generated automatically during `npm run sync`
+
+**Default:** `enabled: false` (keyword search only, no API key required)
+
+See [Semantic Search](/docs-semantic-search) for detailed documentation.
+
+---
+
 ## MCP Server Configuration

 HTTP-based Model Context Protocol server for AI tool integration (Cursor, Claude Desktop).
--- a/TASK.md
+++ b/TASK.md
@@ -4,10 +4,18 @@

 ## Current Status

-v2.10.0 ready. Semantic search with vector embeddings added to complement keyword search.
+v2.10.1 ready. Semantic search now optional via siteConfig.semanticSearch.enabled toggle.

 ## Completed

+- [x] Optional semantic search configuration
+  - [x] Added `SemanticSearchConfig` interface to `siteConfig.ts`
+  - [x] Added `semanticSearch.enabled` toggle (default: true)
+  - [x] Updated `SearchModal.tsx` to conditionally show mode toggle
+  - [x] Updated `sync-posts.ts` to skip embedding generation when disabled
+  - [x] Updated `docs-semantic-search.md` with enable/disable section
+  - [x] Updated `docs.md` with semantic search configuration note
+
 - [x] Semantic search with vector embeddings
  - [x] Dual search modes: Keyword (exact match) and Semantic (meaning-based)
  - [x] Toggle between modes in search modal (Cmd+K) with TextAa and Brain icons
--- a/changelog.md
+++ b/changelog.md
@@ -4,6 +4,28 @@ All notable changes to this project will be documented in this file.

 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

+## [2.10.1] - 2026-01-05
+
+### Added
+
+- Optional semantic search configuration via `siteConfig.semanticSearch`
+  - New `enabled` toggle (default: `false` to avoid blocking forks without API key)
+  - When disabled, search modal shows only keyword search (no mode toggle)
+  - Embedding generation skipped during sync when disabled (saves API costs)
+  - Existing embeddings preserved in database when disabled (no data loss)
+  - Tab key shortcut hints hidden when semantic search is disabled
+  - Dashboard config generator includes semantic search toggle
+
+### Technical
+
+- New `SemanticSearchConfig` interface in `src/config/siteConfig.ts`
+- Updated `src/components/SearchModal.tsx` to conditionally render mode toggle
+- Updated `scripts/sync-posts.ts` to check config before embedding generation
+- Updated `src/pages/Dashboard.tsx` with semantic search config option
+- Updated `FORK_CONFIG.md` with semantic search configuration section
+- Updated `fork-config.json.example` with semanticSearch option
+- Updated documentation: `docs-semantic-search.md`, `docs.md`
+
 ## [2.10.0] - 2026-01-05

 ### Added
--- a/content/pages/changelog-page.md
+++ b/content/pages/changelog-page.md
@@ -11,6 +11,29 @@ docsSectionOrder: 4

 All notable changes to this project.

+## v2.10.1
+
+Released January 5, 2026
+
+**Optional semantic search configuration**
+
+Semantic search can now be disabled via `siteConfig.semanticSearch.enabled`:
+
+```typescript
+semanticSearch: {
+  enabled: false, // Disable semantic search, use keyword only
+},
+```
+
+When disabled:
+- Search modal shows only keyword search (no mode toggle)
+- Embedding generation skipped during sync (saves API costs)
+- Existing embeddings preserved in database (no data loss)
+
+Default is `enabled: false` (keyword search only, no API key required). Set to `true` and configure OPENAI_API_KEY to enable semantic search.
+
+Updated files: `src/config/siteConfig.ts`, `src/components/SearchModal.tsx`, `scripts/sync-posts.ts`, `src/pages/Dashboard.tsx`, `FORK_CONFIG.md`, `fork-config.json.example`, `content/pages/docs-semantic-search.md`, `content/pages/docs.md`
+
 ## v2.10.0

 Released January 5, 2026
--- a/content/pages/docs-semantic-search.md
+++ b/content/pages/docs-semantic-search.md
@@ -87,6 +87,31 @@ If the key is not configured:
 - Keyword search continues to work normally
 - Sync script skips embedding generation

+### Enable/Disable Semantic Search
+
+Semantic search is **disabled by default** to avoid requiring API keys for forks. Enable it via `src/config/siteConfig.ts`:
+
+```typescript
+semanticSearch: {
+  enabled: true, // Enable semantic search (requires OPENAI_API_KEY)
+},
+```
+
+When disabled (default):
+- Search modal shows only keyword search (no mode toggle)
+- Embedding generation skipped during sync (saves API costs)
+- No OpenAI API key required
+
+When enabled:
+- Search modal shows both Keyword and Semantic modes
+- Embeddings generated during `npm run sync`
+- Requires OPENAI_API_KEY in Convex
+
+To enable semantic search:
+1. Set `semanticSearch.enabled: true` in siteConfig.ts
+2. Set `OPENAI_API_KEY` in Convex: `npx convex env set OPENAI_API_KEY sk-xxx`
+3. Run `npm run sync` to generate embeddings
+
 ### How embeddings are generated

 When you run `npm run sync`:
--- a/content/pages/docs.md
+++ b/content/pages/docs.md
@@ -98,8 +98,11 @@ Press `Command+K` (Mac) or `Ctrl+K` (Windows/Linux) to open the search modal. Cl
 - Result snippets with context around matches
 - Distinguishes between posts and pages
 - Works with all four themes
+- Two search modes: [Keyword](/docs-search) (exact match) and [Semantic](/docs-semantic-search) (meaning-based)

-Search uses Convex full text search indexes. No configuration needed.
+Search uses Convex full text search indexes. No configuration needed for keyword search.
+
+**Semantic search configuration:** Requires `OPENAI_API_KEY` in Convex. Can be disabled via `siteConfig.semanticSearch.enabled: false`. See [Semantic Search](/docs-semantic-search) for details.

 ## Copy Page dropdown

--- a/files.md
+++ b/files.md
@@ -35,7 +35,7 @@ A brief description of each file in the codebase.

 | File            | Description                                                                                                                                                                                                                                                                                                                                                                                              |
 | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `siteConfig.ts` | Centralized site configuration (name, logo, blog page, posts display with homepage post limit and read more link, featured section with configurable title via featuredTitle, GitHub contributions, nav order, inner page logo settings, hardcoded navigation items for React routes, GitHub repository config for AI service raw URLs, font family configuration, right sidebar configuration, footer configuration with markdown support, social footer configuration, homepage configuration, AI chat configuration, aiDashboard configuration with multi-model support for text chat and image generation, newsletter configuration with admin and notifications, contact form configuration, weekly digest configuration, stats page configuration with public/private toggle, dashboard configuration with optional WorkOS authentication via requireAuth, image lightbox configuration with enabled toggle) |
+| `siteConfig.ts` | Centralized site configuration (name, logo, blog page, posts display with homepage post limit and read more link, featured section with configurable title via featuredTitle, GitHub contributions, nav order, inner page logo settings, hardcoded navigation items for React routes, GitHub repository config for AI service raw URLs, font family configuration, right sidebar configuration, footer configuration with markdown support, social footer configuration, homepage configuration, AI chat configuration, aiDashboard configuration with multi-model support for text chat and image generation, newsletter configuration with admin and notifications, contact form configuration, weekly digest configuration, stats page configuration with public/private toggle, dashboard configuration with optional WorkOS authentication via requireAuth, image lightbox configuration with enabled toggle, semantic search configuration with enabled toggle) |

 ### Pages (`src/pages/`)

--- a/fork-config.json.example
+++ b/fork-config.json.example
@@ -179,6 +179,9 @@
  "dashboard": {
    "enabled": true,
    "requireAuth": false
+  },
+  "semanticSearch": {
+    "enabled": false
  }
 }

--- a/public/llms.txt
+++ b/public/llms.txt
@@ -1,6 +1,6 @@
 # llms.txt - Information for AI assistants and LLMs
 # Learn more: https://llmstxt.org/
-# Last updated: 2026-01-05T18:54:36.241Z
+# Last updated: 2026-01-06T02:32:19.579Z

 > Your content is instantly available to browsers, LLMs, and AI agents.

--- a/scripts/sync-posts.ts
+++ b/scripts/sync-posts.ts
@@ -4,6 +4,7 @@ import matter from "gray-matter";
 import { ConvexHttpClient } from "convex/browser";
 import { api } from "../convex/_generated/api";
 import dotenv from "dotenv";
+import { siteConfig } from "../src/config/siteConfig";

 // Load environment variables based on SYNC_ENV
 const isProduction = process.env.SYNC_ENV === "production";
@@ -375,22 +376,26 @@ async function syncPosts() {
    }
  }

-  // Generate embeddings for semantic search (if OPENAI_API_KEY is configured)
-  console.log("\nGenerating embeddings for semantic search...");
-  try {
-    const embeddingResult = await client.action(
-      api.embeddings.generateMissingEmbeddings,
-      {}
-    );
-    if (embeddingResult.skipped) {
-      console.log("  Skipped: OPENAI_API_KEY not configured");
-    } else {
-      console.log(`  Posts: ${embeddingResult.postsProcessed} embeddings generated`);
-      console.log(`  Pages: ${embeddingResult.pagesProcessed} embeddings generated`);
+  // Generate embeddings for semantic search (if enabled in siteConfig and OPENAI_API_KEY is configured)
+  if (siteConfig.semanticSearch?.enabled === false) {
+    console.log("\nSkipping embedding generation (semantic search disabled in siteConfig)");
+  } else {
+    console.log("\nGenerating embeddings for semantic search...");
+    try {
+      const embeddingResult = await client.action(
+        api.embeddings.generateMissingEmbeddings,
+        {}
+      );
+      if (embeddingResult.skipped) {
+        console.log("  Skipped: OPENAI_API_KEY not configured");
+      } else {
+        console.log(`  Posts: ${embeddingResult.postsProcessed} embeddings generated`);
+        console.log(`  Pages: ${embeddingResult.pagesProcessed} embeddings generated`);
+      }
+    } catch (error) {
+      // Non-fatal - continue even if embedding generation fails
+      console.log("  Warning: Could not generate embeddings:", error);
    }
-  } catch (error) {
-    // Non-fatal - continue even if embedding generation fails
-    console.log("  Warning: Could not generate embeddings:", error);
  }

  // Generate static raw markdown files in public/raw/
--- a/src/components/SearchModal.tsx
+++ b/src/components/SearchModal.tsx
@@ -11,6 +11,7 @@ import {
  TextAa,
  Brain,
 } from "@phosphor-icons/react";
+import { siteConfig } from "../config/siteConfig";

 interface SearchModalProps {
  isOpen: boolean;
@@ -31,6 +32,9 @@ interface SearchResult {
 }

 export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
+  // Check if semantic search is enabled in siteConfig
+  const semanticEnabled = siteConfig.semanticSearch?.enabled !== false;
+
  const [searchQuery, setSearchQuery] = useState("");
  const [selectedIndex, setSelectedIndex] = useState(0);
  const [searchMode, setSearchMode] = useState<SearchMode>("keyword");
@@ -100,8 +104,8 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
  // Handle keyboard navigation
  const handleKeyDown = useCallback(
    (e: React.KeyboardEvent) => {
-      // Tab toggles between search modes
-      if (e.key === "Tab") {
+      // Tab toggles between search modes (only if semantic search is enabled)
+      if (e.key === "Tab" && semanticEnabled) {
        e.preventDefault();
        setSearchMode((prev) => (prev === "keyword" ? "semantic" : "keyword"));
        return;
@@ -142,7 +146,7 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
          break;
      }
    },
-    [results, selectedIndex, navigate, onClose, searchMode, searchQuery]
+    [results, selectedIndex, navigate, onClose, searchMode, searchQuery, semanticEnabled]
  );

  // Handle clicking on a result
@@ -168,25 +172,27 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
  return (
    <div className="search-modal-backdrop" onClick={handleBackdropClick}>
      <div className="search-modal">
-        {/* Search mode toggle */}
-        <div className="search-mode-toggle">
-          <button
-            className={`search-mode-btn ${searchMode === "keyword" ? "active" : ""}`}
-            onClick={() => setSearchMode("keyword")}
-            title="Keyword search - matches exact words"
-          >
-            <TextAa size={16} weight="bold" />
-            <span>Keyword</span>
-          </button>
-          <button
-            className={`search-mode-btn ${searchMode === "semantic" ? "active" : ""}`}
-            onClick={() => setSearchMode("semantic")}
-            title="Semantic search - finds similar meaning"
-          >
-            <Brain size={16} weight="bold" />
-            <span>Semantic</span>
-          </button>
-        </div>
+        {/* Search mode toggle - only shown when semantic search is enabled */}
+        {semanticEnabled && (
+          <div className="search-mode-toggle">
+            <button
+              className={`search-mode-btn ${searchMode === "keyword" ? "active" : ""}`}
+              onClick={() => setSearchMode("keyword")}
+              title="Keyword search - matches exact words"
+            >
+              <TextAa size={16} weight="bold" />
+              <span>Keyword</span>
+            </button>
+            <button
+              className={`search-mode-btn ${searchMode === "semantic" ? "active" : ""}`}
+              onClick={() => setSearchMode("semantic")}
+              title="Semantic search - finds similar meaning"
+            >
+              <Brain size={16} weight="bold" />
+              <span>Semantic</span>
+            </button>
+          </div>
+        )}

        {/* Search input */}
        <div className="search-modal-input-wrapper">
@@ -223,9 +229,11 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
                  : "Describe what you're looking for"}
              </p>
              <div className="search-modal-shortcuts">
-                <span className="search-shortcut">
-                  <kbd>Tab</kbd> Switch mode
-                </span>
+                {semanticEnabled && (
+                  <span className="search-shortcut">
+                    <kbd>Tab</kbd> Switch mode
+                  </span>
+                )}
                <span className="search-shortcut">
                  <kbd>↑</kbd>
                  <kbd>↓</kbd> Navigate
@@ -292,9 +300,11 @@ export default function SearchModal({ isOpen, onClose }: SearchModalProps) {
        {/* Footer with keyboard hints */}
        {results && results.length > 0 && (
          <div className="search-modal-footer">
-            <span className="search-footer-hint">
-              <kbd>Tab</kbd> switch mode
-            </span>
+            {semanticEnabled && (
+              <span className="search-footer-hint">
+                <kbd>Tab</kbd> switch mode
+              </span>
+            )}
            <span className="search-footer-hint">
              <kbd>↵</kbd> select
            </span>
--- a/src/config/siteConfig.ts
+++ b/src/config/siteConfig.ts
@@ -236,6 +236,13 @@ export interface ImageLightboxConfig {
  enabled: boolean; // Global toggle for image lightbox feature
 }

+// Semantic search configuration
+// Enables AI-powered search using vector embeddings
+// Requires OPENAI_API_KEY environment variable in Convex dashboard
+export interface SemanticSearchConfig {
+  enabled: boolean; // Global toggle for semantic search feature
+}
+
 // Social link configuration for social footer
 export interface SocialLink {
  platform:
@@ -365,6 +372,9 @@ export interface SiteConfig {

  // AI Dashboard configuration (optional)
  aiDashboard?: AIDashboardConfig;
+
+  // Semantic search configuration (optional)
+  semanticSearch?: SemanticSearchConfig;
 }

 // Default site configuration
@@ -744,6 +754,13 @@ export const siteConfig: SiteConfig = {
      },
    ],
  },
+
+  // Semantic search configuration
+  // Set enabled: true to enable semantic search (requires OPENAI_API_KEY in Convex)
+  // When disabled, only keyword search is available (no API key needed)
+  semanticSearch: {
+    enabled: false, // Set to true to enable semantic search (requires OPENAI_API_KEY)
+  },
 };

 // Export the config as default for easy importing
--- a/src/pages/Dashboard.tsx
+++ b/src/pages/Dashboard.tsx
@@ -4597,6 +4597,8 @@ function ConfigSection({
    mcpServerRequireAuth: siteConfig.mcpServer?.requireAuth || false,
    // Image lightbox
    imageLightboxEnabled: siteConfig.imageLightbox?.enabled !== false,
+    // Semantic search
+    semanticSearchEnabled: siteConfig.semanticSearch?.enabled || false,
  });

  const [copied, setCopied] = useState(false);
@@ -4762,6 +4764,12 @@ export const siteConfig: SiteConfig = {
  imageLightbox: {
    enabled: ${config.imageLightboxEnabled},
  },
+
+  // Semantic search configuration
+  // Set enabled: true to enable AI-powered semantic search (requires OPENAI_API_KEY in Convex)
+  semanticSearch: {
+    enabled: ${config.semanticSearchEnabled},
+  },
 };

 export default siteConfig;
@@ -5573,6 +5581,26 @@ export default siteConfig;
          </div>
        </div>

+        {/* Semantic Search */}
+        <div className="dashboard-config-card">
+          <h3>Semantic Search</h3>
+          <div className="config-field checkbox">
+            <label>
+              <input
+                type="checkbox"
+                checked={config.semanticSearchEnabled}
+                onChange={(e) =>
+                  handleChange("semanticSearchEnabled", e.target.checked)
+                }
+              />
+              <span>Enable semantic search (requires OPENAI_API_KEY in Convex)</span>
+            </label>
+          </div>
+          <p className="config-hint">
+            When enabled, search modal shows both Keyword and Semantic modes. Requires OpenAI API key for embeddings.
+          </p>
+        </div>
+
        {/* Links */}
        <div className="dashboard-config-card">
          <h3>External Links</h3>