x/wiki
1
0
Fork 0
mirror of https://github.com/waynesutton/markdown-site.git synced 2026-01-12 04:09:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: Add semantic search with vector embeddings

Browse Source 
Add vector-based semantic search to complement keyword search.
  Users can toggle between "Keyword" and "Semantic" modes in the
  search modal (Cmd+K, then Tab to switch).

  Semantic search:
  - Uses OpenAI text-embedding-ada-002 (1536 dimensions)
  - Finds content by meaning, not exact words
  - Shows similarity scores as percentages
  - ~300ms latency, ~$0.0001/query
  - Graceful fallback if OPENAI_API_KEY not set

  New files:
  - convex/embeddings.ts - Embedding generation actions
  - convex/embeddingsQueries.ts - Queries/mutations for embeddings
  - convex/semanticSearch.ts - Vector search action
  - convex/semanticSearchQueries.ts - Result hydration queries
  - content/pages/docs-search.md - Keyword search docs
  - content/pages/docs-semantic-search.md - Semantic search docs

  Changes:
  - convex/schema.ts: Add embedding field and by_embedding vectorIndex
  - SearchModal.tsx: Add mode toggle (TextAa/Brain icons)
  - sync-posts.ts: Generate embeddings after content sync
  - global.css: Search mode toggle styles

  Documentation updated:
  - changelog.md, TASK.md, files.md, about.md, home.md

  Configuration:
  npx convex env set OPENAI_API_KEY sk-your-key

  Generated with [Claude Code](https://claude.com/claude-code)

  Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

  Status: Ready to commit. All semantic search files are staged. The TypeScript warnings are pre-existing (unused variables) and don't affect the build.
This commit is contained in:
Wayne Sutton
2026-01-05 18:30:48 -08:00
parent 83411ec1b2
commit 5a8df46681
58 changed files with 7024 additions and 2527 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
package.json
View File

@@ -16,6 +16,8 @@
"sync:discovery:prod": "SYNC_ENV=production npx tsx scripts/sync-discovery-files.ts",
"sync:all": "npm run sync && npm run sync:discovery",
"sync:all:prod": "npm run sync:prod && npm run sync:discovery:prod",
"export:db": "npx tsx scripts/export-db-posts.ts",
"export:db:prod": "SYNC_ENV=production npx tsx scripts/export-db-posts.ts",
"import": "npx tsx scripts/import-url.ts",
"configure": "npx tsx scripts/configure-fork.ts",
"newsletter:send": "npx tsx scripts/send-newsletter.ts",
@@ -26,9 +28,9 @@
},
"dependencies": {
"@anthropic-ai/sdk": "^0.71.2",
"@google/genai": "^1.0.1",
"@convex-dev/aggregate": "^0.2.0",
"@convex-dev/workos": "^0.0.1",
"@google/genai": "^1.0.1",
"@mendable/firecrawl-js": "^1.21.1",
"@modelcontextprotocol/sdk": "^1.0.0",
"@phosphor-icons/react": "^2.1.10",
@@ -44,18 +46,23 @@
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-markdown": "^9.0.1",
"react-quill": "^2.0.0",
"react-router-dom": "^6.22.0",
"react-syntax-highlighter": "^15.5.0",
"rehype-raw": "^7.0.0",
"rehype-sanitize": "^6.0.0",
"remark-breaks": "^4.0.0",
"remark-gfm": "^4.0.0"
"remark-gfm": "^4.0.0",
"showdown": "^2.1.0",
"turndown": "^7.2.2"
},
"devDependencies": {
"@types/node": "^25.0.2",
"@types/react": "^18.2.56",
"@types/react-dom": "^18.2.19",
"@types/react-syntax-highlighter": "^15.5.11",
"@types/showdown": "^2.0.6",
"@types/turndown": "^5.0.6",
"@typescript-eslint/eslint-plugin": "^7.0.2",
"@typescript-eslint/parser": "^7.0.2",
"@vitejs/plugin-react": "^4.2.1",