feat: markdown sync v2 complete

Complete markdown synchronization system with dashboard UI, sync server, type safety, and security improvements.
This commit is contained in:
Wayne Sutton
2025-12-30 00:29:07 -08:00
parent 02bc3c2973
commit 689951d344
13 changed files with 523 additions and 15 deletions

View File

@@ -0,0 +1,288 @@
---
name: Admin
overview: Build a comprehensive admin dashboard at /dashboard for managing posts, pages, site configuration, newsletter, imports, stats, and sync operations. Uses existing UI patterns from Write.tsx and NewsletterAdmin.tsx with a config generator approach for siteConfig.ts and downloadable markdown for content editing.
todos: []
---
# Admin Dashboard Implementation Plan
## Architecture Overview
```mermaid
flowchart TB
subgraph dashboard [Dashboard /dashboard]
LeftSidebar[Left Sidebar Nav]
Header[Header with Sync/Search]
MainContent[Main Content Area]
RightSidebar[Right Sidebar - Editor Only]
end
subgraph sections [Dashboard Sections]
Posts[Posts List/Edit]
Pages[Pages List/Edit]
Write[Write Clone]
Agent[AI Agent]
Newsletter[Newsletter Admin]
Import[Firecrawl Import]
Config[Config Generator]
Stats[Stats Clone]
Sync[Sync Commands]
end
LeftSidebar --> sections
MainContent --> sections
```
## Key Decisions
| Feature | Approach | Rationale |
| ---------------- | ------------------------------ | ------------------------------------------------------ |
| Content Editing | Generate downloadable markdown | Keeps markdown files as source of truth, safe workflow |
| Config Editing | Config generator UI | Visual form outputs TypeScript code for siteConfig.ts |
| Newsletter Admin | Move into dashboard | Consolidates admin features |
| Sync Commands | Copy-to-clipboard buttons | Safe, uses existing CLI scripts |
---
## Phase 1: Dashboard Foundation
**Goal:** Create the base dashboard layout with navigation structure
**Files to create/modify:**
- Create [`src/pages/Dashboard.tsx`](src/pages/Dashboard.tsx) - Main dashboard component
- Create [`src/components/dashboard/DashboardLayout.tsx`](src/components/dashboard/DashboardLayout.tsx) - Layout wrapper
- Create [`src/components/dashboard/DashboardSidebar.tsx`](src/components/dashboard/DashboardSidebar.tsx) - Left sidebar navigation
- Create [`src/components/dashboard/DashboardHeader.tsx`](src/components/dashboard/DashboardHeader.tsx) - Header with sync buttons and search
- Modify [`src/App.tsx`](src/App.tsx) - Add /dashboard route
- Add dashboard styles to [`src/styles/global.css`](src/styles/global.css)
**Key patterns to follow:**
- Use existing Write.tsx sidebar structure as template
- Use Phosphor icons (primary), Lucide (backup)
- Match existing theme CSS variables
---
## Phase 2: Posts & Pages List Views
**Goal:** WordPress-style list views with edit/view/publish actions
**Files to create/modify:**
- Create [`src/components/dashboard/PostsList.tsx`](src/components/dashboard/PostsList.tsx) - Posts list with actions
- Create [`src/components/dashboard/PagesList.tsx`](src/components/dashboard/PagesList.tsx) - Pages list with actions
- Create [`convex/dashboard.ts`](convex/dashboard.ts) - Dashboard queries (getAllPostsAdmin, getAllPagesAdmin, togglePublished)
**Features:**
- Table/list view with columns: Title, Date, Edit, View, Published
- Click row to edit
- Pagination (Prev/Next)
- Published toggle (updates Convex, requires re-sync to persist to markdown)
---
## Phase 3: Post/Page Editor View
**Goal:** Markdown editor with live preview and frontmatter sidebar
**Files to create/modify:**
- Create [`src/components/dashboard/ContentEditor.tsx`](src/components/dashboard/ContentEditor.tsx) - Main editor component
- Create [`src/components/dashboard/MarkdownPreview.tsx`](src/components/dashboard/MarkdownPreview.tsx) - Live preview renderer
- Create [`src/components/dashboard/FrontmatterEditor.tsx`](src/components/dashboard/FrontmatterEditor.tsx) - Right sidebar form
- Create [`src/utils/frontmatterParser.ts`](src/utils/frontmatterParser.ts) - Parse/generate frontmatter
**Features:**
- Toggle between "Markdown view" and "Live view"
- Right sidebar with all frontmatter fields (from POST_FIELDS/PAGE_FIELDS in Write.tsx)
- "Copy Markdown" button - copies full file with frontmatter
- "Download File" button - downloads as .md file
- Image path helper (shows available images in /public/images)
---
## Phase 4: Config Generator
**Goal:** Visual form to generate siteConfig.ts code
**Files to create/modify:**
- Create [`src/components/dashboard/ConfigGenerator.tsx`](src/components/dashboard/ConfigGenerator.tsx) - Main config form
- Create [`src/components/dashboard/ConfigSection.tsx`](src/components/dashboard/ConfigSection.tsx) - Reusable section component
- Create [`src/utils/configCodeGenerator.ts`](src/utils/configCodeGenerator.ts) - Generate TypeScript code
**Features:**
- Form sections for each siteConfig area (basic info, blog page, footer, newsletter, etc.)
- Pre-populated with current siteConfig values
- Live TypeScript code preview
- "Copy Config" button
- Instructions for saving to siteConfig.ts
---
## Phase 5: Newsletter Admin Migration
**Goal:** Move NewsletterAdmin.tsx functionality into dashboard
**Files to modify:**
- Integrate existing [`src/pages/NewsletterAdmin.tsx`](src/pages/NewsletterAdmin.tsx) features
- Create [`src/components/dashboard/NewsletterSection.tsx`](src/components/dashboard/NewsletterSection.tsx) - Container for newsletter features
**Subsections:**
- Subscribers (All, Active, Unsubscribed)
- Send Post
- Write Email
- Recent Sends
- Email Stats
---
## Phase 6: Import UI (Firecrawl)
**Goal:** UI for importing external URLs as markdown posts
**Files to create/modify:**
- Create [`src/components/dashboard/ImportSection.tsx`](src/components/dashboard/ImportSection.tsx) - Import UI
- Create [`convex/import.ts`](convex/import.ts) - Import action using Firecrawl
**Features:**
- URL input field
- "Import" button triggers Firecrawl scrape
- Preview imported content with generated frontmatter
- Edit before copying/downloading
- Uses existing pattern from `scripts/import-url.ts`
---
## Phase 7: Stats Clone
**Goal:** Dashboard version of Stats page
**Files to create/modify:**
- Create [`src/components/dashboard/StatsSection.tsx`](src/components/dashboard/StatsSection.tsx) - Stats display
**Features:**
- Clone Stats.tsx functionality
- Same real-time stats (active visitors, total views, etc.)
- Visitor map (if enabled)
- Does not follow siteConfig.statsPage settings (always visible in dashboard)
---
## Phase 8: Sync Section
**Goal:** UI for all sync commands
**Files to create/modify:**
- Create [`src/components/dashboard/SyncSection.tsx`](src/components/dashboard/SyncSection.tsx) - Sync commands UI
**Commands to expose:**
- `npm run sync` - Sync to dev
- `npm run sync:prod` - Sync to production
- `npm run sync:all` - Sync posts, pages, and discovery files (dev)
- `npm run sync:all:prod` - Sync all to production
**Features:**
- Button for each command (copies to clipboard)
- Description of what each command does
- Link to terminal instructions
---
## Phase 9: Write & Agent Clone
**Goal:** Clone Write.tsx and AI Agent into dashboard sections
**Files to create/modify:**
- Create [`src/components/dashboard/WriteSection.tsx`](src/components/dashboard/WriteSection.tsx) - Write page clone
- Create [`src/components/dashboard/AgentSection.tsx`](src/components/dashboard/AgentSection.tsx) - AI chat section
**Features:**
- Write section: Full Write.tsx functionality (template generation, frontmatter reference)
- Agent section: AIChatView with "write-dashboard" context
- Both work independently from main /write page
---
## Phase 10: Search & Polish
**Goal:** Dashboard search, mobile optimization, final polish
**Files to modify:**
- Add search functionality to DashboardHeader
- Ensure all dashboard components are mobile responsive
- Run TypeScript type checks
- Test all themes (dark, light, tan, cloud)
**Search scope:**
- Post/page titles and content
- Dashboard section names
- Config field names
---
## File Structure Summary
```
src/
├── pages/
│ └── Dashboard.tsx # Main dashboard page
├── components/
│ └── dashboard/
│ ├── DashboardLayout.tsx # Layout wrapper
│ ├── DashboardSidebar.tsx # Left nav
│ ├── DashboardHeader.tsx # Header with sync/search
│ ├── PostsList.tsx # Posts list view
│ ├── PagesList.tsx # Pages list view
│ ├── ContentEditor.tsx # Markdown editor
│ ├── MarkdownPreview.tsx # Live preview
│ ├── FrontmatterEditor.tsx # Right sidebar form
│ ├── ConfigGenerator.tsx # Config UI
│ ├── ConfigSection.tsx # Config form section
│ ├── NewsletterSection.tsx # Newsletter admin
│ ├── ImportSection.tsx # Firecrawl import
│ ├── StatsSection.tsx # Stats clone
│ ├── SyncSection.tsx # Sync commands
│ ├── WriteSection.tsx # Write clone
│ └── AgentSection.tsx # AI chat
├── utils/
│ ├── frontmatterParser.ts # Frontmatter utilities
│ └── configCodeGenerator.ts # Config code generation
convex/
├── dashboard.ts # Dashboard queries/mutations
└── import.ts # Firecrawl import action
```
---
## Implementation Notes
1. **No breaking changes** - Existing pages (/write, /newsletter-admin, /stats) continue to work
2. **Convex patterns** - Use indexed queries, idempotent mutations, avoid write conflicts
3. **Theme support** - All components use CSS variables from global.css
4. **Mobile first** - All components responsive across breakpoints
5. **Type safety** - Full TypeScript with Convex validators
6. **Login placeholder** - Bottom left link, no auth implementation yet

View File

@@ -108,6 +108,7 @@ export const siteConfig: SiteConfig = {
// Featured section
featuredViewMode: "cards", // 'list' or 'cards'
featuredTitle: "Get started:", // Featured section title (e.g., "Get started:", "Featured", "Popular")
showViewToggle: true,
// Logo gallery (set enabled: false to hide)
@@ -175,6 +176,26 @@ export const siteConfig: SiteConfig = {
branch: "main", // Default branch
contentPath: "public/raw", // Path to raw markdown files
},
// Stats page configuration (optional)
statsPage: {
enabled: true, // Global toggle for stats page
showInNav: true, // Show link in navigation (controlled via hardcodedNavItems)
},
// Image lightbox configuration (optional)
imageLightbox: {
enabled: true, // Enable click-to-magnify for images in posts/pages
},
// MCP Server configuration (optional)
mcpServer: {
enabled: true, // Global toggle for MCP server
endpoint: "/mcp", // Endpoint path
publicRateLimit: 50, // Requests per minute for public access
authenticatedRateLimit: 1000, // Requests per minute with API key
requireAuth: false, // Require API key for all requests
},
};
```
@@ -711,6 +732,141 @@ See [How to setup WorkOS](https://www.markdown.fast/how-to-setup-workos) for com
See [How to use the Markdown sync dashboard](https://www.markdown.fast/how-to-use-the-markdown-sync-dashboard) for complete usage guide.
### Dashboard Sync Server
The dashboard includes a sync server feature that allows executing sync commands directly from the browser UI without opening a terminal.
**Setup:**
1. Start the sync server locally:
```bash
npm run sync-server
```
2. The server runs on `localhost:3001` and is automatically detected by the dashboard
3. Optional: Set `SYNC_TOKEN` environment variable for authentication
**Features:**
- Execute sync commands from dashboard UI
- Real-time output streaming in dashboard terminal view
- Server status indicator (online/offline)
- Whitelisted commands only (sync, sync:prod, sync:discovery, sync:discovery:prod, sync:all, sync:all:prod)
---
## Stats Page Configuration
Control access to the `/stats` route for viewing site analytics.
### In fork-config.json
```json
{
"statsPage": {
"enabled": true,
"showInNav": true
}
}
```
### Manual Configuration
In `src/config/siteConfig.ts`:
```typescript
statsPage: {
enabled: true, // Global toggle for stats page
showInNav: true, // Show link in navigation (controlled via hardcodedNavItems)
},
```
**Note:** Navigation visibility is controlled via `hardcodedNavItems` configuration. Set `showInNav: false` on the stats nav item to hide it.
---
## Image Lightbox Configuration
Enable click-to-magnify functionality for images in blog posts and pages.
### In fork-config.json
```json
{
"imageLightbox": {
"enabled": true
}
}
```
### Manual Configuration
In `src/config/siteConfig.ts`:
```typescript
imageLightbox: {
enabled: true, // Enable click-to-magnify for images
},
```
**Features:**
- Click any image in a post/page to open in full-screen lightbox
- Dark backdrop with close button (X icon)
- Keyboard support: Press Escape to close
- Click outside image (backdrop) to close
- Alt text displayed as caption below image
- Images show pointer cursor (`zoom-in`) when enabled
---
## MCP Server Configuration
HTTP-based Model Context Protocol server for AI tool integration (Cursor, Claude Desktop).
### In fork-config.json
```json
{
"mcpServer": {
"enabled": true,
"endpoint": "/mcp",
"publicRateLimit": 50,
"authenticatedRateLimit": 1000,
"requireAuth": false
}
}
```
### Manual Configuration
In `src/config/siteConfig.ts`:
```typescript
mcpServer: {
enabled: true, // Global toggle for MCP server
endpoint: "/mcp", // Endpoint path
publicRateLimit: 50, // Requests per minute for public access
authenticatedRateLimit: 1000, // Requests per minute with API key
requireAuth: false, // Require API key for all requests
},
```
**Environment Variables:**
Set `MCP_API_KEY` in Netlify environment variables for authenticated access.
**Features:**
- Accessible 24/7 at `https://yoursite.com/mcp`
- Public access with Netlify built-in rate limiting (50 req/min per IP)
- Optional API key authentication for higher limits (1000 req/min)
- Read-only access to blog posts, pages, homepage, and search
- 7 tools: `list_posts`, `get_post`, `list_pages`, `get_page`, `get_homepage`, `search_content`, `export_all`
- JSON-RPC 2.0 protocol over HTTP POST
See [How to Use the MCP Server](https://www.markdown.fast/how-to-use-mcp-server) for client configuration examples.
---
## Contact Form Configuration

View File

@@ -4,6 +4,25 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [2.0.0] - 2025-12-29
### Added
- Markdown sync v2 complete
- Full markdown content synchronization system
- Real-time sync from markdown files to Convex database
- Dashboard UI for content management
- Sync server for executing sync commands from UI
- Complete type safety across all Convex functions
- Security improvements and optimizations
### Technical
- Optimized `recordPageView` mutation to reduce unnecessary reads
- All mutations follow Convex best practices for write conflict prevention
- Type-safe Convex functions with proper validators
- Security review completed with all endpoints properly secured
## [1.47.0] - 2025-12-29
### Added

View File

@@ -1,5 +1,5 @@
---
title: "How to setup WorkOS"
title: "How to setup WorkOS with Markdown Sync"
description: "Step-by-step guide to configure WorkOS AuthKit authentication for your markdown blog dashboard. WorkOS is optional and can be enabled in siteConfig.ts."
date: "2025-12-29"
slug: "how-to-setup-workos"

View File

@@ -1,5 +1,5 @@
---
title: "How to use Firecrawl"
title: "How to use Firecrawl with Markdown Sync"
description: "Import external articles as markdown posts using Firecrawl. Get your API key and configure environment variables for local imports and AI chat."
date: "2025-12-26"
slug: "how-to-use-firecrawl"

View File

@@ -9,6 +9,26 @@ layout: "sidebar"
All notable changes to this project.
![](https://img.shields.io/badge/License-MIT-yellow.svg)
## v2.0.0
Released December 29, 2025
**Markdown sync v2 complete**
- Full markdown content synchronization system
- Real-time sync from markdown files to Convex database
- Dashboard UI for content management
- Sync server for executing sync commands from UI
- Complete type safety across all Convex functions
- Security improvements and optimizations
**Technical details:**
- Optimized `recordPageView` mutation to reduce unnecessary reads
- All mutations follow Convex best practices for write conflict prevention
- Type-safe Convex functions with proper validators
- Security review completed with all endpoints properly secured
## v1.47.0
Released December 29, 2025

View File

@@ -97,17 +97,20 @@ export const recordPageView = mutation({
sessionId: args.sessionId,
timestamp: now,
});
// Get document for aggregate components (required for insertIfDoesNotExist)
const doc = await ctx.db.get(id);
if (!doc) {
return null;
}
// Update aggregates with the new page view
if (doc) {
await pageViewsByPath.insertIfDoesNotExist(ctx, doc);
await totalPageViews.insertIfDoesNotExist(ctx, doc);
// Only insert into unique visitors aggregate if this is a new session
if (isNewVisitor) {
await uniqueVisitors.insertIfDoesNotExist(ctx, doc);
}
}
return null;
},

View File

@@ -201,15 +201,17 @@ Markdown files for static pages like About, Projects, Contact, Changelog.
## Scripts (`scripts/`)
**Markdown sync v2 complete** - Full markdown content synchronization system with real-time sync from markdown files to Convex database, dashboard UI for content management, and sync server for executing sync commands from UI.
| File | Description |
| ------------------------- | ----------------------------------------------------- |
| `sync-posts.ts` | Syncs markdown files to Convex at build time |
| `sync-posts.ts` | Syncs markdown files to Convex at build time (markdown sync v2) |
| `sync-discovery-files.ts` | Updates AGENTS.md and llms.txt with current app data |
| `import-url.ts` | Imports external URLs as markdown posts (Firecrawl) |
| `configure-fork.ts` | Automated fork configuration (reads fork-config.json) |
| `send-newsletter.ts` | CLI tool for sending newsletter posts (npm run newsletter:send <slug>). Calls scheduleSendPostNewsletter mutation directly. |
| `send-newsletter-stats.ts` | CLI tool for sending weekly stats summary (npm run newsletter:send:stats). Calls scheduleSendStatsSummary mutation directly. |
| `sync-server.ts` | Local HTTP server for executing sync commands from Dashboard UI. Runs on localhost:3001 with optional token authentication. Whitelisted commands only. |
| `sync-server.ts` | Local HTTP server for executing sync commands from Dashboard UI. Runs on localhost:3001 with optional token authentication. Whitelisted commands only. Part of markdown sync v2. |
### Sync Commands

BIN
public/images/templates.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 422 KiB

View File

@@ -8,6 +8,26 @@ Date: 2025-12-30
All notable changes to this project.
![](https://img.shields.io/badge/License-MIT-yellow.svg)
## v2.0.0
Released December 29, 2025
**Markdown sync v2 complete**
- Full markdown content synchronization system
- Real-time sync from markdown files to Convex database
- Dashboard UI for content management
- Sync server for executing sync commands from UI
- Complete type safety across all Convex functions
- Security improvements and optimizations
**Technical details:**
- Optimized `recordPageView` mutation to reduce unnecessary reads
- All mutations follow Convex best practices for write conflict prevention
- Type-safe Convex functions with proper validators
- Security review completed with all endpoints properly secured
## v1.47.0
Released December 29, 2025

View File

@@ -1,4 +1,4 @@
# How to setup WorkOS
# How to setup WorkOS with Markdown Sync
> Step-by-step guide to configure WorkOS AuthKit authentication for your markdown blog dashboard. WorkOS is optional and can be enabled in siteConfig.ts.

View File

@@ -1,4 +1,4 @@
# How to use Firecrawl
# How to use Firecrawl with Markdown Sync
> Import external articles as markdown posts using Firecrawl. Get your API key and configure environment variables for local imports and AI chat.

View File

@@ -4,7 +4,7 @@ This is the homepage index of all published content.
## Blog Posts (16)
- **[How to setup WorkOS](/raw/how-to-setup-workos.md)** - Step-by-step guide to configure WorkOS AuthKit authentication for your markdown blog dashboard. WorkOS is optional and can be enabled in siteConfig.ts.
- **[How to setup WorkOS with Markdown Sync](/raw/how-to-setup-workos.md)** - Step-by-step guide to configure WorkOS AuthKit authentication for your markdown blog dashboard. WorkOS is optional and can be enabled in siteConfig.ts.
- Date: 2025-12-29 | Reading time: 10 min read | Tags: workos, authentication, tutorial, dashboard
- **[How to use the Markdown sync dashboard](/raw/how-to-use-the-markdown-sync-dashboard.md)** - Learn how to use the dashboard at /dashboard to manage content, configure your site, and sync markdown files without leaving your browser.
- Date: 2025-12-29 | Reading time: 8 min read | Tags: dashboard, tutorial, content-management
@@ -12,7 +12,7 @@ This is the homepage index of all published content.
- Date: 2025-12-28 | Reading time: 5 min read | Tags: mcp, cursor, ai, tutorial, netlify
- **[How to use AgentMail with Markdown Sync](/raw/how-to-use-agentmail.md)** - Complete guide to setting up AgentMail for newsletters and contact forms in your markdown blog
- Date: 2025-12-27 | Reading time: 6 min read | Tags: agentmail, newsletter, email, setup
- **[How to use Firecrawl](/raw/how-to-use-firecrawl.md)** - Import external articles as markdown posts using Firecrawl. Get your API key and configure environment variables for local imports and AI chat.
- **[How to use Firecrawl with Markdown Sync](/raw/how-to-use-firecrawl.md)** - Import external articles as markdown posts using Firecrawl. Get your API key and configure environment variables for local imports and AI chat.
- Date: 2025-12-26 | Reading time: 2 min read | Tags: tutorial, firecrawl, import
- **[Happy holidays and thank you](/raw/happy-holidays-2025.md)** - A quick note of thanks for stars, forks, and feedback. More AI-first publishing features coming in 2026.
- Date: 2025-12-25 | Reading time: 2 min read | Tags: updates, community, ai