fix: add edge functions for dynamic RSS, sitemap, and API proxying to Convex HTTP endpoints

This commit is contained in:
Wayne Sutton
2025-12-14 17:00:46 -08:00
parent c6c12ecd58
commit 078fbe6698
13 changed files with 316 additions and 107 deletions

View File

@@ -24,7 +24,7 @@ A minimalist markdown site built with React, Convex, and Vite. Optimized for SEO
- `/api/posts` - JSON list of all posts for agents
- `/api/post?slug=xxx` - Single post JSON or markdown
- `/rss-full.xml` - Full content RSS for LLM ingestion
- Copy Page dropdown for sharing to ChatGPT, Claude, Cursor, VS Code
- Copy Page dropdown for sharing to ChatGPT, Claude
## Getting Started
@@ -202,9 +202,8 @@ npx convex deploy
- Publish directory: `dist`
4. Add environment variable:
- `CONVEX_DEPLOY_KEY` - Generate from [Convex Dashboard](https://dashboard.convex.dev) > Project Settings > Deploy Key
5. Update `netlify.toml` with your production Convex HTTP URL (replace `YOUR_CONVEX_DEPLOYMENT`)
The `CONVEX_DEPLOY_KEY` lets Netlify automatically deploy functions and set `VITE_CONVEX_URL` on each build.
The `CONVEX_DEPLOY_KEY` lets Netlify automatically deploy functions and set `VITE_CONVEX_URL` on each build. Edge functions dynamically proxy RSS, sitemap, and API requests to your Convex HTTP endpoints.
**Build issues?** Netlify sets `NODE_ENV=production` which skips devDependencies. The `--include=dev` flag fixes this. See [netlify-deploy-fix.md](./netlify-deploy-fix.md) for detailed troubleshooting.
@@ -219,6 +218,11 @@ markdown-site/
│ ├── rss.ts # RSS feed generation
│ └── schema.ts # Database schema
├── netlify/ # Netlify edge functions
│ └── edge-functions/
│ ├── rss.ts # RSS feed proxy
│ ├── sitemap.ts # Sitemap proxy
│ ├── api.ts # API endpoint proxy
│ └── botMeta.ts # OG crawler detection
├── public/ # Static assets
│ ├── images/ # Blog images and OG images
│ ├── robots.txt # Crawler rules

10
TASK.md
View File

@@ -2,7 +2,7 @@
## Current Status
v1.0.0 ready for deployment. Build passes. TypeScript verified.
v1.1.0 ready for deployment. Build passes. TypeScript verified.
## Completed
@@ -26,13 +26,15 @@ v1.0.0 ready for deployment. Build passes. TypeScript verified.
- [x] Netlify build configuration verified
- [x] SPA 404 fallback configured
- [x] Mobile responsive design
- [x] Edge functions for dynamic Convex HTTP proxying
- [x] Vite dev server proxy for local development
## Deployment Steps
1. Run `npx convex dev` to initialize Convex
2. Update `netlify.toml` with your Convex deployment URL
3. Set `VITE_CONVEX_URL` in Netlify environment variables
4. Connect repo to Netlify and deploy
2. Set `CONVEX_DEPLOY_KEY` in Netlify environment variables
3. Connect repo to Netlify and deploy
4. Edge functions automatically handle RSS, sitemap, and API routes
## Future Enhancements

View File

@@ -4,6 +4,27 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [1.1.0] - 2025-12-14
### Added
- Netlify Edge Functions for dynamic Convex HTTP proxying
- `rss.ts` proxies `/rss.xml` and `/rss-full.xml`
- `sitemap.ts` proxies `/sitemap.xml`
- `api.ts` proxies `/api/posts` and `/api/post`
- Vite dev server proxy for RSS, sitemap, and API endpoints
### Changed
- Replaced hardcoded Convex URLs in netlify.toml with edge functions
- Edge functions dynamically read `VITE_CONVEX_URL` from environment
- Updated setup guide, docs, and README with edge function documentation
### Fixed
- RSS feeds and sitemap now work without manual URL configuration
- Local development properly proxies API routes to Convex
## [1.0.0] - 2025-12-14
### Added
@@ -25,7 +46,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- API endpoints for LLM access:
- `/api/posts` - JSON list of all posts
- `/api/post?slug=xxx` - Single post as JSON or markdown
- Copy Page dropdown for sharing to ChatGPT, Claude, Cursor, VS Code
- Copy Page dropdown for sharing to ChatGPT, Claude
- Open Graph and Twitter Card meta tags
- Netlify edge function for social media crawler detection
- Build-time markdown sync from `content/blog/` to Convex

View File

@@ -165,30 +165,18 @@ The HTTP URL uses `.convex.site` instead of `.convex.cloud`:
https://happy-animal-123.convex.site
```
## Step 6: Configure Netlify Redirects
## Step 6: Verify Edge Functions
Open `netlify.toml` and replace `YOUR_CONVEX_DEPLOYMENT` with your actual deployment name:
The blog uses Netlify Edge Functions to dynamically proxy RSS, sitemap, and API requests to your Convex HTTP endpoints. No manual URL configuration is needed.
```toml
# Before
[[redirects]]
from = "/rss.xml"
to = "https://YOUR_CONVEX_DEPLOYMENT.convex.site/rss.xml"
Edge functions in `netlify/edge-functions/`:
# After (example)
[[redirects]]
from = "/rss.xml"
to = "https://happy-animal-123.convex.site/rss.xml"
```
- `rss.ts` - Proxies `/rss.xml` and `/rss-full.xml`
- `sitemap.ts` - Proxies `/sitemap.xml`
- `api.ts` - Proxies `/api/posts` and `/api/post`
- `botMeta.ts` - Serves Open Graph HTML to social media crawlers
Update all redirect rules with your deployment name:
- `/rss.xml`
- `/rss-full.xml`
- `/sitemap.xml`
- `/api/posts`
- `/api/post`
- `/meta/post`
These functions automatically read `VITE_CONVEX_URL` from your environment and convert it to the Convex HTTP site URL (`.cloud` becomes `.site`).
## Step 7: Deploy to Netlify
@@ -495,9 +483,10 @@ Your blog includes these API endpoints for search engines and AI:
### RSS/Sitemap not working
1. Verify `netlify.toml` has correct Convex URL
2. Check that Convex functions are deployed
3. Test the Convex HTTP URL directly in browser
1. Verify `VITE_CONVEX_URL` is set in Netlify environment variables
2. Check that Convex HTTP endpoints are deployed (`npx convex deploy`)
3. Test the Convex HTTP URL directly: `https://your-deployment.convex.site/rss.xml`
4. Verify edge functions exist in `netlify/edge-functions/`
### Build failures on Netlify
@@ -545,6 +534,12 @@ markdown-site/
│ ├── posts.ts # Post queries/mutations
│ ├── rss.ts # RSS feed generation
│ └── schema.ts # Database schema
├── netlify/
│ └── edge-functions/ # Netlify edge functions
│ ├── rss.ts # RSS proxy
│ ├── sitemap.ts # Sitemap proxy
│ ├── api.ts # API proxy
│ └── botMeta.ts # OG crawler detection
├── public/ # Static assets
│ ├── robots.txt # Crawler rules
│ └── llms.txt # AI agent discovery

View File

@@ -39,6 +39,12 @@ markdown-site/
│ ├── pages.ts # Page queries/mutations
│ ├── http.ts # API endpoints
│ └── rss.ts # RSS generation
├── netlify/
│ └── edge-functions/ # Netlify edge functions
│ ├── rss.ts # RSS proxy
│ ├── sitemap.ts # Sitemap proxy
│ ├── api.ts # API proxy
│ └── botMeta.ts # OG crawler detection
├── src/
│ ├── components/ # React components
│ ├── context/ # Theme context
@@ -186,9 +192,11 @@ body {
### Netlify setup
1. Connect GitHub repo to Netlify
2. Set build command: `npm run deploy`
3. Set publish directory: `dist`
4. Add env variable: `VITE_CONVEX_URL`
2. Build command: `npm ci --include=dev && npx convex deploy --cmd 'npm run build'`
3. Publish directory: `dist`
4. Add env variable: `CONVEX_DEPLOY_KEY` (from Convex Dashboard)
The build automatically sets `VITE_CONVEX_URL` from the deploy key.
### Convex production
@@ -196,16 +204,9 @@ body {
npx convex deploy
```
### netlify.toml
### Edge functions
Replace `YOUR_CONVEX_DEPLOYMENT` with your deployment name:
```toml
[[redirects]]
from = "/rss.xml"
to = "https://YOUR_DEPLOYMENT.convex.site/rss.xml"
status = 200
```
RSS, sitemap, and API routes are handled by Netlify Edge Functions in `netlify/edge-functions/`. They dynamically read `VITE_CONVEX_URL` from the environment. No manual URL configuration needed.
## Convex schema
@@ -250,12 +251,13 @@ export default defineSchema({
**RSS/Sitemap errors**
- Verify `netlify.toml` Convex URL
- Test Convex HTTP URL directly
- Check Convex function deployment
- Verify `VITE_CONVEX_URL` is set in Netlify
- Test Convex HTTP URL: `https://your-deployment.convex.site/rss.xml`
- Check edge functions in `netlify/edge-functions/`
**Build failures**
- Verify `VITE_CONVEX_URL` is set
- Verify `CONVEX_DEPLOY_KEY` is set in Netlify
- Ensure `@types/node` is in devDependencies
- Build command must include `--include=dev`
- Check Node.js version (18+)
- Review Netlify build logs

View File

@@ -2,9 +2,10 @@ import { httpAction } from "./_generated/server";
import { api } from "./_generated/api";
// Site configuration for RSS feed
const SITE_URL = "https://your-blog.netlify.app";
const SITE_TITLE = "Wayne Sutton";
const SITE_DESCRIPTION = "Developer and writer. Building with Convex and AI.";
const SITE_URL = process.env.SITE_URL || "https://markdowncms.netlify.app";
const SITE_TITLE = "Markdown Site";
const SITE_DESCRIPTION =
"An open source markdown site powered by Convex and Netlify.";
// Escape XML special characters
function escapeXml(text: string): string {

View File

@@ -112,9 +112,12 @@ Markdown files for static pages like About, Projects, Contact.
## Netlify (`netlify/edge-functions/`)
| File | Description |
| ------------ | ------------------------------------------------ |
| `botMeta.ts` | Edge function for social media crawler detection |
| File | Description |
| ------------ | ----------------------------------------------------- |
| `botMeta.ts` | Edge function for social media crawler detection |
| `rss.ts` | Proxies `/rss.xml` and `/rss-full.xml` to Convex HTTP |
| `sitemap.ts` | Proxies `/sitemap.xml` to Convex HTTP |
| `api.ts` | Proxies `/api/posts` and `/api/post` to Convex HTTP |
## Public Assets (`public/`)

View File

@@ -40,10 +40,7 @@
<!-- Twitter Card -->
<meta name="twitter:card" content="summary_large_image" />
<meta property="twitter:domain" content="markdowncms.netlify.app" />
<meta
property="twitter:url"
content="https://markdowncms.netlify.app/"
/>
<meta property="twitter:url" content="https://markdowncms.netlify.app/" />
<meta
name="twitter:title"
content="Markdown Site - Real-time Site with Convex"

View File

@@ -5,54 +5,37 @@
[build.environment]
NODE_VERSION = "20"
# Convex HTTP endpoints
# RSS feeds
[[redirects]]
from = "/rss.xml"
to = "https://agreeable-trout-200.convex.site/rss.xml"
status = 200
force = true
[[redirects]]
from = "/rss-full.xml"
to = "https://agreeable-trout-200.convex.site/rss-full.xml"
status = 200
force = true
# Sitemap for search engines
[[redirects]]
from = "/sitemap.xml"
to = "https://agreeable-trout-200.convex.site/sitemap.xml"
status = 200
force = true
# API endpoints for LLMs and agents
[[redirects]]
from = "/api/posts"
to = "https://agreeable-trout-200.convex.site/api/posts"
status = 200
force = true
[[redirects]]
from = "/api/post"
to = "https://agreeable-trout-200.convex.site/api/post"
status = 200
force = true
# Open Graph metadata endpoint
[[redirects]]
from = "/meta/post"
to = "https://agreeable-trout-200.convex.site/meta/post"
status = 200
force = true
# SPA fallback for client-side routing (must be last)
[[redirects]]
from = "/*"
to = "/index.html"
status = 200
# Edge function for Open Graph bot detection
# Edge functions for dynamic Convex HTTP proxying
# RSS feeds
[[edge_functions]]
path = "/rss.xml"
function = "rss"
[[edge_functions]]
path = "/rss-full.xml"
function = "rss"
# Sitemap
[[edge_functions]]
path = "/sitemap.xml"
function = "sitemap"
# API endpoints
[[edge_functions]]
path = "/api/posts"
function = "api"
[[edge_functions]]
path = "/api/post"
function = "api"
# Open Graph bot detection (catches all other routes)
[[edge_functions]]
path = "/*"
function = "botMeta"

View File

@@ -0,0 +1,61 @@
import type { Context } from "@netlify/edge-functions";
// Edge function to proxy API endpoints to Convex HTTP
export default async function handler(
request: Request,
_context: Context,
): Promise<Response> {
const convexUrl =
Deno.env.get("VITE_CONVEX_URL") || Deno.env.get("CONVEX_URL");
if (!convexUrl) {
return new Response(
JSON.stringify({ error: "Configuration error: Missing Convex URL" }),
{
status: 500,
headers: { "Content-Type": "application/json" },
},
);
}
// Construct the Convex site URL for the HTTP endpoint
const convexSiteUrl = convexUrl.replace(".cloud", ".site");
const url = new URL(request.url);
const targetUrl = `${convexSiteUrl}${url.pathname}${url.search}`;
try {
const response = await fetch(targetUrl, {
headers: {
Accept: "application/json",
},
});
if (!response.ok) {
return new Response(JSON.stringify({ error: "API endpoint error" }), {
status: response.status,
headers: { "Content-Type": "application/json" },
});
}
const data = await response.text();
const contentType =
response.headers.get("Content-Type") || "application/json";
return new Response(data, {
headers: {
"Content-Type": contentType,
"Cache-Control": "public, max-age=300, s-maxage=600",
"Access-Control-Allow-Origin": "*",
},
});
} catch {
return new Response(JSON.stringify({ error: "Failed to fetch from API" }), {
status: 502,
headers: { "Content-Type": "application/json" },
});
}
}
export const config = {
path: ["/api/posts", "/api/post"],
};

View File

@@ -0,0 +1,53 @@
import type { Context } from "@netlify/edge-functions";
// Edge function to proxy RSS feed to Convex HTTP endpoint
export default async function handler(
request: Request,
_context: Context,
): Promise<Response> {
const convexUrl =
Deno.env.get("VITE_CONVEX_URL") || Deno.env.get("CONVEX_URL");
if (!convexUrl) {
return new Response("Configuration error: Missing Convex URL", {
status: 500,
});
}
// Construct the Convex site URL for the HTTP endpoint
const convexSiteUrl = convexUrl.replace(".cloud", ".site");
const url = new URL(request.url);
const targetUrl = `${convexSiteUrl}${url.pathname}`;
try {
const response = await fetch(targetUrl, {
headers: {
Accept: "application/rss+xml",
},
});
if (!response.ok) {
return new Response("RSS feed not available", {
status: response.status,
headers: { "Content-Type": "text/plain" },
});
}
const xml = await response.text();
return new Response(xml, {
headers: {
"Content-Type": "application/rss+xml; charset=utf-8",
"Cache-Control": "public, max-age=3600, s-maxage=7200",
},
});
} catch {
return new Response("Failed to fetch RSS feed", {
status: 502,
headers: { "Content-Type": "text/plain" },
});
}
}
export const config = {
path: ["/rss.xml", "/rss-full.xml"],
};

View File

@@ -0,0 +1,52 @@
import type { Context } from "@netlify/edge-functions";
// Edge function to proxy sitemap to Convex HTTP endpoint
export default async function handler(
request: Request,
_context: Context,
): Promise<Response> {
const convexUrl =
Deno.env.get("VITE_CONVEX_URL") || Deno.env.get("CONVEX_URL");
if (!convexUrl) {
return new Response("Configuration error: Missing Convex URL", {
status: 500,
});
}
// Construct the Convex site URL for the HTTP endpoint
const convexSiteUrl = convexUrl.replace(".cloud", ".site");
const targetUrl = `${convexSiteUrl}/sitemap.xml`;
try {
const response = await fetch(targetUrl, {
headers: {
Accept: "application/xml",
},
});
if (!response.ok) {
return new Response("Sitemap not available", {
status: response.status,
headers: { "Content-Type": "text/plain" },
});
}
const xml = await response.text();
return new Response(xml, {
headers: {
"Content-Type": "application/xml; charset=utf-8",
"Cache-Control": "public, max-age=3600, s-maxage=7200",
},
});
} catch {
return new Response("Failed to fetch sitemap", {
status: 502,
headers: { "Content-Type": "text/plain" },
});
}
}
export const config = {
path: "/sitemap.xml",
};

View File

@@ -1,11 +1,46 @@
import { defineConfig } from "vite";
import { defineConfig, loadEnv } from "vite";
import react from "@vitejs/plugin-react";
// https://vitejs.dev/config/
export default defineConfig({
plugins: [react()],
build: {
outDir: "dist",
},
});
export default defineConfig(({ mode }) => {
const env = loadEnv(mode, process.cwd(), "");
// Convert Convex cloud URL to HTTP site URL
const convexUrl = env.VITE_CONVEX_URL || "";
const convexSiteUrl = convexUrl.replace(".cloud", ".site");
return {
plugins: [react()],
build: {
outDir: "dist",
},
server: {
proxy: {
// Proxy RSS and sitemap to Convex HTTP endpoints in development
"/rss.xml": {
target: convexSiteUrl,
changeOrigin: true,
},
"/rss-full.xml": {
target: convexSiteUrl,
changeOrigin: true,
},
"/sitemap.xml": {
target: convexSiteUrl,
changeOrigin: true,
},
"/api/posts": {
target: convexSiteUrl,
changeOrigin: true,
},
"/api/post": {
target: convexSiteUrl,
changeOrigin: true,
},
"/meta/post": {
target: convexSiteUrl,
changeOrigin: true,
},
},
},
};
});