fix(stats): resolve write conflicts in activeSessions table

- Add 10-second dedup window to heartbeat mutation for idempotency - Add frontend debouncing (5s) with refs to prevent duplicate calls - Track pending heartbeat state to block overlapping async calls - Early return when session already updated with same path Backend (convex/stats.ts): - Add HEARTBEAT_DEDUP_MS constant (10 seconds) - Check if session was recently updated before patching Frontend (src/hooks/usePageTracking.ts): - Add isHeartbeatPending, lastHeartbeatTime, lastHeartbeatPath refs - Wrap sendHeartbeat in useCallback with debounce logic - Import useCallback from React Fixes write conflict retries shown in Convex dashboard for stats:heartbeat
2026-01-12 04:09:14 +00:00 · 2025-12-15 12:52:34 -08:00
parent b280cb4605
commit 97081dc82d
5 changed files with 641 additions and 29 deletions
--- a/prds/howstatsworks.md
+++ b/prds/howstatsworks.md
@@ -0,0 +1,284 @@
+# How stats work
+
+This document explains the real-time analytics system for the markdown site.
+
+## Overview
+
+The stats page at `/stats` shows live visitor data and page view counts. All stats update automatically via Convex subscriptions. No page refresh required.
+
+## Data flow
+
+1. Visitor loads any page
+2. `usePageTracking` hook fires on mount and path change
+3. Page view event recorded to `pageViews` table
+4. Session heartbeat sent to `activeSessions` table
+5. Stats page queries both tables and displays results
+
+## Database tables
+
+### pageViews
+
+Stores individual view events. Uses the event records pattern to avoid write conflicts.
+
+| Field | Type | Purpose |
+|-------|------|---------|
+| path | string | URL path visited |
+| pageType | string | "home", "blog", "page", or "stats" |
+| sessionId | string | Anonymous UUID per browser |
+| timestamp | number | Unix timestamp of visit |
+
+Indexes:
+- `by_path` for filtering by page
+- `by_timestamp` for ordering
+- `by_session_path` for deduplication
+
+### activeSessions
+
+Tracks who is currently on the site. Sessions expire after 2 minutes without a heartbeat.
+
+| Field | Type | Purpose |
+|-------|------|---------|
+| sessionId | string | Anonymous UUID per browser |
+| currentPath | string | Page visitor is currently viewing |
+| lastSeen | number | Last heartbeat timestamp |
+
+Indexes:
+- `by_sessionId` for upserts
+- `by_lastSeen` for cleanup queries
+
+## Frontend tracking
+
+The `usePageTracking` hook in `src/hooks/usePageTracking.ts` handles all client-side tracking.
+
+### Session ID generation
+
+Each browser gets a persistent UUID stored in localStorage. No cookies, no PII.
+
+```typescript
+const SESSION_ID_KEY = "markdown_blog_session_id";
+
+function getSessionId(): string {
+  let sessionId = localStorage.getItem(SESSION_ID_KEY);
+  if (!sessionId) {
+    sessionId = generateSessionId();
+    localStorage.setItem(SESSION_ID_KEY, sessionId);
+  }
+  return sessionId;
+}
+```
+
+### Page view recording
+
+Records a view when the URL path changes. Deduplication happens server-side.
+
+```typescript
+useEffect(() => {
+  const path = location.pathname;
+  if (lastRecordedPath.current !== path) {
+    lastRecordedPath.current = path;
+    recordPageView({ path, pageType: getPageType(path), sessionId });
+  }
+}, [location.pathname, recordPageView]);
+```
+
+### Heartbeat system
+
+Sends a ping every 30 seconds while the page is open. This powers the "Active Now" count.
+
+```typescript
+const HEARTBEAT_INTERVAL_MS = 30 * 1000;
+
+useEffect(() => {
+  const sendHeartbeat = () => {
+    heartbeat({ sessionId, currentPath: path });
+  };
+  sendHeartbeat();
+  const intervalId = setInterval(sendHeartbeat, HEARTBEAT_INTERVAL_MS);
+  return () => clearInterval(intervalId);
+}, [location.pathname, heartbeat]);
+```
+
+## Backend mutations
+
+### recordPageView
+
+Located in `convex/stats.ts`. Records view events with deduplication.
+
+Deduplication window: 30 minutes. Same session viewing same path within 30 minutes counts as 1 view.
+
+```typescript
+export const recordPageView = mutation({
+  args: {
+    path: v.string(),
+    pageType: v.string(),
+    sessionId: v.string(),
+  },
+  returns: v.null(),
+  handler: async (ctx, args) => {
+    const dedupCutoff = Date.now() - DEDUP_WINDOW_MS;
+
+    const recentView = await ctx.db
+      .query("pageViews")
+      .withIndex("by_session_path", (q) =>
+        q.eq("sessionId", args.sessionId).eq("path", args.path)
+      )
+      .order("desc")
+      .first();
+
+    if (recentView && recentView.timestamp > dedupCutoff) {
+      return null;
+    }
+
+    await ctx.db.insert("pageViews", {
+      path: args.path,
+      pageType: args.pageType,
+      sessionId: args.sessionId,
+      timestamp: Date.now(),
+    });
+
+    return null;
+  },
+});
+```
+
+### heartbeat
+
+Creates or updates a session record. Uses indexed lookup for upsert.
+
+```typescript
+export const heartbeat = mutation({
+  args: {
+    sessionId: v.string(),
+    currentPath: v.string(),
+  },
+  returns: v.null(),
+  handler: async (ctx, args) => {
+    const existingSession = await ctx.db
+      .query("activeSessions")
+      .withIndex("by_sessionId", (q) => q.eq("sessionId", args.sessionId))
+      .first();
+
+    if (existingSession) {
+      await ctx.db.patch(existingSession._id, {
+        currentPath: args.currentPath,
+        lastSeen: Date.now(),
+      });
+    } else {
+      await ctx.db.insert("activeSessions", {
+        sessionId: args.sessionId,
+        currentPath: args.currentPath,
+        lastSeen: Date.now(),
+      });
+    }
+
+    return null;
+  },
+});
+```
+
+## Backend query
+
+### getStats
+
+Returns all stats for the `/stats` page. Single query, real-time subscription.
+
+What it returns:
+
+| Field | Type | Description |
+|-------|------|-------------|
+| activeVisitors | number | Sessions with heartbeat in last 2 minutes |
+| activeByPath | array | Breakdown of active visitors by current page |
+| totalPageViews | number | All recorded views since tracking started |
+| uniqueVisitors | number | Count of distinct session IDs |
+| publishedPosts | number | Blog posts with `published: true` |
+| publishedPages | number | Static pages with `published: true` |
+| trackingSince | number or null | Timestamp of earliest view event |
+| pageStats | array | Views per page with title and type |
+
+### Title matching
+
+The query matches URL paths to post/page titles by slug:
+
+```typescript
+const slug = path.startsWith("/") ? path.slice(1) : path;
+const post = posts.find((p) => p.slug === slug);
+const page = pages.find((p) => p.slug === slug);
+
+if (post) {
+  title = post.title;
+  pageType = "blog";
+} else if (page) {
+  title = page.title;
+  pageType = "page";
+}
+```
+
+## Cleanup cron
+
+Stale sessions are cleaned up every 5 minutes via cron job in `convex/crons.ts`.
+
+```typescript
+crons.interval(
+  "cleanup stale sessions",
+  { minutes: 5 },
+  internal.stats.cleanupStaleSessions,
+  {}
+);
+```
+
+The cleanup mutation deletes sessions older than 2 minutes:
+
+```typescript
+const cutoff = Date.now() - SESSION_TIMEOUT_MS;
+const staleSessions = await ctx.db
+  .query("activeSessions")
+  .withIndex("by_lastSeen", (q) => q.lt("lastSeen", cutoff))
+  .collect();
+
+await Promise.all(staleSessions.map((s) => ctx.db.delete(s._id)));
+```
+
+## How new content appears in stats
+
+When you add a new markdown post or page and sync it to Convex:
+
+1. **Post/page counts update instantly.** The `publishedPosts` and `publishedPages` values come from live queries to the `posts` and `pages` tables.
+
+2. **Views appear after first visit.** A page only shows in "Views by Page" after someone visits it.
+
+3. **Titles resolve automatically.** The `getStats` query matches paths to slugs, so new content gets its proper title displayed.
+
+No manual configuration required. Sync content, and stats track it.
+
+## Privacy
+
+- No cookies
+- No PII stored
+- Session IDs are random UUIDs
+- No IP addresses logged
+- No fingerprinting
+- Data stays in your Convex deployment
+
+## Configuration constants
+
+| Constant | Value | Location |
+|----------|-------|----------|
+| DEDUP_WINDOW_MS | 30 minutes | convex/stats.ts |
+| SESSION_TIMEOUT_MS | 2 minutes | convex/stats.ts |
+| HEARTBEAT_INTERVAL_MS | 30 seconds | src/hooks/usePageTracking.ts |
+
+## Files involved
+
+| File | Purpose |
+|------|---------|
+| `convex/stats.ts` | All stats mutations and queries |
+| `convex/schema.ts` | Table definitions for pageViews and activeSessions |
+| `convex/crons.ts` | Scheduled cleanup job |
+| `src/hooks/usePageTracking.ts` | Client-side tracking hook |
+| `src/pages/Stats.tsx` | Stats page UI |
+
+## Related documentation
+
+- [Convex event records pattern](https://docs.convex.dev/understanding/best-practices/)
+- [Preventing write conflicts](https://docs.convex.dev/error#1)
+
--- a/prds/howtoavoidwriteconflicts.md
+++ b/prds/howtoavoidwriteconflicts.md
@@ -0,0 +1,144 @@
+# How to Avoid Write Conflicts in Markdown Blog
+
+This document explains the write conflict issue that occurred in the `activeSessions` table and how it was resolved.
+
+## Problem
+
+The Convex dashboard showed write conflicts in the `stats:heartbeat` mutation affecting the `activeSessions` table. Conflicts were happening every minute with retry counts of 0 and 1, indicating parallel mutations were competing to update the same documents.
+
+## Root Cause
+
+The original implementation had two issues:
+
+1. **Backend**: The heartbeat mutation queried for an existing session, then patched or inserted without checking if an update was actually needed
+2. **Frontend**: The `usePageTracking` hook could send duplicate heartbeats when:
+   - Path changed (immediate heartbeat + interval overlap)
+   - React StrictMode caused double effect invocations
+   - Multiple tabs were open
+
+## Solution
+
+### Backend Changes (convex/stats.ts)
+
+Added a 10-second dedup window to make the heartbeat mutation idempotent:
+
+```typescript
+const HEARTBEAT_DEDUP_MS = 10 * 1000;
+
+export const heartbeat = mutation({
+  args: {
+    sessionId: v.string(),
+    currentPath: v.string(),
+  },
+  returns: v.null(),
+  handler: async (ctx, args) => {
+    const now = Date.now();
+
+    const existingSession = await ctx.db
+      .query("activeSessions")
+      .withIndex("by_sessionId", (q) => q.eq("sessionId", args.sessionId))
+      .first();
+
+    if (existingSession) {
+      // Early return if same path and recently updated
+      if (
+        existingSession.currentPath === args.currentPath &&
+        now - existingSession.lastSeen < HEARTBEAT_DEDUP_MS
+      ) {
+        return null;
+      }
+
+      await ctx.db.patch(existingSession._id, {
+        currentPath: args.currentPath,
+        lastSeen: now,
+      });
+      return null;
+    }
+
+    await ctx.db.insert("activeSessions", {
+      sessionId: args.sessionId,
+      currentPath: args.currentPath,
+      lastSeen: now,
+    });
+
+    return null;
+  },
+});
+```
+
+### Frontend Changes (src/hooks/usePageTracking.ts)
+
+Added debouncing with refs to prevent duplicate mutation calls:
+
+```typescript
+const HEARTBEAT_DEBOUNCE_MS = 5 * 1000;
+
+// Track heartbeat state to prevent duplicate calls
+const isHeartbeatPending = useRef(false);
+const lastHeartbeatTime = useRef(0);
+const lastHeartbeatPath = useRef<string | null>(null);
+
+const sendHeartbeat = useCallback(
+  async (path: string) => {
+    const sessionId = sessionIdRef.current;
+    if (!sessionId) return;
+
+    const now = Date.now();
+
+    // Skip if heartbeat is already pending
+    if (isHeartbeatPending.current) {
+      return;
+    }
+
+    // Skip if same path and sent recently
+    if (
+      lastHeartbeatPath.current === path &&
+      now - lastHeartbeatTime.current < HEARTBEAT_DEBOUNCE_MS
+    ) {
+      return;
+    }
+
+    isHeartbeatPending.current = true;
+    lastHeartbeatTime.current = now;
+    lastHeartbeatPath.current = path;
+
+    try {
+      await heartbeatMutation({ sessionId, currentPath: path });
+    } catch {
+      // Silently fail
+    } finally {
+      isHeartbeatPending.current = false;
+    }
+  },
+  [heartbeatMutation]
+);
+```
+
+## Key Patterns Applied
+
+| Pattern | Implementation | Purpose |
+|---------|---------------|---------|
+| Backend idempotency | 10-second dedup window | Skip updates if recently updated with same data |
+| Frontend debouncing | 5-second debounce with refs | Prevent rapid duplicate calls |
+| Pending state tracking | `isHeartbeatPending` ref | Block overlapping async calls |
+| Indexed queries | `by_sessionId` index | Efficient document lookup |
+
+## Files Changed
+
+- `convex/stats.ts`: Added `HEARTBEAT_DEDUP_MS` constant and early return logic
+- `src/hooks/usePageTracking.ts`: Added debouncing refs and `useCallback` wrapper
+
+## Monitoring
+
+Check the Convex dashboard at Health > Insights > Insight Breakdown to verify:
+
+- Write conflict retries should drop to near zero
+- Function latency should stabilize
+- No permanent failures in error logs
+
+## References
+
+- Convex Write Conflicts: https://docs.convex.dev/error#1
+- Optimistic Concurrency Control: https://docs.convex.dev/database/advanced/occ
+- Convex Best Practices: https://docs.convex.dev/understanding/best-practices/
+