x/wiki
1
0
Fork 0
mirror of https://github.com/waynesutton/markdown-site.git synced 2026-01-12 04:09:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(stats): resolve write conflicts in activeSessions table

Browse Source 
- Add 10-second dedup window to heartbeat mutation for idempotency
- Add frontend debouncing (5s) with refs to prevent duplicate calls
- Track pending heartbeat state to block overlapping async calls
- Early return when session already updated with same path

Backend (convex/stats.ts):
- Add HEARTBEAT_DEDUP_MS constant (10 seconds)
- Check if session was recently updated before patching

Frontend (src/hooks/usePageTracking.ts):
- Add isHeartbeatPending, lastHeartbeatTime, lastHeartbeatPath refs
- Wrap sendHeartbeat in useCallback with debounce logic
- Import useCallback from React

Fixes write conflict retries shown in Convex dashboard for stats:heartbeat
This commit is contained in:
Wayne Sutton
2025-12-15 12:52:34 -08:00
parent b280cb4605
commit 97081dc82d
5 changed files with 641 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

138
.cursor/rules/convex-write-conflicts.mdc
View File

@@ -624,6 +624,138 @@ Some write conflicts are expected and acceptable:
In these cases, ensure your mutations are idempotent and let Convex's retry mechanism handle the conflicts. Add user-facing retry logic if needed.
## App-Specific Patterns: Markdown Blog
This app uses specific patterns to avoid write conflicts in the `activeSessions` table for real-time visitor tracking.
### Heartbeat Mutation with Dedup Window
The `stats:heartbeat` mutation uses a 10-second dedup window to prevent rapid updates:
```typescript
// convex/stats.ts
const HEARTBEAT_DEDUP_MS = 10 * 1000;
export const heartbeat = mutation({
args: {
sessionId: v.string(),
currentPath: v.string(),
},
returns: v.null(),
handler: async (ctx, args) => {
const now = Date.now();
// Find existing session by sessionId using index
const existingSession = await ctx.db
.query("activeSessions")
.withIndex("by_sessionId", (q) => q.eq("sessionId", args.sessionId))
.first();
if (existingSession) {
// Early return if same path and recently updated (idempotent)
if (
existingSession.currentPath === args.currentPath &&
now - existingSession.lastSeen < HEARTBEAT_DEDUP_MS
) {
return null;
}
// Patch directly with new data
await ctx.db.patch(existingSession._id, {
currentPath: args.currentPath,
lastSeen: now,
});
return null;
}
// Create new session only if none exists
await ctx.db.insert("activeSessions", {
sessionId: args.sessionId,
currentPath: args.currentPath,
lastSeen: now,
});
return null;
},
});
```
### Frontend Debouncing with Refs
The `usePageTracking` hook uses refs to prevent duplicate heartbeat calls:
```typescript
// src/hooks/usePageTracking.ts
const HEARTBEAT_DEBOUNCE_MS = 5 * 1000;
export function usePageTracking(): void {
const heartbeatMutation = useMutation(api.stats.heartbeat);
// Track heartbeat state to prevent duplicate calls
const isHeartbeatPending = useRef(false);
const lastHeartbeatTime = useRef(0);
const lastHeartbeatPath = useRef<string | null>(null);
const sendHeartbeat = useCallback(
async (path: string) => {
const sessionId = sessionIdRef.current;
if (!sessionId) return;
const now = Date.now();
// Skip if heartbeat is already pending
if (isHeartbeatPending.current) {
return;
}
// Skip if same path and sent recently (debounce)
if (
lastHeartbeatPath.current === path &&
now - lastHeartbeatTime.current < HEARTBEAT_DEBOUNCE_MS
) {
return;
}
isHeartbeatPending.current = true;
lastHeartbeatTime.current = now;
lastHeartbeatPath.current = path;
try {
await heartbeatMutation({
sessionId,
currentPath: path,
});
} catch {
// Silently fail
} finally {
isHeartbeatPending.current = false;
}
},
[heartbeatMutation],
);
// Send heartbeat on interval and path change
useEffect(() => {
const path = location.pathname;
sendHeartbeat(path);
const intervalId = setInterval(() => {
sendHeartbeat(path);
}, HEARTBEAT_INTERVAL_MS);
return () => clearInterval(intervalId);
}, [location.pathname, sendHeartbeat]);
}
```
### Key Patterns Used
1. **Backend idempotency**: Early return when session was updated within 10 seconds with same path
2. **Frontend debouncing**: 5-second debounce window using refs
3. **Pending state tracking**: `isHeartbeatPending` ref prevents overlapping calls
4. **Indexed queries**: Uses `by_sessionId` index for efficient lookups
5. **Event records pattern**: Page views use `pageViews` table instead of counters
## Summary
**Key Takeaway:** The less you read before writing, the fewer conflicts you'll have. Design your mutations to write directly when possible, and structure your data model to avoid concurrent writes to the same documents.
@@ -632,6 +764,8 @@ In these cases, ensure your mutations are idempotent and let Convex's retry mech
1. Patch directly without reading first (most common fix)
2. Use indexed queries to minimize read scope
3. Debounce rapid user inputs (300-500ms)
3. Debounce rapid user inputs (300-500ms for typing, 5s for heartbeats)
4. Use event records for high-frequency counters
5. Monitor your dashboard for conflict patterns
5. Add backend dedup windows for idempotency (10s for heartbeats)
6. Use refs to track pending mutations and prevent duplicates
7. Monitor your dashboard for conflict patterns

31
convex/stats.ts
View File

@@ -7,6 +7,9 @@ const DEDUP_WINDOW_MS = 30 * 60 * 1000;
// Session timeout: 2 minutes in milliseconds
const SESSION_TIMEOUT_MS = 2 * 60 * 1000;
// Heartbeat dedup window: 10 seconds (prevents write conflicts from rapid calls)
const HEARTBEAT_DEDUP_MS = 10 * 1000;
/**
* Record a page view event.
* Idempotent: same session viewing same path within 30min = 1 view.
@@ -51,6 +54,7 @@ export const recordPageView = mutation({
/**
* Update active session heartbeat.
* Creates or updates session with current path and timestamp.
* Idempotent: skips update if recently updated with same path (prevents write conflicts).
*/
export const heartbeat = mutation({
args: {
@@ -61,27 +65,36 @@ export const heartbeat = mutation({
handler: async (ctx, args) => {
const now = Date.now();
// Find existing session by sessionId
// Find existing session by sessionId using index
const existingSession = await ctx.db
.query("activeSessions")
.withIndex("by_sessionId", (q) => q.eq("sessionId", args.sessionId))
.first();
if (existingSession) {
// Update existing session
// Early return if same path and recently updated (idempotent - prevents write conflicts)
if (
existingSession.currentPath === args.currentPath &&
now - existingSession.lastSeen < HEARTBEAT_DEDUP_MS
) {
return null;
}
// Patch directly with new data
await ctx.db.patch(existingSession._id, {
currentPath: args.currentPath,
lastSeen: now,
});
} else {
// Create new session
await ctx.db.insert("activeSessions", {
sessionId: args.sessionId,
currentPath: args.currentPath,
lastSeen: now,
});
return null;
}
// Create new session only if none exists
await ctx.db.insert("activeSessions", {
sessionId: args.sessionId,
currentPath: args.currentPath,
lastSeen: now,
});
return null;
},
});

284
prds/howstatsworks.md Normal file
View File

@@ -0,0 +1,284 @@
# How stats work
This document explains the real-time analytics system for the markdown site.
## Overview
The stats page at `/stats` shows live visitor data and page view counts. All stats update automatically via Convex subscriptions. No page refresh required.
## Data flow
1. Visitor loads any page
2. `usePageTracking` hook fires on mount and path change
3. Page view event recorded to `pageViews` table
4. Session heartbeat sent to `activeSessions` table
5. Stats page queries both tables and displays results
## Database tables
### pageViews
Stores individual view events. Uses the event records pattern to avoid write conflicts.
| Field | Type | Purpose |
|-------|------|---------|
| path | string | URL path visited |
| pageType | string | "home", "blog", "page", or "stats" |
| sessionId | string | Anonymous UUID per browser |
| timestamp | number | Unix timestamp of visit |
Indexes:
- `by_path` for filtering by page
- `by_timestamp` for ordering
- `by_session_path` for deduplication
### activeSessions
Tracks who is currently on the site. Sessions expire after 2 minutes without a heartbeat.
| Field | Type | Purpose |
|-------|------|---------|
| sessionId | string | Anonymous UUID per browser |
| currentPath | string | Page visitor is currently viewing |
| lastSeen | number | Last heartbeat timestamp |
Indexes:
- `by_sessionId` for upserts
- `by_lastSeen` for cleanup queries
## Frontend tracking
The `usePageTracking` hook in `src/hooks/usePageTracking.ts` handles all client-side tracking.
### Session ID generation
Each browser gets a persistent UUID stored in localStorage. No cookies, no PII.
```typescript
const SESSION_ID_KEY = "markdown_blog_session_id";
function getSessionId(): string {
let sessionId = localStorage.getItem(SESSION_ID_KEY);
if (!sessionId) {
sessionId = generateSessionId();
localStorage.setItem(SESSION_ID_KEY, sessionId);
}
return sessionId;
}
```
### Page view recording
Records a view when the URL path changes. Deduplication happens server-side.
```typescript
useEffect(() => {
const path = location.pathname;
if (lastRecordedPath.current !== path) {
lastRecordedPath.current = path;
recordPageView({ path, pageType: getPageType(path), sessionId });
}
}, [location.pathname, recordPageView]);
```
### Heartbeat system
Sends a ping every 30 seconds while the page is open. This powers the "Active Now" count.
```typescript
const HEARTBEAT_INTERVAL_MS = 30 * 1000;
useEffect(() => {
const sendHeartbeat = () => {
heartbeat({ sessionId, currentPath: path });
};
sendHeartbeat();
const intervalId = setInterval(sendHeartbeat, HEARTBEAT_INTERVAL_MS);
return () => clearInterval(intervalId);
}, [location.pathname, heartbeat]);
```
## Backend mutations
### recordPageView
Located in `convex/stats.ts`. Records view events with deduplication.
Deduplication window: 30 minutes. Same session viewing same path within 30 minutes counts as 1 view.
```typescript
export const recordPageView = mutation({
args: {
path: v.string(),
pageType: v.string(),
sessionId: v.string(),
},
returns: v.null(),
handler: async (ctx, args) => {
const dedupCutoff = Date.now() - DEDUP_WINDOW_MS;
const recentView = await ctx.db
.query("pageViews")
.withIndex("by_session_path", (q) =>
q.eq("sessionId", args.sessionId).eq("path", args.path)
)
.order("desc")
.first();
if (recentView && recentView.timestamp > dedupCutoff) {
return null;
}
await ctx.db.insert("pageViews", {
path: args.path,
pageType: args.pageType,
sessionId: args.sessionId,
timestamp: Date.now(),
});
return null;
},
});
```
### heartbeat
Creates or updates a session record. Uses indexed lookup for upsert.
```typescript
export const heartbeat = mutation({
args: {
sessionId: v.string(),
currentPath: v.string(),
},
returns: v.null(),
handler: async (ctx, args) => {
const existingSession = await ctx.db
.query("activeSessions")
.withIndex("by_sessionId", (q) => q.eq("sessionId", args.sessionId))
.first();
if (existingSession) {
await ctx.db.patch(existingSession._id, {
currentPath: args.currentPath,
lastSeen: Date.now(),
});
} else {
await ctx.db.insert("activeSessions", {
sessionId: args.sessionId,
currentPath: args.currentPath,
lastSeen: Date.now(),
});
}
return null;
},
});
```
## Backend query
### getStats
Returns all stats for the `/stats` page. Single query, real-time subscription.
What it returns:
| Field | Type | Description |
|-------|------|-------------|
| activeVisitors | number | Sessions with heartbeat in last 2 minutes |
| activeByPath | array | Breakdown of active visitors by current page |
| totalPageViews | number | All recorded views since tracking started |
| uniqueVisitors | number | Count of distinct session IDs |
| publishedPosts | number | Blog posts with `published: true` |
| publishedPages | number | Static pages with `published: true` |
| trackingSince | number or null | Timestamp of earliest view event |
| pageStats | array | Views per page with title and type |
### Title matching
The query matches URL paths to post/page titles by slug:
```typescript
const slug = path.startsWith("/") ? path.slice(1) : path;
const post = posts.find((p) => p.slug === slug);
const page = pages.find((p) => p.slug === slug);
if (post) {
title = post.title;
pageType = "blog";
} else if (page) {
title = page.title;
pageType = "page";
}
```
## Cleanup cron
Stale sessions are cleaned up every 5 minutes via cron job in `convex/crons.ts`.
```typescript
crons.interval(
"cleanup stale sessions",
{ minutes: 5 },
internal.stats.cleanupStaleSessions,
{}
);
```
The cleanup mutation deletes sessions older than 2 minutes:
```typescript
const cutoff = Date.now() - SESSION_TIMEOUT_MS;
const staleSessions = await ctx.db
.query("activeSessions")
.withIndex("by_lastSeen", (q) => q.lt("lastSeen", cutoff))
.collect();
await Promise.all(staleSessions.map((s) => ctx.db.delete(s._id)));
```
## How new content appears in stats
When you add a new markdown post or page and sync it to Convex:
1. **Post/page counts update instantly.** The `publishedPosts` and `publishedPages` values come from live queries to the `posts` and `pages` tables.
2. **Views appear after first visit.** A page only shows in "Views by Page" after someone visits it.
3. **Titles resolve automatically.** The `getStats` query matches paths to slugs, so new content gets its proper title displayed.
No manual configuration required. Sync content, and stats track it.
## Privacy
- No cookies
- No PII stored
- Session IDs are random UUIDs
- No IP addresses logged
- No fingerprinting
- Data stays in your Convex deployment
## Configuration constants
| Constant | Value | Location |
|----------|-------|----------|
| DEDUP_WINDOW_MS | 30 minutes | convex/stats.ts |
| SESSION_TIMEOUT_MS | 2 minutes | convex/stats.ts |
| HEARTBEAT_INTERVAL_MS | 30 seconds | src/hooks/usePageTracking.ts |
## Files involved
| File | Purpose |
|------|---------|
| `convex/stats.ts` | All stats mutations and queries |
| `convex/schema.ts` | Table definitions for pageViews and activeSessions |
| `convex/crons.ts` | Scheduled cleanup job |
| `src/hooks/usePageTracking.ts` | Client-side tracking hook |
| `src/pages/Stats.tsx` | Stats page UI |
## Related documentation
- [Convex event records pattern](https://docs.convex.dev/understanding/best-practices/)
- [Preventing write conflicts](https://docs.convex.dev/error#1)

144
prds/howtoavoidwriteconflicts.md Normal file
View File

@@ -0,0 +1,144 @@
# How to Avoid Write Conflicts in Markdown Blog
This document explains the write conflict issue that occurred in the `activeSessions` table and how it was resolved.
## Problem
The Convex dashboard showed write conflicts in the `stats:heartbeat` mutation affecting the `activeSessions` table. Conflicts were happening every minute with retry counts of 0 and 1, indicating parallel mutations were competing to update the same documents.
## Root Cause
The original implementation had two issues:
1. **Backend**: The heartbeat mutation queried for an existing session, then patched or inserted without checking if an update was actually needed
2. **Frontend**: The `usePageTracking` hook could send duplicate heartbeats when:
- Path changed (immediate heartbeat + interval overlap)
- React StrictMode caused double effect invocations
- Multiple tabs were open
## Solution
### Backend Changes (convex/stats.ts)
Added a 10-second dedup window to make the heartbeat mutation idempotent:
```typescript
const HEARTBEAT_DEDUP_MS = 10 * 1000;
export const heartbeat = mutation({
args: {
sessionId: v.string(),
currentPath: v.string(),
},
returns: v.null(),
handler: async (ctx, args) => {
const now = Date.now();
const existingSession = await ctx.db
.query("activeSessions")
.withIndex("by_sessionId", (q) => q.eq("sessionId", args.sessionId))
.first();
if (existingSession) {
// Early return if same path and recently updated
if (
existingSession.currentPath === args.currentPath &&
now - existingSession.lastSeen < HEARTBEAT_DEDUP_MS
) {
return null;
}
await ctx.db.patch(existingSession._id, {
currentPath: args.currentPath,
lastSeen: now,
});
return null;
}
await ctx.db.insert("activeSessions", {
sessionId: args.sessionId,
currentPath: args.currentPath,
lastSeen: now,
});
return null;
},
});
```
### Frontend Changes (src/hooks/usePageTracking.ts)
Added debouncing with refs to prevent duplicate mutation calls:
```typescript
const HEARTBEAT_DEBOUNCE_MS = 5 * 1000;
// Track heartbeat state to prevent duplicate calls
const isHeartbeatPending = useRef(false);
const lastHeartbeatTime = useRef(0);
const lastHeartbeatPath = useRef<string | null>(null);
const sendHeartbeat = useCallback(
async (path: string) => {
const sessionId = sessionIdRef.current;
if (!sessionId) return;
const now = Date.now();
// Skip if heartbeat is already pending
if (isHeartbeatPending.current) {
return;
}
// Skip if same path and sent recently
if (
lastHeartbeatPath.current === path &&
now - lastHeartbeatTime.current < HEARTBEAT_DEBOUNCE_MS
) {
return;
}
isHeartbeatPending.current = true;
lastHeartbeatTime.current = now;
lastHeartbeatPath.current = path;
try {
await heartbeatMutation({ sessionId, currentPath: path });
} catch {
// Silently fail
} finally {
isHeartbeatPending.current = false;
}
},
[heartbeatMutation]
);
```
## Key Patterns Applied
| Pattern | Implementation | Purpose |
|---------|---------------|---------|
| Backend idempotency | 10-second dedup window | Skip updates if recently updated with same data |
| Frontend debouncing | 5-second debounce with refs | Prevent rapid duplicate calls |
| Pending state tracking | `isHeartbeatPending` ref | Block overlapping async calls |
| Indexed queries | `by_sessionId` index | Efficient document lookup |
## Files Changed
- `convex/stats.ts`: Added `HEARTBEAT_DEDUP_MS` constant and early return logic
- `src/hooks/usePageTracking.ts`: Added debouncing refs and `useCallback` wrapper
## Monitoring
Check the Convex dashboard at Health > Insights > Insight Breakdown to verify:
- Write conflict retries should drop to near zero
- Function latency should stabilize
- No permanent failures in error logs
## References
- Convex Write Conflicts: https://docs.convex.dev/error#1
- Optimistic Concurrency Control: https://docs.convex.dev/database/advanced/occ
- Convex Best Practices: https://docs.convex.dev/understanding/best-practices/

73
src/hooks/usePageTracking.ts
View File

@@ -1,4 +1,4 @@
import { useEffect, useRef } from "react";
import { useEffect, useRef, useCallback } from "react";
import { useMutation } from "convex/react";
import { useLocation } from "react-router-dom";
import { api } from "../../convex/_generated/api";
@@ -6,6 +6,9 @@ import { api } from "../../convex/_generated/api";
// Heartbeat interval: 30 seconds
const HEARTBEAT_INTERVAL_MS = 30 * 1000;
// Minimum time between heartbeats to prevent write conflicts: 5 seconds
const HEARTBEAT_DEBOUNCE_MS = 5 * 1000;
// Session ID key in localStorage
const SESSION_ID_KEY = "markdown_blog_session_id";
@@ -56,17 +59,61 @@ function getPageType(path: string): string {
export function usePageTracking(): void {
const location = useLocation();
const recordPageView = useMutation(api.stats.recordPageView);
const heartbeat = useMutation(api.stats.heartbeat);
const heartbeatMutation = useMutation(api.stats.heartbeat);
// Track if we've recorded view for current path
const lastRecordedPath = useRef<string | null>(null);
const sessionIdRef = useRef<string | null>(null);
// Track heartbeat state to prevent duplicate calls and write conflicts
const isHeartbeatPending = useRef(false);
const lastHeartbeatTime = useRef(0);
const lastHeartbeatPath = useRef<string | null>(null);
// Initialize session ID
useEffect(() => {
sessionIdRef.current = getSessionId();
}, []);
// Debounced heartbeat function to prevent write conflicts
const sendHeartbeat = useCallback(
async (path: string) => {
const sessionId = sessionIdRef.current;
if (!sessionId) return;
const now = Date.now();
// Skip if heartbeat is already pending
if (isHeartbeatPending.current) {
return;
}
// Skip if same path and sent recently (debounce)
if (
lastHeartbeatPath.current === path &&
now - lastHeartbeatTime.current < HEARTBEAT_DEBOUNCE_MS
) {
return;
}
isHeartbeatPending.current = true;
lastHeartbeatTime.current = now;
lastHeartbeatPath.current = path;
try {
await heartbeatMutation({
sessionId,
currentPath: path,
});
} catch {
// Silently fail - analytics shouldn't break the app
} finally {
isHeartbeatPending.current = false;
}
},
[heartbeatMutation]
);
// Record page view when path changes
useEffect(() => {
const path = location.pathname;
@@ -91,28 +138,18 @@ export function usePageTracking(): void {
// Send heartbeat on interval and on path change
useEffect(() => {
const path = location.pathname;
const sessionId = sessionIdRef.current;
if (!sessionId) return;
// Send initial heartbeat
const sendHeartbeat = () => {
heartbeat({
sessionId,
currentPath: path,
}).catch(() => {
// Silently fail
});
};
sendHeartbeat();
// Send initial heartbeat for this path
sendHeartbeat(path);
// Set up interval for ongoing heartbeats
const intervalId = setInterval(sendHeartbeat, HEARTBEAT_INTERVAL_MS);
const intervalId = setInterval(() => {
sendHeartbeat(path);
}, HEARTBEAT_INTERVAL_MS);
return () => {
clearInterval(intervalId);
};
}, [location.pathname, heartbeat]);
}, [location.pathname, sendHeartbeat]);
}