x/wiki
1
0
Fork 0
mirror of https://github.com/waynesutton/markdown-site.git synced 2026-01-12 04:09:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: use direct counting for stats until aggregates are backfilled

Browse Source 
The aggregate components only contained new page views (after installation),
not the ~6000+ historical views. Changed getStats to use direct counting
from pageViews table to ensure all historical data is displayed correctly.

This is a temporary fix until we implement chunked backfilling to handle
the large dataset without exceeding memory limits.
This commit is contained in:
Wayne Sutton
2025-12-20 14:54:11 -08:00
parent d562e1ede8
commit 0057194701
2 changed files with 34 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
convex/stats.ts
View File

@@ -210,11 +210,21 @@ export const getStats = query({
.map(([path, count]) => ({ path, count }))
.sort((a, b) => b.count - a.count);
// Use aggregate component for total page views count: O(log n) instead of O(n)
const totalPageViewsCount = await totalPageViews.count(ctx);
// Use aggregate component for unique visitors count: O(log n) instead of O(n)
const uniqueVisitorsCount = await uniqueVisitors.count(ctx);
// Get all page views for direct counting (always accurate)
// We use direct counting until aggregates are fully backfilled
const allPageViews = await ctx.db.query("pageViews").collect();
const totalPageViewsCount = allPageViews.length;
// Count unique sessions from the views
const uniqueSessions = new Set(allPageViews.map((v) => v.sessionId));
const uniqueVisitorsCount = uniqueSessions.size;
// Count views per path from the raw data
const pathCountsFromDb: Record<string, number> = {};
for (const view of allPageViews) {
pathCountsFromDb[view.path] = (pathCountsFromDb[view.path] || 0) + 1;
}
const allPaths = Object.keys(pathCountsFromDb);
// Get earliest page view for tracking since date (single doc fetch)
const firstView = await ctx.db
@@ -235,18 +245,9 @@ export const getStats = query({
.withIndex("by_published", (q) => q.eq("published", true))
.collect();
// Get unique paths from pageViews (needed to build pageStats)
// We still need to iterate for path list, but use aggregate for per-path counts
const allPaths = new Set<string>();
const pathViewsFromDb = await ctx.db.query("pageViews").collect();
for (const view of pathViewsFromDb) {
allPaths.add(view.path);
}
// Build page stats using aggregate counts per path: O(log n) per path
const pageStatsPromises = Array.from(allPaths).map(async (path) => {
// Use aggregate namespace count for this path
const views = await pageViewsByPath.count(ctx, { namespace: path });
// Build page stats using direct counts (always accurate)
const pageStatsPromises = allPaths.map(async (path) => {
const views = pathCountsFromDb[path] || 0;
// Match path to post or page for title
const slug = path.startsWith("/") ? path.slice(1) : path;

16
public/raw/changelog.md
View File

@@ -7,6 +7,22 @@ Date: 2025-12-20
All notable changes to this project.
## v1.11.0
Released December 20, 2025
**Aggregate component for efficient stats**
- Replaced O(n) table scans with O(log n) aggregate counts
- Uses `@convex-dev/aggregate` package for TableAggregate
- Three aggregates: totalPageViews, pageViewsByPath, uniqueVisitors
- Backfill mutation for existing page view data
- Updated `convex/convex.config.ts` with aggregate component registration
- Updated `convex/stats.ts` to use aggregate counts in getStats query
- Updated `prds/howstatsworks.md` with old vs new implementation comparison
Performance improvement: Stats queries now use pre-computed counts instead of scanning all page view records.
## v1.10.0
Released December 20, 2025