Results, Sorting & Pagination
Pagination Performance
Why skip() gets expensive on large collections, why offset pages drift, and when a cursor is the only honest option.
Three problems, one lesson: skip cost, page drift, count cost. They show up together on a large orders collection.
Why skip() is slow
skip(n) does not jump to slot n in O(1). The server walks n keys in the sort order and discards them, then returns limit. Page 1 (skip(0)) is an index range. Page 10,000 (skip(99990)) is walking ~100k keys every request. CPU and IO grow with the page number. A cursor's $gt starts at the last key — work per page stays ~limit.
Why pages drift
Offset is 'items 21–30 of the current sort'. A new cheapest product inserts at the top: old item 21 is now 22, and the user sees a duplicate or a gap. A cursor is 'after this _id'. Already-seen documents stay seen. That is why feeds and activity APIs use cursors even when the collection is not huge.
The count
countDocuments(filter) for 'Page 4 of 80,221' is a second query that may scan as much as the listing. Dropping the total, or approximating, is a product decision. Cursor APIs usually return { items, next } and no total.
- Small, random-access table (jump to page 7) → offset is honest.
- Infinite scroll, notifications, 'load more orders' → cursor.
- Need both jump-to-page and millions of rows → unusual; often a search engine or a capped window, not Mongo skip.
Interview question
Why can skip() become inefficient for large datasets?
skip(n) still traverses n documents or index keys and throws them away. The deeper the page, the more wasted work, every time. Cursor pagination filters to keys after the last seen value so each page costs about limit, not skip+limit. Offset also shifts when new rows insert. I use skip for small admin lists and cursors for anything that can grow.