Services
Technology Web App Development Mobile App Development Windows App Development Database Development UI/UX Development Cloud Development SEO Digital Marketing Guest Posting Content Writing Artificial Intelligence Data Science ERP Development CRM Development CMS Development LMS Development Business Listing Blockchain Technology Startup Promotion AI Tools Integration
Products
Data Converter Cleaner TUC - Unit Converter Import Export Survey Manager Tweet Controller
Developer
Article Blog Forum Interview Beginner News Startup Quiz
Users
Career
Apply Job Internship
Pricing
Login Sign Up Get a Quote

interview

Home Interviews How would you limit memory use when paginating a large object store result set? – MindStick
Ask Question

1 Answers

ICSM Computer
ICSM Computer 08 Jul 2025
Report Abuse

To limit memory use when paginating a large result set in IndexedDB (especially with 10,000+ records), avoid loading all data into memory (.toArray()) and instead use streamed access via cursors or Dexie.js pagination.

Here’s how you do it efficiently:

Strategy: Page data without loading everything

Avoid This:

// Bad: loads everything into memory
const all = await db.users.toArray();
const page = all.slice(offset, offset + limit);

Solution 1: Pagination with Dexie.js (Efficient)

If you're using Dexie.js, use offset().limit():

const pageSize = 100;
const currentPage = 2; // (starts from 0)

const pageData = await db.users
  .orderBy('id') // must be indexed
  .offset(currentPage * pageSize)
  .limit(pageSize)
  .toArray();
  • Only loads 1 page worth of records into memory
  • Uses indexed cursor internally
  • Very fast and memory-safe

Solution 2: Using a Cursor in Vanilla IndexedDB

Cursors allow you to stream records without full memory load:

function paginateWithCursor(db, storeName, page, pageSize) {
  return new Promise((resolve, reject) => {
    const tx = db.transaction([storeName], "readonly");
    const store = tx.objectStore(storeName);
    const request = store.openCursor();

    const start = page * pageSize;
    const end = start + pageSize;

    let result = [];
    let count = 0;

    request.onsuccess = function (event) {
      const cursor = event.target.result;
      if (!cursor) return resolve(result);

      if (count >= start && count < end) {
        result.push(cursor.value);
      }

      count++;
      if (count < end) {
        cursor.continue();
      } else {
        resolve(result); // stop early
      }
    };

    request.onerror = () => reject(request.error);
  });
}
  • Loads only a page-sized portion
  • Prevents high memory use
  • No need for .toArray()

Tips for Optimal Pagination

Tip Why
Use .offset().limit() on indexed field Indexed traversal avoids full scan
Use orderBy() only on indexed columns Otherwise, Dexie can’t paginate efficiently
Use .count() to get total items without loading them For pagination UI
Avoid .filter() or .map() after .toArray() Triggers full memory load

Optional: Count Total Items

For pagination UI:

const totalItems = await db.users.count();
const totalPages = Math.ceil(totalItems / pageSize);

Summary

To limit memory use while paginating in IndexedDB:

  • Use offset().limit() with indexed orderBy() (Dexie)
  • Use a cursor to stream only the needed page (Vanilla)
  • Avoid .toArray() on large datasets

Would you like this turned into a reusable Dexie pagination function with total count?

advertising
Markdown for AI

A clean, structured version of this page for AI assistants and LLMs.

Open .md