interview

home / developersection / interviews / how do you optimize large queries in indexeddb (e.g., 10,000+ records)?

Ask Question
ICSM Computer

IT-Hardware & Networking

Ravi Vishwakarma is a dedicated Software Developer with a passion for crafting efficient and innovative solutions. With a keen eye for detail and years of experience, he excels in developing robust software systems that meet client needs. His expertise spans across multiple programming languages and technologies, making him a valuable asset in any software development project.

Follow
Can you answer this question?
Answer

1 Answers

ICSM Computer

ICSM Computer

07-Jul-2025

Optimizing large queries (10,000+ records) in IndexedDB requires strategic use of indexes, efficient querying, and memory management. 

1. Use Indexes Properly

Define indexes on frequently queried fields to prevent full scans.

db.version(1).stores({
  users: '++id, name, email, age, [email+age]' // compound index example
});

Then use:

db.users.where('email').equals('a@example.com').toArray();

Avoid .filter() or .toArray().filter(), which loads everything into memory.

2. Query in Chunks (Pagination or Batching)

Instead of loading 10,000 records at once:

Paginated Query (by offset and limit):

const pageSize = 1000;
const page = 0; // or 1, 2, 3...
const results = await db.users.offset(page * pageSize).limit(pageSize).toArray();

Cursor with batching:

const results = [];
let count = 0;

db.users
  .orderBy('id')
  .each((item, cursor) => {
    results.push(item);
    count++;
    if (count >= 1000) {
      cursor.stop(); // break the loop early
    }
  });

3. Use Compound Indexes

If your queries use multiple conditions, define compound indexes:

db.users.where('[status+createdAt]').between([1, from], [1, to]);

This avoids filtering in memory and leverages B-tree search.

4. Avoid .toArray().filter(...) on Large Data

This loads all data into memory and is slow for 10K+ records. Instead, use where, equals, between, startsWith, or anyOf when possible.

Bad:

const results = (await db.users.toArray()).filter(u => u.age > 25);

Good:

const results = await db.users.where('age').above(25).toArray();

5. Use eachPrimaryKey or keys() if Only IDs Are Needed

Faster than retrieving full objects:

await db.users.where('status').equals('active').primaryKeys();

6. Use .modify() Instead of Fetch + Update

Bulk modify records efficiently:

await db.users.where('age').below(18).modify(user => {
  user.status = 'minor';
});

7. Use Lazy Iteration or Generators (Memory Friendly)

Process records one-by-one instead of loading all at once.

await db.users.each(user => {
  // Process user
});

8. Compact Data (Avoid Overhead)

If you store large payloads (e.g., images, logs), consider:

  • Storing metadata in IndexedDB and actual files/blobs in Blob or file system (browser-permitting)
  • Compressing large JSON payloads before saving

9. Upgrade to Dexie.js for Performance Features

Dexie optimizes IndexedDB internally and offers:

  • Transaction support
  • Easier compound query syntax
  • Observable support for live updates

10. Measure and Profile

Use Chrome DevTools > Application > IndexedDB tab to inspect query time, or log time manually:

const start = performance.now();
const data = await db.users.where('active').equals(true).toArray();
console.log("Query time:", performance.now() - start);

Summary Table

Optimization Use Case
Indexed fields For fast lookup
Pagination / Cursors For large datasets
Compound indexes For multi-field queries
.each() or .modify() Low-memory processing
Avoid .toArray().filter() Prevent memory overload
Dexie.js Easier + faster queries
MindStick Training
MindStick Training

Liked By

We use cookies to ensure you have the best browsing experience on our website. By using our site, you acknowledge that you have read and understood our Cookie Policy & Privacy Policy