---
title: "How do you handle API pagination when fetching data?"  
description: "How do you handle API pagination when fetching data?"  
author: "Anubhav Sharma"  
published: 2025-11-06  
updated: 2025-11-18  
canonical: https://www.mindstick.com/forum/161986/how-do-you-handle-api-pagination-when-fetching-data  
category: "python"  
tags: ["python-3.4", "Python 3"]  
reading_time: 3 minutes  

---

# How do you handle API pagination when fetching data?

**How do you [handle API](https://www.mindstick.com/interview/34408/how-do-you-handle-api-authentication-in-python-bearer-token) [pagination](https://www.mindstick.com/blog/265/pagination-in-php) when fetching [data](https://www.mindstick.com/articles/13050/salesforce-aiming-to-dominate-predictive-analytics-with-data-science)?**

## Replies

### Reply by ICSM Computer

### 1. Understand the Pagination Type

Different APIs use different pagination styles. Common ones:

1. **Offset-based pagination**\ Example: `/products?limit=20&offset=40`\ You fetch the next page by increasing the offset.
2. **Page-based pagination**\ Example: `/products?page=3&pageSize=20`\ You increment the page number.
3. **Cursor-based pagination (recommended for large data)**\ Example: `/products?cursor=abc123`\ [API](https://www.mindstick.com/articles/12641/instagram-api-upgraded-to-facebook-graph) returns a `nextCursor` to fetch the next batch.
4. **Token-based / Continuation Token**\ Used in cloud APIs (Azure, AWS).\ Example: `?continuationToken=xyz`.

### 2. Handle Pagination Loop

Keep fetching data until there is no next page.

## Example Logic:

```cs
var allItems = new List<Item>();
string nextPage = "1";

while (!string.IsNullOrEmpty(nextPage))
{
    var response = await Api.GetData(nextPage);

    allItems.AddRange(response.Items);

    nextPage = response.NextPage;   // could be page, offset, cursor, token
}
```

### 3. Handle Limits and API Constraints

When implementing pagination:

- Respect **rate limits** (429 Too Many Requests)
- Use **async** to avoid blocking
- Implement **backoff retries** (e.g., 1s → 2s → 4s)
- Keep page size reasonable (20–100 items)

### 4. Example JSON Response from a Paginated API

```plaintext
{
  "items": [...],
  "page": 2,
  "pageSize": 20,
  "hasMore": true,
  "nextPage": 3
}
```

### 5. (Frontend UI)

Here is **how to [handle](https://www.mindstick.com/articles/311004/suede-skillet-handle-cover) API pagination using JavaScript**, with clean, production-ready examples for all common pagination types.

#### 1. Page-based Pagination (page=1,2,3...)

```javascript
async function fetchAllPages() {
  let page = 1;
  let allData = [];
  let hasMore = true;

  while (hasMore) {
    const res = await fetch(`/api/products?page=${page}&pageSize=20`);
    const data = await res.json();

    allData.push(...data.items);

    hasMore = data.hasMore;       // API flag
    page++;
  }

  console.log(allData);
}
```

#### 2. Offset-based Pagination (limit & offset)

```javascript
async function fetchWithOffset() {
  let offset = 0;
  const limit = 20;
  let allItems = [];

  while (true) {
    const res = await fetch(`/api/products?limit=${limit}&offset=${offset}`);
    const data = await res.json();

    allItems.push(...data.items);

    if (data.items.length < limit) break; // No more pages

    offset += limit;
  }

  console.log(allItems);
}
```

#### 3. Cursor-based Pagination (most scalable)

API response example:

```plaintext
{
  "items": [...],
  "nextCursor": "abc123"
}
```

JavaScript code:

```javascript
async function fetchCursorPagination() {
  let cursor = null;
  let allItems = [];
  let url = "/api/list?limit=20";

  while (true) {
    const res = await fetch(cursor ? `${url}&cursor=${cursor}` : url);
    const data = await res.json();

    allItems.push(...data.items);

    if (!data.nextCursor) break; // No more pages
    cursor = data.nextCursor;
  }

  console.log(allItems);
}
```

#### 4. Pagination With Async/Await + Error Handling

```javascript
async function safeFetch(url) {
  try {
    const response = await fetch(url);

    if (!response.ok) {
      throw new Error(`HTTP: ${response.status}`);
    }

    return await response.json();
  } catch (err) {
    console.error("API Error:", err);
  }
}
```

#### 5. Infinite Scroll (Frontend UI)

```javascript
let page = 1;
window.addEventListener("scroll", async () => {
  if (window.innerHeight + window.scrollY >= document.body.offsetHeight) {
    const res = await fetch(`/api/products?page=${page++}`);
    const data = await res.json();

    data.items.forEach(item => {
      document.querySelector("#list").innerHTML += `<li>${item.name}</li>`;
    });
  }
});
```

### 6. Best Practices

- Always check for `hasMore` or `nextCursor`
- Use cursor-based pagination for real-time, large lists
- Cache data where possible
- Optimize your UI to load lazily (infinite scroll)


---

Original Source: https://www.mindstick.com/forum/161986/how-do-you-handle-api-pagination-when-fetching-data

Copyright © MindStick Software Pvt. Ltd. This Markdown version is provided for developers, AI systems, and offline reading.