---
title: "Recipe: Walk next_token Pagination Without Loops or Gaps"
canonical: "https://aifitnessapi.com/cookbook/cursor-pagination"
cluster: "Cookbook"
primary_query: "next_token pagination loop oura whoop api code"
last_reviewed: "2026-11-05"
description: "Tested JavaScript that walks Oura and WHOOP next_token pages to the end: same query each page, loop and budget guards, and resume from the last saved page."
publisher: "AIFitnessAPI — independent, not sponsored"
cite_as: "\"Recipe: Walk next_token Pagination Without Loops or Gaps\", AIFitnessAPI, https://aifitnessapi.com/cookbook/cursor-pagination"
---

# Recipe: Walk next_token Pagination Without Loops or Gaps

> A dependency-free async generator that walks a next_token-paginated collection to its last page. Between pages it changes only the cursor parameter, so WHOOP's start and end ride along unchanged as WHOOP requires; it stops on a null, missing or empty token; it refuses to continue when a token comes back twice or the page budget runs out; and it hands each page's next URL to a callback so a crashed walk resumes where it stopped. It ships the two cursor shapes the site's guides document: Oura's data array with next_token sent back as next_token, and WHOOP's next_token sent back as nextToken with a maximum page size of 25.

- Canonical: https://aifitnessapi.com/cookbook/cursor-pagination
- Last reviewed: 2026-11-05
- Publisher: AIFitnessAPI (https://aifitnessapi.com) — independent, not sponsored
- Cite as: "Recipe: Walk next_token Pagination Without Loops or Gaps", AIFitnessAPI, https://aifitnessapi.com/cookbook/cursor-pagination

---

Oura and WHOOP both page their collections with an opaque token, and both are easy to walk wrongly: change the date window between pages and you get a different result set, send the token under the other vendor's parameter name and you get page one forever, trust the upstream never to repeat a token and one bad deploy on their side becomes an unbounded loop on yours. This recipe is the walker, with the two cursor shapes the site's [Oura](/integrate/oura-api) and [WHOOP](/integrate/whoop-api) guides document.

## What the vendors document

| | Oura | WHOOP |
|---|---|---|
| Records | a `data` array | not named in the guide quoted on this site; set it from a captured response |
| Cursor in the response | `next_token`, a string or `null` | `next_token`, empty on the last page |
| Cursor on the next request | `next_token` | `nextToken`, "using the same query parameters" |
| Page size | not covered here | `limit`, default 10, maximum 25 |

Oura's spec says to "implement pagination using the next_token parameter for large result sets"; WHOOP's pagination guide says "you iterated through all pages if the next_token field is empty on the response". The recipe treats `null`, a missing field and an empty string all as the end, because each vendor uses a different one of the three.

## What the recipe does

- **Changes exactly one parameter between pages.** `nextPageUrl` replaces the cursor and keeps everything else, so a WHOOP walk's `start` and `end` cannot drift.
- **Refuses to guess.** WHOOP's records field is left for you to name; calling the walker without it throws instead of returning empty pages.
- **Guards both ways to run forever.** A repeated token throws `PaginationLoopError`; more than `maxPages` throws `PageBudgetError` with the URL to resume from.
- **Checkpoints after processing.** `onPage` runs once your loop body has handled the page, so a crash re-reads the page instead of skipping it.
- **Clamps page size.** `withPageSize` caps WHOOP's `limit` at 25.

## The implementation

The canonical file is `cookbook/cursor-pagination.mjs` in this site's repository, and `cookbook/cursor-pagination.test.mjs` runs against it on every CI run. The listing below is a verbatim copy of that file. It has no runtime dependencies and needs Node 20 or newer; `getJson` is yours, so authentication, retries and rate limiting stay in the layer that already owns them.

```js
/**
 * cursor-pagination.mjs — walk a next_token-paginated collection to the end,
 * without looping forever, dropping a page, or changing the query mid-walk.
 *
 * WHAT THIS IMPLEMENTS
 *   `paginate({ firstUrl, getJson, cursor, maxPages, onPage })` is an async
 *   generator of pages. Between pages it changes exactly one thing, the
 *   cursor parameter, and keeps every other query parameter (WHOOP: "using
 *   the same query parameters", including start and end). It stops when the
 *   response's token is null, absent or empty, and it refuses to continue
 *   when a token repeats or the page budget runs out — the two ways a buggy
 *   upstream or a buggy client turns a backfill into an infinite bill.
 *
 *   Two cursor descriptions ship, each from the vendor's own documentation as
 *   quoted on the site's integration guides:
 *     OURA   response `data` array plus `next_token` (string or null);
 *            request parameter `next_token`
 *     WHOOP  response `next_token` (empty on the last page); request
 *            parameter `nextToken`; page size `limit`, maximum 25
 *   WHOOP's guide quoted on the site does not name the array that holds a
 *   page's records, so WHOOP's `items` is left for you to set from a
 *   captured response rather than guessed here.
 *
 *   `onPage(page)` runs after a page is yielded and processed: persist
 *   `page.nextUrl` there and a crashed walk resumes from `resumeUrl` instead
 *   of from page one.
 *
 * PAGES THIS SUPPORTS
 *   https://aifitnessapi.com/integrate/oura-api
 *   https://aifitnessapi.com/integrate/whoop-api
 *   https://aifitnessapi.com/cookbook/backfill-checkpointer (windows, retries)
 *
 * No dependencies. Node 20+.
 *
 * MIT — from aifitnessapi.com/cookbook
 */

export const OURA = Object.freeze({ name: "oura", requestParam: "next_token", responseField: "next_token", items: (body) => body.data });
export const WHOOP = Object.freeze({ name: "whoop", requestParam: "nextToken", responseField: "next_token", items: null, maxLimit: 25 });

export class PaginationLoopError extends Error {
  constructor(token, page) {
    super(`cursor "${token}" came back a second time on page ${page}: stopping instead of looping`);
    this.name = "PaginationLoopError";
    this.token = token;
    this.page = page;
  }
}

export class PageBudgetError extends Error {
  constructor(maxPages, nextUrl) {
    super(`stopped after ${maxPages} pages with more to read; resume from nextUrl`);
    this.name = "PageBudgetError";
    this.maxPages = maxPages;
    this.nextUrl = nextUrl;
  }
}

/** null, undefined and "" all mean "this was the last page". */
export function isLastToken(token) {
  return token === null || token === undefined || token === "";
}

/** The URL for the page after `url`: same parameters, cursor replaced. */
export function nextPageUrl(url, cursor, token) {
  const u = new URL(url);
  u.searchParams.set(cursor.requestParam, token);
  return u.toString();
}

/** Clamp a requested page size to the vendor's documented maximum, if any. */
export function withPageSize(url, cursor, limit) {
  const u = new URL(url);
  const n = cursor.maxLimit ? Math.min(limit, cursor.maxLimit) : limit;
  u.searchParams.set("limit", String(n));
  return u.toString();
}

/**
 * @param {{ firstUrl: string, resumeUrl?: string|null, getJson: (url: string) => Promise<any>,
 *           cursor: { requestParam: string, responseField: string, items: ((body: any) => any[])|null },
 *           items?: (body: any) => any[], maxPages?: number,
 *           onPage?: (page: { index: number, url: string, items: any[], nextUrl: string|null }) => Promise<void>|void }} opts
 */
export async function* paginate({ firstUrl, resumeUrl = null, getJson, cursor, items, maxPages = 1000, onPage }) {
  const pick = items ?? cursor.items;
  if (typeof pick !== "function") throw new TypeError(`say which field holds ${cursor.name ?? "this API"}'s records: pass items(body)`);
  const seen = new Set();
  let url = resumeUrl ?? firstUrl;
  const startToken = new URL(url).searchParams.get(cursor.requestParam);
  if (startToken) seen.add(startToken);

  for (let index = 0; ; index++) {
    if (index >= maxPages) throw new PageBudgetError(maxPages, url);
    const body = await getJson(url);
    const rows = pick(body);
    if (!Array.isArray(rows)) throw new TypeError(`page ${index} has no record array where items() looked`);
    const token = body?.[cursor.responseField];
    let nextUrl = null;
    if (!isLastToken(token)) {
      if (seen.has(token)) throw new PaginationLoopError(token, index);
      seen.add(token);
      nextUrl = nextPageUrl(url, cursor, token);
    }
    const page = { index, url, items: rows, nextUrl };
    yield page;
    if (onPage) await onPage(page);
    if (!nextUrl) return;
    url = nextUrl;
  }
}

/** Convenience: every record, in order. Prefer `paginate` for large walks. */
export async function collectAll(opts) {
  const out = [];
  for await (const page of paginate(opts)) out.push(...page.items);
  return out;
}
```

## How to adapt it

**Wrap `getJson` with your retry and pacing layer.** The walker stops on a thrown error; [the rate-limit fetcher](/cookbook/rate-limit-fetcher) and [the multi-window pacer](/cookbook/multi-window-rate-pacer) belong underneath it, so a 429 is retried before the walker ever sees it.

**Split long histories into windows first.** Oura's FAQ, as the site's guide quotes it, suggests fetching history in date ranges of about one to three months and paging within each. Plan the windows with [the backfill checkpointer](/cookbook/backfill-checkpointer) and run one walk per window.

**Make page writes idempotent.** Checkpoint-after-processing means a page can be processed twice after a crash; upsert on the record id.

## The contract the tests hold it to

Against fake Oura- and WHOOP-shaped servers: every record arrives once and in order; the second request keeps `start` and `end` and carries `nextToken`, never `next_token`, for WHOOP; null and empty tokens end the walk; a WHOOP walk without `items` is refused; page size clamps to 25 for WHOOP only; a repeated token stops the walk after two requests; the page budget stops it with a resumable URL; a walk that crashes mid-page resumes from the saved URL and finishes with every record exactly once; a page with no record array is an error; and a cursor already on the URL is replaced, not duplicated.

## What it deliberately does not do

It does not retry, back off or pace; it does not split date ranges; and it does not ship a WHOOP records field, because the guide this site quotes does not name one.

## FAQ

### What request parameter carries the WHOOP API's next_token on the next page?

nextToken. WHOOP's collection responses return the cursor in a field named next_token, and its pagination guide says to place that token on the subsequent request using the same query parameters; the site's WHOOP guide spells the request parameter nextToken. So a request with start and end must keep both exactly as they were and add nextToken, and the walk ends when next_token comes back empty. Sending next_token as the parameter name, the way Oura's API expects, is an easy mistake when one client serves both vendors, and the recipe's test fails if the WHOOP cursor ever sends it.

[Permalink](https://aifitnessapi.com/cookbook/cursor-pagination#faq-1)

### How do I stop a pagination loop when an API keeps returning the same cursor?

Remember every token you have followed and stop the moment one repeats. A cursor that comes back twice means the upstream is replaying a page, or your request is not sending the token it thinks it is, and either way continuing is an infinite loop billed against your rate limit. The recipe keeps a set of seen tokens, including the one in a resume URL, and throws a PaginationLoopError naming the token and page. It also enforces a page budget that throws a PageBudgetError carrying the URL to resume from, so a long walk can stop cleanly and pick up later.

[Permalink](https://aifitnessapi.com/cookbook/cursor-pagination#faq-2)

### Where should a resumable pagination walk save its checkpoint?

After the page has been processed, not when it is fetched. The recipe yields each page to your loop body first and calls onPage only when control comes back, so a crash while you are writing page three leaves the checkpoint pointing at page three, and the resumed walk reads it again instead of skipping it. That makes the processing of a page at-least-once, so the write must be idempotent: upsert on the record's id rather than appending. The saved value is the full next URL, which carries the original query and the cursor together, so resuming needs nothing else.

[Permalink](https://aifitnessapi.com/cookbook/cursor-pagination#faq-3)