The list is moving while you read it
Pages are computed per request. If records are created or deleted between page 3 and page 4, the boundary shifts: a deletion pulls a record back onto a page you already read, an insertion pushes one onto a page you have passed. Most of the time this does not matter. When it does — a reconciliation, an export you will diff — mitigate it:Sort by something stable
Sort by something stable
sort_by=name is stable under new writes. most_recent puts new records at the front of page 1 and shuffles everything back by one, which is the worst case for a paged read.Deduplicate by id
Deduplicate by id
Keep a set of ids as you go. A record you have already seen is a shift, not a duplicate to store.
Take fewer, larger pages
Take fewer, larger pages
per_page=100 gives shifts a quarter of the opportunities that per_page=25 does.Read the whole thing, then act
Read the whole thing, then act
Collect first and process after, rather than mutating records as you page through them — editing page 2 while reading page 3 is how a job skips half its input.
A generator that handles both hazards
Do not page to find one record
Walking every page looking for a back-half is expensive and gets worse as the account grows. Filter server-side:total_count on page 1 is the answer; you do not need to walk the collection to get it.
Resuming a long job
Store the page number and the filters together. A page number alone is meaningless against a different sort or filter set, and resuming with it will skip records silently.Analytics pagination is separate
Analytics breakdowns usecurrent_page and per_page, not page, and per_page defaults to 5 rather than 10. Easy to miss when moving between the two families of endpoints. See Metrics.