How to Export Serenity Tweets with Cursor Pagination

If you only need the latest few Serenity posts, the non-paginated endpoint is enough. A page-by-page export is for a different job: building a local archive that can survive network errors, reruns, and later analysis.

In this guide, Serenity refers to the public X account @aleabitoreddit.

TwexAPI's Get All Tweets and Replies by User by Page endpoint returns up to 20 public items per request and includes has_next_page plus next_cursor. That cursor is the key detail. Save it after each successful page, and you can resume from the last clean checkpoint instead of starting over.

When Pagination Helps

Use cursor pagination when you need a repeatable export, not just a quick preview. Common cases include a nightly Serenity archive, a research dataset for later reply analysis, or an incremental job that only needs to fetch what changed since the last run.

For a first pass, export JSON locally. For a production job, write each page to storage as it arrives, deduplicate by tweet_id, and keep the last successful cursor in a small checkpoint table or file.

API Endpoint

POST https://api.twexapi.io/twitter/tweets-replies/page

Authorization: Bearer <your_token>
Content-Type: application/json

The first request only needs screen_name. To continue, pass the next_cursor from the prior response. Stop when has_next_page is false or the response no longer includes a cursor.

curl --request POST \
  --url https://api.twexapi.io/twitter/tweets-replies/page \
  --header 'Authorization: Bearer <your_token>' \
  --header 'Content-Type: application/json' \
  --data '{"screen_name":"aleabitoreddit"}'

Python Cursor Loop

The example below writes each page to JSONL and saves the next cursor to a checkpoint file. That is safer than keeping everything in memory and writing one large file at the end.

1import json
2import os
3from pathlib import Path
4
5import requests
6
7TOKEN = os.environ["TWEXAPI_BEARER_TOKEN"]
8URL = "https://api.twexapi.io/twitter/tweets-replies/page"
9SCREEN_NAME = "aleabitoreddit"
10HEADERS = {"Authorization": f"Bearer {TOKEN}"}
11OUTPUT = Path("serenity-tweets-pages.jsonl")
12CHECKPOINT = Path("serenity-checkpoint.json")
13
14def load_cursor():
15    if not CHECKPOINT.exists():
16        return None
17    return json.loads(CHECKPOINT.read_text(encoding="utf-8")).get("next_cursor")
18
19def save_cursor(cursor):
20    CHECKPOINT.write_text(
21        json.dumps(
22            {"screen_name": SCREEN_NAME, "next_cursor": cursor},
23            ensure_ascii=False,
24            indent=2,
25        ),
26        encoding="utf-8",
27    )
28
29def append_items(items):
30    with OUTPUT.open("a", encoding="utf-8") as output:
31        for item in items:
32            output.write(json.dumps(item, ensure_ascii=False) + "\n")
33
34cursor = load_cursor()
35page_count = 0
36item_count = 0
37
38while True:
39    body = {"screen_name": SCREEN_NAME}
40    if cursor:
41        body["next_cursor"] = cursor
42
43    response = requests.post(URL, headers=HEADERS, json=body, timeout=30)
44    response.raise_for_status()
45    payload = response.json()
46
47    items = [item for item in payload.get("data", []) if item]
48    append_items(items)
49    page_count += 1
50    item_count += len(items)
51
52    next_cursor = payload.get("next_cursor")
53    if not payload.get("has_next_page") or not next_cursor:
54        if CHECKPOINT.exists():
55            CHECKPOINT.unlink()
56        break
57
58    save_cursor(next_cursor)
59    cursor = next_cursor
60
61print(f"Processed {page_count} pages and appended {item_count} items")

Production Notes

Save next_cursor after the current page has been written successfully, not before.
Deduplicate on tweet_id before inserting into a database or analytics table.
Store created_at_datetime, cashtags, language fields, and engagement counts for later analysis.
Keep the raw JSON response for a short retention window before flattening fields. It gives you a way to recover if the schema mapping changes.
Treat deleted or unavailable posts as normal gaps in a public-data archive.

Related Resources

Disclaimer

This guide covers public-data export for research and monitoring. It is not investment advice.