Historical Twitter (X) Data + Archive API — Complete Guide

By Alex Chen•5 min read

Historical Twitter (X) data — anything older than the ~7-day recent-search window — has always been the hardest data class to access. X serves it through Full-Archive Search at $0.005 per post read, capped at 3 million post reads per monthly billing cycle; beyond that you need Enterprise, which has no public price. For most developers, researchers, and analysts, the practical path is a third-party endpoint that fronts the same underlying archive at commodity pricing.

This guide walks the actual query pattern, the pagination shape, cost math at 4 different volumes, and a real whale case study — plus the official X Full-Archive alternative for teams who need it.

01 — Section

Why historical data is the hardest class

The X archive dates back to 2006 — ~1.5 trillion tweets. It's the definitive historical record of public conversation on the platform. Access to it is intentionally gated because it's expensive to serve at scale (index + storage + rate infrastructure).

X's own pricing is pay-per-use with no subscription (docs.x.com/x-api/getting-started/pricing). Recent search covers ~7 days; Full-Archive Search (/2/tweets/search/all) is open to pay-per-use and Enterprise customers and bills $0.005 per post read. Pay-per-use stops at 3 million post reads per monthly billing cycle; larger pulls need Enterprise through an interest form, with no public price.

twitterapi.io exposes the same underlying archive via /twitter/tweet/advanced_search with since: and until: operators — no tier gating, per-tweet pricing. That commoditization is the reason ~90% of academic + analytical historical-tweet workflows now route through this class of provider rather than paying X's per-read rate.

02 — Section

The core query pattern

The magic is the combination of since: + until: date operators inside the query string. Same X native search grammar (docs.x.com/x-api/enterprise-gnip-2.0/fundamentals/search-api) — the twitterapi.io endpoint accepts them identically.

Date format is YYYY-MM-DD in UTC. Range is inclusive on since:, exclusive on until:. Both operators can be combined with min_faves:, min_retweets:, from:, lang:, and other operators for tight filtering.

python

import os, requests

HEADERS = {"X-API-Key": os.environ["TWITTERAPI_IO_KEY"]}
BASE = "https://api.twitterapi.io"

def historical_search(kw: str, since: str, until: str, min_faves: int = 0):
    query_parts = [kw, f"since:{since}", f"until:{until}"]
    if min_faves: query_parts.append(f"min_faves:{min_faves}")
    query = " ".join(query_parts)

    tweets, cursor = [], None
    for _ in range(500):  # safety cap
        params = {"query": query}
        if cursor: params["cursor"] = cursor
        r = requests.get(f"{BASE}/twitter/tweet/advanced_search", headers=HEADERS, params=params, timeout=30)
        r.raise_for_status()
        resp = r.json()
        tweets.extend(resp.get("tweets", []))
        cursor = resp.get("next_cursor")
        if not cursor: break
    return tweets

# Example: all $TSLA mentions with 100+ likes in Jan 2021
tweets = historical_search("$TSLA", "2021-01-01", "2021-02-01", min_faves=100)
print(f"pulled {len(tweets):,} historical tweets")
# Cost per twitterapi.io/pricing: len(tweets) × $0.00015

03 — Section

Worked example: $2.6K for 17.5M tweets

Take a longitudinal cashtag mention study across 2018-2024 (6 years, one cashtag, no engagement threshold) that returns 17.5M tweets, paginated in 7-day windows across ~312 date ranges to keep each cursor run short.

Cost: $2,625 ($0.00015 × 17.5M). Request time: 17.5M tweets at 20 per page is 875,000 requests; at 20 requests per second (any paid account) that is about 12 hours of calls, which you can spread over several days.

Official X equivalent for the same 17.5M archive pull: $87,500 at $0.005 per post read (17.5M × $0.005). Because pay-per-use is capped at 3M post reads per monthly billing cycle, that pull takes at least 6 billing cycles (17.5M ÷ 3M ≈ 5.8), or an Enterprise contract with no public price. ~33× cost delta ($87,500 ÷ $2,625).

This is the class of workflow the twitterapi.io historical path unblocks — quantitative research, longitudinal analysis, backtest data generation — priced so that a single researcher / small team can afford it without enterprise procurement.

04 — Section

Cost math at 4 volumes

Historical pulls compound differently than recent-search work — you're bounded by the volume-of-time-window × keyword-selectivity product, not by monthly rate.

Volume	twitterapi.io ($0.00015/tweet)	X official (pay-per-use, $0.005 per post read)
100K tweets (1 month, mid-selectivity kw)	$15	$500
1M tweets (1 year, common kw)	$150	$5,000
10M tweets (5 years, common kw)	$1,500	$50,000, over the 3M/month cap: 4+ billing cycles or Enterprise
100M tweets (10 years, broad kw)	$15,000	$500,000 at the per-read rate; needs Enterprise (no public price)

Two takeaways: (a) at every volume twitterapi.io costs about 1/33 of X's per-read rate ($0.15 vs $5 per 1K), and above 3 million Post reads per monthly billing cycle the official path also needs several billing cycles or Enterprise; (b) above 100M tweets, both paths become significant cost — but only twitterapi.io lets you self-serve to that volume without an Enterprise contract.

05 — Section

Query construction for large windows

For pulls spanning years, don't submit a single 5-year since:...until:... query — cursor pagination gets slow past ~500K tweets in one range and edge cases start appearing.

Best pattern: split the window into per-day or per-week sub-queries, iterate, batch-write to storage. Each sub-query independently paginates + you get natural checkpointing (resume from failed day rather than restart the whole 5-year pull).

python

# Bulk historical pull — window-sliced pattern.
import os, requests, json
from pathlib import Path
from datetime import date, timedelta

HEADERS = {"X-API-Key": os.environ["TWITTERAPI_IO_KEY"]}
BASE = "https://api.twitterapi.io"
OUT_DIR = Path("historical_pull")
OUT_DIR.mkdir(exist_ok=True)

def date_range(start: date, end: date, step_days: int = 7):
    d = start
    while d < end:
        nxt = min(d + timedelta(days=step_days), end)
        yield d, nxt
        d = nxt

def pull_window(kw: str, since: date, until: date):
    out_path = OUT_DIR / f"{kw.replace(' ', '_')}_{since}.jsonl"
    if out_path.exists() and out_path.stat().st_size > 0:
        return -1  # already done — skip

    query = f"{kw} since:{since} until:{until}"
    n, cursor = 0, None
    with open(out_path, "w") as f:
        for _ in range(500):
            params = {"query": query}
            if cursor: params["cursor"] = cursor
            r = requests.get(f"{BASE}/twitter/tweet/advanced_search", headers=HEADERS, params=params, timeout=30)
            r.raise_for_status()
            resp = r.json()
            for t in resp.get("tweets", []):
                f.write(json.dumps(t) + "\n"); n += 1
            cursor = resp.get("next_cursor")
            if not cursor: break
    return n

total = 0
for since, until in date_range(date(2018, 1, 1), date(2024, 12, 31), step_days=7):
    n = pull_window("$TSLA min_faves:5", since, until)
    if n >= 0:
        total += n
        print(f"  {since}: pulled {n:,}")
print(f"\ntotal historical: {total:,} tweets")

06 — Section

Alternative — X official Full-Archive Search

X's own Full-Archive Search is open to pay-per-use and Enterprise customers. It needs an X developer account with a saved payment card, plus Bearer-token auth via the X Developer Console.

Endpoint: /2/tweets/search/all (Full-Archive) per docs.x.com/x-api/posts/search/introduction.

Pricing: $0.005 per post read on pay-per-use, capped at 3 million post reads per monthly billing cycle; above that, Enterprise via an interest form, with no public price.

Wins when: your organization is already on Enterprise for other X-side workloads, you need X-side compliance signal for regulated industries (financial services, election-integrity research), or you have infrastructure integrations tied to X's own delivery formats (PowerTrack).

Loses when: single research project, small team, cost sensitivity, or a pull that runs past the 3M-post-read monthly cap — the per-read cost and the cap dwarf any technical difference vs the twitterapi.io path.

07 — Section

Common historical workflows

Academic research: longitudinal studies of discourse, sentiment, or network structure across years. Cashtag studies, election-year mention analysis, virality decay curves.

Backtest data for trading systems: cashtag mention counts + sentiment as features for quant strategies. Requires clean minute-level or hour-level historical coverage.

Brand + reputation forensics: retroactive analysis of a crisis event — what did people say about brand X in the 72 hours after event Y, three years ago?

Deleted-tweet research: cross-reference historical pulls against current live-lookup to find tweets that have been removed. See /blog/deleted-tweet-search for the dedicated workflow.

Legal / compliance discovery: subpoena response, regulated-industry archives, defamation cases requiring historical evidence of specific tweets.

python

# End-to-end example: 6-month cashtag longitudinal pull.
import os, requests, json, csv
from pathlib import Path
from datetime import date, timedelta
from collections import Counter

HEADERS = {"X-API-Key": os.environ["TWITTERAPI_IO_KEY"]}
BASE = "https://api.twitterapi.io"
CASHTAG = "$AAPL"
START, END = date(2024, 1, 1), date(2024, 7, 1)
OUT = Path("aapl_historical.jsonl")

def pull_day(d: date) -> int:
    query = f"{CASHTAG} since:{d} until:{d + timedelta(days=1)}"
    n, cursor = 0, None
    with open(OUT, "a") as f:
        for _ in range(500):
            params = {"query": query}
            if cursor: params["cursor"] = cursor
            r = requests.get(f"{BASE}/twitter/tweet/advanced_search", headers=HEADERS, params=params, timeout=30)
            r.raise_for_status()
            resp = r.json()
            for t in resp.get("tweets", []):
                f.write(json.dumps(t) + "\n"); n += 1
            cursor = resp.get("next_cursor")
            if not cursor: break
    return n

totals = {}
d = START
while d < END:
    n = pull_day(d)
    totals[d.isoformat()] = n
    print(f"  {d}: {n:,} tweets")
    d += timedelta(days=1)

total = sum(totals.values())
print(f"\nCashtag {CASHTAG} pull: {total:,} tweets across {(END - START).days} days")
print(f"Cost per twitterapi.io/pricing: {total} × $0.00015 = ${total * 0.00015:.2f}")

# Aggregate daily counts for time-series analysis
with open("aapl_daily_counts.csv", "w") as cf:
    w = csv.writer(cf)
    w.writerow(["date", "tweet_count"])
    for d, n in sorted(totals.items()):
        w.writerow([d, n])

08 — Questions

Questions readers ask

How far back does the historical archive go?

Twitter launched in 2006. Both twitterapi.io's advanced_search with since:/until: and X's Full-Archive Search access tweets from then forward. Practical coverage is best from ~2010 onward — earlier years have thinner data volume.

What's the max window size for a single query?

No hard limit on the since:/until: window itself, but cursor pagination performance degrades past ~500K tweets in one range. Best practice: split multi-year pulls into per-day or per-week sub-queries with the pattern shown above.

Can I filter historical results by engagement?

Yes — combine min_faves:, min_retweets:, min_replies: with since:/until: in the same query. Very useful for signal-only pulls when the full firehose would be too much data + cost.

Are deleted tweets included in historical results?

Tweets deleted before the query runs may or may not appear — depends on X's archive state at query time. For deleted-tweet-specific research, snapshot the archive at time T then re-check at time T+30 for a diff. See /blog/deleted-tweet-search.

How does this compare to Bright Data or Apify historical scrapers?

Bright Data + Apify historical Twitter scrapers exist but are typically more expensive per tweet at meaningful scale + rely on browser automation (breakability + ToS gray-area). twitterapi.io's direct-API path is cheaper per tweet + more stable long-term for pure read workflows.

Can I get retweet + reply chains for a historical tweet?

Yes — once you have the tweet ID from the historical search, /twitter/tweet/retweeters and /twitter/tweet/replies pull the engagement chain for any historical tweet (same pricing pattern applies to those endpoints).

Any ToS concerns pulling millions of historical tweets?

twitterapi.io operates under its own commercial terms for the historical data it fronts — read the terms + typical acceptable-use for redistribution rules if you plan to re-share the raw data. For internal research, analytics, and derived-metric use, the platform is designed for this workload.

09 — Further reading

Continue

Sources & further reading

Stop reading. Start building.

Starter credits cover real testing on real data. Google sign-in, no card, no application queue.

Get an API key