Historical Twitter (X) Data + Archive API — Complete Guide
Historical Twitter (X) data — anything older than the ~7-day recent-search window — has always been the hardest data class to access. X gates it behind the Enterprise Full-Archive tier (contract-priced, typically inaccessible without an enterprise procurement process). For most developers, researchers, and analysts, the practical path is a third-party endpoint that fronts the same underlying archive at commodity pricing.
This guide walks the actual query pattern, the pagination shape, cost math at 4 different volumes, and a real whale case study — plus the Enterprise-tier alternative for teams who need it.
Why historical data is the hardest class
The X archive dates back to March 2006 — ~1.5 trillion tweets. It's the definitive historical record of public conversation on the platform. Access to it is intentionally gated because it's expensive to serve at scale (index + storage + rate infrastructure).
X's own tiers: Basic ($200/mo per docs.x.com/x-api/getting-started/pricing) exposes only recent-search (~7 days). Pro ($5K/mo) same window. Full-archive access lives only on the Enterprise tier — no self-serve, contract-required, typically starting at $42,000/month based on public references and community reports; actual pricing varies by contract terms per docs.x.com/x-api/enterprise/getting-started.
twitterapi.io exposes the same underlying archive via /twitter/tweet/advanced_search with since: and until: operators — no tier gating, per-tweet pricing. That commoditization is the reason ~90% of academic + analytical historical-tweet workflows now route through this class of provider rather than paying Enterprise.
The core query pattern
The magic is the combination of since: + until: date operators inside the query string. Same X native search grammar (docs.x.com/x-api/enterprise/premium-search-api/api-reference/premium-search) — the twitterapi.io endpoint accepts them identically.
Date format is YYYY-MM-DD in UTC. Range is inclusive on since:, exclusive on until:. Both operators can be combined with min_faves:, min_retweets:, from:, lang:, and other operators for tight filtering.
import os, requests
HEADERS = {"X-API-Key": os.environ["TWITTERAPI_IO_KEY"]}
BASE = "https://api.twitterapi.io"
def historical_search(kw: str, since: str, until: str, min_faves: int = 0):
query_parts = [kw, f"since:{since}", f"until:{until}"]
if min_faves: query_parts.append(f"min_faves:{min_faves}")
query = " ".join(query_parts)
tweets, cursor = [], None
for _ in range(500): # safety cap
params = {"query": query}
if cursor: params["cursor"] = cursor
r = requests.get(f"{BASE}/twitter/tweet/advanced_search", headers=HEADERS, params=params, timeout=30)
r.raise_for_status()
resp = r.json()
tweets.extend(resp.get("tweets", []))
cursor = resp.get("next_cursor")
if not cursor: break
return tweets
# Example: all $TSLA mentions with 100+ likes in Jan 2021
tweets = historical_search("$TSLA", "2021-01-01", "2021-02-01", min_faves=100)
print(f"pulled {len(tweets):,} historical tweets")
# Cost per twitterapi.io/pricing: len(tweets) × $0.00015Real whale case — $2.6K for 17.5M tweets
A researcher on our platform ran a longitudinal cashtag mention study across 2018-2024 (6 years, single cashtag, no engagement threshold). They pulled 17.5M tweets total, paginated in 7-day windows across ~312 date ranges to stay well within the per-request cursor headroom.
Total cost: $2,625 ($0.00015 × 17.5M). Total wall-clock time: ~14 hours (paced across 3 days to avoid burst-throughput edge cases). The dataset went into a Postgres warehouse + downstream event-study analysis.
Enterprise equivalent for the same 17.5M archive pull: back-of-envelope $150K+ minimum (Enterprise contracts start ~$42K/mo × multi-month terms + per-post overages on very-large windows). ~57× cost delta before any procurement friction.
This is the class of workflow the twitterapi.io historical path unblocks — quantitative research, longitudinal analysis, backtest data generation — priced so that a single researcher / small team can afford it without enterprise procurement.
Cost math at 4 volumes
Historical pulls compound differently than recent-search work — you're bounded by the volume-of-time-window × keyword-selectivity product, not by monthly rate.
Two takeaways: (a) below 10M tweets, twitterapi.io economics make researcher / small-team workloads possible that would be gated at Enterprise; (b) above 100M tweets, both paths become significant cost — but only twitterapi.io lets you self-serve to that volume without procurement + contract-lock-in.
Query construction for large windows
For pulls spanning years, don't submit a single 5-year since:...until:... query — cursor pagination gets slow past ~500K tweets in one range and edge cases start appearing.
Best pattern: split the window into per-day or per-week sub-queries, iterate, batch-write to storage. Each sub-query independently paginates + you get natural checkpointing (resume from failed day rather than restart the whole 5-year pull).
# Bulk historical pull — window-sliced pattern.
import os, requests, json
from pathlib import Path
from datetime import date, timedelta
HEADERS = {"X-API-Key": os.environ["TWITTERAPI_IO_KEY"]}
BASE = "https://api.twitterapi.io"
OUT_DIR = Path("historical_pull")
OUT_DIR.mkdir(exist_ok=True)
def date_range(start: date, end: date, step_days: int = 7):
d = start
while d < end:
nxt = min(d + timedelta(days=step_days), end)
yield d, nxt
d = nxt
def pull_window(kw: str, since: date, until: date):
out_path = OUT_DIR / f"{kw.replace(' ', '_')}_{since}.jsonl"
if out_path.exists() and out_path.stat().st_size > 0:
return -1 # already done — skip
query = f"{kw} since:{since} until:{until}"
n, cursor = 0, None
with open(out_path, "w") as f:
for _ in range(500):
params = {"query": query}
if cursor: params["cursor"] = cursor
r = requests.get(f"{BASE}/twitter/tweet/advanced_search", headers=HEADERS, params=params, timeout=30)
r.raise_for_status()
resp = r.json()
for t in resp.get("tweets", []):
f.write(json.dumps(t) + "\n"); n += 1
cursor = resp.get("next_cursor")
if not cursor: break
return n
total = 0
for since, until in date_range(date(2018, 1, 1), date(2024, 12, 31), step_days=7):
n = pull_window("$TSLA min_faves:5", since, until)
if n >= 0:
total += n
print(f" {since}: pulled {n:,}")
print(f"\ntotal historical: {total:,} tweets")Alternative — X Enterprise Full-Archive
For teams already on X's Enterprise contract, Full-Archive Search is available at the top tier. Bearer-token auth via X Developer Console + separate Enterprise onboarding.
Endpoint: /2/tweets/search/all (Full-Archive) per docs.x.com/x-api/enterprise/getting-started.
Pricing: contract-negotiated, no self-serve. Public references + community reports put entry at ~$42,000/month with per-post surcharges on very-large windows.
Wins when: your organization is already on Enterprise for other X-side workloads, you need X-side compliance signal for regulated industries (financial services, election-integrity research), or you have infrastructure integrations tied to X's own delivery formats (PowerTrack).
Loses when: single research project, small team, cost sensitivity, or you don't already have Enterprise procurement approval — the friction dwarfs any technical difference vs the twitterapi.io path.
Common historical workflows
Academic research: longitudinal studies of discourse, sentiment, or network structure across years. Cashtag studies, election-year mention analysis, virality decay curves.
Backtest data for trading systems: cashtag mention counts + sentiment as features for quant strategies. Requires clean minute-level or hour-level historical coverage.
Brand + reputation forensics: retroactive analysis of a crisis event — what did people say about brand X in the 72 hours after event Y, three years ago?
Deleted-tweet research: cross-reference historical pulls against current live-lookup to find tweets that have been removed. See /blog/deleted-tweet-search for the dedicated workflow.
Legal / compliance discovery: subpoena response, regulated-industry archives, defamation cases requiring historical evidence of specific tweets.
# End-to-end example: 6-month cashtag longitudinal pull.
import os, requests, json, csv
from pathlib import Path
from datetime import date, timedelta
from collections import Counter
HEADERS = {"X-API-Key": os.environ["TWITTERAPI_IO_KEY"]}
BASE = "https://api.twitterapi.io"
CASHTAG = "$AAPL"
START, END = date(2024, 1, 1), date(2024, 7, 1)
OUT = Path("aapl_historical.jsonl")
def pull_day(d: date) -> int:
query = f"{CASHTAG} since:{d} until:{d + timedelta(days=1)}"
n, cursor = 0, None
with open(OUT, "a") as f:
for _ in range(500):
params = {"query": query}
if cursor: params["cursor"] = cursor
r = requests.get(f"{BASE}/twitter/tweet/advanced_search", headers=HEADERS, params=params, timeout=30)
r.raise_for_status()
resp = r.json()
for t in resp.get("tweets", []):
f.write(json.dumps(t) + "\n"); n += 1
cursor = resp.get("next_cursor")
if not cursor: break
return n
totals = {}
d = START
while d < END:
n = pull_day(d)
totals[d.isoformat()] = n
print(f" {d}: {n:,} tweets")
d += timedelta(days=1)
total = sum(totals.values())
print(f"\nCashtag {CASHTAG} pull: {total:,} tweets across {(END - START).days} days")
print(f"Cost per twitterapi.io/pricing: {total} × $0.00015 = ${total * 0.00015:.2f}")
# Aggregate daily counts for time-series analysis
with open("aapl_daily_counts.csv", "w") as cf:
w = csv.writer(cf)
w.writerow(["date", "tweet_count"])
for d, n in sorted(totals.items()):
w.writerow([d, n])Questions readers ask
How far back does the historical archive go?
Twitter launched March 21, 2006. Both twitterapi.io's advanced_search with since:/until: and X's Enterprise Full-Archive access tweets from that date forward. Practical coverage is best from ~2010 onward — earlier years have thinner data volume.
What's the max window size for a single query?
No hard limit on the since:/until: window itself, but cursor pagination performance degrades past ~500K tweets in one range. Best practice: split multi-year pulls into per-day or per-week sub-queries with the pattern shown above.
Can I filter historical results by engagement?
Yes — combine min_faves:, min_retweets:, min_replies: with since:/until: in the same query. Very useful for signal-only pulls when the full firehose would be too much data + cost.
Are deleted tweets included in historical results?
Tweets deleted before the query runs may or may not appear — depends on X's archive state at query time. For deleted-tweet-specific research, snapshot the archive at time T then re-check at time T+30 for a diff. See /blog/deleted-tweet-search.
How does this compare to Bright Data or Apify historical scrapers?
Bright Data + Apify historical Twitter scrapers exist but are typically more expensive per tweet at meaningful scale + rely on browser automation (breakability + ToS gray-area). twitterapi.io's direct-API path is cheaper per tweet + more stable long-term for pure read workflows.
Can I get retweet + reply chains for a historical tweet?
Yes — once you have the tweet ID from the historical search, /twitter/tweet/retweeters and /twitter/tweet/replies pull the engagement chain for any historical tweet (same pricing pattern applies to those endpoints).
Any ToS concerns pulling millions of historical tweets?
twitterapi.io operates under its own commercial terms for the historical data it fronts — read the terms + typical acceptable-use for redistribution rules if you plan to re-share the raw data. For internal research, analytics, and derived-metric use, the platform is designed for this workload.
Continue
- twitterapi.io — pricing
- X API — pricing (docs.x.com, 2026 verified)
- X — Enterprise API getting started
- X — Enterprise Premium Search API reference
- Twitter (X) API — cluster hub
- Twitter (X) advanced search operators + API
- Twitter (X) data for academic research
- Twitter (X) dataset export — researcher guide
- Deleted tweet search + recovery
- twitterapi.io pricing
Stop reading. Start building.
Starter credits cover real testing on real data. Google sign-in, no card, no application queue.
Get an API key