How to Scrape Twitter (X) Without the Official API

The official API isn't bad — it's just heavy for pure data collection. Three friction points push people to alternatives:

Approval delay: you must create an X developer account and get a project approved before your first authenticated call — typically 1-2 weeks, sometimes rejected.

Cost floor: the cheapest paid tier (Basic) is $200/month even if you make ten calls; the next tier (Pro) is $5,000/month. There's no true pay-as-you-go.

Read cap: all tiers below Enterprise share a 2,000,000 Post-reads/month hard cap. Hit it and you're forced into an Enterprise contract ($42,000+/month).

For a startup, researcher, or indie developer who just needs public tweets, that economics rarely makes sense — hence the search for a no-official-API path.

The cleanest way to get Twitter data without the official API is to call a third-party data API. You don't scrape anything yourself — the provider handles collection and returns structured JSON. TwitterAPI.io is one such service: you sign in with Google, get an API key (no OAuth, no developer approval), and call endpoints like advanced search and user timelines directly.

What you get: ~75 endpoints (tweets, users, followers, mentions, replies, quotes, trends), pay-per-call pricing (~$0.15 per 1,000 tweets, no monthly floor, no read cap), and a $1 trial credit (no card). Setup to first call is about 5 minutes.

Why it's the practical choice: it's reliable (a single vendor maintains the surface, unlike a DIY scraper that breaks on every X change), it's a SaaS API call (cleaner compliance profile than scraping x.com directly), and it scales — you pay only for what you use. The code below is a complete working example.

The catch: it costs money per call (though far less than the official API for most workloads), and it's read-focused — it supports common writes (post/like/retweet/follow) but not DM-send (API DMs trigger account bans).

The truly-free route is scraping X yourself. Two common tools:

snscrape: a Python library that reads X's web responses without login. It was excellent pre-2023, but X's aggressive anti-scrape measures now mean frequent IP bans and rate-limiting. To stay stable you need a rotating proxy pool — which costs money and erodes the "free" advantage. It also occupies a legal grey area (scraping the site vs. calling a SaaS API have different compliance profiles).

Playwright / Puppeteer (headless browser): drives a real browser to load pages and parse the DOM. It works for small volumes but is slow (seconds per page), brittle (breaks whenever X changes its markup, forcing selector rewrites), and ban-prone (triggers captchas and account locks). Realistic ceiling is well under 1,000 items/day before the bans and maintenance overwhelm the value.

When DIY makes sense: a one-off academic sample, a throwaway proof-of-concept, or a learning exercise — situations where reliability doesn't matter and the budget is exactly zero. For anything ongoing or production-grade, almost every team migrates off DIY onto a maintained API.

Here's the full flow with TwitterAPI.io, the Option-1 route:

1. Get an API key: sign in with Google at twitterapi.io and copy the key from the dashboard. The $1 trial credit (~6,000 calls) needs no card.

2. Search or pull a timeline: call /twitter/tweet/advanced_search with a query (keywords, hashtags, from:user, date ranges), or /twitter/user/last_tweets for a specific account's recent posts. Both return structured JSON.

3. Paginate: responses include a cursor; pass it back to fetch the next page until you've collected what you need.

4. Store: write the JSON to your database, CSV, or data warehouse. That's the whole pipeline — no proxies, no browser, no selector maintenance.

The code snippet below does steps 2-3 end to end in Python.

This is the question that matters most for commercial projects, and the answer differs by route.

Calling a third-party SaaS API (Option 1) is the same legal model as using any data vendor — you're making an authorized API call to a service, over HTTPS, governed by that service's terms. It's the cleaner profile. You're still responsible for complying with data-protection law (GDPR, CCPA, PIPL) for any personal data you process and store.

Browser scraping x.com directly (Option 2) is murkier — it may conflict with X's Terms of Service, and the legal treatment of web scraping varies by jurisdiction and purpose (public vs. non-public data, commercial vs. research use). Courts have reached different conclusions in different cases.

Practical guidance: for any commercial or production use, prefer the SaaS-API route and consult your legal team. Collect only the public data you actually need, honor takedown/deletion obligations, and don't store personal data longer than necessary. This is general information, not legal advice.

A quick decision guide:

Choose the third-party API (TwitterAPI.io) if: you need reliability, you're building anything ongoing or production-grade, you want fast setup, you're cost-sensitive but not zero-budget, or you're feeding an AI agent / analytics pipeline. This covers the large majority of real use cases.

Choose DIY scraping if: it's a one-off tiny sample, a learning project, or a throwaway PoC where bans and maintenance don't matter and the budget is exactly $0 — and you accept it'll likely break.

Choose the official X API if: you specifically need write-at-scale, DM-send, the Ads API, or official partner status — none of which the no-official-API routes provide.

For most readers landing on this page, Option 1 is the answer: it removes the official API's friction (no approval, no $200 floor, no read cap) without taking on the fragility of DIY scraping.