Birdclaw (X/Twitter archive tool): what it stores and how to feed it any account

Birdclaw is a TypeScript CLI plus a small web app, published under MIT by Peter Steinberger and explicitly "not affiliated with X Corp" (README). Its tagline is "Your Twitter history, with a longer memory." At the time of writing the repository shows about 1.7k stars, 156 forks and just over 700 commits. The npm and Homebrew builds "retain the public Node.js >=26.5.1 <27 contract", so an older Node will refuse to run it.

The README's own description of the storage model is the clearest statement of what the tool does: "SQLite is the canonical store. Archive imports and live transports converge on the same tables, and FTS5 powers local tweet and DM search." Everything lives under ~/.birdclaw unless BIRDCLAW_HOME points elsewhere, and a backup surface writes "deterministic JSONL shards that round-trip through Git", which is how people version their archive in a private repo. The stored object types are tweets, DMs, likes, bookmarks, profiles, media and follow edges.

The quick start needs no credentials and no network: birdclaw init --demo seeds sample tweets, DMs, profiles and links, birdclaw search tweets "local-first" --limit 3 --json queries them, and birdclaw serve opens the web app at http://localhost:3000. The real workflow replaces the demo with birdclaw import archive ~/Downloads/twitter-archive.zip --json, after which birdclaw sync timeline --limit 100 --refresh --json and birdclaw sync bookmarks --mode auto --limit 100 --refresh --json can top the store up with live data, and birdclaw mcp exposes it to an agent.

What it is not. It is not an X API client in its own right; the README is explicit that "archive and local search work without an X login" and that "live sync delegates to xurl or an existing private bird installation and only runs when requested". It is not a tool for archiving other people's accounts: the import reads your own data export, and the live transports act as whoever is logged in. And it is not a hosted service; nothing leaves your machine unless you deliberately expose the MCP endpoint, which "remains off until its token and public URL are configured" (README). An environment variable, BIRDCLAW_DISABLE_LIVE_WRITES=1, turns off every write path globally for people who want a strictly read-only install.

Every row in the birdclaw database arrives by one of three transports, and the one you use decides which accounts you can cover, whether an X login is involved, what it costs and what it puts at risk. The fourth row is the route this page adds for accounts that are not yours.

The xurl row deserves a precise description because birdclaw depends on it. xurl is X's own command-line tool, maintained by the xdevplatform organisation under MIT, "for interacting with the X (formerly Twitter) API, supporting both OAuth 1.0a and OAuth 2.0 authentication" (github.com/xdevplatform/xurl). Setup is xurl auth apps add my-app --client-id ... --client-secret ... followed by xurl auth oauth2 --app my-app, and from then on xurl /2/users/me or any other v2 path runs against your developer app. It is the correct, supported way to read your own bookmarks and timeline, and it is billed at the official pay-per-use rates whenever it reads posts.

The cookie row is the one to be careful with. The README notes that native web access "handles DM requests using AUTH_TOKEN and CT0 session cookies" and that neither that path nor xurl needs a browser process. A session cookie is your identity: whatever a tool does with it, X sees your account doing it, and scripted traffic from a user session is the kind X's terms prohibit and the kind that gets accounts locked. Treat the cookie path as "my account, my risk", never as a way to read someone else's history.

You request the export on X under Settings and privacy, Your account, Download an archive of your data; X asks for your password and a verification code, then prepares the file and notifies you in-app and by email when it is ready, which is usually within about a day and longer for large accounts. The download link is valid for a limited time, so save the zip somewhere birdclaw can reach it (X Help Center, help.x.com/en/managing-your-account/how-to-download-your-x-archive). X renames settings entries from time to time, so confirm the exact menu labels in your own settings screen.

Inside the zip there is a Your archive.html viewer, an assets/ folder for that viewer, and a data/ folder that holds the actual content. Each data file is a JavaScript assignment rather than plain JSON, in the form window.YTD.tweets.part0 = [ ... ]; strip the left-hand side and you have a JSON array. The files that matter for a search archive are tweets.js (every post you wrote, with engagement counts and entities), like.js (posts you liked, with their text), direct-messages.js (every conversation, with timestamps and participants), follower.js and following.js (the follow graph as account IDs), account.js and profile.js (your handle, creation date and bio) and a media folder for attached images and video. Birdclaw's importer maps those into its tweets, DMs, likes, bookmarks, profiles, media and follow-edge tables and builds the FTS5 index over tweet and DM text.

Two limits follow from the format. First, the archive is a snapshot: a post you write tomorrow is not in it, which is why birdclaw has sync commands at all. Second, and more important for anyone reading this page with another account in mind, there is no archive request for a handle you do not control. X prepares the export for the logged-in account only. If the history you want is somebody else's public timeline, the zip cannot help, and the next two sections are about what can.

bird was Steinberger's earlier command-line client for X. It authenticated with browser session cookies and spoke to the same GraphQL endpoints the x.com web app uses, which is why it needed no developer app and no payment. On 8 February 2026 he posted: "Since lots of people asked, I had to take down bird - it used the Twitter Web API and X now pushes pay-per-use." (x.com/steipete/status/2020295295406358798). The birdclaw README still accepts "an existing private bird installation" as a live transport for people who kept a copy, but there is nothing to install from the public repository, and the behaviour that made it convenient, a logged-in session driven by a script, is the behaviour X's systems flag. The same author's X account was itself temporarily blocked for what X labelled suspicious activity and restored a few hours later, as reported by mezha.net.

The "pay-per-use" the post refers to is X's current developer pricing. The docs.x.com pricing page lists "Posts: Read" at $0.005 per resource, "User: Read" at $0.010 per resource and "Post: Create" at $0.015 per request, with no subscription and a ceiling of 3 million post reads per monthly billing cycle before an Enterprise contract (docs.x.com/x-api/getting-started/pricing, verified 2026). That is the rate birdclaw sync pays through xurl, and it is the rate any other official-API client pays: reading one account's 3,000-post history is 3,000 x $0.005 = $15.00, plus $0.010 for the profile, plus the developer-portal setup before the first call.

Set beside the alternative this page uses, the official rate is the expensive route for reads. Twitterapi.io bills 15 credits per tweet returned on timeline, mentions and search endpoints, $0.15 per 1,000 tweets, with a 15-credit minimum per request; $5.00 per 1,000 official post reads against $0.15 per 1,000 is a ratio of about 33x. The official API buys you X's own support and the documented v2 operator set, and it is the right choice when you need to act as a user with their consent. For the one job this page is about, reading a public account's history into a local database, the ratio is the whole story, and the next section shows the code.

The script at the end of this page reproduces the part of birdclaw that matters for an agent, a local SQLite file with an FTS5 index over tweet text, for any public handle, from four read endpoints on https://api.twitterapi.io. Every call is a GET with your key in the X-API-Key header; there is no OAuth step, no X account and no cookie. The mapping from the archive files birdclaw expects to the endpoint that returns the same thing for someone else's account:

Three response-shape details decide whether a port of birdclaw's importer works first time. /twitter/user/last_tweets is the odd one out: it nests the list at data.tweets while has_next_page and next_cursor stay at the top level, and it wraps errors as status: "error" with the reason in msg rather than an HTTP 404. /twitter/user/mentions and /twitter/user/followers put their lists (tweets, followers) at the top level next to the cursor. Tweet objects use likeCount, retweetCount, replyCount, viewCount, createdAt (classic format, Tue Dec 10 07:00:30 +0000 2024) and author.userName; follower objects use snake_case. /twitter/user/info returns the profile under data and, for a deleted or suspended handle, status: "error" and msg: "user not found" with data null, again never a 404.

The schema below is the minimum that gives you birdclaw-style search: three ordinary tables and one FTS5 virtual table that indexes tweets.text as an external-content table, kept in step by an insert trigger. Because the trigger fires only on a real insert and the script uses INSERT OR IGNORE, re-running it is incremental: posts already archived cost nothing to skip in the database, and the run report tells you how many pages were already known so you can stop paging early on the next run.

The arithmetic is short enough to check by hand. Timeline pages return up to 20 posts, so a 3,000-post history is 3,000 / 20 = 150 calls to /twitter/user/last_tweets. Twitterapi.io bills 15 credits per tweet returned, so 3,000 x 15 = 45,000 credits, and at $1 per 100,000 credits that is $0.45. The profile row adds 18 credits ($0.00018); a page of 20 mentions adds 300 credits ($0.003); a 200-follower page adds between 200 and 600 credits ($0.002 to $0.006), since followers bill 1 to 3 credits per user returned. The same 3,000 post reads at the official pay-per-use rate are 3,000 x $0.005 = $15.00, plus $0.010 for the user read (docs.x.com pricing, verified 2026).

Scale it by ten and nothing changes except the zeros: a 30,000-post account is 1,500 calls, 450,000 credits, $4.50 on twitterapi.io against $150.00 officially. The chart below plots both cases; it is derived from the two published price lists, not a measurement, and the 33x ratio ($5.00 / $0.15 per 1,000 posts) holds at every size because both prices are linear in the number of posts.

Time matters as much as money for a full-history pull. Before your first payment the account is limited to 0.2 requests per second, so 150 calls take about 13 minutes at the script's default 5.1-second pace; once paid the limit rises to 20 requests per second and the same pull is under a minute even at a polite pace (50 on Scale, 500 on Business). The $1 of free credits a new account receives on sign-up, 100,000 credits for accounts that pass the automatic bot check, one per person or network, expiring after 31 days, is enough for two full 3,000-post histories with change, which is a reasonable way to test the script before deciding whether the archive is worth keeping.

Birdclaw's reason to exist is the agent on the other end, and it ships its own Model Context Protocol server for that: birdclaw mcp exposes "read-only cached tweet search and thread tools behind a dedicated token", and the endpoint stays off until that token and a public URL are configured (README). The tools answer from the local SQLite store, so an OpenClaw or Claude session that needs "what did I say about X in 2023" gets it with no network call and no spend. What they cannot do is answer about a timeline that is not in the store, and they cannot see anything newer than the last import or sync.

For live reads of any public account, twitterapi.io hosts an MCP server at https://mcp.twitterapi.io/mcp (Streamable HTTP, API key in the X-API-Key header) and publishes a stdio package, @kaitoinfra/twitterapi-io-mcp-server, run with npx -y @kaitoinfra/twitterapi-io-mcp-server and the key in TWITTERAPI_IO_API_KEY. Both expose the same 12 tools: search_tweets, get_user_info, get_user_about, get_user_followers, get_user_followings, get_user_last_tweets, get_user_mentions, get_tweets_by_ids, get_tweet_replies, get_tweet_quotes, get_tweet_retweeters and a trends tool. get_user_last_tweets and get_user_mentions are the same reads the script in this page makes, billed the same way, which means an agent can either query live or query the archive you built, whichever is cheaper for the question.

A practical pattern is to run the archive script on a schedule (it is incremental, so a daily run pulls only the first page or two) and give the agent both servers: the local one for history and the hosted one for live questions. The configuration for Claude Code is a single command; the stdio form below suits OpenClaw and any other MCP host that spawns local processes. The full setup for Claude, including the Claude.ai connector form, is on the Claude guide linked below.

Rate limits and pacing. Twitterapi.io allows 0.2 requests per second before the first payment, 20 after it, 50 on the Scale plan and 500 on Business, with up to 1,000 on request. The script's PACE_SECONDS default of 5.1 keeps an unpaid key inside the first limit; set it to 0.05 once you have paid. A full history is a one-off: pull it once, store it, and let later runs stop as soon as a page comes back entirely known, which the run report makes visible. Do not re-pull 150 pages every night to find three new posts.

Public accounts only, and protected accounts are off limits. /twitter/user/info returns protected: true for accounts that have locked their posts, and the script exits there, because no read endpoint returns a protected account's timeline and nothing on this page is about getting around that. Deleted posts are gone too: the API returns what X currently serves, so an archive built today will not contain a post deleted yesterday, and the only copy of a deleted post is one you captured earlier. If you need that, the deleted-tweet and history-export guides linked below cover scheduled capture.

Do not share session cookies. The cookie path birdclaw inherited from bird works by copying AUTH_TOKEN and CT0 out of your browser. Anyone holding those values is you, as far as X is concerned, until the session is revoked. Never paste them into a hosted tool, never commit them, and expect X to lock an account whose session suddenly starts issuing scripted traffic; that risk is the reason the author withdrew the tool. Reads through twitterapi.io carry none of it, because there is no X session involved.

Respect the platform and the people in the data. X's terms govern access to its service, and data-protection law applies to personal data whether or not it was public. Keep the archive to the accounts and fields your question needs, note the source and time of each pull (the script stores createdAt as served; add a fetched_at column if provenance matters to you), and delete what you no longer use. A searchable archive makes a question cheap to ask; it does not make every question appropriate.